* [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP
@ 2014-07-05 17:47 Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 01/14] qcow2: Allow "full" discard Max Reitz
` (13 more replies)
0 siblings, 14 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
qemu-img should use QMP commands whenever possible in order to ensure
feature completeness of both online and offline image operations. For
the "commit" command, this is relatively easy, so implement it first
(in the hope that indeed others will follow).
As qemu-img does not have access to QMP (due to QMP being intertwined
with basically everything in qemu), we cannot directly use QMP, but at
least use the functions the corresponding QMP commands are using (which
would be "block-commit", in this case).
I'll still take a look on Kevin's proposal for patch 3 (make use of lazy
refcounts to simplify emptying a snapshot-less qcow2 image), but I still
think we'll need the size-calculation function anyway; that is why I
sent this version which still uses that function and no lazy refcounts.
In case using lazy refcounts turns out to be vastly easier (which I
actually think it will), I'll send an alternative for patch 3; the other
patches should be untouched by the decision which version is used
(except probably the test from patch 13).
v9: Rebase on master
- Patch 5: block_job_ready() is now called block_job_event_ready() and
has been changed in the meantime
- Patch 6: rebased on the fixes for zero-length mirrors
- Patch 12: test number 094 is now taken as well, move to 097
- Patch 13:
- test number 095 is taken, move to 098
- remove blkdebug.conf when test is done
- filter out test directory and image format from the commit output
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/14:[----] [--] 'qcow2: Allow "full" discard'
002/14:[----] [--] 'qcow2: Implement bdrv_make_empty()'
003/14:[----] [--] 'qcow2: Optimize bdrv_make_empty()'
004/14:[----] [--] 'blockjob: Introduce block_job_complete_sync()'
005/14:[0004] [FC] 'blockjob: Add "ready" field'
006/14:[0006] [FC] 'block/mirror: Improve progress report'
007/14:[----] [--] 'qemu-img: Implement commit like QMP'
008/14:[----] [--] 'qemu-img: Empty image after commit'
009/14:[----] [--] 'qemu-img: Enable progress output for commit'
010/14:[----] [--] 'qemu-img: Specify backing file for commit'
011/14:[----] [--] 'iotests: Add _filter_qemu_img_map'
012/14:[0004] [FC] 'iotests: Add test for backing-chain commits'
013/14:[0013] [FC] 'iotests: Add test for qcow2's bdrv_make_empty'
014/14:[----] [-C] 'iotests: Omit length/offset test in 040 and 041'
Max Reitz (14):
qcow2: Allow "full" discard
qcow2: Implement bdrv_make_empty()
qcow2: Optimize bdrv_make_empty()
blockjob: Introduce block_job_complete_sync()
blockjob: Add "ready" field
block/mirror: Improve progress report
qemu-img: Implement commit like QMP
qemu-img: Empty image after commit
qemu-img: Enable progress output for commit
qemu-img: Specify backing file for commit
iotests: Add _filter_qemu_img_map
iotests: Add test for backing-chain commits
iotests: Add test for qcow2's bdrv_make_empty
iotests: Omit length/offset test in 040 and 041
block/Makefile.objs | 2 +-
block/mirror.c | 34 ++--
block/qcow2-cluster.c | 27 ++-
block/qcow2-snapshot.c | 2 +-
block/qcow2.c | 388 ++++++++++++++++++++++++++++++++++++++-
block/qcow2.h | 2 +-
blockjob.c | 42 ++++-
include/block/blockjob.h | 20 ++
qapi/block-core.json | 4 +-
qemu-img-cmds.hx | 4 +-
qemu-img.c | 149 ++++++++++++---
qemu-img.texi | 13 +-
tests/qemu-iotests/040 | 4 +-
tests/qemu-iotests/041 | 3 +-
tests/qemu-iotests/097 | 122 ++++++++++++
tests/qemu-iotests/097.out | 119 ++++++++++++
tests/qemu-iotests/098 | 75 ++++++++
tests/qemu-iotests/098.out | 26 +++
tests/qemu-iotests/common.filter | 7 +
tests/qemu-iotests/group | 2 +
tests/qemu-iotests/iotests.py | 3 +-
21 files changed, 981 insertions(+), 67 deletions(-)
create mode 100755 tests/qemu-iotests/097
create mode 100644 tests/qemu-iotests/097.out
create mode 100755 tests/qemu-iotests/098
create mode 100644 tests/qemu-iotests/098.out
--
2.0.1
^ permalink raw reply [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 01/14] qcow2: Allow "full" discard
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 02/14] qcow2: Implement bdrv_make_empty() Max Reitz
` (12 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Normally, discarded sectors should read back as zero. However, there are
cases in which a sector (or rather cluster) should be discarded as if
they were never written in the first place, that is, reading them should
fall through to the backing file again.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
---
block/qcow2-cluster.c | 27 +++++++++++++++++----------
block/qcow2-snapshot.c | 2 +-
block/qcow2.c | 2 +-
block/qcow2.h | 2 +-
4 files changed, 20 insertions(+), 13 deletions(-)
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 4208dc0..ce52f9b 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1351,7 +1351,7 @@ int qcow2_decompress_cluster(BlockDriverState *bs, uint64_t cluster_offset)
* clusters.
*/
static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
- unsigned int nb_clusters, enum qcow2_discard_type type)
+ unsigned int nb_clusters, enum qcow2_discard_type type, bool full_discard)
{
BDRVQcowState *s = bs->opaque;
uint64_t *l2_table;
@@ -1373,23 +1373,30 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
old_l2_entry = be64_to_cpu(l2_table[l2_index + i]);
/*
- * Make sure that a discarded area reads back as zeroes for v3 images
- * (we cannot do it for v2 without actually writing a zero-filled
- * buffer). We can skip the operation if the cluster is already marked
- * as zero, or if it's unallocated and we don't have a backing file.
+ * If full_discard is false, make sure that a discarded area reads back
+ * as zeroes for v3 images (we cannot do it for v2 without actually
+ * writing a zero-filled buffer). We can skip the operation if the
+ * cluster is already marked as zero, or if it's unallocated and we
+ * don't have a backing file.
*
* TODO We might want to use bdrv_get_block_status(bs) here, but we're
* holding s->lock, so that doesn't work today.
+ *
+ * If full_discard is true, the sector should not read back as zeroes,
+ * but rather fall through to the backing file.
*/
switch (qcow2_get_cluster_type(old_l2_entry)) {
case QCOW2_CLUSTER_UNALLOCATED:
- if (!bs->backing_hd) {
+ if (full_discard || !bs->backing_hd) {
continue;
}
break;
case QCOW2_CLUSTER_ZERO:
- continue;
+ if (!full_discard) {
+ continue;
+ }
+ break;
case QCOW2_CLUSTER_NORMAL:
case QCOW2_CLUSTER_COMPRESSED:
@@ -1401,7 +1408,7 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
/* First remove L2 entries */
qcow2_cache_entry_mark_dirty(s->l2_table_cache, l2_table);
- if (s->qcow_version >= 3) {
+ if (!full_discard && s->qcow_version >= 3) {
l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
} else {
l2_table[l2_index + i] = cpu_to_be64(0);
@@ -1420,7 +1427,7 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
}
int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
- int nb_sectors, enum qcow2_discard_type type)
+ int nb_sectors, enum qcow2_discard_type type, bool full_discard)
{
BDRVQcowState *s = bs->opaque;
uint64_t end_offset;
@@ -1443,7 +1450,7 @@ int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
/* Each L2 table is handled by its own loop iteration */
while (nb_clusters > 0) {
- ret = discard_single_l2(bs, offset, nb_clusters, type);
+ ret = discard_single_l2(bs, offset, nb_clusters, type, full_discard);
if (ret < 0) {
goto fail;
}
diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c
index 0aa9def..c5ea2cd 100644
--- a/block/qcow2-snapshot.c
+++ b/block/qcow2-snapshot.c
@@ -436,7 +436,7 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)
qcow2_discard_clusters(bs, qcow2_vm_state_offset(s),
align_offset(sn->vm_state_size, s->cluster_size)
>> BDRV_SECTOR_BITS,
- QCOW2_DISCARD_NEVER);
+ QCOW2_DISCARD_NEVER, false);
#ifdef DEBUG_ALLOC
{
diff --git a/block/qcow2.c b/block/qcow2.c
index 67e55c9..ed65679 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1868,7 +1868,7 @@ static coroutine_fn int qcow2_co_discard(BlockDriverState *bs,
qemu_co_mutex_lock(&s->lock);
ret = qcow2_discard_clusters(bs, sector_num << BDRV_SECTOR_BITS,
- nb_sectors, QCOW2_DISCARD_REQUEST);
+ nb_sectors, QCOW2_DISCARD_REQUEST, false);
qemu_co_mutex_unlock(&s->lock);
return ret;
}
diff --git a/block/qcow2.h b/block/qcow2.h
index b49424b..2332634 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -519,7 +519,7 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m);
int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
- int nb_sectors, enum qcow2_discard_type type);
+ int nb_sectors, enum qcow2_discard_type type, bool full_discard);
int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors);
int qcow2_expand_zero_clusters(BlockDriverState *bs);
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 02/14] qcow2: Implement bdrv_make_empty()
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 01/14] qcow2: Allow "full" discard Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty() Max Reitz
` (11 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Implement this function by making all clusters in the image file fall
through to the backing file (by using the recently extended discard).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
---
block/qcow2.c | 27 +++++++++++++++++++++++++++
1 file changed, 27 insertions(+)
diff --git a/block/qcow2.c b/block/qcow2.c
index ed65679..cce6fa9 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2010,6 +2010,32 @@ fail:
return ret;
}
+static int qcow2_make_empty(BlockDriverState *bs)
+{
+ int ret = 0;
+ uint64_t start_sector;
+ int sector_step = INT_MAX / BDRV_SECTOR_SIZE;
+
+ for (start_sector = 0; start_sector < bs->total_sectors;
+ start_sector += sector_step)
+ {
+ /* As this function is generally used after committing an external
+ * snapshot, QCOW2_DISCARD_SNAPSHOT seems appropriate. Also, the
+ * default action for this kind of discard is to pass the discard,
+ * which will ideally result in an actually smaller image file, as
+ * is probably desired. */
+ ret = qcow2_discard_clusters(bs, start_sector * BDRV_SECTOR_SIZE,
+ MIN(sector_step,
+ bs->total_sectors - start_sector),
+ QCOW2_DISCARD_SNAPSHOT, true);
+ if (ret < 0) {
+ break;
+ }
+ }
+
+ return ret;
+}
+
static coroutine_fn int qcow2_co_flush_to_os(BlockDriverState *bs)
{
BDRVQcowState *s = bs->opaque;
@@ -2406,6 +2432,7 @@ static BlockDriver bdrv_qcow2 = {
.bdrv_co_discard = qcow2_co_discard,
.bdrv_truncate = qcow2_truncate,
.bdrv_write_compressed = qcow2_write_compressed,
+ .bdrv_make_empty = qcow2_make_empty,
.bdrv_snapshot_create = qcow2_snapshot_create,
.bdrv_snapshot_goto = qcow2_snapshot_goto,
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty()
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 01/14] qcow2: Allow "full" discard Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 02/14] qcow2: Implement bdrv_make_empty() Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-09 21:22 ` Paolo Bonzini
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 04/14] blockjob: Introduce block_job_complete_sync() Max Reitz
` (10 subsequent siblings)
13 siblings, 1 reply; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
bdrv_make_empty() is currently only called if the current image
represents an external snapshot that has been committed to its base
image; it is therefore unlikely to have internal snapshots. In this
case, bdrv_make_empty() can be greatly sped up by creating an empty L1
table and dropping all data clusters at once by recreating the refcount
structure accordingly instead of normally discarding all clusters.
If there are snapshots, fall back to the simple implementation (discard
all clusters).
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
block/qcow2.c | 389 +++++++++++++++++++++++++++++++++++++++++++++++++++++++---
1 file changed, 374 insertions(+), 15 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index cce6fa9..0aba8dd 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2010,27 +2010,386 @@ fail:
return ret;
}
+/*
+ * Creates a reftable pointing to refblocks following right afterwards and an
+ * empty L1 table at the given @offset. @refblocks is the number of refblocks
+ * to create.
+ *
+ * This combination of structures (reftable+refblocks+L1) is here called a
+ * "blob".
+ */
+static int create_refcount_l1(BlockDriverState *bs, uint64_t offset,
+ uint64_t refblocks)
+{
+ BDRVQcowState *s = bs->opaque;
+ uint64_t *reftable = NULL;
+ uint16_t *refblock = NULL;
+ uint64_t reftable_clusters;
+ uint64_t rbi;
+ uint64_t blob_start, blob_end;
+ uint64_t l2_tables, l1_clusters, l1_size2;
+ uint8_t l1_size_and_offset[12];
+ uint64_t rt_offset;
+ int ret, i;
+
+ reftable_clusters = DIV_ROUND_UP(refblocks, s->cluster_size / 8);
+ l2_tables = DIV_ROUND_UP(bs->total_sectors / s->cluster_sectors,
+ s->cluster_size / 8);
+ l1_clusters = DIV_ROUND_UP(l2_tables, s->cluster_size / 8);
+ l1_size2 = l1_clusters << s->cluster_bits;
+
+ blob_start = offset;
+ blob_end = offset + ((reftable_clusters + refblocks + l1_clusters) <<
+ s->cluster_bits);
+
+ ret = qcow2_pre_write_overlap_check(bs, 0, blob_start,
+ blob_end - blob_start);
+ if (ret < 0) {
+ return ret;
+ }
+
+ /* Create the reftable pointing with entries pointing consecutively to the
+ * following clusters */
+ reftable = g_malloc0_n(reftable_clusters, s->cluster_size);
+
+ for (rbi = 0; rbi < refblocks; rbi++) {
+ reftable[rbi] = cpu_to_be64(offset + ((reftable_clusters + rbi) <<
+ s->cluster_bits));
+ }
+
+ ret = bdrv_write(bs->file, offset >> BDRV_SECTOR_BITS, (uint8_t *)reftable,
+ reftable_clusters * s->cluster_sectors);
+ if (ret < 0) {
+ goto out;
+ }
+
+ offset += reftable_clusters << s->cluster_bits;
+
+ /* Keep the reftable, as we will need it for the BDRVQcowState anyway */
+
+ /* Now, create all the refblocks */
+ refblock = g_malloc(s->cluster_size);
+
+ for (rbi = 0; rbi < refblocks; rbi++) {
+ for (i = 0; i < s->cluster_size / 2; i++) {
+ uint64_t cluster_offset = ((rbi << (s->cluster_bits - 1)) + i)
+ << s->cluster_bits;
+
+ /* Only 0 and 1 are possible as refcounts here */
+ refblock[i] = cpu_to_be16(!cluster_offset ||
+ (cluster_offset >= blob_start &&
+ cluster_offset < blob_end));
+ }
+
+ ret = bdrv_write(bs->file, offset >> BDRV_SECTOR_BITS,
+ (uint8_t *)refblock, s->cluster_sectors);
+ if (ret < 0) {
+ goto out;
+ }
+
+ offset += s->cluster_size;
+ }
+
+ g_free(refblock);
+ refblock = NULL;
+
+ /* The L1 table is very simple */
+ ret = bdrv_write_zeroes(bs->file, offset >> BDRV_SECTOR_BITS,
+ l1_clusters * s->cluster_sectors,
+ BDRV_REQ_MAY_UNMAP);
+ if (ret < 0) {
+ goto out;
+ }
+
+ /* Now make sure all changes are stable and clear all caches */
+ bdrv_flush(bs);
+
+ /* This is probably not really necessary, but it cannot hurt, either */
+ ret = qcow2_mark_clean(bs);
+ if (ret < 0) {
+ goto out;
+ }
+
+ ret = qcow2_cache_empty(bs, s->l2_table_cache);
+ if (ret < 0) {
+ goto out;
+ }
+
+ ret = qcow2_cache_empty(bs, s->refcount_block_cache);
+ if (ret < 0) {
+ goto out;
+ }
+
+ /* Modify the image header to point to the new blank L1 table. This will
+ * leak all currently existing data clusters, which is fine. */
+ BLKDBG_EVENT(bs->file, BLKDBG_L1_UPDATE);
+
+ assert(l1_size2 / 8 <= UINT32_MAX);
+ cpu_to_be32w((uint32_t *)&l1_size_and_offset[0], l1_size2 / 8);
+ cpu_to_be64w((uint64_t *)&l1_size_and_offset[4], offset);
+ ret = bdrv_pwrite_sync(bs->file, offsetof(QCowHeader, l1_size),
+ l1_size_and_offset, sizeof(l1_size_and_offset));
+ if (ret < 0) {
+ goto out;
+ }
+
+ /* Adapt the in-memory L1 table accordingly */
+ s->l1_table_offset = offset;
+ s->l1_size = l1_size2 / 8;
+
+ s->l1_table = g_realloc(s->l1_table, l1_size2);
+ memset(s->l1_table, 0, l1_size2);
+
+ /* Modify the image header to point to the refcount table. This will fix all
+ * leaks (unless an error occurs). */
+ BLKDBG_EVENT(bs->file, BLKDBG_REFTABLE_UPDATE);
+
+ rt_offset = cpu_to_be64(blob_start);
+ ret = bdrv_pwrite_sync(bs->file,
+ offsetof(QCowHeader, refcount_table_offset),
+ &rt_offset, sizeof(rt_offset));
+ if (ret < 0) {
+ goto out;
+ }
+
+ /* The image is now clean. The only thing left is to adapt the in-memory
+ * refcount table to match it. */
+ s->refcount_table_offset = blob_start;
+ s->refcount_table_size = reftable_clusters << (s->cluster_bits - 3);
+
+ for (rbi = 0; rbi < refblocks; rbi++) {
+ be64_to_cpus(&reftable[rbi]);
+ }
+
+ g_free(s->refcount_table);
+ s->refcount_table = reftable;
+ reftable = NULL;
+
+out:
+ g_free(refblock);
+ g_free(reftable);
+ return ret;
+}
+
+/*
+ * Calculates the number of clusters required for an L1 table for an image with
+ * the given parameters plus the reftable and the refblocks required to cover
+ * themselves, the L1 table and a given number of clusters @overhead.
+ *
+ * @ts: Total number of guest sectors the image provides
+ * @cb: 1 << @cb is the cluster size in bytes
+ * @spcb: 1 << @spcb is the number of clusters per sector
+ * @overhead: Number of clusters which shall additionally be covered by the
+ * refcount structures
+ */
+static uint64_t minimal_blob_size(uint64_t ts, int cb, int spcb,
+ uint64_t overhead)
+{
+ uint64_t cs = UINT64_C(1) << cb;
+ uint64_t spc = UINT64_C(1) << spcb;
+
+ /* The following statement is a solution for this system of equations:
+ *
+ * Let req_cls be the required number of clusters required for the reftable,
+ * all refblocks and the L1 table.
+ *
+ * rtc be the clusters required for the reftable, rbc the clusters for all
+ * refblocks (i.e., the number of refblocks), l1c the clusters for the L1
+ * table and l2c the clusters for all L2 tables (i.e., the number of L2
+ * tables).
+ *
+ * cs be the cluster size (in bytes), ts the total number of sectors, spc
+ * the number of sectors per cluster and tc the total number of clusters.
+ *
+ * overhead is a number of clusters which should additionally be covered by
+ * the refcount structures (i.e. all clusters before this blob of req_cls
+ * clusters).
+ *
+ * req_cls >= rtc + rbc + l1c
+ * -- should be obvious
+ * rbc = ceil((overhead + req_cls) / (cs / 2))
+ * -- as each refblock holds cs/2 entries
+ * rtc = ceil(rbc / (cs / 8))
+ * -- as each reftable cluster holds cs/8 entries
+ * tc = ceil(ts / spc)
+ * -- should be obvious as well
+ * l2c = ceil(tc / (cs / 8))
+ * -- as each L2 table holds cs/8 entries
+ * l1c = ceil(l2c / (cs / 8))
+ * -- as each L1 table cluster holds cs/8 entries
+ *
+ * The following equation yields a result which satisfies the constraint.
+ * The result may be too high, but is never too low.
+ *
+ * The original calculation (without bitshifting) was:
+ *
+ * DIV_ROUND_UP(overhead * (1 + cs / 8) +
+ * 3 * cs * cs / 16 - 5 * cs / 8 - 1 +
+ * 4 * (ts + spc * cs / 8 - 1) / spc,
+ * cs * cs / 16 - cs / 8 - 1)
+ *
+ */
+
+ return DIV_ROUND_UP(overhead + (overhead << (cb - 3)) +
+ (3 << (2 * cb - 4)) - (5 << (cb - 3)) - 1 +
+ (4 * (ts + (spc << (cb - 3)) - 1) >> spcb),
+ (1 << (2 * cb - 4)) - cs / 8 - 1);
+}
+
static int qcow2_make_empty(BlockDriverState *bs)
{
+ BDRVQcowState *s = bs->opaque;
int ret = 0;
- uint64_t start_sector;
- int sector_step = INT_MAX / BDRV_SECTOR_SIZE;
- for (start_sector = 0; start_sector < bs->total_sectors;
- start_sector += sector_step)
- {
- /* As this function is generally used after committing an external
- * snapshot, QCOW2_DISCARD_SNAPSHOT seems appropriate. Also, the
- * default action for this kind of discard is to pass the discard,
- * which will ideally result in an actually smaller image file, as
- * is probably desired. */
- ret = qcow2_discard_clusters(bs, start_sector * BDRV_SECTOR_SIZE,
- MIN(sector_step,
- bs->total_sectors - start_sector),
- QCOW2_DISCARD_SNAPSHOT, true);
+ if (s->snapshots) {
+ uint64_t start_sector;
+ int sector_step = INT_MAX / BDRV_SECTOR_SIZE;
+
+ /* If there are snapshots, every active cluster has to be discarded */
+
+ for (start_sector = 0; start_sector < bs->total_sectors;
+ start_sector += sector_step)
+ {
+ /* As this function is generally used after committing an external
+ * snapshot, QCOW2_DISCARD_SNAPSHOT seems appropriate. Also, the
+ * default action for this kind of discard is to pass the discard,
+ * which will ideally result in an actually smaller image file, as
+ * is probably desired. */
+ ret = qcow2_discard_clusters(bs, start_sector * BDRV_SECTOR_SIZE,
+ MIN(sector_step,
+ bs->total_sectors - start_sector),
+ QCOW2_DISCARD_SNAPSHOT, true);
+ if (ret < 0) {
+ break;
+ }
+ }
+ } else {
+ uint64_t min_size_1, min_size_2;
+ int64_t blob_start;
+ uint64_t blob_end, real_blob_size, clusters;
+ uint64_t refblocks, reftable_clusters, l2_tables, l1_clusters;
+
+ /* If there are no snapshots, we basically want to create a new empty
+ * image. This function is however not for creating a new image and
+ * renaming it so it shadows the existing but rather for emptying an
+ * image, so do exactly that.
+ *
+ * Therefore, the image should be valid at all points in time and may
+ * at worst leak clusters.
+ *
+ * Any valid qcow2 image requires an L1 table which is large enough to
+ * cover all of the guest cluster range, therefore it is impossible to
+ * simply drop the L1 table (make its size 0) and create a minimal
+ * refcount structure.
+ *
+ * Instead, allocate a blob of data which is large enough to hold a new
+ * refcount structure (refcount table and all blocks) covering the whole
+ * image until the end of that blob, and an empty L1 table covering the
+ * whole guest cluster range. Then these structures are initialized and
+ * the image header is modified to point to them.
+ *
+ * Generally, this blob will be allocated at the end of the image (that
+ * is, its offset will be greater than its size). If that is indeed the
+ * case, the same operation is repeated, but this time the new blob
+ * starts right after the image header which will then indeed lead to a
+ * minimal image. If this is not the case, the image will be nearly
+ * minimal as well, as long as the underlying protocol supports discard.
+ *
+ * Note that this implementation never frees allocated clusters. This is
+ * because in case of success, the current refcounts are invalid anyway;
+ * and in case of failure, it would be too cumbersome to keep track of
+ * all allocated cluster ranges and free them then.
+ *
+ * min_size_1 will contain the number of clusters minimally required for
+ * a blob that starts right after the image header; min_size_2 will
+ * contain the number of clusters minimally required for the blob which
+ * can be allocated based on the existing refcount structure.
+ */
+
+ /* Repeat until a blob could be allocated which is large enough to
+ * contain all structures necessary for describing itself. Allocated
+ * clusters are not freed, even if they are not suitable, as this would
+ * result in exactly the same cluster range being returned during the
+ * retry (which is obviously not desirable). In case of success, the
+ * current refcounts do not matter anyway; and in case of failure, the
+ * clusters are only leaked (which can be easily repaired). */
+ do {
+ uint64_t fci = s->free_cluster_index;
+
+ /* TODO: Optimize, we do not need refblocks for other parts of the
+ * image than the header and this blob */
+ min_size_2 = minimal_blob_size(bs->total_sectors, s->cluster_bits,
+ s->cluster_bits - BDRV_SECTOR_BITS,
+ fci);
+
+ blob_start = qcow2_alloc_clusters(bs,
+ min_size_2 << s->cluster_bits);
+ if (blob_start < 0) {
+ return blob_start;
+ }
+
+ clusters = (blob_start >> s->cluster_bits) + min_size_2;
+ refblocks = DIV_ROUND_UP(clusters, s->cluster_size / 2);
+ reftable_clusters = DIV_ROUND_UP(refblocks, s->cluster_size / 8);
+ l2_tables = DIV_ROUND_UP(bs->total_sectors /
+ s->cluster_sectors,
+ s->cluster_size / 8);
+ l1_clusters = DIV_ROUND_UP(l2_tables, s->cluster_size / 8);
+
+ real_blob_size = reftable_clusters + refblocks + l1_clusters;
+ } while (min_size_2 < real_blob_size);
+
+ ret = create_refcount_l1(bs, blob_start, refblocks);
if (ret < 0) {
- break;
+ return ret;
+ }
+
+ /* The only overhead is the image header */
+ min_size_1 = minimal_blob_size(bs->total_sectors, s->cluster_bits,
+ s->cluster_bits - BDRV_SECTOR_BITS, 1);
+
+ /* If we cannot create a new blob before the current one, just discard
+ * the sectors in between and return. Even if the discard does nothing,
+ * only up to min_size_1 clusters plus the refcount blocks for those
+ * are unused. The worst case is therefore an image of double the size
+ * it needs to be, which is not too bad. */
+ if ((blob_start >> s->cluster_bits) < 1 + min_size_1) {
+ uint64_t sector, end;
+ int step = INT_MAX / BDRV_SECTOR_SIZE;
+
+ end = blob_start >> (s->cluster_bits - BDRV_SECTOR_SIZE);
+
+ /* skip the image header */
+ for (sector = s->cluster_sectors; sector < end; sector += step) {
+ ret = bdrv_discard(bs->file, sector, MIN(step, end - sector));
+ if (ret < 0) {
+ return ret;
+ }
+ }
+
+ blob_end = (blob_start + real_blob_size) << s->cluster_bits;
+ } else {
+ clusters = 1 + min_size_1;
+ refblocks = DIV_ROUND_UP(clusters, s->cluster_size / 2);
+ reftable_clusters = DIV_ROUND_UP(refblocks, s->cluster_size / 8);
+ l2_tables = DIV_ROUND_UP(bs->total_sectors /
+ s->cluster_sectors,
+ s->cluster_size / 8);
+ l1_clusters = DIV_ROUND_UP(l2_tables, s->cluster_size / 8);
+
+ real_blob_size = reftable_clusters + refblocks + l1_clusters;
+
+ assert(min_size_1 >= real_blob_size);
+
+ ret = create_refcount_l1(bs, s->cluster_size, refblocks);
+ if (ret < 0) {
+ return ret;
+ }
+
+ blob_end = (1 + real_blob_size) << s->cluster_bits;
}
+
+ ret = bdrv_truncate(bs->file, blob_end);
}
return ret;
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 04/14] blockjob: Introduce block_job_complete_sync()
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (2 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty() Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field Max Reitz
` (9 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Implement block_job_complete_sync() by doing the exact same thing as
block_job_cancel_sync() does, only with calling block_job_complete()
instead of block_job_cancel().
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
---
blockjob.c | 39 ++++++++++++++++++++++++++++++++-------
include/block/blockjob.h | 15 +++++++++++++++
2 files changed, 47 insertions(+), 7 deletions(-)
diff --git a/blockjob.c b/blockjob.c
index 67a64ea..2c98689 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -152,7 +152,7 @@ void block_job_iostatus_reset(BlockJob *job)
}
}
-struct BlockCancelData {
+struct BlockFinishData {
BlockJob *job;
BlockDriverCompletionFunc *cb;
void *opaque;
@@ -160,19 +160,22 @@ struct BlockCancelData {
int ret;
};
-static void block_job_cancel_cb(void *opaque, int ret)
+static void block_job_finish_cb(void *opaque, int ret)
{
- struct BlockCancelData *data = opaque;
+ struct BlockFinishData *data = opaque;
data->cancelled = block_job_is_cancelled(data->job);
data->ret = ret;
data->cb(data->opaque, ret);
}
-int block_job_cancel_sync(BlockJob *job)
+static int block_job_finish_sync(BlockJob *job,
+ void (*finish)(BlockJob *, Error **errp),
+ Error **errp)
{
- struct BlockCancelData data;
+ struct BlockFinishData data;
BlockDriverState *bs = job->bs;
+ Error *local_err = NULL;
assert(bs->job == job);
@@ -183,15 +186,37 @@ int block_job_cancel_sync(BlockJob *job)
data.cb = job->cb;
data.opaque = job->opaque;
data.ret = -EINPROGRESS;
- job->cb = block_job_cancel_cb;
+ job->cb = block_job_finish_cb;
job->opaque = &data;
- block_job_cancel(job);
+ finish(job, &local_err);
+ if (local_err) {
+ error_propagate(errp, local_err);
+ return -EBUSY;
+ }
while (data.ret == -EINPROGRESS) {
qemu_aio_wait();
}
return (data.cancelled && data.ret == 0) ? -ECANCELED : data.ret;
}
+/* A wrapper around block_job_cancel() taking an Error ** parameter so it may be
+ * used with block_job_finish_sync() without the need for (rather nasty)
+ * function pointer casts there. */
+static void block_job_cancel_err(BlockJob *job, Error **errp)
+{
+ block_job_cancel(job);
+}
+
+int block_job_cancel_sync(BlockJob *job)
+{
+ return block_job_finish_sync(job, &block_job_cancel_err, NULL);
+}
+
+int block_job_complete_sync(BlockJob *job, Error **errp)
+{
+ return block_job_finish_sync(job, &block_job_complete, errp);
+}
+
void block_job_sleep_ns(BlockJob *job, QEMUClockType type, int64_t ns)
{
assert(job->busy);
diff --git a/include/block/blockjob.h b/include/block/blockjob.h
index f3cf63f..88f4be2 100644
--- a/include/block/blockjob.h
+++ b/include/block/blockjob.h
@@ -273,6 +273,21 @@ bool block_job_is_paused(BlockJob *job);
int block_job_cancel_sync(BlockJob *job);
/**
+ * block_job_complete_sync:
+ * @job: The job to be completed.
+ * @errp: Error object which may be set by block_job_complete(); this is not
+ * necessarily set on every error, the job return value has to be
+ * checked as well.
+ *
+ * Synchronously complete the job. The completion callback is called before the
+ * function returns, unless it is NULL (which is permissible when using this
+ * function).
+ *
+ * Returns the return value from the job.
+ */
+int block_job_complete_sync(BlockJob *job, Error **errp);
+
+/**
* block_job_iostatus_reset:
* @job: The job whose I/O status should be reset.
*
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (3 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 04/14] blockjob: Introduce block_job_complete_sync() Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-07 18:53 ` Eric Blake
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report Max Reitz
` (8 subsequent siblings)
13 siblings, 1 reply; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
When a block job signals readiness, this is currently reported only
through QMP. If qemu wants to use block jobs for internal tasks, there
needs to be another way to correctly detect when a block job may be
completed.
For this reason, introduce a bool "ready" which is set when the block
job may be completed.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
blockjob.c | 3 +++
include/block/blockjob.h | 5 +++++
qapi/block-core.json | 4 +++-
3 files changed, 11 insertions(+), 1 deletion(-)
diff --git a/blockjob.c b/blockjob.c
index 2c98689..ff1d78e 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -260,6 +260,7 @@ BlockJobInfo *block_job_query(BlockJob *job)
info->offset = job->offset;
info->speed = job->speed;
info->io_status = job->iostatus;
+ info->ready = job->ready;
return info;
}
@@ -295,6 +296,8 @@ void block_job_event_completed(BlockJob *job, const char *msg)
void block_job_event_ready(BlockJob *job)
{
+ job->ready = true;
+
qapi_event_send_block_job_ready(job->driver->job_type,
bdrv_get_device_name(job->bs),
job->len,
diff --git a/include/block/blockjob.h b/include/block/blockjob.h
index 88f4be2..97658ca 100644
--- a/include/block/blockjob.h
+++ b/include/block/blockjob.h
@@ -91,6 +91,11 @@ struct BlockJob {
*/
bool busy;
+ /**
+ * Set to true when the job is ready to be completed.
+ */
+ bool ready;
+
/** Status that is published by the query-block-jobs QMP API */
BlockDeviceIoStatus iostatus;
diff --git a/qapi/block-core.json b/qapi/block-core.json
index e378653..72ebfef 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -505,12 +505,14 @@
#
# @io-status: the status of the job (since 1.3)
#
+# @ready: true if the job may be completed (since 2.1)
+#
# Since: 1.1
##
{ 'type': 'BlockJobInfo',
'data': {'type': 'str', 'device': 'str', 'len': 'int',
'offset': 'int', 'busy': 'bool', 'paused': 'bool', 'speed': 'int',
- 'io-status': 'BlockDeviceIoStatus'} }
+ 'io-status': 'BlockDeviceIoStatus', 'ready': 'bool'} }
##
# @query-block-jobs:
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (4 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-07 19:13 ` Eric Blake
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 07/14] qemu-img: Implement commit like QMP Max Reitz
` (7 subsequent siblings)
13 siblings, 1 reply; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Instead of taking the total length of the block device as the block
job's length, use the number of dirty sectors. The progress is now the
number of sectors mirrored to the target block device. Note that this
may result in the job's length increasing during operation, which is
however in fact desirable.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/mirror.c | 34 ++++++++++++++++++++++------------
1 file changed, 22 insertions(+), 12 deletions(-)
diff --git a/block/mirror.c b/block/mirror.c
index 6c3ee70..39c52c3 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -45,6 +45,7 @@ typedef struct MirrorBlockJob {
int64_t sector_num;
int64_t granularity;
size_t buf_size;
+ int64_t bdev_length;
unsigned long *cow_bitmap;
BdrvDirtyBitmap *dirty_bitmap;
HBitmapIter hbi;
@@ -54,6 +55,7 @@ typedef struct MirrorBlockJob {
unsigned long *in_flight_bitmap;
int in_flight;
+ int sectors_in_flight;
int ret;
} MirrorBlockJob;
@@ -87,6 +89,7 @@ static void mirror_iteration_done(MirrorOp *op, int ret)
trace_mirror_iteration_done(s, op->sector_num, op->nb_sectors, ret);
s->in_flight--;
+ s->sectors_in_flight -= op->nb_sectors;
iov = op->qiov.iov;
for (i = 0; i < op->qiov.niov; i++) {
MirrorBuffer *buf = (MirrorBuffer *) iov[i].iov_base;
@@ -98,8 +101,11 @@ static void mirror_iteration_done(MirrorOp *op, int ret)
chunk_num = op->sector_num / sectors_per_chunk;
nb_chunks = op->nb_sectors / sectors_per_chunk;
bitmap_clear(s->in_flight_bitmap, chunk_num, nb_chunks);
- if (s->cow_bitmap && ret >= 0) {
- bitmap_set(s->cow_bitmap, chunk_num, nb_chunks);
+ if (ret >= 0) {
+ if (s->cow_bitmap) {
+ bitmap_set(s->cow_bitmap, chunk_num, nb_chunks);
+ }
+ s->common.offset += (uint64_t)op->nb_sectors * BDRV_SECTOR_SIZE;
}
qemu_iovec_destroy(&op->qiov);
@@ -172,7 +178,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
hbitmap_next_sector = s->sector_num;
sector_num = s->sector_num;
sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS;
- end = s->common.len >> BDRV_SECTOR_BITS;
+ end = s->bdev_length / BDRV_SECTOR_SIZE;
/* Extend the QEMUIOVector to include all adjacent blocks that will
* be copied in this operation.
@@ -284,6 +290,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
/* Copy the dirty cluster. */
s->in_flight++;
+ s->sectors_in_flight += nb_sectors;
trace_mirror_one_iteration(s, sector_num, nb_sectors);
bdrv_aio_readv(source, sector_num, &op->qiov, nb_sectors,
mirror_read_complete, op);
@@ -329,11 +336,11 @@ static void coroutine_fn mirror_run(void *opaque)
goto immediate_exit;
}
- s->common.len = bdrv_getlength(bs);
- if (s->common.len < 0) {
- ret = s->common.len;
+ s->bdev_length = bdrv_getlength(bs);
+ if (s->bdev_length < 0) {
+ ret = s->bdev_length;
goto immediate_exit;
- } else if (s->common.len == 0) {
+ } else if (s->bdev_length == 0) {
/* Report BLOCK_JOB_READY and wait for complete. */
block_job_event_ready(&s->common);
s->synced = true;
@@ -344,7 +351,7 @@ static void coroutine_fn mirror_run(void *opaque)
goto immediate_exit;
}
- length = DIV_ROUND_UP(s->common.len, s->granularity);
+ length = DIV_ROUND_UP(s->bdev_length, s->granularity);
s->in_flight_bitmap = bitmap_new(length);
/* If we have no backing file yet in the destination, we cannot let
@@ -364,7 +371,7 @@ static void coroutine_fn mirror_run(void *opaque)
}
}
- end = s->common.len >> BDRV_SECTOR_BITS;
+ end = s->bdev_length / BDRV_SECTOR_SIZE;
s->buf = qemu_blockalign(bs, s->buf_size);
sectors_per_chunk = s->granularity >> BDRV_SECTOR_BITS;
mirror_free_init(s);
@@ -404,6 +411,12 @@ static void coroutine_fn mirror_run(void *opaque)
}
cnt = bdrv_get_dirty_count(bs, s->dirty_bitmap);
+ /* s->common.offset contains the number of bytes already processed so
+ * far, cnt is the number of dirty sectors remaining and
+ * s->sectors_in_flight is the number of sectors currently being
+ * processed; together those are the current total operation length */
+ s->common.len = s->common.offset +
+ (cnt + s->sectors_in_flight) * BDRV_SECTOR_SIZE;
/* Note that even when no rate limit is applied we need to yield
* periodically with no pending I/O so that qemu_aio_flush() returns.
@@ -440,7 +453,6 @@ static void coroutine_fn mirror_run(void *opaque)
* report completion. This way, block-job-cancel will leave
* the target in a consistent state.
*/
- s->common.offset = end * BDRV_SECTOR_SIZE;
if (!s->synced) {
block_job_event_ready(&s->common);
s->synced = true;
@@ -469,8 +481,6 @@ static void coroutine_fn mirror_run(void *opaque)
ret = 0;
trace_mirror_before_sleep(s, cnt, s->synced, delay_ns);
if (!s->synced) {
- /* Publish progress */
- s->common.offset = (end - cnt) * BDRV_SECTOR_SIZE;
block_job_sleep_ns(&s->common, QEMU_CLOCK_REALTIME, delay_ns);
if (block_job_is_cancelled(&s->common)) {
break;
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 07/14] qemu-img: Implement commit like QMP
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (5 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 08/14] qemu-img: Empty image after commit Max Reitz
` (6 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
qemu-img should use QMP commands whenever possible in order to ensure
feature completeness of both online and offline image operations. As
qemu-img itself has no access to QMP (since this would basically require
just everything being linked into qemu-img), imitate QMP's
implementation of block-commit by using commit_active_start() and then
waiting for the block job to finish.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
block/Makefile.objs | 2 +-
qemu-img.c | 83 +++++++++++++++++++++++++++++++++++++++++------------
2 files changed, 65 insertions(+), 20 deletions(-)
diff --git a/block/Makefile.objs b/block/Makefile.objs
index fd88c03..2c37e80 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -9,6 +9,7 @@ block-obj-y += snapshot.o qapi.o
block-obj-$(CONFIG_WIN32) += raw-win32.o win32-aio.o
block-obj-$(CONFIG_POSIX) += raw-posix.o
block-obj-$(CONFIG_LINUX_AIO) += linux-aio.o
+block-obj-y += mirror.o
ifeq ($(CONFIG_POSIX),y)
block-obj-y += nbd.o nbd-client.o sheepdog.o
@@ -22,7 +23,6 @@ endif
common-obj-y += stream.o
common-obj-y += commit.o
-common-obj-y += mirror.o
common-obj-y += backup.o
iscsi.o-cflags := $(LIBISCSI_CFLAGS)
diff --git a/qemu-img.c b/qemu-img.c
index c98896b..26d8b11 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -30,6 +30,7 @@
#include "qemu/osdep.h"
#include "sysemu/sysemu.h"
#include "block/block_int.h"
+#include "block/blockjob.h"
#include "block/qapi.h"
#include <getopt.h>
#include <glib.h>
@@ -717,12 +718,46 @@ fail:
return ret;
}
+typedef struct CommonBlockJobCBInfo {
+ BlockDriverState *bs;
+ Error **errp;
+} CommonBlockJobCBInfo;
+
+static void common_block_job_cb(void *opaque, int ret)
+{
+ CommonBlockJobCBInfo *cbi = opaque;
+
+ if (ret < 0) {
+ error_setg_errno(cbi->errp, -ret, "Block job failed");
+ }
+
+ /* Drop this block job's reference */
+ bdrv_unref(cbi->bs);
+}
+
+static void run_block_job(BlockJob *job, Error **errp)
+{
+ AioContext *aio_context = bdrv_get_aio_context(job->bs);
+
+ do {
+ aio_poll(aio_context, true);
+
+ if (!job->busy && !job->ready) {
+ block_job_resume(job);
+ }
+ } while (!job->ready);
+
+ block_job_complete_sync(job, errp);
+}
+
static int img_commit(int argc, char **argv)
{
int c, ret, flags;
const char *filename, *fmt, *cache;
- BlockDriverState *bs;
+ BlockDriverState *bs, *base_bs;
bool quiet = false;
+ Error *local_err = NULL;
+ CommonBlockJobCBInfo cbi;
fmt = NULL;
cache = BDRV_DEFAULT_CACHE;
@@ -763,29 +798,39 @@ static int img_commit(int argc, char **argv)
if (!bs) {
return 1;
}
- ret = bdrv_commit(bs);
- switch(ret) {
- case 0:
- qprintf(quiet, "Image committed.\n");
- break;
- case -ENOENT:
- error_report("No disk inserted");
- break;
- case -EACCES:
- error_report("Image is read-only");
- break;
- case -ENOTSUP:
- error_report("Image is already committed");
- break;
- default:
- error_report("Error while committing image");
- break;
+
+ /* This is different from QMP, which by default uses the deepest file in the
+ * backing chain (i.e., the very base); however, the traditional behavior of
+ * qemu-img commit is using the immediate backing file. */
+ base_bs = bs->backing_hd;
+ if (!base_bs) {
+ error_set(&local_err, QERR_BASE_NOT_FOUND, "NULL");
+ goto done;
+ }
+
+ cbi = (CommonBlockJobCBInfo){
+ .errp = &local_err,
+ .bs = bs,
+ };
+
+ commit_active_start(bs, base_bs, 0, BLOCKDEV_ON_ERROR_REPORT,
+ common_block_job_cb, &cbi, &local_err);
+ if (local_err) {
+ goto done;
}
+ run_block_job(bs->job, &local_err);
+
+done:
bdrv_unref(bs);
- if (ret) {
+
+ if (local_err) {
+ qerror_report_err(local_err);
+ error_free(local_err);
return 1;
}
+
+ qprintf(quiet, "Image committed.\n");
return 0;
}
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 08/14] qemu-img: Empty image after commit
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (6 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 07/14] qemu-img: Implement commit like QMP Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 09/14] qemu-img: Enable progress output for commit Max Reitz
` (5 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
After the top image has been committed, it should be emptied unless
specified otherwise.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
qemu-img-cmds.hx | 4 ++--
qemu-img.c | 34 +++++++++++++++++++++++++++++++---
qemu-img.texi | 6 +++++-
3 files changed, 38 insertions(+), 6 deletions(-)
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index d029609..b31d81c 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -22,9 +22,9 @@ STEXI
ETEXI
DEF("commit", img_commit,
- "commit [-q] [-f fmt] [-t cache] filename")
+ "commit [-q] [-f fmt] [-t cache] [-d] filename")
STEXI
-@item commit [-q] [-f @var{fmt}] [-t @var{cache}] @var{filename}
+@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-d] @var{filename}
ETEXI
DEF("compare", img_compare,
diff --git a/qemu-img.c b/qemu-img.c
index 26d8b11..47cc94e 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -755,14 +755,14 @@ static int img_commit(int argc, char **argv)
int c, ret, flags;
const char *filename, *fmt, *cache;
BlockDriverState *bs, *base_bs;
- bool quiet = false;
+ bool quiet = false, drop = false;
Error *local_err = NULL;
CommonBlockJobCBInfo cbi;
fmt = NULL;
cache = BDRV_DEFAULT_CACHE;
for(;;) {
- c = getopt(argc, argv, "f:ht:q");
+ c = getopt(argc, argv, "f:ht:dq");
if (c == -1) {
break;
}
@@ -777,6 +777,9 @@ static int img_commit(int argc, char **argv)
case 't':
cache = optarg;
break;
+ case 'd':
+ drop = true;
+ break;
case 'q':
quiet = true;
break;
@@ -787,7 +790,7 @@ static int img_commit(int argc, char **argv)
}
filename = argv[optind++];
- flags = BDRV_O_RDWR;
+ flags = BDRV_O_RDWR | BDRV_O_UNMAP;
ret = bdrv_parse_cache_flags(cache, &flags);
if (ret < 0) {
error_report("Invalid cache option: %s", cache);
@@ -819,7 +822,32 @@ static int img_commit(int argc, char **argv)
goto done;
}
+ /* The block job will swap base_bs and bs (which is not what we really want
+ * here, but okay) and unref base_bs (after the swap, i.e., the old top
+ * image). In order to still be able to empty that top image afterwards,
+ * increment the reference counter here preemptively. */
+ if (!drop) {
+ bdrv_ref(base_bs);
+ }
+
run_block_job(bs->job, &local_err);
+ if (local_err) {
+ goto unref_backing;
+ }
+
+ if (!drop && base_bs->drv->bdrv_make_empty) {
+ ret = base_bs->drv->bdrv_make_empty(base_bs);
+ if (ret) {
+ error_setg_errno(&local_err, -ret, "Could not empty %s",
+ filename);
+ goto unref_backing;
+ }
+ }
+
+unref_backing:
+ if (!drop) {
+ bdrv_unref(base_bs);
+ }
done:
bdrv_unref(bs);
diff --git a/qemu-img.texi b/qemu-img.texi
index 8496f3b..fd8ecd6 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -163,7 +163,7 @@ this case. @var{backing_file} will never be modified unless you use the
The size can also be specified using the @var{size} option with @code{-o},
it doesn't need to be specified separately in this case.
-@item commit [-f @var{fmt}] [-t @var{cache}] @var{filename}
+@item commit [-f @var{fmt}] [-t @var{cache}] [-d] @var{filename}
Commit the changes recorded in @var{filename} in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be
@@ -172,6 +172,10 @@ the backing file, the backing file will not be truncated. If you want the
backing file to match the size of the smaller snapshot, you can safely truncate
it yourself once the commit operation successfully completes.
+The image @var{filename} is emptied after the operation has succeeded. If you do
+not need @var{filename} afterwards and intend to drop it, you may skip emptying
+@var{filename} by specifying the @code{-d} flag.
+
@item compare [-f @var{fmt}] [-F @var{fmt}] [-p] [-s] [-q] @var{filename1} @var{filename2}
Check if two images have the same content. You can compare images with
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 09/14] qemu-img: Enable progress output for commit
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (7 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 08/14] qemu-img: Empty image after commit Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 10/14] qemu-img: Specify backing file " Max Reitz
` (4 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Implement progress output for the commit command by querying the
progress of the block job.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
qemu-img-cmds.hx | 4 ++--
qemu-img.c | 24 ++++++++++++++++++++++--
qemu-img.texi | 2 +-
3 files changed, 25 insertions(+), 5 deletions(-)
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index b31d81c..ea41d4f 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -22,9 +22,9 @@ STEXI
ETEXI
DEF("commit", img_commit,
- "commit [-q] [-f fmt] [-t cache] [-d] filename")
+ "commit [-q] [-f fmt] [-t cache] [-d] [-p] filename")
STEXI
-@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-d] @var{filename}
+@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-d] [-p] @var{filename}
ETEXI
DEF("compare", img_compare,
diff --git a/qemu-img.c b/qemu-img.c
index 47cc94e..8409062 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -742,12 +742,18 @@ static void run_block_job(BlockJob *job, Error **errp)
do {
aio_poll(aio_context, true);
+ qemu_progress_print((float)job->offset / job->len * 100.f, 0);
+
if (!job->busy && !job->ready) {
block_job_resume(job);
}
} while (!job->ready);
block_job_complete_sync(job, errp);
+
+ /* A block job may finish instantaneously without publishing any progress,
+ * so just signal completion here */
+ qemu_progress_print(100.f, 0);
}
static int img_commit(int argc, char **argv)
@@ -755,14 +761,14 @@ static int img_commit(int argc, char **argv)
int c, ret, flags;
const char *filename, *fmt, *cache;
BlockDriverState *bs, *base_bs;
- bool quiet = false, drop = false;
+ bool progress = false, quiet = false, drop = false;
Error *local_err = NULL;
CommonBlockJobCBInfo cbi;
fmt = NULL;
cache = BDRV_DEFAULT_CACHE;
for(;;) {
- c = getopt(argc, argv, "f:ht:dq");
+ c = getopt(argc, argv, "f:ht:dpq");
if (c == -1) {
break;
}
@@ -780,11 +786,20 @@ static int img_commit(int argc, char **argv)
case 'd':
drop = true;
break;
+ case 'p':
+ progress = true;
+ break;
case 'q':
quiet = true;
break;
}
}
+
+ /* Progress is not shown in Quiet mode */
+ if (quiet) {
+ progress = false;
+ }
+
if (optind != argc - 1) {
error_exit("Expecting one image file name");
}
@@ -802,6 +817,9 @@ static int img_commit(int argc, char **argv)
return 1;
}
+ qemu_progress_init(progress, 1.f);
+ qemu_progress_print(0.f, 100);
+
/* This is different from QMP, which by default uses the deepest file in the
* backing chain (i.e., the very base); however, the traditional behavior of
* qemu-img commit is using the immediate backing file. */
@@ -850,6 +868,8 @@ unref_backing:
}
done:
+ qemu_progress_end();
+
bdrv_unref(bs);
if (local_err) {
diff --git a/qemu-img.texi b/qemu-img.texi
index fd8ecd6..5c151af 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -163,7 +163,7 @@ this case. @var{backing_file} will never be modified unless you use the
The size can also be specified using the @var{size} option with @code{-o},
it doesn't need to be specified separately in this case.
-@item commit [-f @var{fmt}] [-t @var{cache}] [-d] @var{filename}
+@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-d] [-p] @var{filename}
Commit the changes recorded in @var{filename} in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 10/14] qemu-img: Specify backing file for commit
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (8 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 09/14] qemu-img: Enable progress output for commit Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 11/14] iotests: Add _filter_qemu_img_map Max Reitz
` (3 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Introduce a new parameter for qemu-img commit which may be used to
explicitly specify the backing file into which an image should be
committed if the backing chain has more than a single layer.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
qemu-img-cmds.hx | 4 ++--
qemu-img.c | 24 +++++++++++++++++-------
qemu-img.texi | 9 ++++++++-
3 files changed, 27 insertions(+), 10 deletions(-)
diff --git a/qemu-img-cmds.hx b/qemu-img-cmds.hx
index ea41d4f..4331949 100644
--- a/qemu-img-cmds.hx
+++ b/qemu-img-cmds.hx
@@ -22,9 +22,9 @@ STEXI
ETEXI
DEF("commit", img_commit,
- "commit [-q] [-f fmt] [-t cache] [-d] [-p] filename")
+ "commit [-q] [-f fmt] [-t cache] [-b base] [-d] [-p] filename")
STEXI
-@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-d] [-p] @var{filename}
+@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
ETEXI
DEF("compare", img_compare,
diff --git a/qemu-img.c b/qemu-img.c
index 8409062..daf4bba 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -759,7 +759,7 @@ static void run_block_job(BlockJob *job, Error **errp)
static int img_commit(int argc, char **argv)
{
int c, ret, flags;
- const char *filename, *fmt, *cache;
+ const char *filename, *fmt, *cache, *base;
BlockDriverState *bs, *base_bs;
bool progress = false, quiet = false, drop = false;
Error *local_err = NULL;
@@ -767,8 +767,9 @@ static int img_commit(int argc, char **argv)
fmt = NULL;
cache = BDRV_DEFAULT_CACHE;
+ base = NULL;
for(;;) {
- c = getopt(argc, argv, "f:ht:dpq");
+ c = getopt(argc, argv, "f:ht:b:dpq");
if (c == -1) {
break;
}
@@ -783,6 +784,11 @@ static int img_commit(int argc, char **argv)
case 't':
cache = optarg;
break;
+ case 'b':
+ base = optarg;
+ /* -b implies -d */
+ drop = true;
+ break;
case 'd':
drop = true;
break;
@@ -820,12 +826,16 @@ static int img_commit(int argc, char **argv)
qemu_progress_init(progress, 1.f);
qemu_progress_print(0.f, 100);
- /* This is different from QMP, which by default uses the deepest file in the
- * backing chain (i.e., the very base); however, the traditional behavior of
- * qemu-img commit is using the immediate backing file. */
- base_bs = bs->backing_hd;
+ if (base) {
+ base_bs = bdrv_find_backing_image(bs, base);
+ } else {
+ /* This is different from QMP, which by default uses the deepest file in
+ * the backing chain (i.e., the very base); however, the traditional
+ * behavior of qemu-img commit is using the immediate backing file. */
+ base_bs = bs->backing_hd;
+ }
if (!base_bs) {
- error_set(&local_err, QERR_BASE_NOT_FOUND, "NULL");
+ error_set(&local_err, QERR_BASE_NOT_FOUND, base ?: "NULL");
goto done;
}
diff --git a/qemu-img.texi b/qemu-img.texi
index 5c151af..04b6c9e 100644
--- a/qemu-img.texi
+++ b/qemu-img.texi
@@ -163,7 +163,7 @@ this case. @var{backing_file} will never be modified unless you use the
The size can also be specified using the @var{size} option with @code{-o},
it doesn't need to be specified separately in this case.
-@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-d] [-p] @var{filename}
+@item commit [-q] [-f @var{fmt}] [-t @var{cache}] [-b @var{base}] [-d] [-p] @var{filename}
Commit the changes recorded in @var{filename} in its base image or backing file.
If the backing file is smaller than the snapshot, then the backing file will be
@@ -176,6 +176,13 @@ The image @var{filename} is emptied after the operation has succeeded. If you do
not need @var{filename} afterwards and intend to drop it, you may skip emptying
@var{filename} by specifying the @code{-d} flag.
+If the backing chain of the given image file @var{filename} has more than one
+layer, the backing file into which the changes will be committed may be
+specified as @var{base} (which has to be part of @var{filename}'s backing
+chain). If @var{base} is not specified, the immediate backing file of the top
+image (which is @var{filename}) will be used. For reasons of consistency,
+explicitly specifying @var{base} will always imply @code{-d}.
+
@item compare [-f @var{fmt}] [-F @var{fmt}] [-p] [-s] [-q] @var{filename1} @var{filename2}
Check if two images have the same content. You can compare images with
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 11/14] iotests: Add _filter_qemu_img_map
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (9 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 10/14] qemu-img: Specify backing file " Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 12/14] iotests: Add test for backing-chain commits Max Reitz
` (2 subsequent siblings)
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
As different image formats most probably map guest addresses to
different host addresses, add a filter to filter the host addresses out;
also, the image filename should be filtered.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/common.filter | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index a04df7f..8b14edb 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -170,5 +170,12 @@ _filter_qmp()
-e 's#^{"QMP":.*}$#QMP_VERSION#'
}
+# filter out offsets and file names from qemu-img map
+_filter_qemu_img_map()
+{
+ sed -e 's/\([0-9a-fx]* *[0-9a-fx]* *\)[0-9a-fx]* */\1/g' \
+ -e 's/Mapped to *//' | _filter_testdir | _filter_imgfmt
+}
+
# make sure this script returns success
/bin/true
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 12/14] iotests: Add test for backing-chain commits
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (10 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 11/14] iotests: Add _filter_qemu_img_map Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 13/14] iotests: Add test for qcow2's bdrv_make_empty Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 14/14] iotests: Omit length/offset test in 040 and 041 Max Reitz
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Add a test for qemu-img commit on backing chains with more than two
images. This test also checks whether the top image is emptied (unless
this is prevented by specifying either -d or -b) and does therefore not
work for qed and vmdk which requires it to be separate from 020.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/097 | 122 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/097.out | 119 +++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 242 insertions(+)
create mode 100755 tests/qemu-iotests/097
create mode 100644 tests/qemu-iotests/097.out
diff --git a/tests/qemu-iotests/097 b/tests/qemu-iotests/097
new file mode 100755
index 0000000..c7a613b
--- /dev/null
+++ b/tests/qemu-iotests/097
@@ -0,0 +1,122 @@
+#!/bin/bash
+#
+# Commit changes into backing chains and empty the top image if the
+# backing image is not explicitly specified
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mreitz@redhat.com
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+tmp=/tmp/$$
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+ _rm_test_img "$TEST_IMG.itmd"
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.pattern
+
+# Any format supporting backing files and bdrv_make_empty
+_supported_fmt qcow qcow2
+_supported_proto file
+_supported_os Linux
+
+
+# Four passes:
+# 0: Two-layer backing chain, commit to upper backing file (implicitly)
+# (in this case, the top image will be emptied)
+# 1: Two-layer backing chain, commit to upper backing file (explicitly)
+# (in this case, the top image will implicitly stay unchanged)
+# 2: Two-layer backing chain, commit to upper backing file (implicitly with -d)
+# (in this case, the top image will explicitly stay unchanged)
+# 3: Two-layer backing chain, commit to lower backing file
+# (in this case, the top image will implicitly stay unchanged)
+#
+# 020 already tests committing, so this only tests whether image chains are
+# working properly and that all images above the base are emptied; therefore,
+# no complicated patterns are necessary
+for i in 0 1 2 3; do
+
+echo
+echo "=== Test pass $i ==="
+echo
+
+TEST_IMG="$TEST_IMG.base" _make_test_img 64M
+TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" 64M
+_make_test_img -b "$TEST_IMG.itmd" 64M
+
+$QEMU_IO -c 'write -P 1 0 192k' "$TEST_IMG.base" | _filter_qemu_io
+$QEMU_IO -c 'write -P 2 64k 128k' "$TEST_IMG.itmd" | _filter_qemu_io
+$QEMU_IO -c 'write -P 3 128k 64k' "$TEST_IMG" | _filter_qemu_io
+
+if [ $i -lt 3 ]; then
+ if [ $i == 0 ]; then
+ # -b "$TEST_IMG.itmd" should be the default (that is, committing to the
+ # first backing file in the chain)
+ $QEMU_IMG commit "$TEST_IMG"
+ elif [ $i == 1 ]; then
+ # explicitly specify the commit target (this should imply -d)
+ $QEMU_IMG commit -b "$TEST_IMG.itmd" "$TEST_IMG"
+ else
+ # do not explicitly specify the commit target, but use -d to leave the
+ # top image unchanged
+ $QEMU_IMG commit -d "$TEST_IMG"
+ fi
+
+ # Bottom should be unchanged
+ $QEMU_IO -c 'read -P 1 0 192k' "$TEST_IMG.base" | _filter_qemu_io
+
+ # Intermediate should contain changes from top
+ $QEMU_IO -c 'read -P 1 0 64k' "$TEST_IMG.itmd" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 2 64k 64k' "$TEST_IMG.itmd" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 3 128k 64k' "$TEST_IMG.itmd" | _filter_qemu_io
+
+ # And in pass 0, the top image should be empty, whereas in both other passes
+ # it should be unchanged (which is both checked by qemu-img map)
+else
+ $QEMU_IMG commit -b "$TEST_IMG.base" "$TEST_IMG"
+
+ # Bottom should contain all changes
+ $QEMU_IO -c 'read -P 1 0 64k' "$TEST_IMG.base" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 2 64k 64k' "$TEST_IMG.base" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 3 128k 64k' "$TEST_IMG.base" | _filter_qemu_io
+
+ # Both top and intermediate should be unchanged
+fi
+
+$QEMU_IMG map "$TEST_IMG.base" | _filter_qemu_img_map
+$QEMU_IMG map "$TEST_IMG.itmd" | _filter_qemu_img_map
+$QEMU_IMG map "$TEST_IMG" | _filter_qemu_img_map
+
+done
+
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/097.out b/tests/qemu-iotests/097.out
new file mode 100644
index 0000000..1cb7764
--- /dev/null
+++ b/tests/qemu-iotests/097.out
@@ -0,0 +1,119 @@
+QA output created by 097
+
+=== Test pass 0 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base'
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.itmd'
+wrote 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 65536
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 0
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 65536
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x20000 TEST_DIR/t.IMGFMT.itmd
+
+=== Test pass 1 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base'
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.itmd'
+wrote 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 65536
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 0
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 65536
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x10000 TEST_DIR/t.IMGFMT.itmd
+0x20000 0x10000 TEST_DIR/t.IMGFMT
+
+=== Test pass 2 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base'
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.itmd'
+wrote 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 65536
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 0
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 65536
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x10000 TEST_DIR/t.IMGFMT.itmd
+0x20000 0x10000 TEST_DIR/t.IMGFMT
+
+=== Test pass 3 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base'
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.itmd'
+wrote 196608/196608 bytes at offset 0
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 65536
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 65536/65536 bytes at offset 0
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 65536
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 131072
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0 0x10000 TEST_DIR/t.IMGFMT.base
+0x10000 0x10000 TEST_DIR/t.IMGFMT.itmd
+0x20000 0x10000 TEST_DIR/t.IMGFMT
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 6e67f61..bdf04cd 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -100,3 +100,4 @@
091 rw auto quick
092 rw auto quick
095 rw auto quick
+097 rw auto backing
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 13/14] iotests: Add test for qcow2's bdrv_make_empty
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (11 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 12/14] iotests: Add test for backing-chain commits Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
2014-07-07 19:15 ` Eric Blake
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 14/14] iotests: Omit length/offset test in 040 and 041 Max Reitz
13 siblings, 1 reply; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
Add a test for qcow2's fast bdrv_make_empty implementation on images
without internal snapshots.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/098 | 75 ++++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/098.out | 26 ++++++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 102 insertions(+)
create mode 100755 tests/qemu-iotests/098
create mode 100644 tests/qemu-iotests/098.out
diff --git a/tests/qemu-iotests/098 b/tests/qemu-iotests/098
new file mode 100755
index 0000000..116f935
--- /dev/null
+++ b/tests/qemu-iotests/098
@@ -0,0 +1,75 @@
+#!/bin/bash
+#
+# Test qcow2's bdrv_make_empty for images without internal snapshots
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mreitz@redhat.com
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+tmp=/tmp/$$
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+ rm -f "$TEST_DIR/blkdebug.conf"
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.pattern
+
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+
+for event in l1_update reftable_update; do
+
+echo
+echo "=== $event ==="
+echo
+
+TEST_IMG="$TEST_IMG.base" _make_test_img 64M
+_make_test_img -b "$TEST_IMG.base" 64M
+
+cat > "$TEST_DIR/blkdebug.conf" <<EOF
+[inject-error]
+event = "$event"
+EOF
+
+$QEMU_IMG commit "blkdebug:$TEST_DIR/blkdebug.conf:$TEST_IMG" 2>&1 \
+ | _filter_testdir | _filter_imgfmt
+
+_check_test_img
+
+done
+
+
+rm -f "$TEST_DIR/blkdebug.conf"
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/098.out b/tests/qemu-iotests/098.out
new file mode 100644
index 0000000..5f5af1b
--- /dev/null
+++ b/tests/qemu-iotests/098.out
@@ -0,0 +1,26 @@
+QA output created by 098
+
+=== l1_update ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base'
+qemu-img: Could not empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
+Leaked cluster 4 refcount=1 reference=0
+Leaked cluster 5 refcount=1 reference=0
+Leaked cluster 6 refcount=1 reference=0
+
+3 leaked clusters were found on the image.
+This means waste of disk space, but no harm to data.
+
+=== reftable_update ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file='TEST_DIR/t.IMGFMT.base'
+qemu-img: Could not empty blkdebug:TEST_DIR/blkdebug.conf:TEST_DIR/t.IMGFMT: Input/output error
+Leaked cluster 3 refcount=1 reference=0
+Leaked cluster 4 refcount=1 reference=0
+Leaked cluster 5 refcount=1 reference=0
+
+3 leaked clusters were found on the image.
+This means waste of disk space, but no harm to data.
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index bdf04cd..dfbd0ed 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -101,3 +101,4 @@
092 rw auto quick
095 rw auto quick
097 rw auto backing
+098 rw auto backing quick
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* [Qemu-devel] [PATCH v9 14/14] iotests: Omit length/offset test in 040 and 041
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
` (12 preceding siblings ...)
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 13/14] iotests: Add test for qcow2's bdrv_make_empty Max Reitz
@ 2014-07-05 17:47 ` Max Reitz
13 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-05 17:47 UTC (permalink / raw)
To: qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi, Max Reitz
As the length of a mirror block job no longer directly depends on the
size of the block device, drop those checks from this test. Instead,
just check whether the final offset equals the block job length.
As 041 uses the wait_until_completed function from iotests.py, the same
applies there as well which in turn affects tests 030, 055 and 056. On
the other hand, a block job's length does not have to be related to the
length of the image file in the first place, so that check was
questionable anyway.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/040 | 4 +---
tests/qemu-iotests/041 | 3 +--
tests/qemu-iotests/iotests.py | 3 +--
3 files changed, 3 insertions(+), 7 deletions(-)
diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index f1e16c1..2b432ad 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -43,8 +43,7 @@ class ImageCommitTestCase(iotests.QMPTestCase):
if event['event'] == 'BLOCK_JOB_COMPLETED':
self.assert_qmp(event, 'data/type', 'commit')
self.assert_qmp(event, 'data/device', 'drive0')
- self.assert_qmp(event, 'data/offset', self.image_len)
- self.assert_qmp(event, 'data/len', self.image_len)
+ self.assert_qmp(event, 'data/offset', event['data']['len'])
if need_ready:
self.assertTrue(ready, "Expecting BLOCK_JOB_COMPLETED event")
completed = True
@@ -52,7 +51,6 @@ class ImageCommitTestCase(iotests.QMPTestCase):
ready = True
self.assert_qmp(event, 'data/type', 'commit')
self.assert_qmp(event, 'data/device', 'drive0')
- self.assert_qmp(event, 'data/len', self.image_len)
self.vm.qmp('block-job-complete', device='drive0')
self.assert_no_active_block_jobs()
diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index 0815e19..d8bb76c 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -52,8 +52,7 @@ class ImageMirroringTestCase(iotests.QMPTestCase):
event = self.cancel_and_wait(drive=drive)
self.assertEquals(event['event'], 'BLOCK_JOB_COMPLETED')
self.assert_qmp(event, 'data/type', 'mirror')
- self.assert_qmp(event, 'data/offset', self.image_len)
- self.assert_qmp(event, 'data/len', self.image_len)
+ self.assert_qmp(event, 'data/offset', event['data']['len'])
def complete_and_wait(self, drive='drive0', wait_ready=True):
'''Complete a block job and wait for it to finish'''
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 39a4cfc..f57f154 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -267,8 +267,7 @@ class QMPTestCase(unittest.TestCase):
self.assert_qmp(event, 'data/device', drive)
self.assert_qmp_absent(event, 'data/error')
if check_offset:
- self.assert_qmp(event, 'data/offset', self.image_len)
- self.assert_qmp(event, 'data/len', self.image_len)
+ self.assert_qmp(event, 'data/offset', event['data']['len'])
completed = True
self.assert_no_active_block_jobs()
--
2.0.1
^ permalink raw reply related [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field Max Reitz
@ 2014-07-07 18:53 ` Eric Blake
2014-07-09 20:21 ` Max Reitz
0 siblings, 1 reply; 23+ messages in thread
From: Eric Blake @ 2014-07-07 18:53 UTC (permalink / raw)
To: Max Reitz, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 1065 bytes --]
On 07/05/2014 11:47 AM, Max Reitz wrote:
> When a block job signals readiness, this is currently reported only
> through QMP. If qemu wants to use block jobs for internal tasks, there
> needs to be another way to correctly detect when a block job may be
> completed.
>
> For this reason, introduce a bool "ready" which is set when the block
> job may be completed.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> blockjob.c | 3 +++
> include/block/blockjob.h | 5 +++++
> qapi/block-core.json | 4 +++-
> 3 files changed, 11 insertions(+), 1 deletion(-)
>
> +++ b/qapi/block-core.json
> @@ -505,12 +505,14 @@
> #
> # @io-status: the status of the job (since 1.3)
> #
> +# @ready: true if the job may be completed (since 2.1)
Isn't this 2.2 now, or are you still shooting for calling this a bug fix
in 2.1 hard freeze?
Depending on the answer,
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report Max Reitz
@ 2014-07-07 19:13 ` Eric Blake
2014-07-09 21:24 ` Paolo Bonzini
0 siblings, 1 reply; 23+ messages in thread
From: Eric Blake @ 2014-07-07 19:13 UTC (permalink / raw)
To: Max Reitz, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 692 bytes --]
On 07/05/2014 11:47 AM, Max Reitz wrote:
> Instead of taking the total length of the block device as the block
> job's length, use the number of dirty sectors. The progress is now the
> number of sectors mirrored to the target block device. Note that this
> may result in the job's length increasing during operation, which is
> however in fact desirable.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> block/mirror.c | 34 ++++++++++++++++++++++------------
> 1 file changed, 22 insertions(+), 12 deletions(-)
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 13/14] iotests: Add test for qcow2's bdrv_make_empty
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 13/14] iotests: Add test for qcow2's bdrv_make_empty Max Reitz
@ 2014-07-07 19:15 ` Eric Blake
0 siblings, 0 replies; 23+ messages in thread
From: Eric Blake @ 2014-07-07 19:15 UTC (permalink / raw)
To: Max Reitz, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 671 bytes --]
On 07/05/2014 11:47 AM, Max Reitz wrote:
> Add a test for qcow2's fast bdrv_make_empty implementation on images
> without internal snapshots.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> tests/qemu-iotests/098 | 75 ++++++++++++++++++++++++++++++++++++++++++++++
> tests/qemu-iotests/098.out | 26 ++++++++++++++++
> tests/qemu-iotests/group | 1 +
> 3 files changed, 102 insertions(+)
> create mode 100755 tests/qemu-iotests/098
> create mode 100644 tests/qemu-iotests/098.out
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field
2014-07-07 18:53 ` Eric Blake
@ 2014-07-09 20:21 ` Max Reitz
0 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-09 20:21 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
On 07.07.2014 20:53, Eric Blake wrote:
> On 07/05/2014 11:47 AM, Max Reitz wrote:
>> When a block job signals readiness, this is currently reported only
>> through QMP. If qemu wants to use block jobs for internal tasks, there
>> needs to be another way to correctly detect when a block job may be
>> completed.
>>
>> For this reason, introduce a bool "ready" which is set when the block
>> job may be completed.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> blockjob.c | 3 +++
>> include/block/blockjob.h | 5 +++++
>> qapi/block-core.json | 4 +++-
>> 3 files changed, 11 insertions(+), 1 deletion(-)
>>
>> +++ b/qapi/block-core.json
>> @@ -505,12 +505,14 @@
>> #
>> # @io-status: the status of the job (since 1.3)
>> #
>> +# @ready: true if the job may be completed (since 2.1)
> Isn't this 2.2 now, or are you still shooting for calling this a bug fix
> in 2.1 hard freeze?
Ah, right, I forgot. This should be 2.2, of course.
Max
> Depending on the answer,
> Reviewed-by: Eric Blake <eblake@redhat.com>
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty()
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty() Max Reitz
@ 2014-07-09 21:22 ` Paolo Bonzini
2014-07-09 21:38 ` Max Reitz
0 siblings, 1 reply; 23+ messages in thread
From: Paolo Bonzini @ 2014-07-09 21:22 UTC (permalink / raw)
To: Max Reitz, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
Il 05/07/2014 19:47, Max Reitz ha scritto:
> bdrv_make_empty() is currently only called if the current image
> represents an external snapshot that has been committed to its base
> image; it is therefore unlikely to have internal snapshots. In this
> case, bdrv_make_empty() can be greatly sped up by creating an empty L1
> table and dropping all data clusters at once by recreating the refcount
> structure accordingly instead of normally discarding all clusters.
>
> If there are snapshots, fall back to the simple implementation (discard
> all clusters).
How much of this code could be reused by bdrv_create?
Paolo
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report
2014-07-07 19:13 ` Eric Blake
@ 2014-07-09 21:24 ` Paolo Bonzini
2014-07-09 21:36 ` Eric Blake
0 siblings, 1 reply; 23+ messages in thread
From: Paolo Bonzini @ 2014-07-09 21:24 UTC (permalink / raw)
To: Eric Blake, Max Reitz, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
Il 07/07/2014 21:13, Eric Blake ha scritto:
> On 07/05/2014 11:47 AM, Max Reitz wrote:
>> Instead of taking the total length of the block device as the block
>> job's length, use the number of dirty sectors. The progress is now the
>> number of sectors mirrored to the target block device. Note that this
>> may result in the job's length increasing during operation, which is
>> however in fact desirable.
>>
>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>> ---
>> block/mirror.c | 34 ++++++++++++++++++++++------------
>> 1 file changed, 22 insertions(+), 12 deletions(-)
>
> Reviewed-by: Eric Blake <eblake@redhat.com>
This is an API change... IIUC the length can become bigger than the
underlying device's size. Eric, how would libvirt expose this to
clients and what are the chances that they get confused?
Paolo
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report
2014-07-09 21:24 ` Paolo Bonzini
@ 2014-07-09 21:36 ` Eric Blake
0 siblings, 0 replies; 23+ messages in thread
From: Eric Blake @ 2014-07-09 21:36 UTC (permalink / raw)
To: Paolo Bonzini, Max Reitz, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 1997 bytes --]
On 07/09/2014 03:24 PM, Paolo Bonzini wrote:
> Il 07/07/2014 21:13, Eric Blake ha scritto:
>> On 07/05/2014 11:47 AM, Max Reitz wrote:
>>> Instead of taking the total length of the block device as the block
>>> job's length, use the number of dirty sectors. The progress is now the
>>> number of sectors mirrored to the target block device. Note that this
>>> may result in the job's length increasing during operation, which is
>>> however in fact desirable.
>>>
>>> Signed-off-by: Max Reitz <mreitz@redhat.com>
>>> ---
>>> block/mirror.c | 34 ++++++++++++++++++++++------------
>>> 1 file changed, 22 insertions(+), 12 deletions(-)
>>
>> Reviewed-by: Eric Blake <eblake@redhat.com>
>
> This is an API change... IIUC the length can become bigger than the
> underlying device's size. Eric, how would libvirt expose this to
> clients and what are the chances that they get confused?
Libvirt has already documented that a job size is unrelated to the block
device size, that it is only an approximation to completion, and that
the completion number may change during operation. The only hard and
fast rule is that the job is finished when the two counters are equal.
I recommended this change precisely because the new semantics are better
than the old, including how libvirt exposes the numbers to the end user
- if the end parameter continues to grow more than the current
parameter, it is DESIRABLE to expose that as sign that the guest is
dirtying pages fast enough to cause problems in converging the block job.
http://libvirt.org/html/libvirt-libvirt.html#virDomainBlockJobInfo
"The following fields provide an indication of block job progress. @cur
indicates the current position and will be between 0 and @end. @end is
the final cursor position for this operation and represents completion.
To approximate progress, divide @cur by @end."
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 23+ messages in thread
* Re: [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty()
2014-07-09 21:22 ` Paolo Bonzini
@ 2014-07-09 21:38 ` Max Reitz
0 siblings, 0 replies; 23+ messages in thread
From: Max Reitz @ 2014-07-09 21:38 UTC (permalink / raw)
To: Paolo Bonzini, qemu-devel; +Cc: Kevin Wolf, Stefan Hajnoczi
On 09.07.2014 23:22, Paolo Bonzini wrote:
> Il 05/07/2014 19:47, Max Reitz ha scritto:
>> bdrv_make_empty() is currently only called if the current image
>> represents an external snapshot that has been committed to its base
>> image; it is therefore unlikely to have internal snapshots. In this
>> case, bdrv_make_empty() can be greatly sped up by creating an empty L1
>> table and dropping all data clusters at once by recreating the refcount
>> structure accordingly instead of normally discarding all clusters.
>>
>> If there are snapshots, fall back to the simple implementation (discard
>> all clusters).
>
> How much of this code could be reused by bdrv_create?
Currently, none. The latest version of Hu Tao's preallocation series
reuses minimal_blob_size().
The point of this code is to empty an image at runtime while keeping it
consistent all the time. bdrv_open() does not have that problem; there
are no pre-existing structures, therefore it can just go and create
them. bdrv_make_empty() has to be careful (in this version) not to
overwrite any data or metadata while relocating the structures.
Kevin proposed another version which just marks the image dirty, clears
the L1 table, relocates it and then writes minimal refcount structures.
This will be shorted and in principle similar to how bdrv_open() works,
but isn't worth sharing either.
Max
^ permalink raw reply [flat|nested] 23+ messages in thread
end of thread, other threads:[~2014-07-09 21:38 UTC | newest]
Thread overview: 23+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-05 17:47 [Qemu-devel] [PATCH v9 00/14] qemu-img: Implement commit like QMP Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 01/14] qcow2: Allow "full" discard Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 02/14] qcow2: Implement bdrv_make_empty() Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 03/14] qcow2: Optimize bdrv_make_empty() Max Reitz
2014-07-09 21:22 ` Paolo Bonzini
2014-07-09 21:38 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 04/14] blockjob: Introduce block_job_complete_sync() Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 05/14] blockjob: Add "ready" field Max Reitz
2014-07-07 18:53 ` Eric Blake
2014-07-09 20:21 ` Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 06/14] block/mirror: Improve progress report Max Reitz
2014-07-07 19:13 ` Eric Blake
2014-07-09 21:24 ` Paolo Bonzini
2014-07-09 21:36 ` Eric Blake
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 07/14] qemu-img: Implement commit like QMP Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 08/14] qemu-img: Empty image after commit Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 09/14] qemu-img: Enable progress output for commit Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 10/14] qemu-img: Specify backing file " Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 11/14] iotests: Add _filter_qemu_img_map Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 12/14] iotests: Add test for backing-chain commits Max Reitz
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 13/14] iotests: Add test for qcow2's bdrv_make_empty Max Reitz
2014-07-07 19:15 ` Eric Blake
2014-07-05 17:47 ` [Qemu-devel] [PATCH v9 14/14] iotests: Omit length/offset test in 040 and 041 Max Reitz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.