* [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment
@ 2015-03-18 20:56 Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 01/10] progress: Allow regressing progress Max Reitz
` (10 more replies)
0 siblings, 11 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
This series adds support to qemu for changing the refcount_bits option
of an existing qcow2 file through the qemu-img amend command.
Originally (up until v7), this series was called
"qcow2: Support refcount orders != 4", but by now that support has
already been merged so all that is left is support for qemu-img amend.
v8:
- Rebase onto master (one contextual conflict in an iotest reference
output, but other than that this only means that the first 15 patches
are dropped)
git-backport-diff against v7:
Key:
[----] : patches are identical
[####] : number of functional differences between upstream/downstream patch
[down] : patch is downstream-only
The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
001/10:[----] [--] 'progress: Allow regressing progress'
002/10:[----] [--] 'block: Add opaque value to the amend CB'
003/10:[----] [-C] 'qcow2: Use error_report() in qcow2_amend_options()'
004/10:[----] [--] 'qcow2: Use abort() instead of assert(false)'
005/10:[----] [--] 'qcow2: Split upgrade/downgrade paths for amend'
006/10:[----] [--] 'qcow2: Use intermediate helper CB for amend'
007/10:[----] [--] 'qcow2: Add function for refcount order amendment'
008/10:[----] [--] 'qcow2: Invoke refcount order amendment function'
009/10:[----] [--] 'qcow2: Point to amend function in check'
010/10:[----] [--] 'iotests: Extend test 112 for qemu-img amend'
Max Reitz (10):
progress: Allow regressing progress
block: Add opaque value to the amend CB
qcow2: Use error_report() in qcow2_amend_options()
qcow2: Use abort() instead of assert(false)
qcow2: Split upgrade/downgrade paths for amend
qcow2: Use intermediate helper CB for amend
qcow2: Add function for refcount order amendment
qcow2: Invoke refcount order amendment function
qcow2: Point to amend function in check
iotests: Extend test 112 for qemu-img amend
block.c | 4 +-
block/qcow2-cluster.c | 14 +-
block/qcow2-refcount.c | 455 +++++++++++++++++++++++++++++++++++++++++++++
block/qcow2.c | 178 ++++++++++++++----
block/qcow2.h | 7 +-
include/block/block.h | 4 +-
include/block/block_int.h | 3 +-
qemu-img.c | 5 +-
tests/qemu-iotests/061.out | 14 +-
tests/qemu-iotests/112 | 109 +++++++++++
tests/qemu-iotests/112.out | 71 +++++++
util/qemu-progress.c | 3 +-
12 files changed, 808 insertions(+), 59 deletions(-)
--
2.1.0
^ permalink raw reply [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 01/10] progress: Allow regressing progress
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 02/10] block: Add opaque value to the amend CB Max Reitz
` (9 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Progress may regress; this should be displayed correctly by
qemu_progress_print().
While touching that area of code, drop the redundant parentheses in the
same condition.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
util/qemu-progress.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/util/qemu-progress.c b/util/qemu-progress.c
index 4ee5cd0..532333e 100644
--- a/util/qemu-progress.c
+++ b/util/qemu-progress.c
@@ -152,7 +152,8 @@ void qemu_progress_print(float delta, int max)
state.current = current;
if (current > (state.last_print + state.min_skip) ||
- (current == 100) || (current == 0)) {
+ current < (state.last_print - state.min_skip) ||
+ current == 100 || current == 0) {
state.last_print = state.current;
state.print();
}
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 02/10] block: Add opaque value to the amend CB
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 01/10] progress: Allow regressing progress Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 03/10] qcow2: Use error_report() in qcow2_amend_options() Max Reitz
` (8 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Add an opaque value which is to be passed to the bdrv_amend_options()
status callback.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block.c | 4 ++--
block/qcow2-cluster.c | 14 ++++++++------
block/qcow2.c | 9 +++++----
block/qcow2.h | 3 ++-
include/block/block.h | 4 ++--
include/block/block_int.h | 3 ++-
qemu-img.c | 5 +++--
7 files changed, 24 insertions(+), 18 deletions(-)
diff --git a/block.c b/block.c
index 0fe97de..46693d1 100644
--- a/block.c
+++ b/block.c
@@ -5927,12 +5927,12 @@ void bdrv_add_before_write_notifier(BlockDriverState *bs,
}
int bdrv_amend_options(BlockDriverState *bs, QemuOpts *opts,
- BlockDriverAmendStatusCB *status_cb)
+ BlockDriverAmendStatusCB *status_cb, void *cb_opaque)
{
if (!bs->drv->bdrv_amend_options) {
return -ENOTSUP;
}
- return bs->drv->bdrv_amend_options(bs, opts, status_cb);
+ return bs->drv->bdrv_amend_options(bs, opts, status_cb, cb_opaque);
}
/* This function will be called by the bdrv_recurse_is_first_non_filter method
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index ed2b44d..56363dd 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1620,7 +1620,8 @@ fail:
static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
int l1_size, int64_t *visited_l1_entries,
int64_t l1_entries,
- BlockDriverAmendStatusCB *status_cb)
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque)
{
BDRVQcowState *s = bs->opaque;
bool is_active_l1 = (l1_table == s->l1_table);
@@ -1646,7 +1647,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
/* unallocated */
(*visited_l1_entries)++;
if (status_cb) {
- status_cb(bs, *visited_l1_entries, l1_entries);
+ status_cb(bs, *visited_l1_entries, l1_entries, cb_opaque);
}
continue;
}
@@ -1787,7 +1788,7 @@ static int expand_zero_clusters_in_l1(BlockDriverState *bs, uint64_t *l1_table,
(*visited_l1_entries)++;
if (status_cb) {
- status_cb(bs, *visited_l1_entries, l1_entries);
+ status_cb(bs, *visited_l1_entries, l1_entries, cb_opaque);
}
}
@@ -1816,7 +1817,8 @@ fail:
* qcow2 version which doesn't yet support metadata zero clusters.
*/
int qcow2_expand_zero_clusters(BlockDriverState *bs,
- BlockDriverAmendStatusCB *status_cb)
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque)
{
BDRVQcowState *s = bs->opaque;
uint64_t *l1_table = NULL;
@@ -1833,7 +1835,7 @@ int qcow2_expand_zero_clusters(BlockDriverState *bs,
ret = expand_zero_clusters_in_l1(bs, s->l1_table, s->l1_size,
&visited_l1_entries, l1_entries,
- status_cb);
+ status_cb, cb_opaque);
if (ret < 0) {
goto fail;
}
@@ -1868,7 +1870,7 @@ int qcow2_expand_zero_clusters(BlockDriverState *bs,
ret = expand_zero_clusters_in_l1(bs, l1_table, s->snapshots[i].l1_size,
&visited_l1_entries, l1_entries,
- status_cb);
+ status_cb, cb_opaque);
if (ret < 0) {
goto fail;
}
diff --git a/block/qcow2.c b/block/qcow2.c
index 32bdf75..fd66bf2 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2597,7 +2597,7 @@ static int qcow2_load_vmstate(BlockDriverState *bs, uint8_t *buf,
* have to be removed.
*/
static int qcow2_downgrade(BlockDriverState *bs, int target_version,
- BlockDriverAmendStatusCB *status_cb)
+ BlockDriverAmendStatusCB *status_cb, void *cb_opaque)
{
BDRVQcowState *s = bs->opaque;
int current_version = s->qcow_version;
@@ -2646,7 +2646,7 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
/* clearing autoclear features is trivial */
s->autoclear_features = 0;
- ret = qcow2_expand_zero_clusters(bs, status_cb);
+ ret = qcow2_expand_zero_clusters(bs, status_cb, cb_opaque);
if (ret < 0) {
return ret;
}
@@ -2661,7 +2661,8 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
}
static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
- BlockDriverAmendStatusCB *status_cb)
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque)
{
BDRVQcowState *s = bs->opaque;
int old_version = s->qcow_version, new_version = old_version;
@@ -2743,7 +2744,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
return ret;
}
} else {
- ret = qcow2_downgrade(bs, new_version, status_cb);
+ ret = qcow2_downgrade(bs, new_version, status_cb, cb_opaque);
if (ret < 0) {
return ret;
}
diff --git a/block/qcow2.h b/block/qcow2.h
index aa6d367..eed26db 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -546,7 +546,8 @@ int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors);
int qcow2_expand_zero_clusters(BlockDriverState *bs,
- BlockDriverAmendStatusCB *status_cb);
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque);
/* qcow2-snapshot.c functions */
int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info);
diff --git a/include/block/block.h b/include/block/block.h
index 4c57d63..aaa31a0 100644
--- a/include/block/block.h
+++ b/include/block/block.h
@@ -290,9 +290,9 @@ int bdrv_check(BlockDriverState *bs, BdrvCheckResult *res, BdrvCheckMode fix);
* block driver; total_work_size may change during the course of the amendment
* operation */
typedef void BlockDriverAmendStatusCB(BlockDriverState *bs, int64_t offset,
- int64_t total_work_size);
+ int64_t total_work_size, void *opaque);
int bdrv_amend_options(BlockDriverState *bs_new, QemuOpts *opts,
- BlockDriverAmendStatusCB *status_cb);
+ BlockDriverAmendStatusCB *status_cb, void *cb_opaque);
/* external snapshots */
bool bdrv_recurse_is_first_non_filter(BlockDriverState *bs,
diff --git a/include/block/block_int.h b/include/block/block_int.h
index dccb092..615e880 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -237,7 +237,8 @@ struct BlockDriver {
BdrvCheckMode fix);
int (*bdrv_amend_options)(BlockDriverState *bs, QemuOpts *opts,
- BlockDriverAmendStatusCB *status_cb);
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque);
void (*bdrv_debug_event)(BlockDriverState *bs, BlkDebugEvent event);
diff --git a/qemu-img.c b/qemu-img.c
index 9dddfbe..e638f2b 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2882,7 +2882,8 @@ out:
}
static void amend_status_cb(BlockDriverState *bs,
- int64_t offset, int64_t total_work_size)
+ int64_t offset, int64_t total_work_size,
+ void *opaque)
{
qemu_progress_print(100.f * offset / total_work_size, 0);
}
@@ -3006,7 +3007,7 @@ static int img_amend(int argc, char **argv)
/* In case the driver does not call amend_status_cb() */
qemu_progress_print(0.f, 0);
- ret = bdrv_amend_options(bs, opts, &amend_status_cb);
+ ret = bdrv_amend_options(bs, opts, &amend_status_cb, NULL);
qemu_progress_print(100.f, 0);
if (ret < 0) {
error_report("Error while amending options: %s", strerror(-ret));
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 03/10] qcow2: Use error_report() in qcow2_amend_options()
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 01/10] progress: Allow regressing progress Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 02/10] block: Add opaque value to the amend CB Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 04/10] qcow2: Use abort() instead of assert(false) Max Reitz
` (7 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2.c | 14 ++++++--------
tests/qemu-iotests/061.out | 14 +++++++-------
2 files changed, 13 insertions(+), 15 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index fd66bf2..d1a0e53 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2691,11 +2691,11 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
} else if (!strcmp(compat, "1.1")) {
new_version = 3;
} else {
- fprintf(stderr, "Unknown compatibility level %s.\n", compat);
+ error_report("Unknown compatibility level %s", compat);
return -EINVAL;
}
} else if (!strcmp(desc->name, BLOCK_OPT_PREALLOC)) {
- fprintf(stderr, "Cannot change preallocation mode.\n");
+ error_report("Cannot change preallocation mode");
return -ENOTSUP;
} else if (!strcmp(desc->name, BLOCK_OPT_SIZE)) {
new_size = qemu_opt_get_size(opts, BLOCK_OPT_SIZE, 0);
@@ -2707,16 +2707,14 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
encrypt = qemu_opt_get_bool(opts, BLOCK_OPT_ENCRYPT,
s->crypt_method);
if (encrypt != !!s->crypt_method) {
- fprintf(stderr, "Changing the encryption flag is not "
- "supported.\n");
+ error_report("Changing the encryption flag is not supported");
return -ENOTSUP;
}
} else if (!strcmp(desc->name, BLOCK_OPT_CLUSTER_SIZE)) {
cluster_size = qemu_opt_get_size(opts, BLOCK_OPT_CLUSTER_SIZE,
cluster_size);
if (cluster_size != s->cluster_size) {
- fprintf(stderr, "Changing the cluster size is not "
- "supported.\n");
+ error_report("Changing the cluster size is not supported");
return -ENOTSUP;
}
} else if (!strcmp(desc->name, BLOCK_OPT_LAZY_REFCOUNTS)) {
@@ -2762,8 +2760,8 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
if (s->use_lazy_refcounts != lazy_refcounts) {
if (lazy_refcounts) {
if (s->qcow_version < 3) {
- fprintf(stderr, "Lazy refcounts only supported with compatibility "
- "level 1.1 and above (use compat=1.1 or greater)\n");
+ error_report("Lazy refcounts only supported with compatibility "
+ "level 1.1 and above (use compat=1.1 or greater)");
return -EINVAL;
}
s->compatible_features |= QCOW2_COMPAT_LAZY_REFCOUNTS;
diff --git a/tests/qemu-iotests/061.out b/tests/qemu-iotests/061.out
index 5ec248f..713ca6b 100644
--- a/tests/qemu-iotests/061.out
+++ b/tests/qemu-iotests/061.out
@@ -281,18 +281,18 @@ No errors were found on the image.
=== Testing invalid configurations ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
+qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
qemu-img: Error while amending options: Invalid argument
-Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
+qemu-img: Lazy refcounts only supported with compatibility level 1.1 and above (use compat=1.1 or greater)
qemu-img: Error while amending options: Invalid argument
-Unknown compatibility level 0.42.
+qemu-img: Unknown compatibility level 0.42
qemu-img: Error while amending options: Invalid argument
qemu-img: Invalid parameter 'foo'
-Changing the cluster size is not supported.
+qemu-img: Changing the cluster size is not supported
qemu-img: Error while amending options: Operation not supported
-Changing the encryption flag is not supported.
+qemu-img: Changing the encryption flag is not supported
qemu-img: Error while amending options: Operation not supported
-Cannot change preallocation mode.
+qemu-img: Cannot change preallocation mode
qemu-img: Error while amending options: Operation not supported
=== Testing correct handling of unset value ===
@@ -300,7 +300,7 @@ qemu-img: Error while amending options: Operation not supported
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
Should work:
Should not work:
-Changing the cluster size is not supported.
+qemu-img: Changing the cluster size is not supported
qemu-img: Error while amending options: Operation not supported
=== Testing zero expansion on inactive clusters ===
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 04/10] qcow2: Use abort() instead of assert(false)
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (2 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 03/10] qcow2: Use error_report() in qcow2_amend_options() Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 05/10] qcow2: Split upgrade/downgrade paths for amend Max Reitz
` (6 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index d1a0e53..da7795c 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2724,9 +2724,9 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
error_report("Cannot change refcount entry width");
return -ENOTSUP;
} else {
- /* if this assertion fails, this probably means a new option was
+ /* if this point is reached, this probably means a new option was
* added without having it covered here */
- assert(false);
+ abort();
}
desc++;
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 05/10] qcow2: Split upgrade/downgrade paths for amend
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (3 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 04/10] qcow2: Use abort() instead of assert(false) Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 06/10] qcow2: Use intermediate helper CB " Max Reitz
` (5 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
If the image version should be upgraded, that is the first we should do;
if it should be downgraded, that is the last we should do. So split the
version change block into an upgrade part at the start and a downgrade
part at the end.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2.c | 31 ++++++++++++++++---------------
1 file changed, 16 insertions(+), 15 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index da7795c..35b91cd 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2732,20 +2732,13 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
desc++;
}
- if (new_version != old_version) {
- if (new_version > old_version) {
- /* Upgrade */
- s->qcow_version = new_version;
- ret = qcow2_update_header(bs);
- if (ret < 0) {
- s->qcow_version = old_version;
- return ret;
- }
- } else {
- ret = qcow2_downgrade(bs, new_version, status_cb, cb_opaque);
- if (ret < 0) {
- return ret;
- }
+ /* Upgrade first (some features may require compat=1.1) */
+ if (new_version > old_version) {
+ s->qcow_version = new_version;
+ ret = qcow2_update_header(bs);
+ if (ret < 0) {
+ s->qcow_version = old_version;
+ return ret;
}
}
@@ -2759,7 +2752,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
if (s->use_lazy_refcounts != lazy_refcounts) {
if (lazy_refcounts) {
- if (s->qcow_version < 3) {
+ if (new_version < 3) {
error_report("Lazy refcounts only supported with compatibility "
"level 1.1 and above (use compat=1.1 or greater)");
return -EINVAL;
@@ -2795,6 +2788,14 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
}
}
+ /* Downgrade last (so unsupported features can be removed before) */
+ if (new_version < old_version) {
+ ret = qcow2_downgrade(bs, new_version, status_cb, cb_opaque);
+ if (ret < 0) {
+ return ret;
+ }
+ }
+
return 0;
}
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 06/10] qcow2: Use intermediate helper CB for amend
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (4 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 05/10] qcow2: Split upgrade/downgrade paths for amend Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for refcount order amendment Max Reitz
` (4 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
If there is more than one time-consuming operation to be performed for
qcow2_amend_options(), we need an intermediate CB which coordinates the
progress of the individual operations and passes the result to the
original status callback.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2.c | 80 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
1 file changed, 79 insertions(+), 1 deletion(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index 35b91cd..f311159 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2660,6 +2660,75 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
return 0;
}
+typedef enum Qcow2AmendOperation {
+ /* This is the value Qcow2AmendHelperCBInfo::last_operation will be
+ * statically initialized to so that the helper CB can discern the first
+ * invocation from an operation change */
+ QCOW2_NO_OPERATION = 0,
+
+ QCOW2_DOWNGRADING,
+} Qcow2AmendOperation;
+
+typedef struct Qcow2AmendHelperCBInfo {
+ /* The code coordinating the amend operations should only modify
+ * these four fields; the rest will be managed by the CB */
+ BlockDriverAmendStatusCB *original_status_cb;
+ void *original_cb_opaque;
+
+ Qcow2AmendOperation current_operation;
+
+ /* Total number of operations to perform (only set once) */
+ int total_operations;
+
+ /* The following fields are managed by the CB */
+
+ /* Number of operations completed */
+ int operations_completed;
+
+ /* Cumulative offset of all completed operations */
+ int64_t offset_completed;
+
+ Qcow2AmendOperation last_operation;
+ int64_t last_work_size;
+} Qcow2AmendHelperCBInfo;
+
+static void qcow2_amend_helper_cb(BlockDriverState *bs,
+ int64_t operation_offset,
+ int64_t operation_work_size, void *opaque)
+{
+ Qcow2AmendHelperCBInfo *info = opaque;
+ int64_t current_work_size;
+ int64_t projected_work_size;
+
+ if (info->current_operation != info->last_operation) {
+ if (info->last_operation != QCOW2_NO_OPERATION) {
+ info->offset_completed += info->last_work_size;
+ info->operations_completed++;
+ }
+
+ info->last_operation = info->current_operation;
+ }
+
+ assert(info->total_operations > 0);
+ assert(info->operations_completed < info->total_operations);
+
+ info->last_work_size = operation_work_size;
+
+ current_work_size = info->offset_completed + operation_work_size;
+
+ /* current_work_size is the total work size for (operations_completed + 1)
+ * operations (which includes this one), so multiply it by the number of
+ * operations not covered and divide it by the number of operations
+ * covered to get a projection for the operations not covered */
+ projected_work_size = current_work_size * (info->total_operations -
+ info->operations_completed - 1)
+ / (info->operations_completed + 1);
+
+ info->original_status_cb(bs, info->offset_completed + operation_offset,
+ current_work_size + projected_work_size,
+ info->original_cb_opaque);
+}
+
static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
BlockDriverAmendStatusCB *status_cb,
void *cb_opaque)
@@ -2674,6 +2743,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
bool encrypt;
int ret;
QemuOptDesc *desc = opts->list->desc;
+ Qcow2AmendHelperCBInfo helper_cb_info;
while (desc && desc->name) {
if (!qemu_opt_find(opts, desc->name)) {
@@ -2732,6 +2802,12 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
desc++;
}
+ helper_cb_info = (Qcow2AmendHelperCBInfo){
+ .original_status_cb = status_cb,
+ .original_cb_opaque = cb_opaque,
+ .total_operations = (new_version < old_version)
+ };
+
/* Upgrade first (some features may require compat=1.1) */
if (new_version > old_version) {
s->qcow_version = new_version;
@@ -2790,7 +2866,9 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
/* Downgrade last (so unsupported features can be removed before) */
if (new_version < old_version) {
- ret = qcow2_downgrade(bs, new_version, status_cb, cb_opaque);
+ helper_cb_info.current_operation = QCOW2_DOWNGRADING;
+ ret = qcow2_downgrade(bs, new_version, &qcow2_amend_helper_cb,
+ &helper_cb_info);
if (ret < 0) {
return ret;
}
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for refcount order amendment
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (5 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 06/10] qcow2: Use intermediate helper CB " Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 08/10] qcow2: Invoke refcount order amendment function Max Reitz
` (3 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Add a function qcow2_change_refcount_order() which allows changing the
refcount order of a qcow2 image.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2-refcount.c | 452 +++++++++++++++++++++++++++++++++++++++++++++++++
block/qcow2.h | 4 +
2 files changed, 456 insertions(+)
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 6cbae1d..f7e5b5e 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -2461,3 +2461,455 @@ int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset,
return 0;
}
+
+/* A pointer to a function of this type is given to walk_over_reftable(). That
+ * function will create refblocks and pass them to a RefblockFinishOp once they
+ * are completed (@refblock). @refblock_empty is set if the refblock is
+ * completely empty.
+ *
+ * Along with the refblock, a corresponding reftable entry is passed, in the
+ * reftable @reftable (which may be reallocated) at @reftable_index.
+ *
+ * @allocated should be set to true if a new cluster has been allocated.
+ */
+typedef int (RefblockFinishOp)(BlockDriverState *bs, uint64_t **reftable,
+ uint64_t reftable_index, uint64_t *reftable_size,
+ void *refblock, bool refblock_empty,
+ bool *allocated, Error **errp);
+
+/**
+ * This "operation" for walk_over_reftable() allocates the refblock on disk (if
+ * it is not empty) and inserts its offset into the new reftable. The size of
+ * this new reftable is increased as required.
+ */
+static int alloc_refblock(BlockDriverState *bs, uint64_t **reftable,
+ uint64_t reftable_index, uint64_t *reftable_size,
+ void *refblock, bool refblock_empty, bool *allocated,
+ Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ int64_t offset;
+
+ if (!refblock_empty && reftable_index >= *reftable_size) {
+ uint64_t *new_reftable;
+ uint64_t new_reftable_size;
+
+ new_reftable_size = ROUND_UP(reftable_index + 1,
+ s->cluster_size / sizeof(uint64_t));
+ if (new_reftable_size > QCOW_MAX_REFTABLE_SIZE / sizeof(uint64_t)) {
+ error_setg(errp,
+ "This operation would make the refcount table grow "
+ "beyond the maximum size supported by QEMU, aborting");
+ return -ENOTSUP;
+ }
+
+ new_reftable = g_try_realloc(*reftable, new_reftable_size *
+ sizeof(uint64_t));
+ if (!new_reftable) {
+ error_setg(errp, "Failed to increase reftable buffer size");
+ return -ENOMEM;
+ }
+
+ memset(new_reftable + *reftable_size, 0,
+ (new_reftable_size - *reftable_size) * sizeof(uint64_t));
+
+ *reftable = new_reftable;
+ *reftable_size = new_reftable_size;
+ }
+
+ if (!refblock_empty && !(*reftable)[reftable_index]) {
+ offset = qcow2_alloc_clusters(bs, s->cluster_size);
+ if (offset < 0) {
+ error_setg_errno(errp, -offset, "Failed to allocate refblock");
+ return offset;
+ }
+ (*reftable)[reftable_index] = offset;
+ *allocated = true;
+ }
+
+ return 0;
+}
+
+/**
+ * This "operation" for walk_over_reftable() writes the refblock to disk at the
+ * offset specified by the new reftable's entry. It does not modify the new
+ * reftable or change any refcounts.
+ */
+static int flush_refblock(BlockDriverState *bs, uint64_t **reftable,
+ uint64_t reftable_index, uint64_t *reftable_size,
+ void *refblock, bool refblock_empty, bool *allocated,
+ Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ int64_t offset;
+ int ret;
+
+ if (reftable_index < *reftable_size && (*reftable)[reftable_index]) {
+ offset = (*reftable)[reftable_index];
+
+ ret = qcow2_pre_write_overlap_check(bs, 0, offset, s->cluster_size);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Overlap check failed");
+ return ret;
+ }
+
+ ret = bdrv_pwrite(bs->file, offset, refblock, s->cluster_size);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to write refblock");
+ return ret;
+ }
+ } else {
+ assert(refblock_empty);
+ }
+
+ return 0;
+}
+
+/**
+ * This function walks over the existing reftable and every referenced refblock;
+ * if @new_set_refcount is non-NULL, it is called for every refcount entry to
+ * create an equal new entry in the passed @new_refblock. Once that
+ * @new_refblock is completely filled, @operation will be called.
+ *
+ * @status_cb and @cb_opaque are used for the amend operation's status callback.
+ * @index is the index of the walk_over_reftable() calls and @total is the total
+ * number of walk_over_reftable() calls per amend operation. Both are used for
+ * calculating the parameters for the status callback.
+ *
+ * @allocated is set to true if a new cluster has been allocated.
+ */
+static int walk_over_reftable(BlockDriverState *bs, uint64_t **new_reftable,
+ uint64_t *new_reftable_index,
+ uint64_t *new_reftable_size,
+ void *new_refblock, int new_refblock_size,
+ int new_refcount_bits,
+ RefblockFinishOp *operation, bool *allocated,
+ Qcow2SetRefcountFunc *new_set_refcount,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque, int index, int total,
+ Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ uint64_t reftable_index;
+ bool new_refblock_empty = true;
+ int refblock_index;
+ int new_refblock_index = 0;
+ int ret;
+
+ for (reftable_index = 0; reftable_index < s->refcount_table_size;
+ reftable_index++)
+ {
+ uint64_t refblock_offset = s->refcount_table[reftable_index]
+ & REFT_OFFSET_MASK;
+
+ status_cb(bs, (uint64_t)index * s->refcount_table_size + reftable_index,
+ (uint64_t)total * s->refcount_table_size, cb_opaque);
+
+ if (refblock_offset) {
+ void *refblock;
+
+ if (offset_into_cluster(s, refblock_offset)) {
+ qcow2_signal_corruption(bs, true, -1, -1, "Refblock offset %#"
+ PRIx64 " unaligned (reftable index: %#"
+ PRIx64 ")", refblock_offset,
+ reftable_index);
+ error_setg(errp,
+ "Image is corrupt (unaligned refblock offset)");
+ return -EIO;
+ }
+
+ ret = qcow2_cache_get(bs, s->refcount_block_cache, refblock_offset,
+ &refblock);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to retrieve refblock");
+ return ret;
+ }
+
+ for (refblock_index = 0; refblock_index < s->refcount_block_size;
+ refblock_index++)
+ {
+ uint64_t refcount;
+
+ if (new_refblock_index >= new_refblock_size) {
+ /* new_refblock is now complete */
+ ret = operation(bs, new_reftable, *new_reftable_index,
+ new_reftable_size, new_refblock,
+ new_refblock_empty, allocated, errp);
+ if (ret < 0) {
+ qcow2_cache_put(bs, s->refcount_block_cache, &refblock);
+ return ret;
+ }
+
+ (*new_reftable_index)++;
+ new_refblock_index = 0;
+ new_refblock_empty = true;
+ }
+
+ refcount = s->get_refcount(refblock, refblock_index);
+ if (new_refcount_bits < 64 && refcount >> new_refcount_bits) {
+ uint64_t offset;
+
+ qcow2_cache_put(bs, s->refcount_block_cache, &refblock);
+
+ offset = ((reftable_index << s->refcount_block_bits)
+ + refblock_index) << s->cluster_bits;
+
+ error_setg(errp, "Cannot decrease refcount entry width to "
+ "%i bits: Cluster at offset %#" PRIx64 " has a "
+ "refcount of %" PRIu64, new_refcount_bits,
+ offset, refcount);
+ return -EINVAL;
+ }
+
+ if (new_set_refcount) {
+ new_set_refcount(new_refblock, new_refblock_index++,
+ refcount);
+ } else {
+ new_refblock_index++;
+ }
+ new_refblock_empty = new_refblock_empty && refcount == 0;
+ }
+
+ ret = qcow2_cache_put(bs, s->refcount_block_cache, &refblock);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to put refblock back into "
+ "the cache");
+ return ret;
+ }
+ } else {
+ /* No refblock means every refcount is 0 */
+ for (refblock_index = 0; refblock_index < s->refcount_block_size;
+ refblock_index++)
+ {
+ if (new_refblock_index >= new_refblock_size) {
+ /* new_refblock is now complete */
+ ret = operation(bs, new_reftable, *new_reftable_index,
+ new_reftable_size, new_refblock,
+ new_refblock_empty, allocated, errp);
+ if (ret < 0) {
+ return ret;
+ }
+
+ (*new_reftable_index)++;
+ new_refblock_index = 0;
+ new_refblock_empty = true;
+ }
+
+ if (new_set_refcount) {
+ new_set_refcount(new_refblock, new_refblock_index++, 0);
+ } else {
+ new_refblock_index++;
+ }
+ }
+ }
+ }
+
+ if (new_refblock_index > 0) {
+ /* Complete the potentially existing partially filled final refblock */
+ if (new_set_refcount) {
+ for (; new_refblock_index < new_refblock_size;
+ new_refblock_index++)
+ {
+ new_set_refcount(new_refblock, new_refblock_index, 0);
+ }
+ }
+
+ ret = operation(bs, new_reftable, *new_reftable_index,
+ new_reftable_size, new_refblock, new_refblock_empty,
+ allocated, errp);
+ if (ret < 0) {
+ return ret;
+ }
+
+ (*new_reftable_index)++;
+ }
+
+ status_cb(bs, (uint64_t)(index + 1) * s->refcount_table_size,
+ (uint64_t)total * s->refcount_table_size, cb_opaque);
+
+ return 0;
+}
+
+int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque, Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ Qcow2GetRefcountFunc *new_get_refcount;
+ Qcow2SetRefcountFunc *new_set_refcount;
+ void *new_refblock = qemu_blockalign(bs->file, s->cluster_size);
+ uint64_t *new_reftable = NULL, new_reftable_size = 0;
+ uint64_t *old_reftable, old_reftable_size, old_reftable_offset;
+ uint64_t new_reftable_index = 0;
+ uint64_t i;
+ int64_t new_reftable_offset = 0, allocated_reftable_size = 0;
+ int new_refblock_size, new_refcount_bits = 1 << refcount_order;
+ int old_refcount_order;
+ int walk_index = 0;
+ int ret;
+ bool new_allocation;
+
+ assert(s->qcow_version >= 3);
+ assert(refcount_order >= 0 && refcount_order <= 6);
+
+ /* see qcow2_open() */
+ new_refblock_size = 1 << (s->cluster_bits - (refcount_order - 3));
+
+ new_get_refcount = get_refcount_funcs[refcount_order];
+ new_set_refcount = set_refcount_funcs[refcount_order];
+
+
+ do {
+ int total_walks;
+
+ new_allocation = false;
+
+ /* At least we have to do this walk and the one which writes the
+ * refblocks; also, at least we have to do this loop here at least
+ * twice (normally), first to do the allocations, and second to
+ * determine that everything is correctly allocated, this then makes
+ * three walks in total */
+ total_walks = MAX(walk_index + 2, 3);
+
+ /* First, allocate the structures so they are present in the refcount
+ * structures */
+ ret = walk_over_reftable(bs, &new_reftable, &new_reftable_index,
+ &new_reftable_size, NULL, new_refblock_size,
+ new_refcount_bits, &alloc_refblock,
+ &new_allocation, NULL, status_cb, cb_opaque,
+ walk_index++, total_walks, errp);
+ if (ret < 0) {
+ goto done;
+ }
+
+ new_reftable_index = 0;
+
+ if (new_allocation) {
+ if (new_reftable_offset) {
+ qcow2_free_clusters(bs, new_reftable_offset,
+ allocated_reftable_size * sizeof(uint64_t),
+ QCOW2_DISCARD_NEVER);
+ }
+
+ new_reftable_offset = qcow2_alloc_clusters(bs, new_reftable_size *
+ sizeof(uint64_t));
+ if (new_reftable_offset < 0) {
+ error_setg_errno(errp, -new_reftable_offset,
+ "Failed to allocate the new reftable");
+ ret = new_reftable_offset;
+ goto done;
+ }
+ allocated_reftable_size = new_reftable_size;
+ }
+ } while (new_allocation);
+
+ /* Second, write the new refblocks */
+ ret = walk_over_reftable(bs, &new_reftable, &new_reftable_index,
+ &new_reftable_size, new_refblock,
+ new_refblock_size, new_refcount_bits,
+ &flush_refblock, &new_allocation, new_set_refcount,
+ status_cb, cb_opaque, walk_index, walk_index + 1,
+ errp);
+ if (ret < 0) {
+ goto done;
+ }
+ assert(!new_allocation);
+
+
+ /* Write the new reftable */
+ ret = qcow2_pre_write_overlap_check(bs, 0, new_reftable_offset,
+ new_reftable_size * sizeof(uint64_t));
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Overlap check failed");
+ goto done;
+ }
+
+ for (i = 0; i < new_reftable_size; i++) {
+ cpu_to_be64s(&new_reftable[i]);
+ }
+
+ ret = bdrv_pwrite(bs->file, new_reftable_offset, new_reftable,
+ new_reftable_size * sizeof(uint64_t));
+
+ for (i = 0; i < new_reftable_size; i++) {
+ be64_to_cpus(&new_reftable[i]);
+ }
+
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to write the new reftable");
+ goto done;
+ }
+
+
+ /* Empty the refcount cache */
+ ret = qcow2_cache_flush(bs, s->refcount_block_cache);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to flush the refblock cache");
+ goto done;
+ }
+
+ /* Update the image header to point to the new reftable; this only updates
+ * the fields which are relevant to qcow2_update_header(); other fields
+ * such as s->refcount_table or s->refcount_bits stay stale for now
+ * (because we have to restore everything if qcow2_update_header() fails) */
+ old_refcount_order = s->refcount_order;
+ old_reftable_size = s->refcount_table_size;
+ old_reftable_offset = s->refcount_table_offset;
+
+ s->refcount_order = refcount_order;
+ s->refcount_table_size = new_reftable_size;
+ s->refcount_table_offset = new_reftable_offset;
+
+ ret = qcow2_update_header(bs);
+ if (ret < 0) {
+ s->refcount_order = old_refcount_order;
+ s->refcount_table_size = old_reftable_size;
+ s->refcount_table_offset = old_reftable_offset;
+ error_setg_errno(errp, -ret, "Failed to update the qcow2 header");
+ goto done;
+ }
+
+ /* Now update the rest of the in-memory information */
+ old_reftable = s->refcount_table;
+ s->refcount_table = new_reftable;
+
+ s->refcount_bits = 1 << refcount_order;
+ s->refcount_max = UINT64_C(1) << (s->refcount_bits - 1);
+ s->refcount_max += s->refcount_max - 1;
+
+ s->refcount_block_bits = s->cluster_bits - (refcount_order - 3);
+ s->refcount_block_size = 1 << s->refcount_block_bits;
+
+ s->get_refcount = new_get_refcount;
+ s->set_refcount = new_set_refcount;
+
+ /* For cleaning up all old refblocks and the old reftable below the "done"
+ * label */
+ new_reftable = old_reftable;
+ new_reftable_size = old_reftable_size;
+ new_reftable_offset = old_reftable_offset;
+
+done:
+ if (new_reftable) {
+ /* On success, new_reftable actually points to the old reftable (and
+ * new_reftable_size is the old reftable's size); but that is just
+ * fine */
+ for (i = 0; i < new_reftable_size; i++) {
+ uint64_t offset = new_reftable[i] & REFT_OFFSET_MASK;
+ if (offset) {
+ qcow2_free_clusters(bs, offset, s->cluster_size,
+ QCOW2_DISCARD_OTHER);
+ }
+ }
+ g_free(new_reftable);
+
+ if (new_reftable_offset > 0) {
+ qcow2_free_clusters(bs, new_reftable_offset,
+ new_reftable_size * sizeof(uint64_t),
+ QCOW2_DISCARD_OTHER);
+ }
+ }
+
+ qemu_vfree(new_refblock);
+ return ret;
+}
diff --git a/block/qcow2.h b/block/qcow2.h
index eed26db..8f34a5c 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -521,6 +521,10 @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset,
int64_t size);
+int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque, Error **errp);
+
/* qcow2-cluster.c functions */
int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,
bool exact_size);
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 08/10] qcow2: Invoke refcount order amendment function
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (6 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for refcount order amendment Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 09/10] qcow2: Point to amend function in check Max Reitz
` (2 subsequent siblings)
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Make use of qcow2_change_refcount_order() to support changing the
refcount order with qemu-img amend.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2.c | 44 +++++++++++++++++++++++++++++++++++---------
1 file changed, 35 insertions(+), 9 deletions(-)
diff --git a/block/qcow2.c b/block/qcow2.c
index f311159..3814361 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2612,13 +2612,7 @@ static int qcow2_downgrade(BlockDriverState *bs, int target_version,
}
if (s->refcount_order != 4) {
- /* we would have to convert the image to a refcount_order == 4 image
- * here; however, since qemu (at the time of writing this) does not
- * support anything different than 4 anyway, there is no point in doing
- * so right now; however, we should error out (if qemu supports this in
- * the future and this code has not been adapted) */
- error_report("qcow2_downgrade: Image refcount orders other than 4 are "
- "currently not supported.");
+ error_report("compat=0.10 requires refcount_bits=16");
return -ENOTSUP;
}
@@ -2666,6 +2660,7 @@ typedef enum Qcow2AmendOperation {
* invocation from an operation change */
QCOW2_NO_OPERATION = 0,
+ QCOW2_CHANGING_REFCOUNT_ORDER,
QCOW2_DOWNGRADING,
} Qcow2AmendOperation;
@@ -2741,6 +2736,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
const char *compat = NULL;
uint64_t cluster_size = s->cluster_size;
bool encrypt;
+ int refcount_bits = s->refcount_bits;
int ret;
QemuOptDesc *desc = opts->list->desc;
Qcow2AmendHelperCBInfo helper_cb_info;
@@ -2791,8 +2787,16 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
lazy_refcounts = qemu_opt_get_bool(opts, BLOCK_OPT_LAZY_REFCOUNTS,
lazy_refcounts);
} else if (!strcmp(desc->name, BLOCK_OPT_REFCOUNT_BITS)) {
- error_report("Cannot change refcount entry width");
- return -ENOTSUP;
+ refcount_bits = qemu_opt_get_number(opts, BLOCK_OPT_REFCOUNT_BITS,
+ refcount_bits);
+
+ if (refcount_bits <= 0 || refcount_bits > 64 ||
+ !is_power_of_2(refcount_bits))
+ {
+ error_report("Refcount width must be a power of two and may "
+ "not exceed 64 bits");
+ return -EINVAL;
+ }
} else {
/* if this point is reached, this probably means a new option was
* added without having it covered here */
@@ -2806,6 +2810,7 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
.original_status_cb = status_cb,
.original_cb_opaque = cb_opaque,
.total_operations = (new_version < old_version)
+ + (s->refcount_bits != refcount_bits)
};
/* Upgrade first (some features may require compat=1.1) */
@@ -2818,6 +2823,27 @@ static int qcow2_amend_options(BlockDriverState *bs, QemuOpts *opts,
}
}
+ if (s->refcount_bits != refcount_bits) {
+ int refcount_order = ffs(refcount_bits) - 1;
+ Error *local_error = NULL;
+
+ if (new_version < 3 && refcount_bits != 16) {
+ error_report("Different refcount widths than 16 bits require "
+ "compatibility level 1.1 or above (use compat=1.1 or "
+ "greater)");
+ return -EINVAL;
+ }
+
+ helper_cb_info.current_operation = QCOW2_CHANGING_REFCOUNT_ORDER;
+ ret = qcow2_change_refcount_order(bs, refcount_order,
+ &qcow2_amend_helper_cb,
+ &helper_cb_info, &local_error);
+ if (ret < 0) {
+ qerror_report_err(local_error);
+ return ret;
+ }
+ }
+
if (backing_file || backing_format) {
ret = qcow2_change_backing_file(bs, backing_file ?: bs->backing_file,
backing_format ?: bs->backing_format);
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 09/10] qcow2: Point to amend function in check
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (7 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 08/10] qcow2: Invoke refcount order amendment function Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 10/10] iotests: Extend test 112 for qemu-img amend Max Reitz
2015-04-24 15:00 ` [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
10 siblings, 0 replies; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
If a reference count is not representable with the current refcount
order, the image check should point to qemu-img amend for increasing the
refcount order. However, qemu-img amend needs write access to the image
which cannot be provided if the image is marked corrupt; and the image
check will not mark the image consistent unless everything actually is
consistent.
Therefore, if an image is marked corrupt and the image check encounters
a reference count overflow, it cannot be fixed by using qemu-img amend
to increase the refcount order. Instead, one has to use qemu-img convert
to create a completely new copy of the image in this case.
Alternatively, we may want to give the user a way of manually removing
the corrupt flag, maybe through qemu-img amend, but this is not part of
this patch.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
---
block/qcow2-refcount.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index f7e5b5e..46f999c 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -1340,6 +1340,9 @@ static int inc_refcounts(BlockDriverState *bs,
if (refcount == s->refcount_max) {
fprintf(stderr, "ERROR: overflow cluster offset=0x%" PRIx64
"\n", cluster_offset);
+ fprintf(stderr, "Use qemu-img amend to increase the refcount entry "
+ "width or qemu-img convert to create a clean copy if the "
+ "image cannot be opened for writing\n");
res->corruptions++;
continue;
}
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 10/10] iotests: Extend test 112 for qemu-img amend
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (8 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 09/10] qcow2: Point to amend function in check Max Reitz
@ 2015-03-18 20:56 ` Max Reitz
2015-05-11 12:49 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2015-04-24 15:00 ` [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
10 siblings, 1 reply; 16+ messages in thread
From: Max Reitz @ 2015-03-18 20:56 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Add tests for conversion between different refcount widths.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/112 | 109 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/112.out | 71 +++++++++++++++++++++++++++++
2 files changed, 180 insertions(+)
diff --git a/tests/qemu-iotests/112 b/tests/qemu-iotests/112
index 3f054a3..34ba06a 100755
--- a/tests/qemu-iotests/112
+++ b/tests/qemu-iotests/112
@@ -180,6 +180,115 @@ $QEMU_IMG snapshot -c foo "$TEST_IMG"
# leaked (refcount=UINT64_MAX reference=1)
_check_test_img
+echo
+echo '=== Amend from refcount_bits=16 to refcount_bits=1 ==='
+echo
+
+_make_test_img 64M
+print_refcount_bits
+
+$QEMU_IO -c 'write 16M 32M' "$TEST_IMG" | _filter_qemu_io
+$QEMU_IMG amend -o refcount_bits=1 "$TEST_IMG"
+_check_test_img
+print_refcount_bits
+
+echo
+echo '=== Amend from refcount_bits=1 to refcount_bits=64 ==='
+echo
+
+$QEMU_IMG amend -o refcount_bits=64 "$TEST_IMG"
+_check_test_img
+print_refcount_bits
+
+echo
+echo '=== Amend to compat=0.10 ==='
+echo
+
+# Should not work because refcount_bits needs to be 16 for compat=0.10
+$QEMU_IMG amend -o compat=0.10 "$TEST_IMG"
+print_refcount_bits
+# Should work
+$QEMU_IMG amend -o compat=0.10,refcount_bits=16 "$TEST_IMG"
+_check_test_img
+print_refcount_bits
+
+# Get back to compat=1.1 and refcount_bits=16
+$QEMU_IMG amend -o compat=1.1 "$TEST_IMG"
+print_refcount_bits
+# Should not work
+$QEMU_IMG amend -o refcount_bits=32,compat=0.10 "$TEST_IMG"
+print_refcount_bits
+
+echo
+echo '=== Amend with snapshot ==='
+echo
+
+$QEMU_IMG snapshot -c foo "$TEST_IMG"
+# Just to have different refcounts across the image
+$QEMU_IO -c 'write 0 16M' "$TEST_IMG" | _filter_qemu_io
+
+# Should not work (may work in the future by first decreasing all refcounts so
+# they fit into the target range by copying them)
+$QEMU_IMG amend -o refcount_bits=1 "$TEST_IMG"
+_check_test_img
+print_refcount_bits
+
+# Should work
+$QEMU_IMG amend -o refcount_bits=2 "$TEST_IMG"
+_check_test_img
+print_refcount_bits
+
+echo
+echo '=== Testing too many references for check ==='
+echo
+
+IMGOPTS="$IMGOPTS,refcount_bits=1" _make_test_img 64M
+print_refcount_bits
+
+# This cluster should be created at 0x50000
+$QEMU_IO -c 'write 0 64k' "$TEST_IMG" | _filter_qemu_io
+# Now make the second L2 entry (the L2 table should be at 0x40000) point to that
+# cluster, so we have two references
+poke_file "$TEST_IMG" $((0x40008)) "\x80\x00\x00\x00\x00\x05\x00\x00"
+
+# This should say "please use amend"
+_check_test_img -r all
+
+# So we do that
+$QEMU_IMG amend -o refcount_bits=2 "$TEST_IMG"
+print_refcount_bits
+
+# And try again
+_check_test_img -r all
+
+echo
+echo '=== Multiple walks necessary during amend ==='
+echo
+
+IMGOPTS="$IMGOPTS,refcount_bits=1,cluster_size=512" _make_test_img 64k
+
+# Cluster 0 is the image header, clusters 1 to 4 are used by the L1 table, a
+# single L2 table, the reftable and a single refblock. This creates 58 data
+# clusters (actually, the L2 table is created here, too), so in total there are
+# then 63 used clusters in the image. With a refcount width of 64, one refblock
+# describes 64 clusters (512 bytes / 64 bits/entry = 64 entries), so this will
+# make the first refblock in the amended image have exactly one free entry.
+$QEMU_IO -c "write 0 $((58 * 512))" "$TEST_IMG" | _filter_qemu_io
+
+# Now change the refcount width; since the first new refblock will have exactly
+# one free entry, that entry will be used to store its own reference. No other
+# refblocks are needed, so then the new reftable will be allocated; since the
+# first new refblock is completely filled up, this will require a new refblock
+# which is why the refcount width changing function will need to run through
+# everything one more time until the allocations are stable.
+# Having more walks than usual should be visible as regressing progress (from
+# 66.67 % (2/3 walks) to 50.00 % (2/4 walks)).
+$QEMU_IMG amend -o refcount_bits=64 -p "$TEST_IMG" | tr '\r' '\n' \
+ | grep -A 1 '66.67'
+print_refcount_bits
+
+_check_test_img
+
# success, all done
echo '*** done'
diff --git a/tests/qemu-iotests/112.out b/tests/qemu-iotests/112.out
index 9a98633..4438076 100644
--- a/tests/qemu-iotests/112.out
+++ b/tests/qemu-iotests/112.out
@@ -81,4 +81,75 @@ Leaked cluster 6 refcount=1 reference=0
2 leaked clusters were found on the image.
This means waste of disk space, but no harm to data.
+
+=== Amend from refcount_bits=16 to refcount_bits=1 ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+refcount bits: 16
+wrote 33554432/33554432 bytes at offset 16777216
+32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+No errors were found on the image.
+refcount bits: 1
+
+=== Amend from refcount_bits=1 to refcount_bits=64 ===
+
+No errors were found on the image.
+refcount bits: 64
+
+=== Amend to compat=0.10 ===
+
+qemu-img: compat=0.10 requires refcount_bits=16
+qemu-img: Error while amending options: Operation not supported
+refcount bits: 64
+No errors were found on the image.
+refcount bits: 16
+refcount bits: 16
+qemu-img: Different refcount widths than 16 bits require compatibility level 1.1 or above (use compat=1.1 or greater)
+qemu-img: Error while amending options: Invalid argument
+refcount bits: 16
+
+=== Amend with snapshot ===
+
+wrote 16777216/16777216 bytes at offset 0
+16 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+qemu-img: Cannot decrease refcount entry width to 1 bits: Cluster at offset 0x50000 has a refcount of 2
+qemu-img: Error while amending options: Invalid argument
+No errors were found on the image.
+refcount bits: 16
+No errors were found on the image.
+refcount bits: 2
+
+=== Testing too many references for check ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+refcount bits: 1
+wrote 65536/65536 bytes at offset 0
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ERROR: overflow cluster offset=0x50000
+Use qemu-img amend to increase the refcount entry width or qemu-img convert to create a clean copy if the image cannot be opened for writing
+
+1 errors were found on the image.
+Data may be corrupted, or further writes to the image may corrupt it.
+refcount bits: 2
+ERROR cluster 5 refcount=1 reference=2
+Repairing cluster 5 refcount=1 reference=2
+Repairing OFLAG_COPIED data cluster: l2_entry=8000000000050000 refcount=2
+Repairing OFLAG_COPIED data cluster: l2_entry=8000000000050000 refcount=2
+The following inconsistencies were found and repaired:
+
+ 0 leaked clusters
+ 3 corruptions
+
+Double checking the fixed image now...
+No errors were found on the image.
+
+=== Multiple walks necessary during amend ===
+
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=65536
+wrote 29696/29696 bytes at offset 0
+29 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+ (66.67/100%)
+ (50.00/100%)
+refcount bits: 64
+No errors were found on the image.
*** done
--
2.1.0
^ permalink raw reply related [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
` (9 preceding siblings ...)
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 10/10] iotests: Extend test 112 for qemu-img amend Max Reitz
@ 2015-04-24 15:00 ` Max Reitz
2015-05-11 12:49 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
10 siblings, 1 reply; 16+ messages in thread
From: Max Reitz @ 2015-04-24 15:00 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi
On 18.03.2015 21:56, Max Reitz wrote:
> This series adds support to qemu for changing the refcount_bits option
> of an existing qcow2 file through the qemu-img amend command.
>
> Originally (up until v7), this series was called
> "qcow2: Support refcount orders != 4", but by now that support has
> already been merged so all that is left is support for qemu-img amend.
>
>
> v8:
> - Rebase onto master (one contextual conflict in an iotest reference
> output, but other than that this only means that the first 15 patches
> are dropped)
Ping; there is a contextual conflict in patch 8 which I found pretty
simple to resolve, but if you want me to send a rebase, so be it.
Max
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v8 10/10] iotests: Extend test 112 for qemu-img amend
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 10/10] iotests: Extend test 112 for qemu-img amend Max Reitz
@ 2015-05-11 12:49 ` Stefan Hajnoczi
0 siblings, 0 replies; 16+ messages in thread
From: Stefan Hajnoczi @ 2015-05-11 12:49 UTC (permalink / raw)
To: Max Reitz; +Cc: Stefan Hajnoczi, qemu-devel, qemu-block
[-- Attachment #1: Type: text/plain, Size: 472 bytes --]
On Wed, Mar 18, 2015 at 04:56:28PM -0400, Max Reitz wrote:
> Add tests for conversion between different refcount widths.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> Reviewed-by: Eric Blake <eblake@redhat.com>
> ---
> tests/qemu-iotests/112 | 109 +++++++++++++++++++++++++++++++++++++++++++++
> tests/qemu-iotests/112.out | 71 +++++++++++++++++++++++++++++
> 2 files changed, 180 insertions(+)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment
2015-04-24 15:00 ` [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
@ 2015-05-11 12:49 ` Stefan Hajnoczi
0 siblings, 0 replies; 16+ messages in thread
From: Stefan Hajnoczi @ 2015-05-11 12:49 UTC (permalink / raw)
To: Max Reitz; +Cc: Kevin Wolf, Stefan Hajnoczi, qemu-devel, qemu-block
[-- Attachment #1: Type: text/plain, Size: 853 bytes --]
On Fri, Apr 24, 2015 at 05:00:18PM +0200, Max Reitz wrote:
> On 18.03.2015 21:56, Max Reitz wrote:
> >This series adds support to qemu for changing the refcount_bits option
> >of an existing qcow2 file through the qemu-img amend command.
> >
> >Originally (up until v7), this series was called
> >"qcow2: Support refcount orders != 4", but by now that support has
> >already been merged so all that is left is support for qemu-img amend.
> >
> >
> >v8:
> >- Rebase onto master (one contextual conflict in an iotest reference
> > output, but other than that this only means that the first 15 patches
> > are dropped)
>
> Ping; there is a contextual conflict in patch 8 which I found pretty simple
> to resolve, but if you want me to send a rebase, so be it.
All patches have my R-b.
This series should go via Kevin.
Stefan
[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for refcount order amendment
2015-06-03 20:13 ` [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for " Max Reitz
@ 2015-06-03 22:10 ` Eric Blake
0 siblings, 0 replies; 16+ messages in thread
From: Eric Blake @ 2015-06-03 22:10 UTC (permalink / raw)
To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi
[-- Attachment #1: Type: text/plain, Size: 530 bytes --]
On 06/03/2015 02:13 PM, Max Reitz wrote:
> Add a function qcow2_change_refcount_order() which allows changing the
> refcount order of a qcow2 image.
>
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
> block/qcow2-refcount.c | 447 +++++++++++++++++++++++++++++++++++++++++++++++++
> block/qcow2.h | 4 +
> 2 files changed, 451 insertions(+)
>
Reviewed-by: Eric Blake <eblake@redhat.com>
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for refcount order amendment
2015-06-03 20:13 [Qemu-devel] [PATCH v8 00/10] qcow2: Support refcount order amendment Max Reitz
@ 2015-06-03 20:13 ` Max Reitz
2015-06-03 22:10 ` Eric Blake
0 siblings, 1 reply; 16+ messages in thread
From: Max Reitz @ 2015-06-03 20:13 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, qemu-devel, Stefan Hajnoczi, Max Reitz
Add a function qcow2_change_refcount_order() which allows changing the
refcount order of a qcow2 image.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/qcow2-refcount.c | 447 +++++++++++++++++++++++++++++++++++++++++++++++++
block/qcow2.h | 4 +
2 files changed, 451 insertions(+)
diff --git a/block/qcow2-refcount.c b/block/qcow2-refcount.c
index 0632fc3..13397a9 100644
--- a/block/qcow2-refcount.c
+++ b/block/qcow2-refcount.c
@@ -2449,3 +2449,450 @@ int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset,
return 0;
}
+
+/* A pointer to a function of this type is given to walk_over_reftable(). That
+ * function will create refblocks and pass them to a RefblockFinishOp once they
+ * are completed (@refblock). @refblock_empty is set if the refblock is
+ * completely empty.
+ *
+ * Along with the refblock, a corresponding reftable entry is passed, in the
+ * reftable @reftable (which may be reallocated) at @reftable_index.
+ *
+ * @allocated should be set to true if a new cluster has been allocated.
+ */
+typedef int (RefblockFinishOp)(BlockDriverState *bs, uint64_t **reftable,
+ uint64_t reftable_index, uint64_t *reftable_size,
+ void *refblock, bool refblock_empty,
+ bool *allocated, Error **errp);
+
+/**
+ * This "operation" for walk_over_reftable() allocates the refblock on disk (if
+ * it is not empty) and inserts its offset into the new reftable. The size of
+ * this new reftable is increased as required.
+ */
+static int alloc_refblock(BlockDriverState *bs, uint64_t **reftable,
+ uint64_t reftable_index, uint64_t *reftable_size,
+ void *refblock, bool refblock_empty, bool *allocated,
+ Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ int64_t offset;
+
+ if (!refblock_empty && reftable_index >= *reftable_size) {
+ uint64_t *new_reftable;
+ uint64_t new_reftable_size;
+
+ new_reftable_size = ROUND_UP(reftable_index + 1,
+ s->cluster_size / sizeof(uint64_t));
+ if (new_reftable_size > QCOW_MAX_REFTABLE_SIZE / sizeof(uint64_t)) {
+ error_setg(errp,
+ "This operation would make the refcount table grow "
+ "beyond the maximum size supported by QEMU, aborting");
+ return -ENOTSUP;
+ }
+
+ new_reftable = g_try_realloc(*reftable, new_reftable_size *
+ sizeof(uint64_t));
+ if (!new_reftable) {
+ error_setg(errp, "Failed to increase reftable buffer size");
+ return -ENOMEM;
+ }
+
+ memset(new_reftable + *reftable_size, 0,
+ (new_reftable_size - *reftable_size) * sizeof(uint64_t));
+
+ *reftable = new_reftable;
+ *reftable_size = new_reftable_size;
+ }
+
+ if (!refblock_empty && !(*reftable)[reftable_index]) {
+ offset = qcow2_alloc_clusters(bs, s->cluster_size);
+ if (offset < 0) {
+ error_setg_errno(errp, -offset, "Failed to allocate refblock");
+ return offset;
+ }
+ (*reftable)[reftable_index] = offset;
+ *allocated = true;
+ }
+
+ return 0;
+}
+
+/**
+ * This "operation" for walk_over_reftable() writes the refblock to disk at the
+ * offset specified by the new reftable's entry. It does not modify the new
+ * reftable or change any refcounts.
+ */
+static int flush_refblock(BlockDriverState *bs, uint64_t **reftable,
+ uint64_t reftable_index, uint64_t *reftable_size,
+ void *refblock, bool refblock_empty, bool *allocated,
+ Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ int64_t offset;
+ int ret;
+
+ if (reftable_index < *reftable_size && (*reftable)[reftable_index]) {
+ offset = (*reftable)[reftable_index];
+
+ ret = qcow2_pre_write_overlap_check(bs, 0, offset, s->cluster_size);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Overlap check failed");
+ return ret;
+ }
+
+ ret = bdrv_pwrite(bs->file, offset, refblock, s->cluster_size);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to write refblock");
+ return ret;
+ }
+ } else {
+ assert(refblock_empty);
+ }
+
+ return 0;
+}
+
+/**
+ * This function walks over the existing reftable and every referenced refblock;
+ * if @new_set_refcount is non-NULL, it is called for every refcount entry to
+ * create an equal new entry in the passed @new_refblock. Once that
+ * @new_refblock is completely filled, @operation will be called.
+ *
+ * @status_cb and @cb_opaque are used for the amend operation's status callback.
+ * @index is the index of the walk_over_reftable() calls and @total is the total
+ * number of walk_over_reftable() calls per amend operation. Both are used for
+ * calculating the parameters for the status callback.
+ *
+ * @allocated is set to true if a new cluster has been allocated.
+ */
+static int walk_over_reftable(BlockDriverState *bs, uint64_t **new_reftable,
+ uint64_t *new_reftable_index,
+ uint64_t *new_reftable_size,
+ void *new_refblock, int new_refblock_size,
+ int new_refcount_bits,
+ RefblockFinishOp *operation, bool *allocated,
+ Qcow2SetRefcountFunc *new_set_refcount,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque, int index, int total,
+ Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ uint64_t reftable_index;
+ bool new_refblock_empty = true;
+ int refblock_index;
+ int new_refblock_index = 0;
+ int ret;
+
+ for (reftable_index = 0; reftable_index < s->refcount_table_size;
+ reftable_index++)
+ {
+ uint64_t refblock_offset = s->refcount_table[reftable_index]
+ & REFT_OFFSET_MASK;
+
+ status_cb(bs, (uint64_t)index * s->refcount_table_size + reftable_index,
+ (uint64_t)total * s->refcount_table_size, cb_opaque);
+
+ if (refblock_offset) {
+ void *refblock;
+
+ if (offset_into_cluster(s, refblock_offset)) {
+ qcow2_signal_corruption(bs, true, -1, -1, "Refblock offset %#"
+ PRIx64 " unaligned (reftable index: %#"
+ PRIx64 ")", refblock_offset,
+ reftable_index);
+ error_setg(errp,
+ "Image is corrupt (unaligned refblock offset)");
+ return -EIO;
+ }
+
+ ret = qcow2_cache_get(bs, s->refcount_block_cache, refblock_offset,
+ &refblock);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to retrieve refblock");
+ return ret;
+ }
+
+ for (refblock_index = 0; refblock_index < s->refcount_block_size;
+ refblock_index++)
+ {
+ uint64_t refcount;
+
+ if (new_refblock_index >= new_refblock_size) {
+ /* new_refblock is now complete */
+ ret = operation(bs, new_reftable, *new_reftable_index,
+ new_reftable_size, new_refblock,
+ new_refblock_empty, allocated, errp);
+ if (ret < 0) {
+ qcow2_cache_put(bs, s->refcount_block_cache, &refblock);
+ return ret;
+ }
+
+ (*new_reftable_index)++;
+ new_refblock_index = 0;
+ new_refblock_empty = true;
+ }
+
+ refcount = s->get_refcount(refblock, refblock_index);
+ if (new_refcount_bits < 64 && refcount >> new_refcount_bits) {
+ uint64_t offset;
+
+ qcow2_cache_put(bs, s->refcount_block_cache, &refblock);
+
+ offset = ((reftable_index << s->refcount_block_bits)
+ + refblock_index) << s->cluster_bits;
+
+ error_setg(errp, "Cannot decrease refcount entry width to "
+ "%i bits: Cluster at offset %#" PRIx64 " has a "
+ "refcount of %" PRIu64, new_refcount_bits,
+ offset, refcount);
+ return -EINVAL;
+ }
+
+ if (new_set_refcount) {
+ new_set_refcount(new_refblock, new_refblock_index++,
+ refcount);
+ } else {
+ new_refblock_index++;
+ }
+ new_refblock_empty = new_refblock_empty && refcount == 0;
+ }
+
+ qcow2_cache_put(bs, s->refcount_block_cache, &refblock);
+ } else {
+ /* No refblock means every refcount is 0 */
+ for (refblock_index = 0; refblock_index < s->refcount_block_size;
+ refblock_index++)
+ {
+ if (new_refblock_index >= new_refblock_size) {
+ /* new_refblock is now complete */
+ ret = operation(bs, new_reftable, *new_reftable_index,
+ new_reftable_size, new_refblock,
+ new_refblock_empty, allocated, errp);
+ if (ret < 0) {
+ return ret;
+ }
+
+ (*new_reftable_index)++;
+ new_refblock_index = 0;
+ new_refblock_empty = true;
+ }
+
+ if (new_set_refcount) {
+ new_set_refcount(new_refblock, new_refblock_index++, 0);
+ } else {
+ new_refblock_index++;
+ }
+ }
+ }
+ }
+
+ if (new_refblock_index > 0) {
+ /* Complete the potentially existing partially filled final refblock */
+ if (new_set_refcount) {
+ for (; new_refblock_index < new_refblock_size;
+ new_refblock_index++)
+ {
+ new_set_refcount(new_refblock, new_refblock_index, 0);
+ }
+ }
+
+ ret = operation(bs, new_reftable, *new_reftable_index,
+ new_reftable_size, new_refblock, new_refblock_empty,
+ allocated, errp);
+ if (ret < 0) {
+ return ret;
+ }
+
+ (*new_reftable_index)++;
+ }
+
+ status_cb(bs, (uint64_t)(index + 1) * s->refcount_table_size,
+ (uint64_t)total * s->refcount_table_size, cb_opaque);
+
+ return 0;
+}
+
+int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque, Error **errp)
+{
+ BDRVQcowState *s = bs->opaque;
+ Qcow2GetRefcountFunc *new_get_refcount;
+ Qcow2SetRefcountFunc *new_set_refcount;
+ void *new_refblock = qemu_blockalign(bs->file, s->cluster_size);
+ uint64_t *new_reftable = NULL, new_reftable_size = 0;
+ uint64_t *old_reftable, old_reftable_size, old_reftable_offset;
+ uint64_t new_reftable_index = 0;
+ uint64_t i;
+ int64_t new_reftable_offset = 0, allocated_reftable_size = 0;
+ int new_refblock_size, new_refcount_bits = 1 << refcount_order;
+ int old_refcount_order;
+ int walk_index = 0;
+ int ret;
+ bool new_allocation;
+
+ assert(s->qcow_version >= 3);
+ assert(refcount_order >= 0 && refcount_order <= 6);
+
+ /* see qcow2_open() */
+ new_refblock_size = 1 << (s->cluster_bits - (refcount_order - 3));
+
+ new_get_refcount = get_refcount_funcs[refcount_order];
+ new_set_refcount = set_refcount_funcs[refcount_order];
+
+
+ do {
+ int total_walks;
+
+ new_allocation = false;
+
+ /* At least we have to do this walk and the one which writes the
+ * refblocks; also, at least we have to do this loop here at least
+ * twice (normally), first to do the allocations, and second to
+ * determine that everything is correctly allocated, this then makes
+ * three walks in total */
+ total_walks = MAX(walk_index + 2, 3);
+
+ /* First, allocate the structures so they are present in the refcount
+ * structures */
+ ret = walk_over_reftable(bs, &new_reftable, &new_reftable_index,
+ &new_reftable_size, NULL, new_refblock_size,
+ new_refcount_bits, &alloc_refblock,
+ &new_allocation, NULL, status_cb, cb_opaque,
+ walk_index++, total_walks, errp);
+ if (ret < 0) {
+ goto done;
+ }
+
+ new_reftable_index = 0;
+
+ if (new_allocation) {
+ if (new_reftable_offset) {
+ qcow2_free_clusters(bs, new_reftable_offset,
+ allocated_reftable_size * sizeof(uint64_t),
+ QCOW2_DISCARD_NEVER);
+ }
+
+ new_reftable_offset = qcow2_alloc_clusters(bs, new_reftable_size *
+ sizeof(uint64_t));
+ if (new_reftable_offset < 0) {
+ error_setg_errno(errp, -new_reftable_offset,
+ "Failed to allocate the new reftable");
+ ret = new_reftable_offset;
+ goto done;
+ }
+ allocated_reftable_size = new_reftable_size;
+ }
+ } while (new_allocation);
+
+ /* Second, write the new refblocks */
+ ret = walk_over_reftable(bs, &new_reftable, &new_reftable_index,
+ &new_reftable_size, new_refblock,
+ new_refblock_size, new_refcount_bits,
+ &flush_refblock, &new_allocation, new_set_refcount,
+ status_cb, cb_opaque, walk_index, walk_index + 1,
+ errp);
+ if (ret < 0) {
+ goto done;
+ }
+ assert(!new_allocation);
+
+
+ /* Write the new reftable */
+ ret = qcow2_pre_write_overlap_check(bs, 0, new_reftable_offset,
+ new_reftable_size * sizeof(uint64_t));
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Overlap check failed");
+ goto done;
+ }
+
+ for (i = 0; i < new_reftable_size; i++) {
+ cpu_to_be64s(&new_reftable[i]);
+ }
+
+ ret = bdrv_pwrite(bs->file, new_reftable_offset, new_reftable,
+ new_reftable_size * sizeof(uint64_t));
+
+ for (i = 0; i < new_reftable_size; i++) {
+ be64_to_cpus(&new_reftable[i]);
+ }
+
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to write the new reftable");
+ goto done;
+ }
+
+
+ /* Empty the refcount cache */
+ ret = qcow2_cache_flush(bs, s->refcount_block_cache);
+ if (ret < 0) {
+ error_setg_errno(errp, -ret, "Failed to flush the refblock cache");
+ goto done;
+ }
+
+ /* Update the image header to point to the new reftable; this only updates
+ * the fields which are relevant to qcow2_update_header(); other fields
+ * such as s->refcount_table or s->refcount_bits stay stale for now
+ * (because we have to restore everything if qcow2_update_header() fails) */
+ old_refcount_order = s->refcount_order;
+ old_reftable_size = s->refcount_table_size;
+ old_reftable_offset = s->refcount_table_offset;
+
+ s->refcount_order = refcount_order;
+ s->refcount_table_size = new_reftable_size;
+ s->refcount_table_offset = new_reftable_offset;
+
+ ret = qcow2_update_header(bs);
+ if (ret < 0) {
+ s->refcount_order = old_refcount_order;
+ s->refcount_table_size = old_reftable_size;
+ s->refcount_table_offset = old_reftable_offset;
+ error_setg_errno(errp, -ret, "Failed to update the qcow2 header");
+ goto done;
+ }
+
+ /* Now update the rest of the in-memory information */
+ old_reftable = s->refcount_table;
+ s->refcount_table = new_reftable;
+
+ s->refcount_bits = 1 << refcount_order;
+ s->refcount_max = UINT64_C(1) << (s->refcount_bits - 1);
+ s->refcount_max += s->refcount_max - 1;
+
+ s->refcount_block_bits = s->cluster_bits - (refcount_order - 3);
+ s->refcount_block_size = 1 << s->refcount_block_bits;
+
+ s->get_refcount = new_get_refcount;
+ s->set_refcount = new_set_refcount;
+
+ /* For cleaning up all old refblocks and the old reftable below the "done"
+ * label */
+ new_reftable = old_reftable;
+ new_reftable_size = old_reftable_size;
+ new_reftable_offset = old_reftable_offset;
+
+done:
+ if (new_reftable) {
+ /* On success, new_reftable actually points to the old reftable (and
+ * new_reftable_size is the old reftable's size); but that is just
+ * fine */
+ for (i = 0; i < new_reftable_size; i++) {
+ uint64_t offset = new_reftable[i] & REFT_OFFSET_MASK;
+ if (offset) {
+ qcow2_free_clusters(bs, offset, s->cluster_size,
+ QCOW2_DISCARD_OTHER);
+ }
+ }
+ g_free(new_reftable);
+
+ if (new_reftable_offset > 0) {
+ qcow2_free_clusters(bs, new_reftable_offset,
+ new_reftable_size * sizeof(uint64_t),
+ QCOW2_DISCARD_OTHER);
+ }
+ }
+
+ qemu_vfree(new_refblock);
+ return ret;
+}
diff --git a/block/qcow2.h b/block/qcow2.h
index 8d63af3..2273dd3 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -527,6 +527,10 @@ int qcow2_check_metadata_overlap(BlockDriverState *bs, int ign, int64_t offset,
int qcow2_pre_write_overlap_check(BlockDriverState *bs, int ign, int64_t offset,
int64_t size);
+int qcow2_change_refcount_order(BlockDriverState *bs, int refcount_order,
+ BlockDriverAmendStatusCB *status_cb,
+ void *cb_opaque, Error **errp);
+
/* qcow2-cluster.c functions */
int qcow2_grow_l1_table(BlockDriverState *bs, uint64_t min_size,
bool exact_size);
--
2.4.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
end of thread, other threads:[~2015-06-03 22:10 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-03-18 20:56 [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 01/10] progress: Allow regressing progress Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 02/10] block: Add opaque value to the amend CB Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 03/10] qcow2: Use error_report() in qcow2_amend_options() Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 04/10] qcow2: Use abort() instead of assert(false) Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 05/10] qcow2: Split upgrade/downgrade paths for amend Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 06/10] qcow2: Use intermediate helper CB " Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for refcount order amendment Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 08/10] qcow2: Invoke refcount order amendment function Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 09/10] qcow2: Point to amend function in check Max Reitz
2015-03-18 20:56 ` [Qemu-devel] [PATCH v8 10/10] iotests: Extend test 112 for qemu-img amend Max Reitz
2015-05-11 12:49 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2015-04-24 15:00 ` [Qemu-devel] [PATCH v8 00/10] qcow2: Allow refcount_bits amendment Max Reitz
2015-05-11 12:49 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2015-06-03 20:13 [Qemu-devel] [PATCH v8 00/10] qcow2: Support refcount order amendment Max Reitz
2015-06-03 20:13 ` [Qemu-devel] [PATCH v8 07/10] qcow2: Add function for " Max Reitz
2015-06-03 22:10 ` Eric Blake
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.