* [PATCH 0/6] block: add block-dirty-bitmap-populate job
@ 2020-02-25 0:56 John Snow
2020-02-25 0:56 ` [PATCH 1/6] block: add bitmap-populate job John Snow
` (5 more replies)
0 siblings, 6 replies; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
Hi,
This is a new (very small) block job that writes a pattern into a
bitmap. The only pattern implemented is the top allocation information.
This can be used to "recover" an incremental bitmap chain if an external
snapshot was taken without creating a new bitmap first: any writes made
to the image will be reflected by the allocation status and can be
written back into a bitmap.
This is useful for e.g. libvirt managing backup chains if a user creates
an external snapshot outside of libvirt.
Patches 1-2: The new job.
Patch 3: iotest prerequisite
Patch 4-5: completely optional cleanup.
Patch 6: Test.
John Snow (6):
block: add bitmap-populate job
qmp: expose block-dirty-bitmap-populate
iotests: move bitmap helpers into their own file
iotests: add hmp helper with logging
qmp.py: change event_wait to use a dict
iotests: add 287 for block-dirty-bitmap-populate
qapi/block-core.json | 66 +
qapi/job.json | 2 +-
qapi/transaction.json | 2 +
include/block/block_int.h | 21 +
block/bitmap-alloc.c | 207 ++
blockdev.c | 78 +
blockjob.c | 3 +-
block/Makefile.objs | 1 +
python/qemu/machine.py | 10 +-
tests/qemu-iotests/040 | 12 +-
tests/qemu-iotests/257 | 110 +-
tests/qemu-iotests/260 | 5 +-
tests/qemu-iotests/287 | 233 ++
tests/qemu-iotests/287.out | 4544 +++++++++++++++++++++++++++++++++
tests/qemu-iotests/bitmaps.py | 131 +
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 34 +-
17 files changed, 5321 insertions(+), 139 deletions(-)
create mode 100644 block/bitmap-alloc.c
create mode 100755 tests/qemu-iotests/287
create mode 100644 tests/qemu-iotests/287.out
create mode 100644 tests/qemu-iotests/bitmaps.py
--
2.21.1
^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 1/6] block: add bitmap-populate job
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
@ 2020-02-25 0:56 ` John Snow
2020-02-25 16:04 ` Vladimir Sementsov-Ogievskiy
2020-02-25 0:56 ` [PATCH 2/6] qmp: expose block-dirty-bitmap-populate John Snow
` (4 subsequent siblings)
5 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
This job copies the allocation map into a bitmap. It's a job because
there's no guarantee that allocation interrogation will be quick (or
won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
It was designed with different possible population patterns in mind,
but only top layer allocation was implemented for now.
Signed-off-by: John Snow <jsnow@redhat.com>
---
qapi/block-core.json | 48 +++++++++
qapi/job.json | 2 +-
include/block/block_int.h | 21 ++++
block/bitmap-alloc.c | 207 ++++++++++++++++++++++++++++++++++++++
blockjob.c | 3 +-
block/Makefile.objs | 1 +
6 files changed, 280 insertions(+), 2 deletions(-)
create mode 100644 block/bitmap-alloc.c
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 85e27bb61f..df1797681a 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2245,6 +2245,54 @@
{ 'command': 'block-dirty-bitmap-merge',
'data': 'BlockDirtyBitmapMerge' }
+##
+# @BitmapPattern:
+#
+# An enumeration of possible patterns that can be written into a bitmap.
+#
+# @allocation-top: The allocation status of the top layer
+# of the attached storage node.
+#
+# Since: 5.0
+##
+{ 'enum': 'BitmapPattern',
+ 'data': ['allocation-top'] }
+
+##
+# @BlockDirtyBitmapPopulate:
+#
+# @job-id: identifier for the newly-created block job.
+#
+# @pattern: What pattern should be written into the bitmap?
+#
+# @on-error: the action to take if an error is encountered on a bitmap's
+# attached node, default 'report'.
+# 'stop' and 'enospc' can only be used if the block device supports
+# io-status (see BlockInfo).
+#
+# @auto-finalize: When false, this job will wait in a PENDING state after it has
+# finished its work, waiting for @block-job-finalize before
+# making any block graph changes.
+# When true, this job will automatically
+# perform its abort or commit actions.
+# Defaults to true.
+#
+# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it
+# has completely ceased all work, and awaits @block-job-dismiss.
+# When true, this job will automatically disappear from the query
+# list without user intervention.
+# Defaults to true.
+#
+# Since: 5.0
+##
+{ 'struct': 'BlockDirtyBitmapPopulate',
+ 'base': 'BlockDirtyBitmap',
+ 'data': { 'job-id': 'str',
+ 'pattern': 'BitmapPattern',
+ '*on-error': 'BlockdevOnError',
+ '*auto-finalize': 'bool',
+ '*auto-dismiss': 'bool' } }
+
##
# @BlockDirtyBitmapSha256:
#
diff --git a/qapi/job.json b/qapi/job.json
index 5e658281f5..5f496d4630 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -22,7 +22,7 @@
# Since: 1.7
##
{ 'enum': 'JobType',
- 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+ 'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'bitmap-populate'] }
##
# @JobStatus:
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 6f9fd5e20e..a5884b597e 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1215,6 +1215,27 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
BlockCompletionFunc *cb, void *opaque,
JobTxn *txn, Error **errp);
+/*
+ * bitpop_job_create: Create a new bitmap population job.
+ *
+ * @job_id: The id of the newly-created job.
+ * @bs: Block device associated with the @target_bitmap.
+ * @target_bitmap: The bitmap to populate.
+ * @on_error: What to do if an error on @bs is encountered.
+ * @creation_flags: Flags that control the behavior of the Job lifetime.
+ * See @BlockJobCreateFlags
+ * @cb: Completion function for the job.
+ * @opaque: Opaque pointer value passed to @cb.
+ * @txn: Transaction that this job is part of (may be NULL).
+ */
+BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
+ BdrvDirtyBitmap *target_bitmap,
+ BitmapPattern pattern,
+ BlockdevOnError on_error,
+ int creation_flags,
+ BlockCompletionFunc *cb, void *opaque,
+ JobTxn *txn, Error **errp);
+
void hmp_drive_add_node(Monitor *mon, const char *optstr);
BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
new file mode 100644
index 0000000000..47d542dc12
--- /dev/null
+++ b/block/bitmap-alloc.c
@@ -0,0 +1,207 @@
+/*
+ * Async Dirty Bitmap Populator
+ *
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ * John Snow <jsnow@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "trace.h"
+#include "block/block.h"
+#include "block/block_int.h"
+#include "block/blockjob_int.h"
+#include "block/block_backup.h"
+#include "block/block-copy.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/ratelimit.h"
+#include "qemu/cutils.h"
+#include "sysemu/block-backend.h"
+#include "qemu/bitmap.h"
+#include "qemu/error-report.h"
+
+typedef struct BitpopBlockJob {
+ BlockJob common;
+ BlockDriverState *bs;
+ BdrvDirtyBitmap *target_bitmap;
+ BdrvDirtyBitmap *new_bitmap;
+ BlockdevOnError on_error;
+ uint64_t len;
+} BitpopBlockJob;
+
+static const BlockJobDriver bitpop_job_driver;
+
+static void bitpop_commit(Job *job)
+{
+ BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
+
+ bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
+ NULL, true);
+}
+
+/* no abort needed; just clean without committing. */
+
+static void bitpop_clean(Job *job)
+{
+ BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
+
+ bdrv_release_dirty_bitmap(s->new_bitmap);
+ bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
+}
+
+static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int error)
+{
+ return block_job_error_action(&job->common, job->on_error, true, error);
+}
+
+static bool coroutine_fn yield_and_check(Job *job)
+{
+ if (job_is_cancelled(job)) {
+ return true;
+ }
+
+ job_sleep_ns(job, 0);
+
+ if (job_is_cancelled(job)) {
+ return true;
+ }
+
+ return false;
+}
+
+static int coroutine_fn bitpop_run(Job *job, Error **errp)
+{
+ BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
+ int ret = 0;
+ int64_t offset;
+ int64_t count;
+ int64_t bytes;
+
+ for (offset = 0; offset < s->len; ) {
+ if (yield_and_check(job)) {
+ ret = -ECANCELED;
+ break;
+ }
+
+ bytes = s->len - offset;
+ ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
+ if (ret < 0) {
+ if (bitpop_error_action(s, -ret) == BLOCK_ERROR_ACTION_REPORT) {
+ break;
+ }
+ continue;
+ }
+
+ if (!count) {
+ ret = 0;
+ break;
+ }
+
+ if (ret) {
+ bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
+ ret = 0;
+ }
+
+ job_progress_update(job, count);
+ offset += count;
+ }
+
+ return ret;
+}
+
+static const BlockJobDriver bitpop_job_driver = {
+ .job_driver = {
+ .instance_size = sizeof(BitpopBlockJob),
+ .job_type = JOB_TYPE_BITMAP_POPULATE,
+ .free = block_job_free,
+ .user_resume = block_job_user_resume,
+ .run = bitpop_run,
+ .commit = bitpop_commit,
+ .clean = bitpop_clean,
+ }
+};
+
+
+BlockJob *bitpop_job_create(
+ const char *job_id,
+ BlockDriverState *bs,
+ BdrvDirtyBitmap *target_bitmap,
+ BitmapPattern pattern,
+ BlockdevOnError on_error,
+ int creation_flags,
+ BlockCompletionFunc *cb,
+ void *opaque,
+ JobTxn *txn,
+ Error **errp)
+{
+ int64_t len;
+ BitpopBlockJob *job = NULL;
+ int64_t cluster_size;
+ BdrvDirtyBitmap *new_bitmap = NULL;
+
+ assert(bs);
+ assert(target_bitmap);
+
+ if (!bdrv_is_inserted(bs)) {
+ error_setg(errp, "Device is not inserted: %s",
+ bdrv_get_device_name(bs));
+ return NULL;
+ }
+
+ if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
+ return NULL;
+ }
+
+ if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
+ return NULL;
+ }
+
+ if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
+ error_setg(errp, "Unrecognized bitmap pattern");
+ return NULL;
+ }
+
+ len = bdrv_getlength(bs);
+ if (len < 0) {
+ error_setg_errno(errp, -len, "unable to get length for '%s'",
+ bdrv_get_device_name(bs));
+ return NULL;
+ }
+
+ /* NB: new bitmap is anonymous and enabled */
+ cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
+ new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
+ if (!new_bitmap) {
+ return NULL;
+ }
+
+ /* Take ownership; we reserve the right to write into this on-commit. */
+ bdrv_dirty_bitmap_set_busy(target_bitmap, true);
+
+ job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
+ BLK_PERM_CONSISTENT_READ,
+ BLK_PERM_ALL & ~BLK_PERM_RESIZE,
+ 0, creation_flags,
+ cb, opaque, errp);
+ if (!job) {
+ bdrv_dirty_bitmap_set_busy(target_bitmap, false);
+ bdrv_release_dirty_bitmap(new_bitmap);
+ return NULL;
+ }
+
+ job->bs = bs;
+ job->on_error = on_error;
+ job->target_bitmap = target_bitmap;
+ job->new_bitmap = new_bitmap;
+ job->len = len;
+ job_progress_set_remaining(&job->common.job, job->len);
+
+ return &job->common;
+}
diff --git a/blockjob.c b/blockjob.c
index 5d63b1e89d..7e450372bd 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
return job_type(job) == JOB_TYPE_BACKUP ||
job_type(job) == JOB_TYPE_COMMIT ||
job_type(job) == JOB_TYPE_MIRROR ||
- job_type(job) == JOB_TYPE_STREAM;
+ job_type(job) == JOB_TYPE_STREAM ||
+ job_type(job) == JOB_TYPE_BITMAP_POPULATE;
}
BlockJob *block_job_next(BlockJob *bjob)
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 3bcb35c81d..f3cfc89d90 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
block-obj-y += accounting.o dirty-bitmap.o
block-obj-y += write-threshold.o
block-obj-y += backup.o
+block-obj-y += bitmap-alloc.o
block-obj-$(CONFIG_REPLICATION) += replication.o
block-obj-y += throttle.o copy-on-read.o
block-obj-y += block-copy.o
--
2.21.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 2/6] qmp: expose block-dirty-bitmap-populate
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
2020-02-25 0:56 ` [PATCH 1/6] block: add bitmap-populate job John Snow
@ 2020-02-25 0:56 ` John Snow
2020-02-27 10:44 ` Vladimir Sementsov-Ogievskiy
2020-02-25 0:56 ` [PATCH 3/6] iotests: move bitmap helpers into their own file John Snow
` (3 subsequent siblings)
5 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
This is a new job-creating command.
Signed-off-by: John Snow <jsnow@redhat.com>
---
qapi/block-core.json | 18 ++++++++++
qapi/transaction.json | 2 ++
blockdev.c | 78 +++++++++++++++++++++++++++++++++++++++++++
3 files changed, 98 insertions(+)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index df1797681a..a8be1fb36b 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2293,6 +2293,24 @@
'*auto-finalize': 'bool',
'*auto-dismiss': 'bool' } }
+##
+# @block-dirty-bitmap-populate:
+#
+# Creates a new job that writes a pattern into a dirty bitmap.
+#
+# Since: 5.0
+#
+# Example:
+#
+# -> { "execute": "block-dirty-bitmap-populate",
+# "arguments": { "node": "drive0", "target": "bitmap0",
+# "job-id": "job0", "pattern": "allocate-top" } }
+# <- { "return": {} }
+#
+##
+{ 'command': 'block-dirty-bitmap-populate', 'boxed': true,
+ 'data': 'BlockDirtyBitmapPopulate' }
+
##
# @BlockDirtyBitmapSha256:
#
diff --git a/qapi/transaction.json b/qapi/transaction.json
index 04301f1be7..28521d5c7f 100644
--- a/qapi/transaction.json
+++ b/qapi/transaction.json
@@ -50,6 +50,7 @@
# - @block-dirty-bitmap-enable: since 4.0
# - @block-dirty-bitmap-disable: since 4.0
# - @block-dirty-bitmap-merge: since 4.0
+# - @block-dirty-bitmap-populate: since 5.0
# - @blockdev-backup: since 2.3
# - @blockdev-snapshot: since 2.5
# - @blockdev-snapshot-internal-sync: since 1.7
@@ -67,6 +68,7 @@
'block-dirty-bitmap-enable': 'BlockDirtyBitmap',
'block-dirty-bitmap-disable': 'BlockDirtyBitmap',
'block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge',
+ 'block-dirty-bitmap-populate': 'BlockDirtyBitmapPopulate',
'blockdev-backup': 'BlockdevBackup',
'blockdev-snapshot': 'BlockdevSnapshot',
'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal',
diff --git a/blockdev.c b/blockdev.c
index 011dcfec27..33c0e35399 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2314,6 +2314,67 @@ static void block_dirty_bitmap_remove_commit(BlkActionState *common)
bdrv_release_dirty_bitmap(state->bitmap);
}
+static void block_dirty_bitmap_populate_prepare(BlkActionState *common, Error **errp)
+{
+ BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common);
+ BlockDirtyBitmapPopulate *bitpop;
+ BlockDriverState *bs;
+ AioContext *aio_context;
+ BdrvDirtyBitmap *bmap = NULL;
+ int job_flags = JOB_DEFAULT;
+
+ assert(common->action->type == TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE);
+ bitpop = common->action->u.block_dirty_bitmap_populate.data;
+
+ bs = bdrv_lookup_bs(bitpop->node, bitpop->node, errp);
+ if (!bs) {
+ return;
+ }
+
+ aio_context = bdrv_get_aio_context(bs);
+ aio_context_acquire(aio_context);
+ state->bs = bs;
+
+ bmap = bdrv_find_dirty_bitmap(bs, bitpop->name);
+ if (!bmap) {
+ error_setg(errp, "Bitmap '%s' could not be found", bitpop->name);
+ return;
+ }
+
+ /* Paired with .clean() */
+ bdrv_drained_begin(state->bs);
+
+ if (!bitpop->has_on_error) {
+ bitpop->on_error = BLOCKDEV_ON_ERROR_REPORT;
+ }
+ if (!bitpop->has_auto_finalize) {
+ bitpop->auto_finalize = true;
+ }
+ if (!bitpop->has_auto_dismiss) {
+ bitpop->auto_dismiss = true;
+ }
+
+ if (!bitpop->auto_finalize) {
+ job_flags |= JOB_MANUAL_FINALIZE;
+ }
+ if (!bitpop->auto_dismiss) {
+ job_flags |= JOB_MANUAL_DISMISS;
+ }
+
+ state->job = bitpop_job_create(
+ bitpop->job_id,
+ bs,
+ bmap,
+ bitpop->pattern,
+ bitpop->on_error,
+ job_flags,
+ NULL, NULL,
+ common->block_job_txn,
+ errp);
+
+ aio_context_release(aio_context);
+}
+
static void abort_prepare(BlkActionState *common, Error **errp)
{
error_setg(errp, "Transaction aborted using Abort action");
@@ -2397,6 +2458,13 @@ static const BlkActionOps actions[] = {
.commit = block_dirty_bitmap_remove_commit,
.abort = block_dirty_bitmap_remove_abort,
},
+ [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE] = {
+ .instance_size = sizeof(BlockdevBackupState),
+ .prepare = block_dirty_bitmap_populate_prepare,
+ .commit = blockdev_backup_commit,
+ .abort = blockdev_backup_abort,
+ .clean = blockdev_backup_clean,
+ },
/* Where are transactions for MIRROR, COMMIT and STREAM?
* Although these blockjobs use transaction callbacks like the backup job,
* these jobs do not necessarily adhere to transaction semantics.
@@ -3225,6 +3293,16 @@ void qmp_block_dirty_bitmap_merge(const char *node, const char *target,
do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp);
}
+void qmp_block_dirty_bitmap_populate(BlockDirtyBitmapPopulate *bitpop,
+ Error **errp)
+{
+ TransactionAction action = {
+ .type = TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE,
+ .u.block_dirty_bitmap_populate.data = bitpop,
+ };
+ blockdev_do_action(&action, errp);
+}
+
BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,
const char *name,
Error **errp)
--
2.21.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 3/6] iotests: move bitmap helpers into their own file
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
2020-02-25 0:56 ` [PATCH 1/6] block: add bitmap-populate job John Snow
2020-02-25 0:56 ` [PATCH 2/6] qmp: expose block-dirty-bitmap-populate John Snow
@ 2020-02-25 0:56 ` John Snow
2020-02-27 10:54 ` Vladimir Sementsov-Ogievskiy
2020-02-25 0:56 ` [PATCH 4/6] iotests: add hmp helper with logging John Snow
` (2 subsequent siblings)
5 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/257 | 110 +---------------------------
tests/qemu-iotests/bitmaps.py | 131 ++++++++++++++++++++++++++++++++++
2 files changed, 132 insertions(+), 109 deletions(-)
create mode 100644 tests/qemu-iotests/bitmaps.py
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 004a433b8b..2a81f9e30c 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -24,120 +24,12 @@ import os
import iotests
from iotests import log, qemu_img
+from bitmaps import EmulatedBitmap, GROUPS
SIZE = 64 * 1024 * 1024
GRANULARITY = 64 * 1024
-class Pattern:
- def __init__(self, byte, offset, size=GRANULARITY):
- self.byte = byte
- self.offset = offset
- self.size = size
-
- def bits(self, granularity):
- lower = self.offset // granularity
- upper = (self.offset + self.size - 1) // granularity
- return set(range(lower, upper + 1))
-
-
-class PatternGroup:
- """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
- def __init__(self, patterns):
- self.patterns = patterns
-
- def bits(self, granularity):
- """Calculate the unique bits dirtied by this pattern grouping"""
- res = set()
- for pattern in self.patterns:
- res |= pattern.bits(granularity)
- return res
-
-
-GROUPS = [
- PatternGroup([
- # Batch 0: 4 clusters
- Pattern('0x49', 0x0000000),
- Pattern('0x6c', 0x0100000), # 1M
- Pattern('0x6f', 0x2000000), # 32M
- Pattern('0x76', 0x3ff0000)]), # 64M - 64K
- PatternGroup([
- # Batch 1: 6 clusters (3 new)
- Pattern('0x65', 0x0000000), # Full overwrite
- Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
- Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
- Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
- PatternGroup([
- # Batch 2: 7 clusters (3 new)
- Pattern('0x74', 0x0010000), # Adjacent-right
- Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
- Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
- Pattern('0x67', 0x3fe0000,
- 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
- PatternGroup([
- # Batch 3: 8 clusters (5 new)
- # Carefully chosen such that nothing re-dirties the one cluster
- # that copies out successfully before failure in Group #1.
- Pattern('0xaa', 0x0010000,
- 3*GRANULARITY), # Overwrite and 2x Adjacent-right
- Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
- Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
- Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
-]
-
-
-class EmulatedBitmap:
- def __init__(self, granularity=GRANULARITY):
- self._bits = set()
- self.granularity = granularity
-
- def dirty_bits(self, bits):
- self._bits |= set(bits)
-
- def dirty_group(self, n):
- self.dirty_bits(GROUPS[n].bits(self.granularity))
-
- def clear(self):
- self._bits = set()
-
- def clear_bits(self, bits):
- self._bits -= set(bits)
-
- def clear_bit(self, bit):
- self.clear_bits({bit})
-
- def clear_group(self, n):
- self.clear_bits(GROUPS[n].bits(self.granularity))
-
- @property
- def first_bit(self):
- return sorted(self.bits)[0]
-
- @property
- def bits(self):
- return self._bits
-
- @property
- def count(self):
- return len(self.bits)
-
- def compare(self, qmp_bitmap):
- """
- Print a nice human-readable message checking that a bitmap as reported
- by the QMP interface has as many bits set as we expect it to.
- """
-
- name = qmp_bitmap.get('name', '(anonymous)')
- log("= Checking Bitmap {:s} =".format(name))
-
- want = self.count
- have = qmp_bitmap['count'] // qmp_bitmap['granularity']
-
- log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
- want, have, "OK!" if want == have else "ERROR!"))
- log('')
-
-
class Drive:
"""Represents, vaguely, a drive attached to a VM.
Includes format, graph, and device information."""
diff --git a/tests/qemu-iotests/bitmaps.py b/tests/qemu-iotests/bitmaps.py
new file mode 100644
index 0000000000..522fc25171
--- /dev/null
+++ b/tests/qemu-iotests/bitmaps.py
@@ -0,0 +1,131 @@
+# Bitmap-related helper utilities
+#
+# Copyright (c) 2020 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+# owner=jsnow@redhat.com
+
+from iotests import log
+
+GRANULARITY = 64 * 1024
+
+
+class Pattern:
+ def __init__(self, byte, offset, size=GRANULARITY):
+ self.byte = byte
+ self.offset = offset
+ self.size = size
+
+ def bits(self, granularity):
+ lower = self.offset // granularity
+ upper = (self.offset + self.size - 1) // granularity
+ return set(range(lower, upper + 1))
+
+
+class PatternGroup:
+ """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
+ def __init__(self, patterns):
+ self.patterns = patterns
+
+ def bits(self, granularity):
+ """Calculate the unique bits dirtied by this pattern grouping"""
+ res = set()
+ for pattern in self.patterns:
+ res |= pattern.bits(granularity)
+ return res
+
+
+GROUPS = [
+ PatternGroup([
+ # Batch 0: 4 clusters
+ Pattern('0x49', 0x0000000),
+ Pattern('0x6c', 0x0100000), # 1M
+ Pattern('0x6f', 0x2000000), # 32M
+ Pattern('0x76', 0x3ff0000)]), # 64M - 64K
+ PatternGroup([
+ # Batch 1: 6 clusters (3 new)
+ Pattern('0x65', 0x0000000), # Full overwrite
+ Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
+ Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
+ Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
+ PatternGroup([
+ # Batch 2: 7 clusters (3 new)
+ Pattern('0x74', 0x0010000), # Adjacent-right
+ Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
+ Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
+ Pattern('0x67', 0x3fe0000,
+ 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+ PatternGroup([
+ # Batch 3: 8 clusters (5 new)
+ # Carefully chosen such that nothing re-dirties the one cluster
+ # that copies out successfully before failure in Group #1.
+ Pattern('0xaa', 0x0010000,
+ 3*GRANULARITY), # Overwrite and 2x Adjacent-right
+ Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
+ Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
+ Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
+]
+
+
+class EmulatedBitmap:
+ def __init__(self, granularity=GRANULARITY):
+ self._bits = set()
+ self.granularity = granularity
+
+ def dirty_bits(self, bits):
+ self._bits |= set(bits)
+
+ def dirty_group(self, n):
+ self.dirty_bits(GROUPS[n].bits(self.granularity))
+
+ def clear(self):
+ self._bits = set()
+
+ def clear_bits(self, bits):
+ self._bits -= set(bits)
+
+ def clear_bit(self, bit):
+ self.clear_bits({bit})
+
+ def clear_group(self, n):
+ self.clear_bits(GROUPS[n].bits(self.granularity))
+
+ @property
+ def first_bit(self):
+ return sorted(self.bits)[0]
+
+ @property
+ def bits(self):
+ return self._bits
+
+ @property
+ def count(self):
+ return len(self.bits)
+
+ def compare(self, qmp_bitmap):
+ """
+ Print a nice human-readable message checking that a bitmap as reported
+ by the QMP interface has as many bits set as we expect it to.
+ """
+
+ name = qmp_bitmap.get('name', '(anonymous)')
+ log("= Checking Bitmap {:s} =".format(name))
+
+ want = self.count
+ have = qmp_bitmap['count'] // qmp_bitmap['granularity']
+
+ log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
+ want, have, "OK!" if want == have else "ERROR!"))
+ log('')
--
2.21.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 4/6] iotests: add hmp helper with logging
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
` (2 preceding siblings ...)
2020-02-25 0:56 ` [PATCH 3/6] iotests: move bitmap helpers into their own file John Snow
@ 2020-02-25 0:56 ` John Snow
2020-02-27 10:57 ` Vladimir Sementsov-Ogievskiy
2020-02-25 0:56 ` [PATCH 5/6] qmp.py: change event_wait to use a dict John Snow
2020-02-25 0:56 ` [PATCH 6/6] iotests: add 287 for block-dirty-bitmap-populate John Snow
5 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
Just a mild cleanup while I was here.
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/iotests.py | 18 +++++++++++-------
1 file changed, 11 insertions(+), 7 deletions(-)
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 8815052eb5..5d2990a0e4 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -525,23 +525,27 @@ def add_incoming(self, addr):
self._args.append(addr)
return self
+ def hmp(self, command_line, log=False):
+ cmd = 'human-monitor-command'
+ kwargs = { 'command-line': command_line }
+ if log:
+ return self.qmp_log(cmd, **kwargs)
+ return self.qmp(cmd, **kwargs)
+
def pause_drive(self, drive, event=None):
'''Pause drive r/w operations'''
if not event:
self.pause_drive(drive, "read_aio")
self.pause_drive(drive, "write_aio")
return
- self.qmp('human-monitor-command',
- command_line='qemu-io %s "break %s bp_%s"' % (drive, event, drive))
+ self.hmp('qemu-io %s "break %s bp_%s"' % (drive, event, drive))
def resume_drive(self, drive):
- self.qmp('human-monitor-command',
- command_line='qemu-io %s "remove_break bp_%s"' % (drive, drive))
+ self.hmp('qemu-io %s "remove_break bp_%s"' % (drive, drive))
- def hmp_qemu_io(self, drive, cmd):
+ def hmp_qemu_io(self, drive, cmd, log=False):
'''Write to a given drive using an HMP command'''
- return self.qmp('human-monitor-command',
- command_line='qemu-io %s "%s"' % (drive, cmd))
+ return self.hmp('qemu-io %s "%s"' % (drive, cmd), log=log)
def flatten_qmp_object(self, obj, output=None, basestr=''):
if output is None:
--
2.21.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 5/6] qmp.py: change event_wait to use a dict
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
` (3 preceding siblings ...)
2020-02-25 0:56 ` [PATCH 4/6] iotests: add hmp helper with logging John Snow
@ 2020-02-25 0:56 ` John Snow
2020-02-27 11:25 ` Vladimir Sementsov-Ogievskiy
2020-02-25 0:56 ` [PATCH 6/6] iotests: add 287 for block-dirty-bitmap-populate John Snow
5 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
It's easier to work with than a list of tuples, because we can check the
keys for membership.
Signed-off-by: John Snow <jsnow@redhat.com>
---
python/qemu/machine.py | 10 +++++-----
tests/qemu-iotests/040 | 12 ++++++------
tests/qemu-iotests/260 | 5 +++--
tests/qemu-iotests/iotests.py | 16 ++++++++--------
4 files changed, 22 insertions(+), 21 deletions(-)
diff --git a/python/qemu/machine.py b/python/qemu/machine.py
index 183d8f3d38..748de5f322 100644
--- a/python/qemu/machine.py
+++ b/python/qemu/machine.py
@@ -476,21 +476,21 @@ def event_wait(self, name, timeout=60.0, match=None):
timeout: QEMUMonitorProtocol.pull_event timeout parameter.
match: Optional match criteria. See event_match for details.
"""
- return self.events_wait([(name, match)], timeout)
+ return self.events_wait({name: match}, timeout)
def events_wait(self, events, timeout=60.0):
"""
events_wait waits for and returns a named event from QMP with a timeout.
- events: a sequence of (name, match_criteria) tuples.
+ events: a mapping containing {name: match_criteria}.
The match criteria are optional and may be None.
See event_match for details.
timeout: QEMUMonitorProtocol.pull_event timeout parameter.
"""
def _match(event):
- for name, match in events:
- if event['event'] == name and self.event_match(event, match):
- return True
+ name = event['event']
+ if name in events:
+ return self.event_match(event, events[name])
return False
# Search cached events
diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
index 32c82b4ec6..90b59081ff 100755
--- a/tests/qemu-iotests/040
+++ b/tests/qemu-iotests/040
@@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase):
def run_job(self, expected_events, error_pauses_job=False):
match_device = {'data': {'device': 'job0'}}
- events = [
- ('BLOCK_JOB_COMPLETED', match_device),
- ('BLOCK_JOB_CANCELLED', match_device),
- ('BLOCK_JOB_ERROR', match_device),
- ('BLOCK_JOB_READY', match_device),
- ]
+ events = {
+ 'BLOCK_JOB_COMPLETED': match_device,
+ 'BLOCK_JOB_CANCELLED': match_device,
+ 'BLOCK_JOB_ERROR': match_device,
+ 'BLOCK_JOB_READY': match_device,
+ }
completed = False
log = []
diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
index 30c0de380d..b2fb045ddd 100755
--- a/tests/qemu-iotests/260
+++ b/tests/qemu-iotests/260
@@ -65,8 +65,9 @@ def test(persistent, restart):
vm.qmp_log('block-commit', device='drive0', top=top,
filters=[iotests.filter_qmp_testfiles])
- ev = vm.events_wait((('BLOCK_JOB_READY', None),
- ('BLOCK_JOB_COMPLETED', None)))
+ ev = vm.events_wait({
+ 'BLOCK_JOB_READY': None,
+ 'BLOCK_JOB_COMPLETED': None })
log(filter_qmp_event(ev))
if (ev['event'] == 'BLOCK_JOB_COMPLETED'):
vm.shutdown()
diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
index 5d2990a0e4..3390fab021 100644
--- a/tests/qemu-iotests/iotests.py
+++ b/tests/qemu-iotests/iotests.py
@@ -604,14 +604,14 @@ def run_job(self, job, auto_finalize=True, auto_dismiss=False,
"""
match_device = {'data': {'device': job}}
match_id = {'data': {'id': job}}
- events = [
- ('BLOCK_JOB_COMPLETED', match_device),
- ('BLOCK_JOB_CANCELLED', match_device),
- ('BLOCK_JOB_ERROR', match_device),
- ('BLOCK_JOB_READY', match_device),
- ('BLOCK_JOB_PENDING', match_id),
- ('JOB_STATUS_CHANGE', match_id)
- ]
+ events = {
+ 'BLOCK_JOB_COMPLETED': match_device,
+ 'BLOCK_JOB_CANCELLED': match_device,
+ 'BLOCK_JOB_ERROR': match_device,
+ 'BLOCK_JOB_READY': match_device,
+ 'BLOCK_JOB_PENDING': match_id,
+ 'JOB_STATUS_CHANGE': match_id,
+ }
error = None
while True:
ev = filter_qmp_event(self.events_wait(events, timeout=wait))
--
2.21.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 6/6] iotests: add 287 for block-dirty-bitmap-populate
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
` (4 preceding siblings ...)
2020-02-25 0:56 ` [PATCH 5/6] qmp.py: change event_wait to use a dict John Snow
@ 2020-02-25 0:56 ` John Snow
5 siblings, 0 replies; 20+ messages in thread
From: John Snow @ 2020-02-25 0:56 UTC (permalink / raw)
To: qemu-devel
Cc: Kevin Wolf, vsementsov, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, pkrempa, Cleber Rosa, John Snow
Give block-dirty-bitmap-populate a workout.
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/287 | 233 ++
tests/qemu-iotests/287.out | 4544 ++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 4778 insertions(+)
create mode 100755 tests/qemu-iotests/287
create mode 100644 tests/qemu-iotests/287.out
diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
new file mode 100755
index 0000000000..0ab58dc011
--- /dev/null
+++ b/tests/qemu-iotests/287
@@ -0,0 +1,233 @@
+#!/usr/bin/env python3
+#
+# Test block-dirty-bitmap-populate
+#
+# Copyright (c) 2020 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+# owner=jsnow@redhat.com
+
+from collections import namedtuple
+import itertools
+import math
+import os
+
+import iotests
+from iotests import log, qemu_img
+from bitmaps import EmulatedBitmap, GROUPS
+
+SIZE = 64 * 1024 * 1024
+GRANULARITY = 64 * 1024
+
+
+class Drive:
+ def __init__(self, path, vm):
+ self.path = path
+ self.vm = vm
+ self.fmt = None
+ self.size = None
+ self.node = None
+
+ def img_create(self, fmt, size):
+ self.fmt = fmt
+ self.size = size
+ iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+
+
+def block_dirty_bitmap_populate(vm, node, bitmap, job_id, pattern, **kwargs):
+ # Strip any arguments explicitly nulled by the caller:
+ kwargs = {key: val for key, val in kwargs.items() if val is not None}
+ result = vm.qmp_log('block-dirty-bitmap-populate',
+ node=node,
+ name=bitmap,
+ job_id=job_id,
+ pattern=pattern,
+ **kwargs)
+ return result
+
+
+def populate(drive, bitmap, job_id, pattern='allocation-top', **kwargs):
+ kwargs.setdefault('pattern', pattern)
+ kwargs.setdefault('auto-finalize', False)
+ kwargs.setdefault('auto-dismiss', False)
+ ret = block_dirty_bitmap_populate(drive.vm, drive.node,
+ bitmap, job_id, **kwargs)
+ return {
+ 'id': job_id,
+ 'auto-finalize': kwargs['auto-finalize'],
+ 'auto-dismiss': kwargs['auto-dismiss'],
+ 'return': ret,
+ }
+
+
+def perform_writes(drive, n, filter_node_name=None):
+ log("-- Write #{:d}:".format(n))
+ node_name = filter_node_name or drive.node
+ for pattern in GROUPS[n].patterns:
+ cmd = "write -P{:s} 0x{:07x} 0x{:x}".format(
+ pattern.byte,
+ pattern.offset,
+ pattern.size)
+ drive.vm.hmp_qemu_io(node_name, cmd, log=True)
+ log('')
+
+
+TestConfig = namedtuple('TestConfig', [
+ 'base_pattern',
+ 'disabled',
+ 'pre_writes',
+ 'mid_writes',
+ 'cancel',
+ 'post_writes',
+])
+
+
+def test_bitmap_populate(config):
+ """
+ Test bitmap populate.
+
+ :param base_pattern: Write a base pattern?
+ :param disabled: Disable the target bitmap?
+ :param pre_writes: Write a pattern after bitmap creation, but before job?
+ :param mid_writes: Write a pattern before job finalizes?
+ :param cancel: Cancel the job instead of finalizing it?
+ :param post_writes: Write a pattern after the job?
+ """
+ with iotests.FilePaths(['img']) as (img_path,), iotests.VM() as vm:
+ log("\n=== Bitmap Populate {:s} ===\n".format(str(config)))
+
+ log('-- Prepare image & VM:')
+ drive0 = Drive(img_path, vm=vm)
+ drive0.img_create(iotests.imgfmt, SIZE)
+ vm.add_device("{},id=scsi0".format(iotests.get_virtio_scsi_device()))
+ vm.launch()
+
+ file_config = {
+ 'driver': 'file',
+ 'filename': drive0.path
+ }
+
+ drive0.node = 'drive0'
+ vm.qmp_log('blockdev-add',
+ filters=[iotests.filter_qmp_testfiles],
+ node_name=drive0.node,
+ driver=drive0.fmt,
+ file=file_config)
+ log('')
+
+
+ # Step 0: Prepare & Base Allocation Pattern
+
+ if config.base_pattern:
+ perform_writes(drive0, 0)
+
+
+ # Step 1: Add test bitmap
+
+ log('-- Add Bitmap:')
+ vm.qmp_log('block-dirty-bitmap-add',
+ node=drive0.node,
+ name="target",
+ granularity=GRANULARITY,
+ disabled=config.disabled)
+ ebitmap = EmulatedBitmap()
+ log('')
+
+ # Step 2: Pre-Writes
+
+ if config.pre_writes:
+ perform_writes(drive0, 1)
+ if not config.disabled:
+ ebitmap.dirty_group(1)
+ bitmap = vm.get_bitmap(drive0.node, 'target')
+ ebitmap.compare(bitmap)
+
+
+ # Step 3: Launch job & Mid-Writes
+
+ log('-- Test block-dirty-bitmap-populate (bitpop0):')
+ def pre_finalize():
+ # Writes issued prior to job finalization:
+ if config.mid_writes:
+ perform_writes(drive0, 2)
+ if not config.disabled:
+ ebitmap.dirty_group(2)
+
+ job = populate(drive0, 'target', 'bitpop0')
+ assert job['return'] == {'return': {}}
+ vm.run_job(job['id'],
+ auto_dismiss=job['auto-dismiss'],
+ auto_finalize=job['auto-finalize'],
+ pre_finalize=pre_finalize,
+ cancel=config.cancel)
+ log('')
+
+
+ # Step 4: Post-job verification
+
+ if not config.cancel:
+ # Any writes made prior to the job finishing should now be visible.
+ if config.base_pattern:
+ ebitmap.dirty_group(0)
+ if config.pre_writes:
+ ebitmap.dirty_group(1)
+ if config.mid_writes:
+ ebitmap.dirty_group(2)
+
+ bitmap = vm.get_bitmap(drive0.node, 'target')
+ ebitmap.compare(bitmap)
+
+
+ # Step 5: Post-Writes
+
+ if config.post_writes:
+ perform_writes(drive0, 3)
+ if not config.disabled:
+ ebitmap.dirty_group(3)
+
+
+ # Step 6: Final Verification
+
+ log('-- Verification:')
+ bitmaps = vm.query_bitmaps()
+ log({'bitmaps': bitmaps}, indent=2)
+ log('')
+ bitmap = vm.get_bitmap(drive0.node, 'target', bitmaps=bitmaps)
+ ebitmap.compare(bitmap)
+
+
+ log('-- Cleanup:')
+ vm.qmp_log("block-dirty-bitmap-remove",
+ node=drive0.node, name="target")
+
+ bitmaps = vm.query_bitmaps()
+ if bitmaps:
+ log("ERROR: bitmaps unaccounted for:")
+ log(bitmaps)
+ else:
+ log('OK: All bitmaps removed')
+ vm.shutdown()
+ log('')
+
+
+def main():
+ for args in itertools.product((True, False), repeat=6):
+ cfg = TestConfig(*args)
+ test_bitmap_populate(cfg)
+
+
+if __name__ == '__main__':
+ iotests.script_main(main, supported_fmts=['qcow2'],
+ supported_protocols=['file'])
diff --git a/tests/qemu-iotests/287.out b/tests/qemu-iotests/287.out
new file mode 100644
index 0000000000..7c0afc7192
--- /dev/null
+++ b/tests/qemu-iotests/287.out
@@ -0,0 +1,4544 @@
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 262144,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 262144,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 917504,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 14 dirty sectors; have 14. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 8 dirty sectors; have 8. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 262144,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 917504,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 14 dirty sectors; have 14. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 917504,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 14 dirty sectors; have 14. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 8 dirty sectors; have 8. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 524288,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 8 dirty sectors; have 8. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+ "bitmaps": {
+ "drive0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "target",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 0317667695..5e80d7deab 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -293,3 +293,4 @@
283 auto quick
284 rw
286 rw quick
+287 rw
--
2.21.1
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 1/6] block: add bitmap-populate job
2020-02-25 0:56 ` [PATCH 1/6] block: add bitmap-populate job John Snow
@ 2020-02-25 16:04 ` Vladimir Sementsov-Ogievskiy
2020-02-25 20:41 ` John Snow
0 siblings, 1 reply; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-25 16:04 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
25.02.2020 3:56, John Snow wrote:
> This job copies the allocation map into a bitmap. It's a job because
> there's no guarantee that allocation interrogation will be quick (or
> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
>
> It was designed with different possible population patterns in mind,
> but only top layer allocation was implemented for now.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> qapi/block-core.json | 48 +++++++++
> qapi/job.json | 2 +-
> include/block/block_int.h | 21 ++++
> block/bitmap-alloc.c | 207 ++++++++++++++++++++++++++++++++++++++
> blockjob.c | 3 +-
> block/Makefile.objs | 1 +
> 6 files changed, 280 insertions(+), 2 deletions(-)
> create mode 100644 block/bitmap-alloc.c
>
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 85e27bb61f..df1797681a 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2245,6 +2245,54 @@
> { 'command': 'block-dirty-bitmap-merge',
> 'data': 'BlockDirtyBitmapMerge' }
>
> +##
> +# @BitmapPattern:
> +#
> +# An enumeration of possible patterns that can be written into a bitmap.
> +#
> +# @allocation-top: The allocation status of the top layer
> +# of the attached storage node.
> +#
> +# Since: 5.0
> +##
> +{ 'enum': 'BitmapPattern',
> + 'data': ['allocation-top'] }
> +
> +##
> +# @BlockDirtyBitmapPopulate:
> +#
> +# @job-id: identifier for the newly-created block job.
> +#
> +# @pattern: What pattern should be written into the bitmap?
> +#
> +# @on-error: the action to take if an error is encountered on a bitmap's
> +# attached node, default 'report'.
> +# 'stop' and 'enospc' can only be used if the block device supports
> +# io-status (see BlockInfo).
> +#
> +# @auto-finalize: When false, this job will wait in a PENDING state after it has
> +# finished its work, waiting for @block-job-finalize before
> +# making any block graph changes.
sounds a bit strange in context of bitmap-population job
> +# When true, this job will automatically
> +# perform its abort or commit actions.
> +# Defaults to true.
> +#
> +# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it
> +# has completely ceased all work, and awaits @block-job-dismiss.
> +# When true, this job will automatically disappear from the query
> +# list without user intervention.
> +# Defaults to true.
> +#
> +# Since: 5.0
> +##
> +{ 'struct': 'BlockDirtyBitmapPopulate',
> + 'base': 'BlockDirtyBitmap',
> + 'data': { 'job-id': 'str',
> + 'pattern': 'BitmapPattern',
> + '*on-error': 'BlockdevOnError',
> + '*auto-finalize': 'bool',
> + '*auto-dismiss': 'bool' } }
> +
> ##
> # @BlockDirtyBitmapSha256:
> #
> diff --git a/qapi/job.json b/qapi/job.json
> index 5e658281f5..5f496d4630 100644
> --- a/qapi/job.json
> +++ b/qapi/job.json
> @@ -22,7 +22,7 @@
> # Since: 1.7
> ##
> { 'enum': 'JobType',
> - 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
> + 'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'bitmap-populate'] }
>
> ##
> # @JobStatus:
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index 6f9fd5e20e..a5884b597e 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -1215,6 +1215,27 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
> BlockCompletionFunc *cb, void *opaque,
> JobTxn *txn, Error **errp);
>
> +/*
> + * bitpop_job_create: Create a new bitmap population job.
> + *
> + * @job_id: The id of the newly-created job.
> + * @bs: Block device associated with the @target_bitmap.
> + * @target_bitmap: The bitmap to populate.
> + * @on_error: What to do if an error on @bs is encountered.
> + * @creation_flags: Flags that control the behavior of the Job lifetime.
> + * See @BlockJobCreateFlags
> + * @cb: Completion function for the job.
> + * @opaque: Opaque pointer value passed to @cb.
> + * @txn: Transaction that this job is part of (may be NULL).
> + */
> +BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
> + BdrvDirtyBitmap *target_bitmap,
> + BitmapPattern pattern,
> + BlockdevOnError on_error,
> + int creation_flags,
> + BlockCompletionFunc *cb, void *opaque,
> + JobTxn *txn, Error **errp);
> +
> void hmp_drive_add_node(Monitor *mon, const char *optstr);
>
> BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
> diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
> new file mode 100644
> index 0000000000..47d542dc12
> --- /dev/null
> +++ b/block/bitmap-alloc.c
> @@ -0,0 +1,207 @@
> +/*
> + * Async Dirty Bitmap Populator
> + *
> + * Copyright (C) 2020 Red Hat, Inc.
> + *
> + * Authors:
> + * John Snow <jsnow@redhat.com>
> + *
> + * This work is licensed under the terms of the GNU GPL, version 2 or later.
> + * See the COPYING file in the top-level directory.
> + *
> + */
> +
> +#include "qemu/osdep.h"
> +
> +#include "trace.h"
> +#include "block/block.h"
> +#include "block/block_int.h"
> +#include "block/blockjob_int.h"
> +#include "block/block_backup.h"
> +#include "block/block-copy.h"
I hope, not all includes are needed :)
> +#include "qapi/error.h"
> +#include "qapi/qmp/qerror.h"
> +#include "qemu/ratelimit.h"
> +#include "qemu/cutils.h"
> +#include "sysemu/block-backend.h"
> +#include "qemu/bitmap.h"
> +#include "qemu/error-report.h"
> +
> +typedef struct BitpopBlockJob {
> + BlockJob common;
> + BlockDriverState *bs;
> + BdrvDirtyBitmap *target_bitmap;
> + BdrvDirtyBitmap *new_bitmap;
> + BlockdevOnError on_error;
> + uint64_t len;
> +} BitpopBlockJob;
> +
> +static const BlockJobDriver bitpop_job_driver;
> +
> +static void bitpop_commit(Job *job)
> +{
> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
> +
> + bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
> + NULL, true);
Hmm, so you populate new_bitmap, and then merge to target. Why can't we work
directly with target bitmap? The most probable case is that libvirt will
create bitmap specifically to use as target in this operation, or not?
Hmm, just to make it possible to cancel the job and keep the target bitmap in
original state? Is it really needed? I think on failure target bitmap will be
removed anyway..
> +}
> +
> +/* no abort needed; just clean without committing. */
> +
> +static void bitpop_clean(Job *job)
> +{
> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
> +
> + bdrv_release_dirty_bitmap(s->new_bitmap);
> + bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
> +}
> +
> +static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int error)
> +{
> + return block_job_error_action(&job->common, job->on_error, true, error);
> +}
> +
> +static bool coroutine_fn yield_and_check(Job *job)
> +{
> + if (job_is_cancelled(job)) {
> + return true;
> + }
> +
> + job_sleep_ns(job, 0);
> +
> + if (job_is_cancelled(job)) {
> + return true;
> + }
> +
> + return false;
> +}
> +
> +static int coroutine_fn bitpop_run(Job *job, Error **errp)
> +{
> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
> + int ret = 0;
> + int64_t offset;
> + int64_t count;
> + int64_t bytes;
> +
> + for (offset = 0; offset < s->len; ) {
> + if (yield_and_check(job)) {
> + ret = -ECANCELED;
> + break;
> + }
> +
> + bytes = s->len - offset;
> + ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
> + if (ret < 0) {
> + if (bitpop_error_action(s, -ret) == BLOCK_ERROR_ACTION_REPORT) {
> + break;
> + }
> + continue;
> + }
> +
> + if (!count) {
> + ret = 0;
Hmm, I think it's impossible case.. If so, better to make an assertion or ignore..
> + break;
> + }
> +
> + if (ret) {
> + bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
> + ret = 0;
> + }
> +
> + job_progress_update(job, count);
> + offset += count;
> + }
> +
> + return ret;
> +}
> +
> +static const BlockJobDriver bitpop_job_driver = {
> + .job_driver = {
> + .instance_size = sizeof(BitpopBlockJob),
> + .job_type = JOB_TYPE_BITMAP_POPULATE,
> + .free = block_job_free,
> + .user_resume = block_job_user_resume,
> + .run = bitpop_run,
> + .commit = bitpop_commit,
> + .clean = bitpop_clean,
> + }
> +};
> +
> +
> +BlockJob *bitpop_job_create(
> + const char *job_id,
> + BlockDriverState *bs,
> + BdrvDirtyBitmap *target_bitmap,
> + BitmapPattern pattern,
> + BlockdevOnError on_error,
> + int creation_flags,
> + BlockCompletionFunc *cb,
> + void *opaque,
> + JobTxn *txn,
> + Error **errp)
> +{
> + int64_t len;
> + BitpopBlockJob *job = NULL;
> + int64_t cluster_size;
> + BdrvDirtyBitmap *new_bitmap = NULL;
> +
> + assert(bs);
> + assert(target_bitmap);
> +
> + if (!bdrv_is_inserted(bs)) {
> + error_setg(errp, "Device is not inserted: %s",
> + bdrv_get_device_name(bs));
> + return NULL;
> + }
Why this?
> +
> + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
> + return NULL;
> + }
and this?
> +
> + if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
> + return NULL;
> + }
> +
> + if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
> + error_setg(errp, "Unrecognized bitmap pattern");
> + return NULL;
> + }
> +
> + len = bdrv_getlength(bs);
> + if (len < 0) {
> + error_setg_errno(errp, -len, "unable to get length for '%s'",
> + bdrv_get_device_name(bs));
> + return NULL;
> + }
> +
> + /* NB: new bitmap is anonymous and enabled */
> + cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> + new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> + if (!new_bitmap) {
> + return NULL;
> + }
> +
> + /* Take ownership; we reserve the right to write into this on-commit. */
> + bdrv_dirty_bitmap_set_busy(target_bitmap, true);
Honestly, I still have bad understanding about how should we use dirty bitmap mutex,
but note that bdrv_dirty_bitmap_set_busy locks the mutex. And it is (may be) possible,
that busy status of the bitmap is changed after bdrv_dirty_bitmap_check but before
bdrv_dirty_bitmap_set_busy. So, more correct would be do both operation under one
critical section. Still, I don't know is the situation possible.
> +
> + job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
> + BLK_PERM_CONSISTENT_READ,
Do we need it? We are not going to read..
> + BLK_PERM_ALL & ~BLK_PERM_RESIZE,
> + 0, creation_flags,
> + cb, opaque, errp);
> + if (!job) {
> + bdrv_dirty_bitmap_set_busy(target_bitmap, false);
> + bdrv_release_dirty_bitmap(new_bitmap);
> + return NULL;
> + }
> +
> + job->bs = bs;
> + job->on_error = on_error;
> + job->target_bitmap = target_bitmap;
> + job->new_bitmap = new_bitmap;
> + job->len = len;
> + job_progress_set_remaining(&job->common.job, job->len);
> +
> + return &job->common;
> +}
> diff --git a/blockjob.c b/blockjob.c
> index 5d63b1e89d..7e450372bd 100644
> --- a/blockjob.c
> +++ b/blockjob.c
> @@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
> return job_type(job) == JOB_TYPE_BACKUP ||
> job_type(job) == JOB_TYPE_COMMIT ||
> job_type(job) == JOB_TYPE_MIRROR ||
> - job_type(job) == JOB_TYPE_STREAM;
> + job_type(job) == JOB_TYPE_STREAM ||
> + job_type(job) == JOB_TYPE_BITMAP_POPULATE;
> }
>
> BlockJob *block_job_next(BlockJob *bjob)
> diff --git a/block/Makefile.objs b/block/Makefile.objs
> index 3bcb35c81d..f3cfc89d90 100644
> --- a/block/Makefile.objs
> +++ b/block/Makefile.objs
> @@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
> block-obj-y += accounting.o dirty-bitmap.o
> block-obj-y += write-threshold.o
> block-obj-y += backup.o
> +block-obj-y += bitmap-alloc.o
> block-obj-$(CONFIG_REPLICATION) += replication.o
> block-obj-y += throttle.o copy-on-read.o
> block-obj-y += block-copy.o
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/6] block: add bitmap-populate job
2020-02-25 16:04 ` Vladimir Sementsov-Ogievskiy
@ 2020-02-25 20:41 ` John Snow
2020-02-26 5:07 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-25 20:41 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
On 2/25/20 11:04 AM, Vladimir Sementsov-Ogievskiy wrote:
> 25.02.2020 3:56, John Snow wrote:
>> This job copies the allocation map into a bitmap. It's a job because
>> there's no guarantee that allocation interrogation will be quick (or
>> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
>>
>> It was designed with different possible population patterns in mind,
>> but only top layer allocation was implemented for now.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> qapi/block-core.json | 48 +++++++++
>> qapi/job.json | 2 +-
>> include/block/block_int.h | 21 ++++
>> block/bitmap-alloc.c | 207 ++++++++++++++++++++++++++++++++++++++
>> blockjob.c | 3 +-
>> block/Makefile.objs | 1 +
>> 6 files changed, 280 insertions(+), 2 deletions(-)
>> create mode 100644 block/bitmap-alloc.c
>>
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index 85e27bb61f..df1797681a 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -2245,6 +2245,54 @@
>> { 'command': 'block-dirty-bitmap-merge',
>> 'data': 'BlockDirtyBitmapMerge' }
>> +##
>> +# @BitmapPattern:
>> +#
>> +# An enumeration of possible patterns that can be written into a bitmap.
>> +#
>> +# @allocation-top: The allocation status of the top layer
>> +# of the attached storage node.
>> +#
>> +# Since: 5.0
>> +##
>> +{ 'enum': 'BitmapPattern',
>> + 'data': ['allocation-top'] }
>> +
>> +##
>> +# @BlockDirtyBitmapPopulate:
>> +#
>> +# @job-id: identifier for the newly-created block job.
>> +#
>> +# @pattern: What pattern should be written into the bitmap?
>> +#
>> +# @on-error: the action to take if an error is encountered on a bitmap's
>> +# attached node, default 'report'.
>> +# 'stop' and 'enospc' can only be used if the block device
>> supports
>> +# io-status (see BlockInfo).
>> +#
>> +# @auto-finalize: When false, this job will wait in a PENDING state
>> after it has
>> +# finished its work, waiting for @block-job-finalize
>> before
>> +# making any block graph changes.
>
> sounds a bit strange in context of bitmap-population job
>
Yeah, you're right. Copy-pasted for "consistency".
>> +# When true, this job will automatically
>> +# perform its abort or commit actions.
>> +# Defaults to true.
>> +#
>> +# @auto-dismiss: When false, this job will wait in a CONCLUDED state
>> after it
>> +# has completely ceased all work, and awaits
>> @block-job-dismiss.
>> +# When true, this job will automatically disappear
>> from the query
>> +# list without user intervention.
>> +# Defaults to true.
>> +#
>> +# Since: 5.0
>> +##
>> +{ 'struct': 'BlockDirtyBitmapPopulate',
>> + 'base': 'BlockDirtyBitmap',
>> + 'data': { 'job-id': 'str',
>> + 'pattern': 'BitmapPattern',
>> + '*on-error': 'BlockdevOnError',
>> + '*auto-finalize': 'bool',
>> + '*auto-dismiss': 'bool' } }
>> +
>> ##
>> # @BlockDirtyBitmapSha256:
>> #
>> diff --git a/qapi/job.json b/qapi/job.json
>> index 5e658281f5..5f496d4630 100644
>> --- a/qapi/job.json
>> +++ b/qapi/job.json
>> @@ -22,7 +22,7 @@
>> # Since: 1.7
>> ##
>> { 'enum': 'JobType',
>> - 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
>> + 'data': ['commit', 'stream', 'mirror', 'backup', 'create',
>> 'bitmap-populate'] }
>> ##
>> # @JobStatus:
>> diff --git a/include/block/block_int.h b/include/block/block_int.h
>> index 6f9fd5e20e..a5884b597e 100644
>> --- a/include/block/block_int.h
>> +++ b/include/block/block_int.h
>> @@ -1215,6 +1215,27 @@ BlockJob *backup_job_create(const char *job_id,
>> BlockDriverState *bs,
>> BlockCompletionFunc *cb, void *opaque,
>> JobTxn *txn, Error **errp);
>> +/*
>> + * bitpop_job_create: Create a new bitmap population job.
>> + *
>> + * @job_id: The id of the newly-created job.
>> + * @bs: Block device associated with the @target_bitmap.
>> + * @target_bitmap: The bitmap to populate.
>> + * @on_error: What to do if an error on @bs is encountered.
>> + * @creation_flags: Flags that control the behavior of the Job lifetime.
>> + * See @BlockJobCreateFlags
>> + * @cb: Completion function for the job.
>> + * @opaque: Opaque pointer value passed to @cb.
>> + * @txn: Transaction that this job is part of (may be NULL).
>> + */
>> +BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
>> + BdrvDirtyBitmap *target_bitmap,
>> + BitmapPattern pattern,
>> + BlockdevOnError on_error,
>> + int creation_flags,
>> + BlockCompletionFunc *cb, void *opaque,
>> + JobTxn *txn, Error **errp);
>> +
>> void hmp_drive_add_node(Monitor *mon, const char *optstr);
>> BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
>> diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
>> new file mode 100644
>> index 0000000000..47d542dc12
>> --- /dev/null
>> +++ b/block/bitmap-alloc.c
>> @@ -0,0 +1,207 @@
>> +/*
>> + * Async Dirty Bitmap Populator
>> + *
>> + * Copyright (C) 2020 Red Hat, Inc.
>> + *
>> + * Authors:
>> + * John Snow <jsnow@redhat.com>
>> + *
>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>> later.
>> + * See the COPYING file in the top-level directory.
>> + *
>> + */
>> +
>> +#include "qemu/osdep.h"
>> +
>> +#include "trace.h"
>> +#include "block/block.h"
>> +#include "block/block_int.h"
>> +#include "block/blockjob_int.h"
>> +#include "block/block_backup.h"
>> +#include "block/block-copy.h"
>
> I hope, not all includes are needed :)
Whoops, no, of course not. I copied the skeleton from backup, as you can
tell ;)
>
>> +#include "qapi/error.h"
>> +#include "qapi/qmp/qerror.h"
>> +#include "qemu/ratelimit.h"
>> +#include "qemu/cutils.h"
>> +#include "sysemu/block-backend.h"
>> +#include "qemu/bitmap.h"
>> +#include "qemu/error-report.h"
>> +
>> +typedef struct BitpopBlockJob {
>> + BlockJob common;
>> + BlockDriverState *bs;
>> + BdrvDirtyBitmap *target_bitmap;
>> + BdrvDirtyBitmap *new_bitmap;
>> + BlockdevOnError on_error;
>> + uint64_t len;
>> +} BitpopBlockJob;
>> +
>> +static const BlockJobDriver bitpop_job_driver;
>> +
>> +static void bitpop_commit(Job *job)
>> +{
>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>> +
>> + bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
>> + NULL, true);
>
> Hmm, so you populate new_bitmap, and then merge to target. Why can't we
> work
> directly with target bitmap? The most probable case is that libvirt will
> create bitmap specifically to use as target in this operation, or not?
>
Most likely case, yes. Odds are very good it will be a brand new bitmap.
However, we already have a creation command -- I didn't want to make a
second job-version of the command and then maintain two interfaces, so I
made it a "merge into existing" style command instead.
> Hmm, just to make it possible to cancel the job and keep the target
> bitmap in
> original state? Is it really needed? I think on failure target bitmap
> will be
> removed anyway..
>
You caught me being *lazy*. I copy the bitmap so I can unconditionally
enable it to catch in-flight writes without having to create block graph
modifications.
But, yes, to undo changes if we cancel.
I didn't want to make a job that was not able to be canceled. The
alternative is pursuing the design where we allow new bitmaps only --
because then on cancel we can just delete them.
>> +}
>> +
>> +/* no abort needed; just clean without committing. */
>> +
>> +static void bitpop_clean(Job *job)
>> +{
>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>> +
>> + bdrv_release_dirty_bitmap(s->new_bitmap);
>> + bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
>> +}
>> +
>> +static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int
>> error)
>> +{
>> + return block_job_error_action(&job->common, job->on_error, true,
>> error);
>> +}
>> +
>> +static bool coroutine_fn yield_and_check(Job *job)
>> +{
>> + if (job_is_cancelled(job)) {
>> + return true;
>> + }
>> +
>> + job_sleep_ns(job, 0);
>> +
>> + if (job_is_cancelled(job)) {
>> + return true;
>> + }
>> +
>> + return false;
>> +}
>> +
>> +static int coroutine_fn bitpop_run(Job *job, Error **errp)
>> +{
>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>> + int ret = 0;
>> + int64_t offset;
>> + int64_t count;
>> + int64_t bytes;
>> +
>> + for (offset = 0; offset < s->len; ) {
>> + if (yield_and_check(job)) {
>> + ret = -ECANCELED;
>> + break;
>> + }
>> +
>> + bytes = s->len - offset;
>> + ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
>> + if (ret < 0) {
>> + if (bitpop_error_action(s, -ret) ==
>> BLOCK_ERROR_ACTION_REPORT) {
>> + break;
>> + }
>> + continue;
>> + }
>> +
>> + if (!count) {
>> + ret = 0;
>
> Hmm, I think it's impossible case.. If so, better to make an assertion
> or ignore..
>
OK, agreed.
>> + break;
>> + }
>> +
>> + if (ret) {
>> + bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
>> + ret = 0;
>> + }
>> +
>> + job_progress_update(job, count);
>> + offset += count;
>> + }
>> +
>> + return ret;
>> +}
>> +
>> +static const BlockJobDriver bitpop_job_driver = {
>> + .job_driver = {
>> + .instance_size = sizeof(BitpopBlockJob),
>> + .job_type = JOB_TYPE_BITMAP_POPULATE,
>> + .free = block_job_free,
>> + .user_resume = block_job_user_resume,
>> + .run = bitpop_run,
>> + .commit = bitpop_commit,
>> + .clean = bitpop_clean,
>> + }
>> +};
>> +
>> +
>> +BlockJob *bitpop_job_create(
>> + const char *job_id,
>> + BlockDriverState *bs,
>> + BdrvDirtyBitmap *target_bitmap,
>> + BitmapPattern pattern,
>> + BlockdevOnError on_error,
>> + int creation_flags,
>> + BlockCompletionFunc *cb,
>> + void *opaque,
>> + JobTxn *txn,
>> + Error **errp)
>> +{
>> + int64_t len;
>> + BitpopBlockJob *job = NULL;
>> + int64_t cluster_size;
>> + BdrvDirtyBitmap *new_bitmap = NULL;
>> +
>> + assert(bs);
>> + assert(target_bitmap);
>> +
>> + if (!bdrv_is_inserted(bs)) {
>> + error_setg(errp, "Device is not inserted: %s",
>> + bdrv_get_device_name(bs));
>> + return NULL;
>> + }
>
> Why this?
>
I assumed there was nothing to read the allocation map *of* if there
wasn't a media present.
Am I mistaken?
>> +
>> + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
>> + return NULL;
>> + }
>
> and this?
>
Copy-paste: I don't understand if I want a new op blocker, to re-use an
op-blocker, or to have no op blocker.
Genuinely I have no idea. I should left a review comment here, I forgot
about this part, sorry.
>> +
>> + if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT,
>> errp)) {
>> + return NULL;
>> + }
>> +
>> + if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
>> + error_setg(errp, "Unrecognized bitmap pattern");
>> + return NULL;
>> + }
>> +
>> + len = bdrv_getlength(bs);
>> + if (len < 0) {
>> + error_setg_errno(errp, -len, "unable to get length for '%s'",
>> + bdrv_get_device_name(bs));
>> + return NULL;
>> + }
>> +
>> + /* NB: new bitmap is anonymous and enabled */
>> + cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
>> + new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
>> + if (!new_bitmap) {
>> + return NULL;
>> + }
>> +
>> + /* Take ownership; we reserve the right to write into this
>> on-commit. */
>> + bdrv_dirty_bitmap_set_busy(target_bitmap, true);
>
> Honestly, I still have bad understanding about how should we use dirty
> bitmap mutex,
> but note that bdrv_dirty_bitmap_set_busy locks the mutex. And it is (may
> be) possible,
> that busy status of the bitmap is changed after bdrv_dirty_bitmap_check
> but before
> bdrv_dirty_bitmap_set_busy. So, more correct would be do both operation
> under one
> critical section. Still, I don't know is the situation possible.
>
Aren't we under the BQL here? Can we be pre-empted? :(
>> +
>> + job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
>> + BLK_PERM_CONSISTENT_READ,
>
> Do we need it? We are not going to read..
>
Copy-paste / leftover from an earlier draft where I was trying to
achieve atomicity. It can be removed if we don't want the stricter
atomicity.
>> + BLK_PERM_ALL & ~BLK_PERM_RESIZE,
>> + 0, creation_flags,
>> + cb, opaque, errp);
>> + if (!job) {
>> + bdrv_dirty_bitmap_set_busy(target_bitmap, false);
>> + bdrv_release_dirty_bitmap(new_bitmap);
>> + return NULL;
>> + }
>> +
>> + job->bs = bs;
>> + job->on_error = on_error;
>> + job->target_bitmap = target_bitmap;
>> + job->new_bitmap = new_bitmap;
>> + job->len = len;
>> + job_progress_set_remaining(&job->common.job, job->len);
>> +
>> + return &job->common;
>> +}
>> diff --git a/blockjob.c b/blockjob.c
>> index 5d63b1e89d..7e450372bd 100644
>> --- a/blockjob.c
>> +++ b/blockjob.c
>> @@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
>> return job_type(job) == JOB_TYPE_BACKUP ||
>> job_type(job) == JOB_TYPE_COMMIT ||
>> job_type(job) == JOB_TYPE_MIRROR ||
>> - job_type(job) == JOB_TYPE_STREAM;
>> + job_type(job) == JOB_TYPE_STREAM ||
>> + job_type(job) == JOB_TYPE_BITMAP_POPULATE;
>> }
>> BlockJob *block_job_next(BlockJob *bjob)
>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>> index 3bcb35c81d..f3cfc89d90 100644
>> --- a/block/Makefile.objs
>> +++ b/block/Makefile.objs
>> @@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
>> block-obj-y += accounting.o dirty-bitmap.o
>> block-obj-y += write-threshold.o
>> block-obj-y += backup.o
>> +block-obj-y += bitmap-alloc.o
>> block-obj-$(CONFIG_REPLICATION) += replication.o
>> block-obj-y += throttle.o copy-on-read.o
>> block-obj-y += block-copy.o
>>
>
>
Thanks for the review. I'll start making changes, but won't send V2 just
yet.
--js
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/6] block: add bitmap-populate job
2020-02-25 20:41 ` John Snow
@ 2020-02-26 5:07 ` Vladimir Sementsov-Ogievskiy
2020-02-26 19:11 ` John Snow
0 siblings, 1 reply; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-26 5:07 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
25.02.2020 23:41, John Snow wrote:
>
>
> On 2/25/20 11:04 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 25.02.2020 3:56, John Snow wrote:
>>> This job copies the allocation map into a bitmap. It's a job because
>>> there's no guarantee that allocation interrogation will be quick (or
>>> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
>>>
>>> It was designed with different possible population patterns in mind,
>>> but only top layer allocation was implemented for now.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> qapi/block-core.json | 48 +++++++++
>>> qapi/job.json | 2 +-
>>> include/block/block_int.h | 21 ++++
>>> block/bitmap-alloc.c | 207 ++++++++++++++++++++++++++++++++++++++
>>> blockjob.c | 3 +-
>>> block/Makefile.objs | 1 +
>>> 6 files changed, 280 insertions(+), 2 deletions(-)
>>> create mode 100644 block/bitmap-alloc.c
>>>
>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>> index 85e27bb61f..df1797681a 100644
>>> --- a/qapi/block-core.json
>>> +++ b/qapi/block-core.json
>>> @@ -2245,6 +2245,54 @@
>>> { 'command': 'block-dirty-bitmap-merge',
>>> 'data': 'BlockDirtyBitmapMerge' }
>>> +##
>>> +# @BitmapPattern:
>>> +#
>>> +# An enumeration of possible patterns that can be written into a bitmap.
>>> +#
>>> +# @allocation-top: The allocation status of the top layer
>>> +# of the attached storage node.
>>> +#
>>> +# Since: 5.0
>>> +##
>>> +{ 'enum': 'BitmapPattern',
>>> + 'data': ['allocation-top'] }
>>> +
>>> +##
>>> +# @BlockDirtyBitmapPopulate:
>>> +#
>>> +# @job-id: identifier for the newly-created block job.
>>> +#
>>> +# @pattern: What pattern should be written into the bitmap?
>>> +#
>>> +# @on-error: the action to take if an error is encountered on a bitmap's
>>> +# attached node, default 'report'.
>>> +# 'stop' and 'enospc' can only be used if the block device
>>> supports
>>> +# io-status (see BlockInfo).
>>> +#
>>> +# @auto-finalize: When false, this job will wait in a PENDING state
>>> after it has
>>> +# finished its work, waiting for @block-job-finalize
>>> before
>>> +# making any block graph changes.
>>
>> sounds a bit strange in context of bitmap-population job
>>
>
> Yeah, you're right. Copy-pasted for "consistency".
>
>>> +# When true, this job will automatically
>>> +# perform its abort or commit actions.
>>> +# Defaults to true.
>>> +#
>>> +# @auto-dismiss: When false, this job will wait in a CONCLUDED state
>>> after it
>>> +# has completely ceased all work, and awaits
>>> @block-job-dismiss.
>>> +# When true, this job will automatically disappear
>>> from the query
>>> +# list without user intervention.
>>> +# Defaults to true.
>>> +#
>>> +# Since: 5.0
>>> +##
>>> +{ 'struct': 'BlockDirtyBitmapPopulate',
>>> + 'base': 'BlockDirtyBitmap',
>>> + 'data': { 'job-id': 'str',
>>> + 'pattern': 'BitmapPattern',
>>> + '*on-error': 'BlockdevOnError',
>>> + '*auto-finalize': 'bool',
>>> + '*auto-dismiss': 'bool' } }
>>> +
>>> ##
>>> # @BlockDirtyBitmapSha256:
>>> #
>>> diff --git a/qapi/job.json b/qapi/job.json
>>> index 5e658281f5..5f496d4630 100644
>>> --- a/qapi/job.json
>>> +++ b/qapi/job.json
>>> @@ -22,7 +22,7 @@
>>> # Since: 1.7
>>> ##
>>> { 'enum': 'JobType',
>>> - 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
>>> + 'data': ['commit', 'stream', 'mirror', 'backup', 'create',
>>> 'bitmap-populate'] }
>>> ##
>>> # @JobStatus:
>>> diff --git a/include/block/block_int.h b/include/block/block_int.h
>>> index 6f9fd5e20e..a5884b597e 100644
>>> --- a/include/block/block_int.h
>>> +++ b/include/block/block_int.h
>>> @@ -1215,6 +1215,27 @@ BlockJob *backup_job_create(const char *job_id,
>>> BlockDriverState *bs,
>>> BlockCompletionFunc *cb, void *opaque,
>>> JobTxn *txn, Error **errp);
>>> +/*
>>> + * bitpop_job_create: Create a new bitmap population job.
>>> + *
>>> + * @job_id: The id of the newly-created job.
>>> + * @bs: Block device associated with the @target_bitmap.
>>> + * @target_bitmap: The bitmap to populate.
>>> + * @on_error: What to do if an error on @bs is encountered.
>>> + * @creation_flags: Flags that control the behavior of the Job lifetime.
>>> + * See @BlockJobCreateFlags
>>> + * @cb: Completion function for the job.
>>> + * @opaque: Opaque pointer value passed to @cb.
>>> + * @txn: Transaction that this job is part of (may be NULL).
>>> + */
>>> +BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
>>> + BdrvDirtyBitmap *target_bitmap,
>>> + BitmapPattern pattern,
>>> + BlockdevOnError on_error,
>>> + int creation_flags,
>>> + BlockCompletionFunc *cb, void *opaque,
>>> + JobTxn *txn, Error **errp);
>>> +
>>> void hmp_drive_add_node(Monitor *mon, const char *optstr);
>>> BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
>>> diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
>>> new file mode 100644
>>> index 0000000000..47d542dc12
>>> --- /dev/null
>>> +++ b/block/bitmap-alloc.c
>>> @@ -0,0 +1,207 @@
>>> +/*
>>> + * Async Dirty Bitmap Populator
>>> + *
>>> + * Copyright (C) 2020 Red Hat, Inc.
>>> + *
>>> + * Authors:
>>> + * John Snow <jsnow@redhat.com>
>>> + *
>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>> later.
>>> + * See the COPYING file in the top-level directory.
>>> + *
>>> + */
>>> +
>>> +#include "qemu/osdep.h"
>>> +
>>> +#include "trace.h"
>>> +#include "block/block.h"
>>> +#include "block/block_int.h"
>>> +#include "block/blockjob_int.h"
>>> +#include "block/block_backup.h"
>>> +#include "block/block-copy.h"
>>
>> I hope, not all includes are needed :)
>
> Whoops, no, of course not. I copied the skeleton from backup, as you can
> tell ;)
>
>>
>>> +#include "qapi/error.h"
>>> +#include "qapi/qmp/qerror.h"
>>> +#include "qemu/ratelimit.h"
>>> +#include "qemu/cutils.h"
>>> +#include "sysemu/block-backend.h"
>>> +#include "qemu/bitmap.h"
>>> +#include "qemu/error-report.h"
>>> +
>>> +typedef struct BitpopBlockJob {
>>> + BlockJob common;
>>> + BlockDriverState *bs;
>>> + BdrvDirtyBitmap *target_bitmap;
>>> + BdrvDirtyBitmap *new_bitmap;
>>> + BlockdevOnError on_error;
>>> + uint64_t len;
>>> +} BitpopBlockJob;
>>> +
>>> +static const BlockJobDriver bitpop_job_driver;
>>> +
>>> +static void bitpop_commit(Job *job)
>>> +{
>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>> +
>>> + bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
>>> + NULL, true);
>>
>> Hmm, so you populate new_bitmap, and then merge to target. Why can't we
>> work
>> directly with target bitmap? The most probable case is that libvirt will
>> create bitmap specifically to use as target in this operation, or not?
>>
>
> Most likely case, yes. Odds are very good it will be a brand new bitmap.
>
> However, we already have a creation command -- I didn't want to make a
> second job-version of the command and then maintain two interfaces, so I
> made it a "merge into existing" style command instead.
>
>> Hmm, just to make it possible to cancel the job and keep the target
>> bitmap in
>> original state? Is it really needed? I think on failure target bitmap
>> will be
>> removed anyway..
>>
>
> You caught me being *lazy*. I copy the bitmap so I can unconditionally
> enable it to catch in-flight writes without having to create block graph
> modifications.
>
> But, yes, to undo changes if we cancel.
>
> I didn't want to make a job that was not able to be canceled. The
> alternative is pursuing the design where we allow new bitmaps only --
> because then on cancel we can just delete them.
On backup job (and any other) we can't rollback target changes on cancel.
So, I think it would be OK to take same semantics for the new job, keeping in
mind that it would be most probable usage case and no sense in creating
additional bitmaps. And if caller needs to use existent non-empty bitmap as
target and wants correct cancel, it always can create additional bitmap by
itself and then merge it to actual target.
And why new? It's up to user, what to use as target. And user knows, that on
failure or cancel target becomes invalid and will workaround this if needed.
>
>>> +}
>>> +
>>> +/* no abort needed; just clean without committing. */
>>> +
>>> +static void bitpop_clean(Job *job)
>>> +{
>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>> +
>>> + bdrv_release_dirty_bitmap(s->new_bitmap);
>>> + bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
>>> +}
>>> +
>>> +static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int
>>> error)
>>> +{
>>> + return block_job_error_action(&job->common, job->on_error, true,
>>> error);
>>> +}
>>> +
>>> +static bool coroutine_fn yield_and_check(Job *job)
>>> +{
>>> + if (job_is_cancelled(job)) {
>>> + return true;
>>> + }
>>> +
>>> + job_sleep_ns(job, 0);
>>> +
>>> + if (job_is_cancelled(job)) {
>>> + return true;
>>> + }
>>> +
>>> + return false;
>>> +}
>>> +
>>> +static int coroutine_fn bitpop_run(Job *job, Error **errp)
>>> +{
>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>> + int ret = 0;
>>> + int64_t offset;
>>> + int64_t count;
>>> + int64_t bytes;
>>> +
>>> + for (offset = 0; offset < s->len; ) {
>>> + if (yield_and_check(job)) {
>>> + ret = -ECANCELED;
>>> + break;
>>> + }
>>> +
>>> + bytes = s->len - offset;
>>> + ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
>>> + if (ret < 0) {
>>> + if (bitpop_error_action(s, -ret) ==
>>> BLOCK_ERROR_ACTION_REPORT) {
>>> + break;
>>> + }
>>> + continue;
>>> + }
>>> +
>>> + if (!count) {
>>> + ret = 0;
>>
>> Hmm, I think it's impossible case.. If so, better to make an assertion
>> or ignore..
>>
>
> OK, agreed.
>
>>> + break;
>>> + }
>>> +
>>> + if (ret) {
>>> + bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
>>> + ret = 0;
>>> + }
>>> +
>>> + job_progress_update(job, count);
>>> + offset += count;
>>> + }
>>> +
>>> + return ret;
>>> +}
>>> +
>>> +static const BlockJobDriver bitpop_job_driver = {
>>> + .job_driver = {
>>> + .instance_size = sizeof(BitpopBlockJob),
>>> + .job_type = JOB_TYPE_BITMAP_POPULATE,
>>> + .free = block_job_free,
>>> + .user_resume = block_job_user_resume,
>>> + .run = bitpop_run,
>>> + .commit = bitpop_commit,
>>> + .clean = bitpop_clean,
>>> + }
>>> +};
>>> +
>>> +
>>> +BlockJob *bitpop_job_create(
>>> + const char *job_id,
>>> + BlockDriverState *bs,
>>> + BdrvDirtyBitmap *target_bitmap,
>>> + BitmapPattern pattern,
>>> + BlockdevOnError on_error,
>>> + int creation_flags,
>>> + BlockCompletionFunc *cb,
>>> + void *opaque,
>>> + JobTxn *txn,
>>> + Error **errp)
>>> +{
>>> + int64_t len;
>>> + BitpopBlockJob *job = NULL;
>>> + int64_t cluster_size;
>>> + BdrvDirtyBitmap *new_bitmap = NULL;
>>> +
>>> + assert(bs);
>>> + assert(target_bitmap);
>>> +
>>> + if (!bdrv_is_inserted(bs)) {
>>> + error_setg(errp, "Device is not inserted: %s",
>>> + bdrv_get_device_name(bs));
>>> + return NULL;
>>> + }
>>
>> Why this?
>>
>
> I assumed there was nothing to read the allocation map *of* if there
> wasn't a media present.
>
> Am I mistaken?
is_inserted checks existing of bs->drv, but bitmap operations actually
doesn't need any drv.. Hmm. I'm not against this check anyway.
>
>>> +
>>> + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
>>> + return NULL;
>>> + }
>>
>> and this?
>>
>
> Copy-paste: I don't understand if I want a new op blocker, to re-use an
> op-blocker, or to have no op blocker.
>
> Genuinely I have no idea. I should left a review comment here, I forgot
> about this part, sorry.
I'm for no op blocker. As I understand, op-blockers are old and should be
replaced by permissions and frozen children. So, if we don't know, do we
need any blocking here, better go on without it. Also, op-blockers can't
block forbidden usage through filters anyway.
>
>>> +
>>> + if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT,
>>> errp)) {
>>> + return NULL;
>>> + }
>>> +
>>> + if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
>>> + error_setg(errp, "Unrecognized bitmap pattern");
>>> + return NULL;
>>> + }
>>> +
>>> + len = bdrv_getlength(bs);
>>> + if (len < 0) {
>>> + error_setg_errno(errp, -len, "unable to get length for '%s'",
>>> + bdrv_get_device_name(bs));
>>> + return NULL;
>>> + }
>>> +
>>> + /* NB: new bitmap is anonymous and enabled */
>>> + cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
>>> + new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
>>> + if (!new_bitmap) {
>>> + return NULL;
>>> + }
>>> +
>>> + /* Take ownership; we reserve the right to write into this
>>> on-commit. */
>>> + bdrv_dirty_bitmap_set_busy(target_bitmap, true);
>>
>> Honestly, I still have bad understanding about how should we use dirty
>> bitmap mutex,
>> but note that bdrv_dirty_bitmap_set_busy locks the mutex. And it is (may
>> be) possible,
>> that busy status of the bitmap is changed after bdrv_dirty_bitmap_check
>> but before
>> bdrv_dirty_bitmap_set_busy. So, more correct would be do both operation
>> under one
>> critical section. Still, I don't know is the situation possible.
>>
>
> Aren't we under the BQL here? Can we be pre-empted? :(
Seems we are. But, as it's said above dirty_bitmap_mutex declaration:
/* Writing to the list requires the BQL _and_ the dirty_bitmap_mutex.
* Reading from the list can be done with either the BQL or the
* dirty_bitmap_mutex. Modifying a bitmap only requires
* dirty_bitmap_mutex. */
It means, that another thread may modify bitmap (for example its 'busy' field)
taking only dirty_bitmap_mutex, which will lead to the case I described.
>
>>> +
>>> + job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
>>> + BLK_PERM_CONSISTENT_READ,
>>
>> Do we need it? We are not going to read..
>>
>
> Copy-paste / leftover from an earlier draft where I was trying to
> achieve atomicity. It can be removed if we don't want the stricter
> atomicity.
>
>>> + BLK_PERM_ALL & ~BLK_PERM_RESIZE,
>>> + 0, creation_flags,
>>> + cb, opaque, errp);
>>> + if (!job) {
>>> + bdrv_dirty_bitmap_set_busy(target_bitmap, false);
>>> + bdrv_release_dirty_bitmap(new_bitmap);
>>> + return NULL;
>>> + }
>>> +
>>> + job->bs = bs;
>>> + job->on_error = on_error;
>>> + job->target_bitmap = target_bitmap;
>>> + job->new_bitmap = new_bitmap;
>>> + job->len = len;
>>> + job_progress_set_remaining(&job->common.job, job->len);
>>> +
>>> + return &job->common;
>>> +}
>>> diff --git a/blockjob.c b/blockjob.c
>>> index 5d63b1e89d..7e450372bd 100644
>>> --- a/blockjob.c
>>> +++ b/blockjob.c
>>> @@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
>>> return job_type(job) == JOB_TYPE_BACKUP ||
>>> job_type(job) == JOB_TYPE_COMMIT ||
>>> job_type(job) == JOB_TYPE_MIRROR ||
>>> - job_type(job) == JOB_TYPE_STREAM;
>>> + job_type(job) == JOB_TYPE_STREAM ||
>>> + job_type(job) == JOB_TYPE_BITMAP_POPULATE;
>>> }
>>> BlockJob *block_job_next(BlockJob *bjob)
>>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>>> index 3bcb35c81d..f3cfc89d90 100644
>>> --- a/block/Makefile.objs
>>> +++ b/block/Makefile.objs
>>> @@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
>>> block-obj-y += accounting.o dirty-bitmap.o
>>> block-obj-y += write-threshold.o
>>> block-obj-y += backup.o
>>> +block-obj-y += bitmap-alloc.o
>>> block-obj-$(CONFIG_REPLICATION) += replication.o
>>> block-obj-y += throttle.o copy-on-read.o
>>> block-obj-y += block-copy.o
>>>
>>
>>
>
> Thanks for the review. I'll start making changes, but won't send V2 just
> yet.
>
> --js
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/6] block: add bitmap-populate job
2020-02-26 5:07 ` Vladimir Sementsov-Ogievskiy
@ 2020-02-26 19:11 ` John Snow
2020-02-27 6:11 ` Vladimir Sementsov-Ogievskiy
0 siblings, 1 reply; 20+ messages in thread
From: John Snow @ 2020-02-26 19:11 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
On 2/26/20 12:07 AM, Vladimir Sementsov-Ogievskiy wrote:
> 25.02.2020 23:41, John Snow wrote:
>>
>>
>> On 2/25/20 11:04 AM, Vladimir Sementsov-Ogievskiy wrote:
>>> 25.02.2020 3:56, John Snow wrote:
>>>> This job copies the allocation map into a bitmap. It's a job because
>>>> there's no guarantee that allocation interrogation will be quick (or
>>>> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
>>>>
>>>> It was designed with different possible population patterns in mind,
>>>> but only top layer allocation was implemented for now.
>>>>
>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>> ---
>>>> qapi/block-core.json | 48 +++++++++
>>>> qapi/job.json | 2 +-
>>>> include/block/block_int.h | 21 ++++
>>>> block/bitmap-alloc.c | 207
>>>> ++++++++++++++++++++++++++++++++++++++
>>>> blockjob.c | 3 +-
>>>> block/Makefile.objs | 1 +
>>>> 6 files changed, 280 insertions(+), 2 deletions(-)
>>>> create mode 100644 block/bitmap-alloc.c
>>>>
>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>>> index 85e27bb61f..df1797681a 100644
>>>> --- a/qapi/block-core.json
>>>> +++ b/qapi/block-core.json
>>>> @@ -2245,6 +2245,54 @@
>>>> { 'command': 'block-dirty-bitmap-merge',
>>>> 'data': 'BlockDirtyBitmapMerge' }
>>>> +##
>>>> +# @BitmapPattern:
>>>> +#
>>>> +# An enumeration of possible patterns that can be written into a
>>>> bitmap.
>>>> +#
>>>> +# @allocation-top: The allocation status of the top layer
>>>> +# of the attached storage node.
>>>> +#
>>>> +# Since: 5.0
>>>> +##
>>>> +{ 'enum': 'BitmapPattern',
>>>> + 'data': ['allocation-top'] }
>>>> +
>>>> +##
>>>> +# @BlockDirtyBitmapPopulate:
>>>> +#
>>>> +# @job-id: identifier for the newly-created block job.
>>>> +#
>>>> +# @pattern: What pattern should be written into the bitmap?
>>>> +#
>>>> +# @on-error: the action to take if an error is encountered on a
>>>> bitmap's
>>>> +# attached node, default 'report'.
>>>> +# 'stop' and 'enospc' can only be used if the block device
>>>> supports
>>>> +# io-status (see BlockInfo).
>>>> +#
>>>> +# @auto-finalize: When false, this job will wait in a PENDING state
>>>> after it has
>>>> +# finished its work, waiting for @block-job-finalize
>>>> before
>>>> +# making any block graph changes.
>>>
>>> sounds a bit strange in context of bitmap-population job
>>>
>>
>> Yeah, you're right. Copy-pasted for "consistency".
>>
>>>> +# When true, this job will automatically
>>>> +# perform its abort or commit actions.
>>>> +# Defaults to true.
>>>> +#
>>>> +# @auto-dismiss: When false, this job will wait in a CONCLUDED state
>>>> after it
>>>> +# has completely ceased all work, and awaits
>>>> @block-job-dismiss.
>>>> +# When true, this job will automatically disappear
>>>> from the query
>>>> +# list without user intervention.
>>>> +# Defaults to true.
>>>> +#
>>>> +# Since: 5.0
>>>> +##
>>>> +{ 'struct': 'BlockDirtyBitmapPopulate',
>>>> + 'base': 'BlockDirtyBitmap',
>>>> + 'data': { 'job-id': 'str',
>>>> + 'pattern': 'BitmapPattern',
>>>> + '*on-error': 'BlockdevOnError',
>>>> + '*auto-finalize': 'bool',
>>>> + '*auto-dismiss': 'bool' } }
>>>> +
>>>> ##
>>>> # @BlockDirtyBitmapSha256:
>>>> #
>>>> diff --git a/qapi/job.json b/qapi/job.json
>>>> index 5e658281f5..5f496d4630 100644
>>>> --- a/qapi/job.json
>>>> +++ b/qapi/job.json
>>>> @@ -22,7 +22,7 @@
>>>> # Since: 1.7
>>>> ##
>>>> { 'enum': 'JobType',
>>>> - 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
>>>> + 'data': ['commit', 'stream', 'mirror', 'backup', 'create',
>>>> 'bitmap-populate'] }
>>>> ##
>>>> # @JobStatus:
>>>> diff --git a/include/block/block_int.h b/include/block/block_int.h
>>>> index 6f9fd5e20e..a5884b597e 100644
>>>> --- a/include/block/block_int.h
>>>> +++ b/include/block/block_int.h
>>>> @@ -1215,6 +1215,27 @@ BlockJob *backup_job_create(const char *job_id,
>>>> BlockDriverState *bs,
>>>> BlockCompletionFunc *cb, void *opaque,
>>>> JobTxn *txn, Error **errp);
>>>> +/*
>>>> + * bitpop_job_create: Create a new bitmap population job.
>>>> + *
>>>> + * @job_id: The id of the newly-created job.
>>>> + * @bs: Block device associated with the @target_bitmap.
>>>> + * @target_bitmap: The bitmap to populate.
>>>> + * @on_error: What to do if an error on @bs is encountered.
>>>> + * @creation_flags: Flags that control the behavior of the Job
>>>> lifetime.
>>>> + * See @BlockJobCreateFlags
>>>> + * @cb: Completion function for the job.
>>>> + * @opaque: Opaque pointer value passed to @cb.
>>>> + * @txn: Transaction that this job is part of (may be NULL).
>>>> + */
>>>> +BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
>>>> + BdrvDirtyBitmap *target_bitmap,
>>>> + BitmapPattern pattern,
>>>> + BlockdevOnError on_error,
>>>> + int creation_flags,
>>>> + BlockCompletionFunc *cb, void *opaque,
>>>> + JobTxn *txn, Error **errp);
>>>> +
>>>> void hmp_drive_add_node(Monitor *mon, const char *optstr);
>>>> BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
>>>> diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
>>>> new file mode 100644
>>>> index 0000000000..47d542dc12
>>>> --- /dev/null
>>>> +++ b/block/bitmap-alloc.c
>>>> @@ -0,0 +1,207 @@
>>>> +/*
>>>> + * Async Dirty Bitmap Populator
>>>> + *
>>>> + * Copyright (C) 2020 Red Hat, Inc.
>>>> + *
>>>> + * Authors:
>>>> + * John Snow <jsnow@redhat.com>
>>>> + *
>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>> later.
>>>> + * See the COPYING file in the top-level directory.
>>>> + *
>>>> + */
>>>> +
>>>> +#include "qemu/osdep.h"
>>>> +
>>>> +#include "trace.h"
>>>> +#include "block/block.h"
>>>> +#include "block/block_int.h"
>>>> +#include "block/blockjob_int.h"
>>>> +#include "block/block_backup.h"
>>>> +#include "block/block-copy.h"
>>>
>>> I hope, not all includes are needed :)
>>
>> Whoops, no, of course not. I copied the skeleton from backup, as you can
>> tell ;)
>>
>>>
>>>> +#include "qapi/error.h"
>>>> +#include "qapi/qmp/qerror.h"
>>>> +#include "qemu/ratelimit.h"
>>>> +#include "qemu/cutils.h"
>>>> +#include "sysemu/block-backend.h"
>>>> +#include "qemu/bitmap.h"
>>>> +#include "qemu/error-report.h"
>>>> +
>>>> +typedef struct BitpopBlockJob {
>>>> + BlockJob common;
>>>> + BlockDriverState *bs;
>>>> + BdrvDirtyBitmap *target_bitmap;
>>>> + BdrvDirtyBitmap *new_bitmap;
>>>> + BlockdevOnError on_error;
>>>> + uint64_t len;
>>>> +} BitpopBlockJob;
>>>> +
>>>> +static const BlockJobDriver bitpop_job_driver;
>>>> +
>>>> +static void bitpop_commit(Job *job)
>>>> +{
>>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>>> +
>>>> + bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
>>>> + NULL, true);
>>>
>>> Hmm, so you populate new_bitmap, and then merge to target. Why can't we
>>> work
>>> directly with target bitmap? The most probable case is that libvirt will
>>> create bitmap specifically to use as target in this operation, or not?
>>>
>>
>> Most likely case, yes. Odds are very good it will be a brand new bitmap.
>>
>> However, we already have a creation command -- I didn't want to make a
>> second job-version of the command and then maintain two interfaces, so I
>> made it a "merge into existing" style command instead.
>>
>>> Hmm, just to make it possible to cancel the job and keep the target
>>> bitmap in
>>> original state? Is it really needed? I think on failure target bitmap
>>> will be
>>> removed anyway..
>>>
>>
>> You caught me being *lazy*. I copy the bitmap so I can unconditionally
>> enable it to catch in-flight writes without having to create block graph
>> modifications.
>>
>> But, yes, to undo changes if we cancel.
>>
>> I didn't want to make a job that was not able to be canceled. The
>> alternative is pursuing the design where we allow new bitmaps only --
>> because then on cancel we can just delete them.
>
> On backup job (and any other) we can't rollback target changes on cancel.
> So, I think it would be OK to take same semantics for the new job,
> keeping in
> mind that it would be most probable usage case and no sense in creating
> additional bitmaps. And if caller needs to use existent non-empty bitmap as
> target and wants correct cancel, it always can create additional bitmap by
> itself and then merge it to actual target.
>
While backup can't roll back data changes, it does avoid modifying
bitmaps if it didn't succeed. This is a bitmap-centric job and we have
the capability to do a complete rollback.
i.e. I am thinking more about consistency with bitmap behavior than I am
consistency with job behavior.
That this job becomes the only one to be able to be *fully* reversed on
cancel is unique to jobs -- but is shared by all transactionable bitmap
commands.
> And why new? It's up to user, what to use as target. And user knows,
> that on
> failure or cancel target becomes invalid and will workaround this if
> needed.
>
I think you're advocating for writing directly into the bitmap and just
accepting that it's trashed afterwards, but I don't like this behavior
because it limits what we can do with this job in the future.
Is this just a matter of taste? Since every other bitmap command offers
some kind of rollback I felt more comfortable doing the same here.
I'm worried that if I decide to treat the result bitmap as disposable,
that I will need to provide greater atomicity for the allocation status
read. At the moment I rely on the temporary bitmap being *enabled* to
make sure that if any new writes happen in regions I have already read
that they are reflected in the bitmap anyway.
If the user passes a disabled bitmap, that guarantee is lost -- we lose
point in time semantics entirely. Writes that occur further in the file
are captured, but ones that occur earlier are missed.
If I make a temporary enabled bitmap, I catch everything.
I could prohibit *disabled* bitmaps from being passed in to this job,
but that again makes it of a more limited use for other patterns in the
future.
So I guess I would prefer to just leave this as-is for now if it's not
that harmful.
>>
>>>> +}
>>>> +
>>>> +/* no abort needed; just clean without committing. */
>>>> +
>>>> +static void bitpop_clean(Job *job)
>>>> +{
>>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>>> +
>>>> + bdrv_release_dirty_bitmap(s->new_bitmap);
>>>> + bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
>>>> +}
>>>> +
>>>> +static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int
>>>> error)
>>>> +{
>>>> + return block_job_error_action(&job->common, job->on_error, true,
>>>> error);
>>>> +}
>>>> +
>>>> +static bool coroutine_fn yield_and_check(Job *job)
>>>> +{
>>>> + if (job_is_cancelled(job)) {
>>>> + return true;
>>>> + }
>>>> +
>>>> + job_sleep_ns(job, 0);
>>>> +
>>>> + if (job_is_cancelled(job)) {
>>>> + return true;
>>>> + }
>>>> +
>>>> + return false;
>>>> +}
>>>> +
>>>> +static int coroutine_fn bitpop_run(Job *job, Error **errp)
>>>> +{
>>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>>> + int ret = 0;
>>>> + int64_t offset;
>>>> + int64_t count;
>>>> + int64_t bytes;
>>>> +
>>>> + for (offset = 0; offset < s->len; ) {
>>>> + if (yield_and_check(job)) {
>>>> + ret = -ECANCELED;
>>>> + break;
>>>> + }
>>>> +
>>>> + bytes = s->len - offset;
>>>> + ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
>>>> + if (ret < 0) {
>>>> + if (bitpop_error_action(s, -ret) ==
>>>> BLOCK_ERROR_ACTION_REPORT) {
>>>> + break;
>>>> + }
>>>> + continue;
>>>> + }
>>>> +
>>>> + if (!count) {
>>>> + ret = 0;
>>>
>>> Hmm, I think it's impossible case.. If so, better to make an assertion
>>> or ignore..
>>>
>>
>> OK, agreed.
>>
>>>> + break;
>>>> + }
>>>> +
>>>> + if (ret) {
>>>> + bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
>>>> + ret = 0;
>>>> + }
>>>> +
>>>> + job_progress_update(job, count);
>>>> + offset += count;
>>>> + }
>>>> +
>>>> + return ret;
>>>> +}
>>>> +
>>>> +static const BlockJobDriver bitpop_job_driver = {
>>>> + .job_driver = {
>>>> + .instance_size = sizeof(BitpopBlockJob),
>>>> + .job_type = JOB_TYPE_BITMAP_POPULATE,
>>>> + .free = block_job_free,
>>>> + .user_resume = block_job_user_resume,
>>>> + .run = bitpop_run,
>>>> + .commit = bitpop_commit,
>>>> + .clean = bitpop_clean,
>>>> + }
>>>> +};
>>>> +
>>>> +
>>>> +BlockJob *bitpop_job_create(
>>>> + const char *job_id,
>>>> + BlockDriverState *bs,
>>>> + BdrvDirtyBitmap *target_bitmap,
>>>> + BitmapPattern pattern,
>>>> + BlockdevOnError on_error,
>>>> + int creation_flags,
>>>> + BlockCompletionFunc *cb,
>>>> + void *opaque,
>>>> + JobTxn *txn,
>>>> + Error **errp)
>>>> +{
>>>> + int64_t len;
>>>> + BitpopBlockJob *job = NULL;
>>>> + int64_t cluster_size;
>>>> + BdrvDirtyBitmap *new_bitmap = NULL;
>>>> +
>>>> + assert(bs);
>>>> + assert(target_bitmap);
>>>> +
>>>> + if (!bdrv_is_inserted(bs)) {
>>>> + error_setg(errp, "Device is not inserted: %s",
>>>> + bdrv_get_device_name(bs));
>>>> + return NULL;
>>>> + }
>>>
>>> Why this?
>>>
>>
>> I assumed there was nothing to read the allocation map *of* if there
>> wasn't a media present.
>>
>> Am I mistaken?
>
> is_inserted checks existing of bs->drv, but bitmap operations actually
> doesn't need any drv.. Hmm. I'm not against this check anyway.
>
Maybe it would be clearer to say this:
if (pattern = BITMAP_PATTERN_ALLOCATION_TOP) {
if (!bdrv_is_inserted(bs)) {
...
}
} else {
...
}
We need to make sure bs->drv exists for the allocation map reading --
not for every theoretical pattern, but for this one we do.
>>
>>>> +
>>>> + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
>>>> + return NULL;
>>>> + }
>>>
>>> and this?
>>>
>>
>> Copy-paste: I don't understand if I want a new op blocker, to re-use an
>> op-blocker, or to have no op blocker.
>>
>> Genuinely I have no idea. I should left a review comment here, I forgot
>> about this part, sorry.
>
> I'm for no op blocker. As I understand, op-blockers are old and should be
> replaced by permissions and frozen children. So, if we don't know, do we
> need any blocking here, better go on without it. Also, op-blockers can't
> block forbidden usage through filters anyway.
>
Sounds good to me, honestly.
>>
>>>> +
>>>> + if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT,
>>>> errp)) {
>>>> + return NULL;
>>>> + }
>>>> +
>>>> + if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
>>>> + error_setg(errp, "Unrecognized bitmap pattern");
>>>> + return NULL;
>>>> + }
>>>> +
>>>> + len = bdrv_getlength(bs);
>>>> + if (len < 0) {
>>>> + error_setg_errno(errp, -len, "unable to get length for '%s'",
>>>> + bdrv_get_device_name(bs));
>>>> + return NULL;
>>>> + }
>>>> +
>>>> + /* NB: new bitmap is anonymous and enabled */
>>>> + cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
>>>> + new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL,
>>>> errp);
>>>> + if (!new_bitmap) {
>>>> + return NULL;
>>>> + }
>>>> +
>>>> + /* Take ownership; we reserve the right to write into this
>>>> on-commit. */
>>>> + bdrv_dirty_bitmap_set_busy(target_bitmap, true);
>>>
>>> Honestly, I still have bad understanding about how should we use dirty
>>> bitmap mutex,
>>> but note that bdrv_dirty_bitmap_set_busy locks the mutex. And it is (may
>>> be) possible,
>>> that busy status of the bitmap is changed after bdrv_dirty_bitmap_check
>>> but before
>>> bdrv_dirty_bitmap_set_busy. So, more correct would be do both operation
>>> under one
>>> critical section. Still, I don't know is the situation possible.
>>>
>>
>> Aren't we under the BQL here? Can we be pre-empted? :(
>
> Seems we are. But, as it's said above dirty_bitmap_mutex declaration:
>
> /* Writing to the list requires the BQL _and_ the dirty_bitmap_mutex.
> * Reading from the list can be done with either the BQL or the
> * dirty_bitmap_mutex. Modifying a bitmap only requires
> * dirty_bitmap_mutex. */
>
> It means, that another thread may modify bitmap (for example its 'busy'
> field)
> taking only dirty_bitmap_mutex, which will lead to the case I described.
>
Alright, I'll add the locking change. Atomic check-and-lock for the busy
bit sounds like a good model anyway.
>>
>>>> +
>>>> + job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
>>>> + BLK_PERM_CONSISTENT_READ,
>>>
>>> Do we need it? We are not going to read..
>>>
>>
>> Copy-paste / leftover from an earlier draft where I was trying to
>> achieve atomicity. It can be removed if we don't want the stricter
>> atomicity.
>>
I guess I really don't need READ here. I guess ~RESIZE is the only
important one.
>>>> + BLK_PERM_ALL & ~BLK_PERM_RESIZE,
>>>> + 0, creation_flags,
>>>> + cb, opaque, errp);
>>>> + if (!job) {
>>>> + bdrv_dirty_bitmap_set_busy(target_bitmap, false);
>>>> + bdrv_release_dirty_bitmap(new_bitmap);
>>>> + return NULL;
>>>> + }
>>>> +
>>>> + job->bs = bs;
>>>> + job->on_error = on_error;
>>>> + job->target_bitmap = target_bitmap;
>>>> + job->new_bitmap = new_bitmap;
>>>> + job->len = len;
>>>> + job_progress_set_remaining(&job->common.job, job->len);
>>>> +
>>>> + return &job->common;
>>>> +}
>>>> diff --git a/blockjob.c b/blockjob.c
>>>> index 5d63b1e89d..7e450372bd 100644
>>>> --- a/blockjob.c
>>>> +++ b/blockjob.c
>>>> @@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
>>>> return job_type(job) == JOB_TYPE_BACKUP ||
>>>> job_type(job) == JOB_TYPE_COMMIT ||
>>>> job_type(job) == JOB_TYPE_MIRROR ||
>>>> - job_type(job) == JOB_TYPE_STREAM;
>>>> + job_type(job) == JOB_TYPE_STREAM ||
>>>> + job_type(job) == JOB_TYPE_BITMAP_POPULATE;
>>>> }
>>>> BlockJob *block_job_next(BlockJob *bjob)
>>>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>>>> index 3bcb35c81d..f3cfc89d90 100644
>>>> --- a/block/Makefile.objs
>>>> +++ b/block/Makefile.objs
>>>> @@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
>>>> block-obj-y += accounting.o dirty-bitmap.o
>>>> block-obj-y += write-threshold.o
>>>> block-obj-y += backup.o
>>>> +block-obj-y += bitmap-alloc.o
>>>> block-obj-$(CONFIG_REPLICATION) += replication.o
>>>> block-obj-y += throttle.o copy-on-read.o
>>>> block-obj-y += block-copy.o
>>>>
>>>
>>>
>>
Thanks!
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/6] block: add bitmap-populate job
2020-02-26 19:11 ` John Snow
@ 2020-02-27 6:11 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:39 ` John Snow
0 siblings, 1 reply; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-27 6:11 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
26.02.2020 22:11, John Snow wrote:
>
>
> On 2/26/20 12:07 AM, Vladimir Sementsov-Ogievskiy wrote:
>> 25.02.2020 23:41, John Snow wrote:
>>>
>>>
>>> On 2/25/20 11:04 AM, Vladimir Sementsov-Ogievskiy wrote:
>>>> 25.02.2020 3:56, John Snow wrote:
>>>>> This job copies the allocation map into a bitmap. It's a job because
>>>>> there's no guarantee that allocation interrogation will be quick (or
>>>>> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
>>>>>
>>>>> It was designed with different possible population patterns in mind,
>>>>> but only top layer allocation was implemented for now.
>>>>>
>>>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>>>> ---
>>>>> qapi/block-core.json | 48 +++++++++
>>>>> qapi/job.json | 2 +-
>>>>> include/block/block_int.h | 21 ++++
>>>>> block/bitmap-alloc.c | 207
>>>>> ++++++++++++++++++++++++++++++++++++++
>>>>> blockjob.c | 3 +-
>>>>> block/Makefile.objs | 1 +
>>>>> 6 files changed, 280 insertions(+), 2 deletions(-)
>>>>> create mode 100644 block/bitmap-alloc.c
>>>>>
>>>>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>>>>> index 85e27bb61f..df1797681a 100644
>>>>> --- a/qapi/block-core.json
>>>>> +++ b/qapi/block-core.json
>>>>> @@ -2245,6 +2245,54 @@
>>>>> { 'command': 'block-dirty-bitmap-merge',
>>>>> 'data': 'BlockDirtyBitmapMerge' }
>>>>> +##
>>>>> +# @BitmapPattern:
>>>>> +#
>>>>> +# An enumeration of possible patterns that can be written into a
>>>>> bitmap.
>>>>> +#
>>>>> +# @allocation-top: The allocation status of the top layer
>>>>> +# of the attached storage node.
>>>>> +#
>>>>> +# Since: 5.0
>>>>> +##
>>>>> +{ 'enum': 'BitmapPattern',
>>>>> + 'data': ['allocation-top'] }
>>>>> +
>>>>> +##
>>>>> +# @BlockDirtyBitmapPopulate:
>>>>> +#
>>>>> +# @job-id: identifier for the newly-created block job.
>>>>> +#
>>>>> +# @pattern: What pattern should be written into the bitmap?
>>>>> +#
>>>>> +# @on-error: the action to take if an error is encountered on a
>>>>> bitmap's
>>>>> +# attached node, default 'report'.
>>>>> +# 'stop' and 'enospc' can only be used if the block device
>>>>> supports
>>>>> +# io-status (see BlockInfo).
>>>>> +#
>>>>> +# @auto-finalize: When false, this job will wait in a PENDING state
>>>>> after it has
>>>>> +# finished its work, waiting for @block-job-finalize
>>>>> before
>>>>> +# making any block graph changes.
>>>>
>>>> sounds a bit strange in context of bitmap-population job
>>>>
>>>
>>> Yeah, you're right. Copy-pasted for "consistency".
>>>
>>>>> +# When true, this job will automatically
>>>>> +# perform its abort or commit actions.
>>>>> +# Defaults to true.
>>>>> +#
>>>>> +# @auto-dismiss: When false, this job will wait in a CONCLUDED state
>>>>> after it
>>>>> +# has completely ceased all work, and awaits
>>>>> @block-job-dismiss.
>>>>> +# When true, this job will automatically disappear
>>>>> from the query
>>>>> +# list without user intervention.
>>>>> +# Defaults to true.
>>>>> +#
>>>>> +# Since: 5.0
>>>>> +##
>>>>> +{ 'struct': 'BlockDirtyBitmapPopulate',
>>>>> + 'base': 'BlockDirtyBitmap',
>>>>> + 'data': { 'job-id': 'str',
>>>>> + 'pattern': 'BitmapPattern',
>>>>> + '*on-error': 'BlockdevOnError',
>>>>> + '*auto-finalize': 'bool',
>>>>> + '*auto-dismiss': 'bool' } }
>>>>> +
>>>>> ##
>>>>> # @BlockDirtyBitmapSha256:
>>>>> #
>>>>> diff --git a/qapi/job.json b/qapi/job.json
>>>>> index 5e658281f5..5f496d4630 100644
>>>>> --- a/qapi/job.json
>>>>> +++ b/qapi/job.json
>>>>> @@ -22,7 +22,7 @@
>>>>> # Since: 1.7
>>>>> ##
>>>>> { 'enum': 'JobType',
>>>>> - 'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
>>>>> + 'data': ['commit', 'stream', 'mirror', 'backup', 'create',
>>>>> 'bitmap-populate'] }
>>>>> ##
>>>>> # @JobStatus:
>>>>> diff --git a/include/block/block_int.h b/include/block/block_int.h
>>>>> index 6f9fd5e20e..a5884b597e 100644
>>>>> --- a/include/block/block_int.h
>>>>> +++ b/include/block/block_int.h
>>>>> @@ -1215,6 +1215,27 @@ BlockJob *backup_job_create(const char *job_id,
>>>>> BlockDriverState *bs,
>>>>> BlockCompletionFunc *cb, void *opaque,
>>>>> JobTxn *txn, Error **errp);
>>>>> +/*
>>>>> + * bitpop_job_create: Create a new bitmap population job.
>>>>> + *
>>>>> + * @job_id: The id of the newly-created job.
>>>>> + * @bs: Block device associated with the @target_bitmap.
>>>>> + * @target_bitmap: The bitmap to populate.
>>>>> + * @on_error: What to do if an error on @bs is encountered.
>>>>> + * @creation_flags: Flags that control the behavior of the Job
>>>>> lifetime.
>>>>> + * See @BlockJobCreateFlags
>>>>> + * @cb: Completion function for the job.
>>>>> + * @opaque: Opaque pointer value passed to @cb.
>>>>> + * @txn: Transaction that this job is part of (may be NULL).
>>>>> + */
>>>>> +BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
>>>>> + BdrvDirtyBitmap *target_bitmap,
>>>>> + BitmapPattern pattern,
>>>>> + BlockdevOnError on_error,
>>>>> + int creation_flags,
>>>>> + BlockCompletionFunc *cb, void *opaque,
>>>>> + JobTxn *txn, Error **errp);
>>>>> +
>>>>> void hmp_drive_add_node(Monitor *mon, const char *optstr);
>>>>> BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
>>>>> diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
>>>>> new file mode 100644
>>>>> index 0000000000..47d542dc12
>>>>> --- /dev/null
>>>>> +++ b/block/bitmap-alloc.c
>>>>> @@ -0,0 +1,207 @@
>>>>> +/*
>>>>> + * Async Dirty Bitmap Populator
>>>>> + *
>>>>> + * Copyright (C) 2020 Red Hat, Inc.
>>>>> + *
>>>>> + * Authors:
>>>>> + * John Snow <jsnow@redhat.com>
>>>>> + *
>>>>> + * This work is licensed under the terms of the GNU GPL, version 2 or
>>>>> later.
>>>>> + * See the COPYING file in the top-level directory.
>>>>> + *
>>>>> + */
>>>>> +
>>>>> +#include "qemu/osdep.h"
>>>>> +
>>>>> +#include "trace.h"
>>>>> +#include "block/block.h"
>>>>> +#include "block/block_int.h"
>>>>> +#include "block/blockjob_int.h"
>>>>> +#include "block/block_backup.h"
>>>>> +#include "block/block-copy.h"
>>>>
>>>> I hope, not all includes are needed :)
>>>
>>> Whoops, no, of course not. I copied the skeleton from backup, as you can
>>> tell ;)
>>>
>>>>
>>>>> +#include "qapi/error.h"
>>>>> +#include "qapi/qmp/qerror.h"
>>>>> +#include "qemu/ratelimit.h"
>>>>> +#include "qemu/cutils.h"
>>>>> +#include "sysemu/block-backend.h"
>>>>> +#include "qemu/bitmap.h"
>>>>> +#include "qemu/error-report.h"
>>>>> +
>>>>> +typedef struct BitpopBlockJob {
>>>>> + BlockJob common;
>>>>> + BlockDriverState *bs;
>>>>> + BdrvDirtyBitmap *target_bitmap;
>>>>> + BdrvDirtyBitmap *new_bitmap;
>>>>> + BlockdevOnError on_error;
>>>>> + uint64_t len;
>>>>> +} BitpopBlockJob;
>>>>> +
>>>>> +static const BlockJobDriver bitpop_job_driver;
>>>>> +
>>>>> +static void bitpop_commit(Job *job)
>>>>> +{
>>>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>>>> +
>>>>> + bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
>>>>> + NULL, true);
>>>>
>>>> Hmm, so you populate new_bitmap, and then merge to target. Why can't we
>>>> work
>>>> directly with target bitmap? The most probable case is that libvirt will
>>>> create bitmap specifically to use as target in this operation, or not?
>>>>
>>>
>>> Most likely case, yes. Odds are very good it will be a brand new bitmap.
>>>
>>> However, we already have a creation command -- I didn't want to make a
>>> second job-version of the command and then maintain two interfaces, so I
>>> made it a "merge into existing" style command instead.
>>>
>>>> Hmm, just to make it possible to cancel the job and keep the target
>>>> bitmap in
>>>> original state? Is it really needed? I think on failure target bitmap
>>>> will be
>>>> removed anyway..
>>>>
>>>
>>> You caught me being *lazy*. I copy the bitmap so I can unconditionally
>>> enable it to catch in-flight writes without having to create block graph
>>> modifications.
>>>
>>> But, yes, to undo changes if we cancel.
>>>
>>> I didn't want to make a job that was not able to be canceled. The
>>> alternative is pursuing the design where we allow new bitmaps only --
>>> because then on cancel we can just delete them.
>>
>> On backup job (and any other) we can't rollback target changes on cancel.
>> So, I think it would be OK to take same semantics for the new job,
>> keeping in
>> mind that it would be most probable usage case and no sense in creating
>> additional bitmaps. And if caller needs to use existent non-empty bitmap as
>> target and wants correct cancel, it always can create additional bitmap by
>> itself and then merge it to actual target.
>>
>
> While backup can't roll back data changes, it does avoid modifying
> bitmaps if it didn't succeed. This is a bitmap-centric job and we have
> the capability to do a complete rollback.
>
> i.e. I am thinking more about consistency with bitmap behavior than I am
> consistency with job behavior.
>
> That this job becomes the only one to be able to be *fully* reversed on
> cancel is unique to jobs -- but is shared by all transactionable bitmap
> commands.
Hmm, yes.. But it will be useless without a possibility to put job-completion
into transaction. Still we'll need such thing anyway at some point.
>
>> And why new? It's up to user, what to use as target. And user knows,
>> that on
>> failure or cancel target becomes invalid and will workaround this if
>> needed.
>>
>
> I think you're advocating for writing directly into the bitmap and just
> accepting that it's trashed afterwards, but I don't like this behavior
> because it limits what we can do with this job in the future.
>
> Is this just a matter of taste? Since every other bitmap command offers
> some kind of rollback I felt more comfortable doing the same here.
OK
>
> I'm worried that if I decide to treat the result bitmap as disposable,
> that I will need to provide greater atomicity for the allocation status
> read. At the moment I rely on the temporary bitmap being *enabled* to
> make sure that if any new writes happen in regions I have already read
> that they are reflected in the bitmap anyway.
>
> If the user passes a disabled bitmap, that guarantee is lost -- we lose
> point in time semantics entirely. Writes that occur further in the file
> are captured, but ones that occur earlier are missed.
Still, if user pass disabled bitmap, it will be invalid immediately after
job finish.
>
> If I make a temporary enabled bitmap, I catch everything.
>
> I could prohibit *disabled* bitmaps from being passed in to this job,
> but that again makes it of a more limited use for other patterns in the
> future.
>
> So I guess I would prefer to just leave this as-is for now if it's not
> that harmful.
OK, I agreed, because of semantics of other bitmap-related commands.
>
>>>
>>>>> +}
>>>>> +
>>>>> +/* no abort needed; just clean without committing. */
>>>>> +
>>>>> +static void bitpop_clean(Job *job)
>>>>> +{
>>>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>>>> +
>>>>> + bdrv_release_dirty_bitmap(s->new_bitmap);
>>>>> + bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
>>>>> +}
>>>>> +
>>>>> +static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int
>>>>> error)
>>>>> +{
>>>>> + return block_job_error_action(&job->common, job->on_error, true,
>>>>> error);
>>>>> +}
>>>>> +
>>>>> +static bool coroutine_fn yield_and_check(Job *job)
>>>>> +{
>>>>> + if (job_is_cancelled(job)) {
>>>>> + return true;
>>>>> + }
>>>>> +
>>>>> + job_sleep_ns(job, 0);
>>>>> +
>>>>> + if (job_is_cancelled(job)) {
>>>>> + return true;
>>>>> + }
>>>>> +
>>>>> + return false;
>>>>> +}
>>>>> +
>>>>> +static int coroutine_fn bitpop_run(Job *job, Error **errp)
>>>>> +{
>>>>> + BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
>>>>> + int ret = 0;
>>>>> + int64_t offset;
>>>>> + int64_t count;
>>>>> + int64_t bytes;
>>>>> +
>>>>> + for (offset = 0; offset < s->len; ) {
>>>>> + if (yield_and_check(job)) {
>>>>> + ret = -ECANCELED;
>>>>> + break;
>>>>> + }
>>>>> +
>>>>> + bytes = s->len - offset;
>>>>> + ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
>>>>> + if (ret < 0) {
>>>>> + if (bitpop_error_action(s, -ret) ==
>>>>> BLOCK_ERROR_ACTION_REPORT) {
>>>>> + break;
>>>>> + }
>>>>> + continue;
>>>>> + }
>>>>> +
>>>>> + if (!count) {
>>>>> + ret = 0;
>>>>
>>>> Hmm, I think it's impossible case.. If so, better to make an assertion
>>>> or ignore..
>>>>
>>>
>>> OK, agreed.
>>>
>>>>> + break;
>>>>> + }
>>>>> +
>>>>> + if (ret) {
>>>>> + bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
>>>>> + ret = 0;
>>>>> + }
>>>>> +
>>>>> + job_progress_update(job, count);
>>>>> + offset += count;
>>>>> + }
>>>>> +
>>>>> + return ret;
>>>>> +}
>>>>> +
>>>>> +static const BlockJobDriver bitpop_job_driver = {
>>>>> + .job_driver = {
>>>>> + .instance_size = sizeof(BitpopBlockJob),
>>>>> + .job_type = JOB_TYPE_BITMAP_POPULATE,
>>>>> + .free = block_job_free,
>>>>> + .user_resume = block_job_user_resume,
>>>>> + .run = bitpop_run,
>>>>> + .commit = bitpop_commit,
>>>>> + .clean = bitpop_clean,
>>>>> + }
>>>>> +};
>>>>> +
>>>>> +
>>>>> +BlockJob *bitpop_job_create(
>>>>> + const char *job_id,
>>>>> + BlockDriverState *bs,
>>>>> + BdrvDirtyBitmap *target_bitmap,
>>>>> + BitmapPattern pattern,
>>>>> + BlockdevOnError on_error,
>>>>> + int creation_flags,
>>>>> + BlockCompletionFunc *cb,
>>>>> + void *opaque,
>>>>> + JobTxn *txn,
>>>>> + Error **errp)
>>>>> +{
>>>>> + int64_t len;
>>>>> + BitpopBlockJob *job = NULL;
>>>>> + int64_t cluster_size;
>>>>> + BdrvDirtyBitmap *new_bitmap = NULL;
>>>>> +
>>>>> + assert(bs);
>>>>> + assert(target_bitmap);
>>>>> +
>>>>> + if (!bdrv_is_inserted(bs)) {
>>>>> + error_setg(errp, "Device is not inserted: %s",
>>>>> + bdrv_get_device_name(bs));
>>>>> + return NULL;
>>>>> + }
>>>>
>>>> Why this?
>>>>
>>>
>>> I assumed there was nothing to read the allocation map *of* if there
>>> wasn't a media present.
>>>
>>> Am I mistaken?
>>
>> is_inserted checks existing of bs->drv, but bitmap operations actually
>> doesn't need any drv.. Hmm. I'm not against this check anyway.
>>
>
> Maybe it would be clearer to say this:
>
> if (pattern = BITMAP_PATTERN_ALLOCATION_TOP) {
> if (!bdrv_is_inserted(bs)) {
> ...
> }
> } else {
> ...
> }
>
> We need to make sure bs->drv exists for the allocation map reading --
> not for every theoretical pattern, but for this one we do.
We can leave this as is now, I don't want to care much. Still I just doubt that
this check is needed in any job. What are the cases when we have bs without drv?
Corrupted qcow2? And where is guarantee that we will not lose drv during the job?
>
>>>
>>>>> +
>>>>> + if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
>>>>> + return NULL;
>>>>> + }
>>>>
>>>> and this?
>>>>
>>>
>>> Copy-paste: I don't understand if I want a new op blocker, to re-use an
>>> op-blocker, or to have no op blocker.
>>>
>>> Genuinely I have no idea. I should left a review comment here, I forgot
>>> about this part, sorry.
>>
>> I'm for no op blocker. As I understand, op-blockers are old and should be
>> replaced by permissions and frozen children. So, if we don't know, do we
>> need any blocking here, better go on without it. Also, op-blockers can't
>> block forbidden usage through filters anyway.
>>
>
> Sounds good to me, honestly.
>
>>>
>>>>> +
>>>>> + if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT,
>>>>> errp)) {
>>>>> + return NULL;
>>>>> + }
>>>>> +
>>>>> + if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
>>>>> + error_setg(errp, "Unrecognized bitmap pattern");
>>>>> + return NULL;
>>>>> + }
>>>>> +
>>>>> + len = bdrv_getlength(bs);
>>>>> + if (len < 0) {
>>>>> + error_setg_errno(errp, -len, "unable to get length for '%s'",
>>>>> + bdrv_get_device_name(bs));
>>>>> + return NULL;
>>>>> + }
>>>>> +
>>>>> + /* NB: new bitmap is anonymous and enabled */
>>>>> + cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
>>>>> + new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL,
>>>>> errp);
>>>>> + if (!new_bitmap) {
>>>>> + return NULL;
>>>>> + }
>>>>> +
>>>>> + /* Take ownership; we reserve the right to write into this
>>>>> on-commit. */
>>>>> + bdrv_dirty_bitmap_set_busy(target_bitmap, true);
>>>>
>>>> Honestly, I still have bad understanding about how should we use dirty
>>>> bitmap mutex,
>>>> but note that bdrv_dirty_bitmap_set_busy locks the mutex. And it is (may
>>>> be) possible,
>>>> that busy status of the bitmap is changed after bdrv_dirty_bitmap_check
>>>> but before
>>>> bdrv_dirty_bitmap_set_busy. So, more correct would be do both operation
>>>> under one
>>>> critical section. Still, I don't know is the situation possible.
>>>>
>>>
>>> Aren't we under the BQL here? Can we be pre-empted? :(
>>
>> Seems we are. But, as it's said above dirty_bitmap_mutex declaration:
>>
>> /* Writing to the list requires the BQL _and_ the dirty_bitmap_mutex.
>> * Reading from the list can be done with either the BQL or the
>> * dirty_bitmap_mutex. Modifying a bitmap only requires
>> * dirty_bitmap_mutex. */
>>
>> It means, that another thread may modify bitmap (for example its 'busy'
>> field)
>> taking only dirty_bitmap_mutex, which will lead to the case I described.
>>
>
> Alright, I'll add the locking change. Atomic check-and-lock for the busy
> bit sounds like a good model anyway.
>
>>>
>>>>> +
>>>>> + job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
>>>>> + BLK_PERM_CONSISTENT_READ,
>>>>
>>>> Do we need it? We are not going to read..
>>>>
>>>
>>> Copy-paste / leftover from an earlier draft where I was trying to
>>> achieve atomicity. It can be removed if we don't want the stricter
>>> atomicity.
>>>
>
> I guess I really don't need READ here. I guess ~RESIZE is the only
> important one.
Agree
>
>>>>> + BLK_PERM_ALL & ~BLK_PERM_RESIZE,
>>>>> + 0, creation_flags,
>>>>> + cb, opaque, errp);
>>>>> + if (!job) {
>>>>> + bdrv_dirty_bitmap_set_busy(target_bitmap, false);
>>>>> + bdrv_release_dirty_bitmap(new_bitmap);
>>>>> + return NULL;
>>>>> + }
>>>>> +
>>>>> + job->bs = bs;
>>>>> + job->on_error = on_error;
>>>>> + job->target_bitmap = target_bitmap;
>>>>> + job->new_bitmap = new_bitmap;
>>>>> + job->len = len;
>>>>> + job_progress_set_remaining(&job->common.job, job->len);
>>>>> +
>>>>> + return &job->common;
>>>>> +}
>>>>> diff --git a/blockjob.c b/blockjob.c
>>>>> index 5d63b1e89d..7e450372bd 100644
>>>>> --- a/blockjob.c
>>>>> +++ b/blockjob.c
>>>>> @@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
>>>>> return job_type(job) == JOB_TYPE_BACKUP ||
>>>>> job_type(job) == JOB_TYPE_COMMIT ||
>>>>> job_type(job) == JOB_TYPE_MIRROR ||
>>>>> - job_type(job) == JOB_TYPE_STREAM;
>>>>> + job_type(job) == JOB_TYPE_STREAM ||
>>>>> + job_type(job) == JOB_TYPE_BITMAP_POPULATE;
>>>>> }
>>>>> BlockJob *block_job_next(BlockJob *bjob)
>>>>> diff --git a/block/Makefile.objs b/block/Makefile.objs
>>>>> index 3bcb35c81d..f3cfc89d90 100644
>>>>> --- a/block/Makefile.objs
>>>>> +++ b/block/Makefile.objs
>>>>> @@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
>>>>> block-obj-y += accounting.o dirty-bitmap.o
>>>>> block-obj-y += write-threshold.o
>>>>> block-obj-y += backup.o
>>>>> +block-obj-y += bitmap-alloc.o
>>>>> block-obj-$(CONFIG_REPLICATION) += replication.o
>>>>> block-obj-y += throttle.o copy-on-read.o
>>>>> block-obj-y += block-copy.o
>>>>>
>>>>
>>>>
>>>
>
> Thanks!
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 2/6] qmp: expose block-dirty-bitmap-populate
2020-02-25 0:56 ` [PATCH 2/6] qmp: expose block-dirty-bitmap-populate John Snow
@ 2020-02-27 10:44 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:48 ` John Snow
0 siblings, 1 reply; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-27 10:44 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
25.02.2020 3:56, John Snow wrote:
> This is a new job-creating command.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> qapi/block-core.json | 18 ++++++++++
> qapi/transaction.json | 2 ++
> blockdev.c | 78 +++++++++++++++++++++++++++++++++++++++++++
> 3 files changed, 98 insertions(+)
>
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index df1797681a..a8be1fb36b 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2293,6 +2293,24 @@
> '*auto-finalize': 'bool',
> '*auto-dismiss': 'bool' } }
>
> +##
> +# @block-dirty-bitmap-populate:
> +#
> +# Creates a new job that writes a pattern into a dirty bitmap.
> +#
> +# Since: 5.0
> +#
> +# Example:
> +#
> +# -> { "execute": "block-dirty-bitmap-populate",
> +# "arguments": { "node": "drive0", "target": "bitmap0",
> +# "job-id": "job0", "pattern": "allocate-top" } }
> +# <- { "return": {} }
> +#
> +##
> +{ 'command': 'block-dirty-bitmap-populate', 'boxed': true,
> + 'data': 'BlockDirtyBitmapPopulate' }
> +
> ##
> # @BlockDirtyBitmapSha256:
> #
> diff --git a/qapi/transaction.json b/qapi/transaction.json
> index 04301f1be7..28521d5c7f 100644
> --- a/qapi/transaction.json
> +++ b/qapi/transaction.json
> @@ -50,6 +50,7 @@
> # - @block-dirty-bitmap-enable: since 4.0
> # - @block-dirty-bitmap-disable: since 4.0
> # - @block-dirty-bitmap-merge: since 4.0
> +# - @block-dirty-bitmap-populate: since 5.0
> # - @blockdev-backup: since 2.3
> # - @blockdev-snapshot: since 2.5
> # - @blockdev-snapshot-internal-sync: since 1.7
> @@ -67,6 +68,7 @@
> 'block-dirty-bitmap-enable': 'BlockDirtyBitmap',
> 'block-dirty-bitmap-disable': 'BlockDirtyBitmap',
> 'block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge',
> + 'block-dirty-bitmap-populate': 'BlockDirtyBitmapPopulate',
> 'blockdev-backup': 'BlockdevBackup',
> 'blockdev-snapshot': 'BlockdevSnapshot',
> 'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal',
> diff --git a/blockdev.c b/blockdev.c
> index 011dcfec27..33c0e35399 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -2314,6 +2314,67 @@ static void block_dirty_bitmap_remove_commit(BlkActionState *common)
> bdrv_release_dirty_bitmap(state->bitmap);
> }
>
> +static void block_dirty_bitmap_populate_prepare(BlkActionState *common, Error **errp)
over80 line (not the only)
> +{
> + BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common);
At first glance using *Backup* looks like a mistake. May be rename it, or at least add a comment.
> + BlockDirtyBitmapPopulate *bitpop;
> + BlockDriverState *bs;
> + AioContext *aio_context;
> + BdrvDirtyBitmap *bmap = NULL;
> + int job_flags = JOB_DEFAULT;
> +
> + assert(common->action->type == TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE);
> + bitpop = common->action->u.block_dirty_bitmap_populate.data;
> +
> + bs = bdrv_lookup_bs(bitpop->node, bitpop->node, errp);
> + if (!bs) {
> + return;
> + }
> +
> + aio_context = bdrv_get_aio_context(bs);
> + aio_context_acquire(aio_context);
> + state->bs = bs;
> +
> + bmap = bdrv_find_dirty_bitmap(bs, bitpop->name);
Could we use block_dirty_bitmap_lookup ?
> + if (!bmap) {
> + error_setg(errp, "Bitmap '%s' could not be found", bitpop->name);
> + return;
aio context lock leaked
> + }
> +
> + /* Paired with .clean() */
> + bdrv_drained_begin(state->bs);
> +
> + if (!bitpop->has_on_error) {
> + bitpop->on_error = BLOCKDEV_ON_ERROR_REPORT;
> + }
> + if (!bitpop->has_auto_finalize) {
> + bitpop->auto_finalize = true;
> + }
> + if (!bitpop->has_auto_dismiss) {
> + bitpop->auto_dismiss = true;
> + }
> +
> + if (!bitpop->auto_finalize) {
> + job_flags |= JOB_MANUAL_FINALIZE;
> + }
> + if (!bitpop->auto_dismiss) {
> + job_flags |= JOB_MANUAL_DISMISS;
> + }
> +
> + state->job = bitpop_job_create(
> + bitpop->job_id,
> + bs,
> + bmap,
> + bitpop->pattern,
> + bitpop->on_error,
> + job_flags,
> + NULL, NULL,
> + common->block_job_txn,
> + errp);
> +
> + aio_context_release(aio_context);
> +}
> +
> static void abort_prepare(BlkActionState *common, Error **errp)
> {
> error_setg(errp, "Transaction aborted using Abort action");
> @@ -2397,6 +2458,13 @@ static const BlkActionOps actions[] = {
> .commit = block_dirty_bitmap_remove_commit,
> .abort = block_dirty_bitmap_remove_abort,
> },
> + [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE] = {
> + .instance_size = sizeof(BlockdevBackupState),
> + .prepare = block_dirty_bitmap_populate_prepare,
> + .commit = blockdev_backup_commit,
> + .abort = blockdev_backup_abort,
> + .clean = blockdev_backup_clean,
> + },
> /* Where are transactions for MIRROR, COMMIT and STREAM?
> * Although these blockjobs use transaction callbacks like the backup job,
> * these jobs do not necessarily adhere to transaction semantics.
> @@ -3225,6 +3293,16 @@ void qmp_block_dirty_bitmap_merge(const char *node, const char *target,
> do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp);
> }
>
> +void qmp_block_dirty_bitmap_populate(BlockDirtyBitmapPopulate *bitpop,
> + Error **errp)
> +{
> + TransactionAction action = {
> + .type = TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE,
> + .u.block_dirty_bitmap_populate.data = bitpop,
> + };
> + blockdev_do_action(&action, errp);
> +}
> +
> BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,
> const char *name,
> Error **errp)
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/6] iotests: move bitmap helpers into their own file
2020-02-25 0:56 ` [PATCH 3/6] iotests: move bitmap helpers into their own file John Snow
@ 2020-02-27 10:54 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:55 ` John Snow
0 siblings, 1 reply; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-27 10:54 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
25.02.2020 3:56, John Snow wrote:
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 110 +---------------------------
> tests/qemu-iotests/bitmaps.py | 131 ++++++++++++++++++++++++++++++++++
> 2 files changed, 132 insertions(+), 109 deletions(-)
> create mode 100644 tests/qemu-iotests/bitmaps.py
>
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> index 004a433b8b..2a81f9e30c 100755
> --- a/tests/qemu-iotests/257
> +++ b/tests/qemu-iotests/257
> @@ -24,120 +24,12 @@ import os
>
> import iotests
> from iotests import log, qemu_img
> +from bitmaps import EmulatedBitmap, GROUPS
Clean code movement, no changes. If test passes, it should be correct :)
The only thing: I'd prefer not exporting global variables and use bitmaps.GROUPS instead (even then, it's not very good interface but..)
with or without it:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
> SIZE = 64 * 1024 * 1024
> GRANULARITY = 64 * 1024
>
>
> -class Pattern:
> - def __init__(self, byte, offset, size=GRANULARITY):
> - self.byte = byte
> - self.offset = offset
> - self.size = size
> -
> - def bits(self, granularity):
> - lower = self.offset // granularity
> - upper = (self.offset + self.size - 1) // granularity
> - return set(range(lower, upper + 1))
> -
> -
> -class PatternGroup:
> - """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
> - def __init__(self, patterns):
> - self.patterns = patterns
> -
> - def bits(self, granularity):
> - """Calculate the unique bits dirtied by this pattern grouping"""
> - res = set()
> - for pattern in self.patterns:
> - res |= pattern.bits(granularity)
> - return res
> -
> -
> -GROUPS = [
> - PatternGroup([
> - # Batch 0: 4 clusters
> - Pattern('0x49', 0x0000000),
> - Pattern('0x6c', 0x0100000), # 1M
> - Pattern('0x6f', 0x2000000), # 32M
> - Pattern('0x76', 0x3ff0000)]), # 64M - 64K
> - PatternGroup([
> - # Batch 1: 6 clusters (3 new)
> - Pattern('0x65', 0x0000000), # Full overwrite
> - Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
> - Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
> - Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
> - PatternGroup([
> - # Batch 2: 7 clusters (3 new)
> - Pattern('0x74', 0x0010000), # Adjacent-right
> - Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
> - Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
> - Pattern('0x67', 0x3fe0000,
> - 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
> - PatternGroup([
> - # Batch 3: 8 clusters (5 new)
> - # Carefully chosen such that nothing re-dirties the one cluster
> - # that copies out successfully before failure in Group #1.
> - Pattern('0xaa', 0x0010000,
> - 3*GRANULARITY), # Overwrite and 2x Adjacent-right
> - Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
> - Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
> - Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
> -]
> -
> -
> -class EmulatedBitmap:
> - def __init__(self, granularity=GRANULARITY):
> - self._bits = set()
> - self.granularity = granularity
> -
> - def dirty_bits(self, bits):
> - self._bits |= set(bits)
> -
> - def dirty_group(self, n):
> - self.dirty_bits(GROUPS[n].bits(self.granularity))
> -
> - def clear(self):
> - self._bits = set()
> -
> - def clear_bits(self, bits):
> - self._bits -= set(bits)
> -
> - def clear_bit(self, bit):
> - self.clear_bits({bit})
> -
> - def clear_group(self, n):
> - self.clear_bits(GROUPS[n].bits(self.granularity))
> -
> - @property
> - def first_bit(self):
> - return sorted(self.bits)[0]
> -
> - @property
> - def bits(self):
> - return self._bits
> -
> - @property
> - def count(self):
> - return len(self.bits)
> -
> - def compare(self, qmp_bitmap):
> - """
> - Print a nice human-readable message checking that a bitmap as reported
> - by the QMP interface has as many bits set as we expect it to.
> - """
> -
> - name = qmp_bitmap.get('name', '(anonymous)')
> - log("= Checking Bitmap {:s} =".format(name))
> -
> - want = self.count
> - have = qmp_bitmap['count'] // qmp_bitmap['granularity']
> -
> - log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
> - want, have, "OK!" if want == have else "ERROR!"))
> - log('')
> -
> -
> class Drive:
> """Represents, vaguely, a drive attached to a VM.
> Includes format, graph, and device information."""
> diff --git a/tests/qemu-iotests/bitmaps.py b/tests/qemu-iotests/bitmaps.py
> new file mode 100644
> index 0000000000..522fc25171
> --- /dev/null
> +++ b/tests/qemu-iotests/bitmaps.py
> @@ -0,0 +1,131 @@
> +# Bitmap-related helper utilities
> +#
> +# Copyright (c) 2020 John Snow for Red Hat, Inc.
> +#
> +# This program is free software; you can redistribute it and/or modify
> +# it under the terms of the GNU General Public License as published by
> +# the Free Software Foundation; either version 2 of the License, or
> +# (at your option) any later version.
> +#
> +# This program is distributed in the hope that it will be useful,
> +# but WITHOUT ANY WARRANTY; without even the implied warranty of
> +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
> +# GNU General Public License for more details.
> +#
> +# You should have received a copy of the GNU General Public License
> +# along with this program. If not, see <http://www.gnu.org/licenses/>.
> +#
> +# owner=jsnow@redhat.com
> +
> +from iotests import log
> +
> +GRANULARITY = 64 * 1024
> +
> +
> +class Pattern:
> + def __init__(self, byte, offset, size=GRANULARITY):
> + self.byte = byte
> + self.offset = offset
> + self.size = size
> +
> + def bits(self, granularity):
> + lower = self.offset // granularity
> + upper = (self.offset + self.size - 1) // granularity
> + return set(range(lower, upper + 1))
> +
> +
> +class PatternGroup:
> + """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
> + def __init__(self, patterns):
> + self.patterns = patterns
> +
> + def bits(self, granularity):
> + """Calculate the unique bits dirtied by this pattern grouping"""
> + res = set()
> + for pattern in self.patterns:
> + res |= pattern.bits(granularity)
> + return res
> +
> +
> +GROUPS = [
> + PatternGroup([
> + # Batch 0: 4 clusters
> + Pattern('0x49', 0x0000000),
> + Pattern('0x6c', 0x0100000), # 1M
> + Pattern('0x6f', 0x2000000), # 32M
> + Pattern('0x76', 0x3ff0000)]), # 64M - 64K
> + PatternGroup([
> + # Batch 1: 6 clusters (3 new)
> + Pattern('0x65', 0x0000000), # Full overwrite
> + Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
> + Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
> + Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
> + PatternGroup([
> + # Batch 2: 7 clusters (3 new)
> + Pattern('0x74', 0x0010000), # Adjacent-right
> + Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
> + Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
> + Pattern('0x67', 0x3fe0000,
> + 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
> + PatternGroup([
> + # Batch 3: 8 clusters (5 new)
> + # Carefully chosen such that nothing re-dirties the one cluster
> + # that copies out successfully before failure in Group #1.
> + Pattern('0xaa', 0x0010000,
> + 3*GRANULARITY), # Overwrite and 2x Adjacent-right
> + Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
> + Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
> + Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
> +]
> +
> +
> +class EmulatedBitmap:
> + def __init__(self, granularity=GRANULARITY):
> + self._bits = set()
> + self.granularity = granularity
> +
> + def dirty_bits(self, bits):
> + self._bits |= set(bits)
> +
> + def dirty_group(self, n):
> + self.dirty_bits(GROUPS[n].bits(self.granularity))
> +
> + def clear(self):
> + self._bits = set()
> +
> + def clear_bits(self, bits):
> + self._bits -= set(bits)
> +
> + def clear_bit(self, bit):
> + self.clear_bits({bit})
> +
> + def clear_group(self, n):
> + self.clear_bits(GROUPS[n].bits(self.granularity))
> +
> + @property
> + def first_bit(self):
> + return sorted(self.bits)[0]
> +
> + @property
> + def bits(self):
> + return self._bits
> +
> + @property
> + def count(self):
> + return len(self.bits)
> +
> + def compare(self, qmp_bitmap):
> + """
> + Print a nice human-readable message checking that a bitmap as reported
> + by the QMP interface has as many bits set as we expect it to.
> + """
> +
> + name = qmp_bitmap.get('name', '(anonymous)')
> + log("= Checking Bitmap {:s} =".format(name))
> +
> + want = self.count
> + have = qmp_bitmap['count'] // qmp_bitmap['granularity']
> +
> + log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
> + want, have, "OK!" if want == have else "ERROR!"))
> + log('')
>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 4/6] iotests: add hmp helper with logging
2020-02-25 0:56 ` [PATCH 4/6] iotests: add hmp helper with logging John Snow
@ 2020-02-27 10:57 ` Vladimir Sementsov-Ogievskiy
0 siblings, 0 replies; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-27 10:57 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
25.02.2020 3:56, John Snow wrote:
> Just a mild cleanup while I was here.
>
> Signed-off-by: John Snow<jsnow@redhat.com>
Great!
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 5/6] qmp.py: change event_wait to use a dict
2020-02-25 0:56 ` [PATCH 5/6] qmp.py: change event_wait to use a dict John Snow
@ 2020-02-27 11:25 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:58 ` John Snow
0 siblings, 1 reply; 20+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2020-02-27 11:25 UTC (permalink / raw)
To: John Snow, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
25.02.2020 3:56, John Snow wrote:
> It's easier to work with than a list of tuples, because we can check the
> keys for membership.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> python/qemu/machine.py | 10 +++++-----
> tests/qemu-iotests/040 | 12 ++++++------
> tests/qemu-iotests/260 | 5 +++--
> tests/qemu-iotests/iotests.py | 16 ++++++++--------
> 4 files changed, 22 insertions(+), 21 deletions(-)
>
> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
> index 183d8f3d38..748de5f322 100644
> --- a/python/qemu/machine.py
> +++ b/python/qemu/machine.py
> @@ -476,21 +476,21 @@ def event_wait(self, name, timeout=60.0, match=None):
> timeout: QEMUMonitorProtocol.pull_event timeout parameter.
> match: Optional match criteria. See event_match for details.
> """
> - return self.events_wait([(name, match)], timeout)
> + return self.events_wait({name: match}, timeout)
>
> def events_wait(self, events, timeout=60.0):
> """
> events_wait waits for and returns a named event from QMP with a timeout.
>
> - events: a sequence of (name, match_criteria) tuples.
> + events: a mapping containing {name: match_criteria}.
> The match criteria are optional and may be None.
> See event_match for details.
> timeout: QEMUMonitorProtocol.pull_event timeout parameter.
> """
> def _match(event):
> - for name, match in events:
> - if event['event'] == name and self.event_match(event, match):
> - return True
> + name = event['event']
> + if name in events:
> + return self.event_match(event, events[name])
> return False
>
> # Search cached events
> diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
> index 32c82b4ec6..90b59081ff 100755
> --- a/tests/qemu-iotests/040
> +++ b/tests/qemu-iotests/040
> @@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase):
>
> def run_job(self, expected_events, error_pauses_job=False):
> match_device = {'data': {'device': 'job0'}}
> - events = [
> - ('BLOCK_JOB_COMPLETED', match_device),
> - ('BLOCK_JOB_CANCELLED', match_device),
> - ('BLOCK_JOB_ERROR', match_device),
> - ('BLOCK_JOB_READY', match_device),
> - ]
> + events = {
> + 'BLOCK_JOB_COMPLETED': match_device,
> + 'BLOCK_JOB_CANCELLED': match_device,
> + 'BLOCK_JOB_ERROR': match_device,
> + 'BLOCK_JOB_READY': match_device,
> + }
>
> completed = False
> log = []
> diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
> index 30c0de380d..b2fb045ddd 100755
> --- a/tests/qemu-iotests/260
> +++ b/tests/qemu-iotests/260
> @@ -65,8 +65,9 @@ def test(persistent, restart):
>
> vm.qmp_log('block-commit', device='drive0', top=top,
> filters=[iotests.filter_qmp_testfiles])
> - ev = vm.events_wait((('BLOCK_JOB_READY', None),
> - ('BLOCK_JOB_COMPLETED', None)))
> + ev = vm.events_wait({
> + 'BLOCK_JOB_READY': None,
> + 'BLOCK_JOB_COMPLETED': None })
may be better to keep it 2-lines. Or, otherwise, put closing "})" on a separate line too.
> log(filter_qmp_event(ev))
> if (ev['event'] == 'BLOCK_JOB_COMPLETED'):
> vm.shutdown()
> diff --git a/tests/qemu-iotests/iotests.py b/tests/qemu-iotests/iotests.py
> index 5d2990a0e4..3390fab021 100644
> --- a/tests/qemu-iotests/iotests.py
> +++ b/tests/qemu-iotests/iotests.py
> @@ -604,14 +604,14 @@ def run_job(self, job, auto_finalize=True, auto_dismiss=False,
> """
> match_device = {'data': {'device': job}}
> match_id = {'data': {'id': job}}
> - events = [
> - ('BLOCK_JOB_COMPLETED', match_device),
> - ('BLOCK_JOB_CANCELLED', match_device),
> - ('BLOCK_JOB_ERROR', match_device),
> - ('BLOCK_JOB_READY', match_device),
> - ('BLOCK_JOB_PENDING', match_id),
> - ('JOB_STATUS_CHANGE', match_id)
> - ]
> + events = {
> + 'BLOCK_JOB_COMPLETED': match_device,
> + 'BLOCK_JOB_CANCELLED': match_device,
> + 'BLOCK_JOB_ERROR': match_device,
> + 'BLOCK_JOB_READY': match_device,
> + 'BLOCK_JOB_PENDING': match_id,
> + 'JOB_STATUS_CHANGE': match_id,
> + }
> error = None
> while True:
> ev = filter_qmp_event(self.events_wait(events, timeout=wait))
>
Not sure that I like new interface more (neither that it is faster), but it works too:
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 1/6] block: add bitmap-populate job
2020-02-27 6:11 ` Vladimir Sementsov-Ogievskiy
@ 2020-03-03 21:39 ` John Snow
0 siblings, 0 replies; 20+ messages in thread
From: John Snow @ 2020-03-03 21:39 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
On 2/27/20 1:11 AM, Vladimir Sementsov-Ogievskiy wrote:
>
> Still, if user pass disabled bitmap, it will be invalid immediately after
> job finish.
Yes ... In truth, I really want to augment this job to provide a defined
point-in-time semantic so it can be fully useful in all circumstances.
At the moment, the point-in-time it provides is "at finalize", which may
or may not be the single most useful semantic. (It matches mirror --
because that was easier.)
The other option is "at start" which matches backup, but will require a
new filter and some changes to permissions. That was more invasive, so I
avoided it for now.
(In the interest of a speedy resolution for libvirt, can we add a
critical_moment=[START|FINALIZE] option to this job *later*?)
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 2/6] qmp: expose block-dirty-bitmap-populate
2020-02-27 10:44 ` Vladimir Sementsov-Ogievskiy
@ 2020-03-03 21:48 ` John Snow
0 siblings, 0 replies; 20+ messages in thread
From: John Snow @ 2020-03-03 21:48 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
On 2/27/20 5:44 AM, Vladimir Sementsov-Ogievskiy wrote:
> 25.02.2020 3:56, John Snow wrote:
>> This is a new job-creating command.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> qapi/block-core.json | 18 ++++++++++
>> qapi/transaction.json | 2 ++
>> blockdev.c | 78 +++++++++++++++++++++++++++++++++++++++++++
>> 3 files changed, 98 insertions(+)
>>
>> diff --git a/qapi/block-core.json b/qapi/block-core.json
>> index df1797681a..a8be1fb36b 100644
>> --- a/qapi/block-core.json
>> +++ b/qapi/block-core.json
>> @@ -2293,6 +2293,24 @@
>> '*auto-finalize': 'bool',
>> '*auto-dismiss': 'bool' } }
>> +##
>> +# @block-dirty-bitmap-populate:
>> +#
>> +# Creates a new job that writes a pattern into a dirty bitmap.
>> +#
>> +# Since: 5.0
>> +#
>> +# Example:
>> +#
>> +# -> { "execute": "block-dirty-bitmap-populate",
>> +# "arguments": { "node": "drive0", "target": "bitmap0",
>> +# "job-id": "job0", "pattern": "allocate-top" } }
>> +# <- { "return": {} }
>> +#
>> +##
>> +{ 'command': 'block-dirty-bitmap-populate', 'boxed': true,
>> + 'data': 'BlockDirtyBitmapPopulate' }
>> +
>> ##
>> # @BlockDirtyBitmapSha256:
>> #
>> diff --git a/qapi/transaction.json b/qapi/transaction.json
>> index 04301f1be7..28521d5c7f 100644
>> --- a/qapi/transaction.json
>> +++ b/qapi/transaction.json
>> @@ -50,6 +50,7 @@
>> # - @block-dirty-bitmap-enable: since 4.0
>> # - @block-dirty-bitmap-disable: since 4.0
>> # - @block-dirty-bitmap-merge: since 4.0
>> +# - @block-dirty-bitmap-populate: since 5.0
>> # - @blockdev-backup: since 2.3
>> # - @blockdev-snapshot: since 2.5
>> # - @blockdev-snapshot-internal-sync: since 1.7
>> @@ -67,6 +68,7 @@
>> 'block-dirty-bitmap-enable': 'BlockDirtyBitmap',
>> 'block-dirty-bitmap-disable': 'BlockDirtyBitmap',
>> 'block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge',
>> + 'block-dirty-bitmap-populate': 'BlockDirtyBitmapPopulate',
>> 'blockdev-backup': 'BlockdevBackup',
>> 'blockdev-snapshot': 'BlockdevSnapshot',
>> 'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal',
>> diff --git a/blockdev.c b/blockdev.c
>> index 011dcfec27..33c0e35399 100644
>> --- a/blockdev.c
>> +++ b/blockdev.c
>> @@ -2314,6 +2314,67 @@ static void
>> block_dirty_bitmap_remove_commit(BlkActionState *common)
>> bdrv_release_dirty_bitmap(state->bitmap);
>> }
>> +static void block_dirty_bitmap_populate_prepare(BlkActionState
>> *common, Error **errp)
>
> over80 line (not the only)
>
OK, will fix all instances.
>> +{
>> + BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState,
>> common, common);
>
> At first glance using *Backup* looks like a mistake. May be rename it,
> or at least add a comment.
>
You're right, it's tricky. A rename would be helpful.
>> + BlockDirtyBitmapPopulate *bitpop;
>> + BlockDriverState *bs;
>> + AioContext *aio_context;
>> + BdrvDirtyBitmap *bmap = NULL;
>> + int job_flags = JOB_DEFAULT;
>> +
>> + assert(common->action->type ==
>> TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE);
>> + bitpop = common->action->u.block_dirty_bitmap_populate.data;
>> +
>> + bs = bdrv_lookup_bs(bitpop->node, bitpop->node, errp);
>> + if (!bs) {
>> + return;
>> + }
>> +
>> + aio_context = bdrv_get_aio_context(bs);
>> + aio_context_acquire(aio_context);
>> + state->bs = bs;
>> +
>> + bmap = bdrv_find_dirty_bitmap(bs, bitpop->name);
>
> Could we use block_dirty_bitmap_lookup ?
>
Yes.
>> + if (!bmap) {
>> + error_setg(errp, "Bitmap '%s' could not be found",
>> bitpop->name);
>> + return;
>
> aio context lock leaked
>
Good spot.
>> + }
>> +
>> + /* Paired with .clean() */
>> + bdrv_drained_begin(state->bs);
>> +
>> + if (!bitpop->has_on_error) {
>> + bitpop->on_error = BLOCKDEV_ON_ERROR_REPORT;
>> + }
>> + if (!bitpop->has_auto_finalize) {
>> + bitpop->auto_finalize = true;
>> + }
>> + if (!bitpop->has_auto_dismiss) {
>> + bitpop->auto_dismiss = true;
>> + }
>> +
>> + if (!bitpop->auto_finalize) {
>> + job_flags |= JOB_MANUAL_FINALIZE;
>> + }
>> + if (!bitpop->auto_dismiss) {
>> + job_flags |= JOB_MANUAL_DISMISS;
>> + }
>> +
>> + state->job = bitpop_job_create(
>> + bitpop->job_id,
>> + bs,
>> + bmap,
>> + bitpop->pattern,
>> + bitpop->on_error,
>> + job_flags,
>> + NULL, NULL,
>> + common->block_job_txn,
>> + errp);
>> +
>> + aio_context_release(aio_context);
>> +}
>> +
>> static void abort_prepare(BlkActionState *common, Error **errp)
>> {
>> error_setg(errp, "Transaction aborted using Abort action");
>> @@ -2397,6 +2458,13 @@ static const BlkActionOps actions[] = {
>> .commit = block_dirty_bitmap_remove_commit,
>> .abort = block_dirty_bitmap_remove_abort,
>> },
>> + [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE] = {
>> + .instance_size = sizeof(BlockdevBackupState),
>> + .prepare = block_dirty_bitmap_populate_prepare,
>> + .commit = blockdev_backup_commit,
>> + .abort = blockdev_backup_abort,
>> + .clean = blockdev_backup_clean,
>> + },
>> /* Where are transactions for MIRROR, COMMIT and STREAM?
>> * Although these blockjobs use transaction callbacks like the
>> backup job,
>> * these jobs do not necessarily adhere to transaction semantics.
>> @@ -3225,6 +3293,16 @@ void qmp_block_dirty_bitmap_merge(const char
>> *node, const char *target,
>> do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp);
>> }
>> +void qmp_block_dirty_bitmap_populate(BlockDirtyBitmapPopulate *bitpop,
>> + Error **errp)
>> +{
>> + TransactionAction action = {
>> + .type = TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE,
>> + .u.block_dirty_bitmap_populate.data = bitpop,
>> + };
>> + blockdev_do_action(&action, errp);
>> +}
>> +
>> BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const
>> char *node,
>> const
>> char *name,
>> Error
>> **errp)
>>
>
>
--
—js
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 3/6] iotests: move bitmap helpers into their own file
2020-02-27 10:54 ` Vladimir Sementsov-Ogievskiy
@ 2020-03-03 21:55 ` John Snow
0 siblings, 0 replies; 20+ messages in thread
From: John Snow @ 2020-03-03 21:55 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
On 2/27/20 5:54 AM, Vladimir Sementsov-Ogievskiy wrote:
>
> Clean code movement, no changes. If test passes, it should be correct :)
>
> The only thing: I'd prefer not exporting global variables and use
> bitmaps.GROUPS instead (even then, it's not very good interface but..)
>
> with or without it:
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Hm, yeah -- it's not great, it's definitely just a bit of a quick
shuffle instead of a more serious effort at a refactor to provide bitmap
services for all tests.
I don't think it's worth the time just yet to make a serious attempt at
making this module bigger -- but maybe there is work that can be done at
providing bitmap management services targeted for use outside of testing
code. (i.e. into the python/ folder somewhere.)
Well, I can just do this:
"
import bitmaps
from bitmaps import EmulatedBitmap
"
and bitmaps.GROUPS makes it a little more obvious in-context at the
expense of one extra import line, so I'll do that.
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 5/6] qmp.py: change event_wait to use a dict
2020-02-27 11:25 ` Vladimir Sementsov-Ogievskiy
@ 2020-03-03 21:58 ` John Snow
0 siblings, 0 replies; 20+ messages in thread
From: John Snow @ 2020-03-03 21:58 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel
Cc: Kevin Wolf, pkrempa, Eduardo Habkost, qemu-block,
Markus Armbruster, Max Reitz, Cleber Rosa
On 2/27/20 6:25 AM, Vladimir Sementsov-Ogievskiy wrote:
> 25.02.2020 3:56, John Snow wrote:
>> It's easier to work with than a list of tuples, because we can check the
>> keys for membership.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> python/qemu/machine.py | 10 +++++-----
>> tests/qemu-iotests/040 | 12 ++++++------
>> tests/qemu-iotests/260 | 5 +++--
>> tests/qemu-iotests/iotests.py | 16 ++++++++--------
>> 4 files changed, 22 insertions(+), 21 deletions(-)
>>
>> diff --git a/python/qemu/machine.py b/python/qemu/machine.py
>> index 183d8f3d38..748de5f322 100644
>> --- a/python/qemu/machine.py
>> +++ b/python/qemu/machine.py
>> @@ -476,21 +476,21 @@ def event_wait(self, name, timeout=60.0,
>> match=None):
>> timeout: QEMUMonitorProtocol.pull_event timeout parameter.
>> match: Optional match criteria. See event_match for details.
>> """
>> - return self.events_wait([(name, match)], timeout)
>> + return self.events_wait({name: match}, timeout)
>> def events_wait(self, events, timeout=60.0):
>> """
>> events_wait waits for and returns a named event from QMP
>> with a timeout.
>> - events: a sequence of (name, match_criteria) tuples.
>> + events: a mapping containing {name: match_criteria}.
>> The match criteria are optional and may be None.
>> See event_match for details.
>> timeout: QEMUMonitorProtocol.pull_event timeout parameter.
>> """
>> def _match(event):
>> - for name, match in events:
>> - if event['event'] == name and self.event_match(event,
>> match):
>> - return True
>> + name = event['event']
>> + if name in events:
>> + return self.event_match(event, events[name])
>> return False
>> # Search cached events
>> diff --git a/tests/qemu-iotests/040 b/tests/qemu-iotests/040
>> index 32c82b4ec6..90b59081ff 100755
>> --- a/tests/qemu-iotests/040
>> +++ b/tests/qemu-iotests/040
>> @@ -485,12 +485,12 @@ class TestErrorHandling(iotests.QMPTestCase):
>> def run_job(self, expected_events, error_pauses_job=False):
>> match_device = {'data': {'device': 'job0'}}
>> - events = [
>> - ('BLOCK_JOB_COMPLETED', match_device),
>> - ('BLOCK_JOB_CANCELLED', match_device),
>> - ('BLOCK_JOB_ERROR', match_device),
>> - ('BLOCK_JOB_READY', match_device),
>> - ]
>> + events = {
>> + 'BLOCK_JOB_COMPLETED': match_device,
>> + 'BLOCK_JOB_CANCELLED': match_device,
>> + 'BLOCK_JOB_ERROR': match_device,
>> + 'BLOCK_JOB_READY': match_device,
>> + }
>> completed = False
>> log = []
>> diff --git a/tests/qemu-iotests/260 b/tests/qemu-iotests/260
>> index 30c0de380d..b2fb045ddd 100755
>> --- a/tests/qemu-iotests/260
>> +++ b/tests/qemu-iotests/260
>> @@ -65,8 +65,9 @@ def test(persistent, restart):
>> vm.qmp_log('block-commit', device='drive0', top=top,
>> filters=[iotests.filter_qmp_testfiles])
>> - ev = vm.events_wait((('BLOCK_JOB_READY', None),
>> - ('BLOCK_JOB_COMPLETED', None)))
>> + ev = vm.events_wait({
>> + 'BLOCK_JOB_READY': None,
>> + 'BLOCK_JOB_COMPLETED': None })
>
> may be better to keep it 2-lines. Or, otherwise, put closing "})" on a
> separate line too.
>
Sure.
>> log(filter_qmp_event(ev))
>> if (ev['event'] == 'BLOCK_JOB_COMPLETED'):
>> vm.shutdown()
>> diff --git a/tests/qemu-iotests/iotests.py
>> b/tests/qemu-iotests/iotests.py
>> index 5d2990a0e4..3390fab021 100644
>> --- a/tests/qemu-iotests/iotests.py
>> +++ b/tests/qemu-iotests/iotests.py
>> @@ -604,14 +604,14 @@ def run_job(self, job, auto_finalize=True,
>> auto_dismiss=False,
>> """
>> match_device = {'data': {'device': job}}
>> match_id = {'data': {'id': job}}
>> - events = [
>> - ('BLOCK_JOB_COMPLETED', match_device),
>> - ('BLOCK_JOB_CANCELLED', match_device),
>> - ('BLOCK_JOB_ERROR', match_device),
>> - ('BLOCK_JOB_READY', match_device),
>> - ('BLOCK_JOB_PENDING', match_id),
>> - ('JOB_STATUS_CHANGE', match_id)
>> - ]
>> + events = {
>> + 'BLOCK_JOB_COMPLETED': match_device,
>> + 'BLOCK_JOB_CANCELLED': match_device,
>> + 'BLOCK_JOB_ERROR': match_device,
>> + 'BLOCK_JOB_READY': match_device,
>> + 'BLOCK_JOB_PENDING': match_id,
>> + 'JOB_STATUS_CHANGE': match_id,
>> + }
>> error = None
>> while True:
>> ev = filter_qmp_event(self.events_wait(events,
>> timeout=wait))
>>
>
>
> Not sure that I like new interface more (neither that it is faster), but
> it works too:
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
It's a little bit of cross-contamination from the JobRunner series I
posted. This patch really belongs with that series instead.
It's not likely to be faster on the order of ~6 events, but logistically
it's just a little cleaner to read.
Thanks for the reviews!
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2020-03-03 21:59 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-02-25 0:56 [PATCH 0/6] block: add block-dirty-bitmap-populate job John Snow
2020-02-25 0:56 ` [PATCH 1/6] block: add bitmap-populate job John Snow
2020-02-25 16:04 ` Vladimir Sementsov-Ogievskiy
2020-02-25 20:41 ` John Snow
2020-02-26 5:07 ` Vladimir Sementsov-Ogievskiy
2020-02-26 19:11 ` John Snow
2020-02-27 6:11 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:39 ` John Snow
2020-02-25 0:56 ` [PATCH 2/6] qmp: expose block-dirty-bitmap-populate John Snow
2020-02-27 10:44 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:48 ` John Snow
2020-02-25 0:56 ` [PATCH 3/6] iotests: move bitmap helpers into their own file John Snow
2020-02-27 10:54 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:55 ` John Snow
2020-02-25 0:56 ` [PATCH 4/6] iotests: add hmp helper with logging John Snow
2020-02-27 10:57 ` Vladimir Sementsov-Ogievskiy
2020-02-25 0:56 ` [PATCH 5/6] qmp.py: change event_wait to use a dict John Snow
2020-02-27 11:25 ` Vladimir Sementsov-Ogievskiy
2020-03-03 21:58 ` John Snow
2020-02-25 0:56 ` [PATCH 6/6] iotests: add 287 for block-dirty-bitmap-populate John Snow
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.