[PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

[PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

[John Snow]

Hi,

This is a new (very small) block job that writes a pattern into a
bitmap. The only pattern implemented is the top allocation information.

This can be used to "recover" an incremental bitmap chain if an external
snapshot was taken without creating a new bitmap first: any writes made
to the image will be reflected by the allocation status and can be
written back into a bitmap.

This is useful for e.g. libvirt managing backup chains if a user creates
an external snapshot outside of libvirt.

v2:
 - Addressed some, but not all feedback
 - Rebased on latest 'job-runner' series; but it's not clear if it
   should be kept.
 - This version doesn't address all of the feedback from v1,
   but I am posting it to the list as an RFC.

John Snow (5):
  block: add bitmap-populate job
  blockdev: combine DriveBackupState and BlockdevBackupState
  qmp: expose block-dirty-bitmap-populate
  iotests: move bitmap helpers into their own file
  iotests: add 287 for block-dirty-bitmap-populate

 qapi/block-core.json          |   66 +
 qapi/job.json                 |    2 +-
 qapi/transaction.json         |    2 +
 include/block/block_int.h     |   21 +
 block/bitmap-alloc.c          |  207 ++
 blockdev.c                    |  104 +-
 blockjob.c                    |    3 +-
 block/Makefile.objs           |    1 +
 tests/qemu-iotests/257        |  110 +-
 tests/qemu-iotests/287        |  242 ++
 tests/qemu-iotests/287.out    | 4544 +++++++++++++++++++++++++++++++++
 tests/qemu-iotests/bitmaps.py |  131 +
 tests/qemu-iotests/group      |    1 +
 13 files changed, 5305 insertions(+), 129 deletions(-)
 create mode 100644 block/bitmap-alloc.c
 create mode 100755 tests/qemu-iotests/287
 create mode 100644 tests/qemu-iotests/287.out
 create mode 100644 tests/qemu-iotests/bitmaps.py

-- 
2.21.1

[PATCH RFC v2 1/5] block: add bitmap-populate job

[John Snow]

This job copies the allocation map into a bitmap. It's a job because
there's no guarantee that allocation interrogation will be quick (or
won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.

It was designed with different possible population patterns in mind,
but only top layer allocation was implemented for now.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 qapi/block-core.json      |  48 +++++++++
 qapi/job.json             |   2 +-
 include/block/block_int.h |  21 ++++
 block/bitmap-alloc.c      | 207 ++++++++++++++++++++++++++++++++++++++
 blockjob.c                |   3 +-
 block/Makefile.objs       |   1 +
 6 files changed, 280 insertions(+), 2 deletions(-)
 create mode 100644 block/bitmap-alloc.c

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 943df1926a..0fb527a9a1 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2202,6 +2202,54 @@
       { 'command': 'block-dirty-bitmap-merge',
         'data': 'BlockDirtyBitmapMerge' }
 
+##
+# @BitmapPattern:
+#
+# An enumeration of possible patterns that can be written into a bitmap.
+#
+# @allocation-top: The allocation status of the top layer
+#                  of the attached storage node.
+#
+# Since: 5.0
+##
+{ 'enum': 'BitmapPattern',
+  'data': ['allocation-top'] }
+
+##
+# @BlockDirtyBitmapPopulate:
+#
+# @job-id: identifier for the newly-created block job.
+#
+# @pattern: What pattern should be written into the bitmap?
+#
+# @on-error: the action to take if an error is encountered on a bitmap's
+#            attached node, default 'report'.
+#            'stop' and 'enospc' can only be used if the block device supports
+#            io-status (see BlockInfo).
+#
+# @auto-finalize: When false, this job will wait in a PENDING state after it has
+#                 finished its work, waiting for @block-job-finalize before
+#                 making any block graph changes.
+#                 When true, this job will automatically
+#                 perform its abort or commit actions.
+#                 Defaults to true.
+#
+# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it
+#                has completely ceased all work, and awaits @block-job-dismiss.
+#                When true, this job will automatically disappear from the query
+#                list without user intervention.
+#                Defaults to true.
+#
+# Since: 5.0
+##
+{ 'struct': 'BlockDirtyBitmapPopulate',
+  'base': 'BlockDirtyBitmap',
+  'data': { 'job-id': 'str',
+            'pattern': 'BitmapPattern',
+            '*on-error': 'BlockdevOnError',
+            '*auto-finalize': 'bool',
+            '*auto-dismiss': 'bool' } }
+
 ##
 # @BlockDirtyBitmapSha256:
 #
diff --git a/qapi/job.json b/qapi/job.json
index 5e658281f5..5f496d4630 100644
--- a/qapi/job.json
+++ b/qapi/job.json
@@ -22,7 +22,7 @@
 # Since: 1.7
 ##
 { 'enum': 'JobType',
-  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
+  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'bitmap-populate'] }
 
 ##
 # @JobStatus:
diff --git a/include/block/block_int.h b/include/block/block_int.h
index df6d0273d6..eb68a5639a 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -1222,6 +1222,27 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
                             BlockCompletionFunc *cb, void *opaque,
                             JobTxn *txn, Error **errp);
 
+/*
+ * bitpop_job_create: Create a new bitmap population job.
+ *
+ * @job_id: The id of the newly-created job.
+ * @bs: Block device associated with the @target_bitmap.
+ * @target_bitmap: The bitmap to populate.
+ * @on_error: What to do if an error on @bs is encountered.
+ * @creation_flags: Flags that control the behavior of the Job lifetime.
+ *                  See @BlockJobCreateFlags
+ * @cb: Completion function for the job.
+ * @opaque: Opaque pointer value passed to @cb.
+ * @txn: Transaction that this job is part of (may be NULL).
+ */
+BlockJob *bitpop_job_create(const char *job_id, BlockDriverState *bs,
+                            BdrvDirtyBitmap *target_bitmap,
+                            BitmapPattern pattern,
+                            BlockdevOnError on_error,
+                            int creation_flags,
+                            BlockCompletionFunc *cb, void *opaque,
+                            JobTxn *txn, Error **errp);
+
 BdrvChild *bdrv_root_attach_child(BlockDriverState *child_bs,
                                   const char *child_name,
                                   const BdrvChildRole *child_role,
diff --git a/block/bitmap-alloc.c b/block/bitmap-alloc.c
new file mode 100644
index 0000000000..47d542dc12
--- /dev/null
+++ b/block/bitmap-alloc.c
@@ -0,0 +1,207 @@
+/*
+ * Async Dirty Bitmap Populator
+ *
+ * Copyright (C) 2020 Red Hat, Inc.
+ *
+ * Authors:
+ *  John Snow <jsnow@redhat.com>
+ *
+ * This work is licensed under the terms of the GNU GPL, version 2 or later.
+ * See the COPYING file in the top-level directory.
+ *
+ */
+
+#include "qemu/osdep.h"
+
+#include "trace.h"
+#include "block/block.h"
+#include "block/block_int.h"
+#include "block/blockjob_int.h"
+#include "block/block_backup.h"
+#include "block/block-copy.h"
+#include "qapi/error.h"
+#include "qapi/qmp/qerror.h"
+#include "qemu/ratelimit.h"
+#include "qemu/cutils.h"
+#include "sysemu/block-backend.h"
+#include "qemu/bitmap.h"
+#include "qemu/error-report.h"
+
+typedef struct BitpopBlockJob {
+    BlockJob common;
+    BlockDriverState *bs;
+    BdrvDirtyBitmap *target_bitmap;
+    BdrvDirtyBitmap *new_bitmap;
+    BlockdevOnError on_error;
+    uint64_t len;
+} BitpopBlockJob;
+
+static const BlockJobDriver bitpop_job_driver;
+
+static void bitpop_commit(Job *job)
+{
+    BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
+
+    bdrv_dirty_bitmap_merge_internal(s->target_bitmap, s->new_bitmap,
+                                     NULL, true);
+}
+
+/* no abort needed; just clean without committing. */
+
+static void bitpop_clean(Job *job)
+{
+    BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
+
+    bdrv_release_dirty_bitmap(s->new_bitmap);
+    bdrv_dirty_bitmap_set_busy(s->target_bitmap, false);
+}
+
+static BlockErrorAction bitpop_error_action(BitpopBlockJob *job, int error)
+{
+    return block_job_error_action(&job->common, job->on_error, true, error);
+}
+
+static bool coroutine_fn yield_and_check(Job *job)
+{
+    if (job_is_cancelled(job)) {
+        return true;
+    }
+
+    job_sleep_ns(job, 0);
+
+    if (job_is_cancelled(job)) {
+        return true;
+    }
+
+    return false;
+}
+
+static int coroutine_fn bitpop_run(Job *job, Error **errp)
+{
+    BitpopBlockJob *s = container_of(job, BitpopBlockJob, common.job);
+    int ret = 0;
+    int64_t offset;
+    int64_t count;
+    int64_t bytes;
+
+    for (offset = 0; offset < s->len; ) {
+        if (yield_and_check(job)) {
+            ret = -ECANCELED;
+            break;
+        }
+
+        bytes = s->len - offset;
+        ret = bdrv_is_allocated(s->bs, offset, bytes, &count);
+        if (ret < 0) {
+            if (bitpop_error_action(s, -ret) == BLOCK_ERROR_ACTION_REPORT) {
+                break;
+            }
+            continue;
+        }
+
+        if (!count) {
+            ret = 0;
+            break;
+        }
+
+        if (ret) {
+            bdrv_set_dirty_bitmap(s->new_bitmap, offset, count);
+            ret = 0;
+        }
+
+        job_progress_update(job, count);
+        offset += count;
+    }
+
+    return ret;
+}
+
+static const BlockJobDriver bitpop_job_driver = {
+    .job_driver = {
+        .instance_size          = sizeof(BitpopBlockJob),
+        .job_type               = JOB_TYPE_BITMAP_POPULATE,
+        .free                   = block_job_free,
+        .user_resume            = block_job_user_resume,
+        .run                    = bitpop_run,
+        .commit                 = bitpop_commit,
+        .clean                  = bitpop_clean,
+    }
+};
+
+
+BlockJob *bitpop_job_create(
+    const char *job_id,
+    BlockDriverState *bs,
+    BdrvDirtyBitmap *target_bitmap,
+    BitmapPattern pattern,
+    BlockdevOnError on_error,
+    int creation_flags,
+    BlockCompletionFunc *cb,
+    void *opaque,
+    JobTxn *txn,
+    Error **errp)
+{
+    int64_t len;
+    BitpopBlockJob *job = NULL;
+    int64_t cluster_size;
+    BdrvDirtyBitmap *new_bitmap = NULL;
+
+    assert(bs);
+    assert(target_bitmap);
+
+    if (!bdrv_is_inserted(bs)) {
+        error_setg(errp, "Device is not inserted: %s",
+                   bdrv_get_device_name(bs));
+        return NULL;
+    }
+
+    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
+        return NULL;
+    }
+
+    if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
+        return NULL;
+    }
+
+    if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
+        error_setg(errp, "Unrecognized bitmap pattern");
+        return NULL;
+    }
+
+    len = bdrv_getlength(bs);
+    if (len < 0) {
+        error_setg_errno(errp, -len, "unable to get length for '%s'",
+                         bdrv_get_device_name(bs));
+        return NULL;
+    }
+
+    /* NB: new bitmap is anonymous and enabled */
+    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
+    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
+    if (!new_bitmap) {
+        return NULL;
+    }
+
+    /* Take ownership; we reserve the right to write into this on-commit. */
+    bdrv_dirty_bitmap_set_busy(target_bitmap, true);
+
+    job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
+                           BLK_PERM_CONSISTENT_READ,
+                           BLK_PERM_ALL & ~BLK_PERM_RESIZE,
+                           0, creation_flags,
+                           cb, opaque, errp);
+    if (!job) {
+        bdrv_dirty_bitmap_set_busy(target_bitmap, false);
+        bdrv_release_dirty_bitmap(new_bitmap);
+        return NULL;
+    }
+
+    job->bs = bs;
+    job->on_error = on_error;
+    job->target_bitmap = target_bitmap;
+    job->new_bitmap = new_bitmap;
+    job->len = len;
+    job_progress_set_remaining(&job->common.job, job->len);
+
+    return &job->common;
+}
diff --git a/blockjob.c b/blockjob.c
index 2affa1844d..991d9167a2 100644
--- a/blockjob.c
+++ b/blockjob.c
@@ -56,7 +56,8 @@ static bool is_block_job(Job *job)
     return job_type(job) == JOB_TYPE_BACKUP ||
            job_type(job) == JOB_TYPE_COMMIT ||
            job_type(job) == JOB_TYPE_MIRROR ||
-           job_type(job) == JOB_TYPE_STREAM;
+           job_type(job) == JOB_TYPE_STREAM ||
+           job_type(job) == JOB_TYPE_BITMAP_POPULATE;
 }
 
 BlockJob *block_job_next(BlockJob *bjob)
diff --git a/block/Makefile.objs b/block/Makefile.objs
index 3635b6b4c1..87357529f3 100644
--- a/block/Makefile.objs
+++ b/block/Makefile.objs
@@ -36,6 +36,7 @@ block-obj-$(CONFIG_LIBSSH) += ssh.o
 block-obj-y += accounting.o dirty-bitmap.o
 block-obj-y += write-threshold.o
 block-obj-y += backup.o
+block-obj-y += bitmap-alloc.o
 block-obj-$(CONFIG_REPLICATION) += replication.o
 block-obj-y += throttle.o copy-on-read.o
 block-obj-y += block-copy.o
-- 
2.21.1

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Eric Blake]

On 5/13/20 10:49 PM, John Snow wrote:
> This job copies the allocation map into a bitmap. It's a job because
> there's no guarantee that allocation interrogation will be quick (or
> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.

retrofitted

> 
> It was designed with different possible population patterns in mind,
> but only top layer allocation was implemented for now.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---

> +++ b/qapi/block-core.json
> @@ -2202,6 +2202,54 @@
>         { 'command': 'block-dirty-bitmap-merge',
>           'data': 'BlockDirtyBitmapMerge' }
>   
> +##
> +# @BitmapPattern:
> +#
> +# An enumeration of possible patterns that can be written into a bitmap.
> +#
> +# @allocation-top: The allocation status of the top layer
> +#                  of the attached storage node.

Presumably, allocated => bits are set.

> +#
> +# Since: 5.0

5.1, now

> +##
> +{ 'enum': 'BitmapPattern',
> +  'data': ['allocation-top'] }
> +
> +##
> +# @BlockDirtyBitmapPopulate:
> +#
> +# @job-id: identifier for the newly-created block job.
> +#
> +# @pattern: What pattern should be written into the bitmap?
> +#
> +# @on-error: the action to take if an error is encountered on a bitmap's
> +#            attached node, default 'report'.
> +#            'stop' and 'enospc' can only be used if the block device supports
> +#            io-status (see BlockInfo).
> +#
> +# @auto-finalize: When false, this job will wait in a PENDING state after it has
> +#                 finished its work, waiting for @block-job-finalize before
> +#                 making any block graph changes.
> +#                 When true, this job will automatically
> +#                 perform its abort or commit actions.
> +#                 Defaults to true.
> +#
> +# @auto-dismiss: When false, this job will wait in a CONCLUDED state after it
> +#                has completely ceased all work, and awaits @block-job-dismiss.
> +#                When true, this job will automatically disappear from the query
> +#                list without user intervention.
> +#                Defaults to true.
> +#
> +# Since: 5.0

5.1

> +##
> +{ 'struct': 'BlockDirtyBitmapPopulate',
> +  'base': 'BlockDirtyBitmap',
> +  'data': { 'job-id': 'str',
> +            'pattern': 'BitmapPattern',
> +            '*on-error': 'BlockdevOnError',
> +            '*auto-finalize': 'bool',
> +            '*auto-dismiss': 'bool' } }
> +
>   ##
>   # @BlockDirtyBitmapSha256:
>   #
> diff --git a/qapi/job.json b/qapi/job.json
> index 5e658281f5..5f496d4630 100644
> --- a/qapi/job.json
> +++ b/qapi/job.json
> @@ -22,7 +22,7 @@
>   # Since: 1.7
>   ##
>   { 'enum': 'JobType',
> -  'data': ['commit', 'stream', 'mirror', 'backup', 'create'] }
> +  'data': ['commit', 'stream', 'mirror', 'backup', 'create', 'bitmap-populate'] }

Missing docs that 'bitmap-populate' is since 5.1.

> +++ b/block/bitmap-alloc.c
> @@ -0,0 +1,207 @@

New file, but no MAINTAINERS update.  It is covered by 'Block layer 
core', but shouldn't it also be covered by 'Dirty Bitmaps'?


> +BlockJob *bitpop_job_create(
> +    const char *job_id,
> +    BlockDriverState *bs,
> +    BdrvDirtyBitmap *target_bitmap,
> +    BitmapPattern pattern,
> +    BlockdevOnError on_error,
> +    int creation_flags,
> +    BlockCompletionFunc *cb,
> +    void *opaque,
> +    JobTxn *txn,
> +    Error **errp)
> +{

> +
> +    /* NB: new bitmap is anonymous and enabled */
> +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> +    if (!new_bitmap) {
> +        return NULL;
> +    }

This means if the guest writes to the disk while the job is ongoing, the 
bitmap will be updated to mark that portion of the bitmap as set, even 
if it was not allocated at the time the job started.  But then again, 
the guest writes are causing allocation, so this seems like the right 
thing to do.

Do we need to worry about the converse case where the job started with 
something allocated but runs in parallel with the guest trimming, such 
that our bitmap marks something as set even though at the conclusion of 
our job it is no longer allocated?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Mon, May 18, 2020 at 15:49:02 -0500, Eric Blake wrote:
> On 5/13/20 10:49 PM, John Snow wrote:

[...]

> > +
> > +    /* NB: new bitmap is anonymous and enabled */
> > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > +    if (!new_bitmap) {
> > +        return NULL;
> > +    }
> 
> This means if the guest writes to the disk while the job is ongoing, the
> bitmap will be updated to mark that portion of the bitmap as set, even if it
> was not allocated at the time the job started.  But then again, the guest
> writes are causing allocation, so this seems like the right thing to do.

Well, this could be made problem of the caller by skipping any newly
allocated sectors to be written to the bitmap. The caller then can
decide whether a snapshot of the allocation map is needed and thus a new
inactive bitmap should be used as the destination, or whether new writes
should be tracked by using an active bitmap.

> Do we need to worry about the converse case where the job started with
> something allocated but runs in parallel with the guest trimming, such that
> our bitmap marks something as set even though at the conclusion of our job
> it is no longer allocated?

Given the semantics above this would conveniently not be a problem of
the population job. If you create a snapshot of the allocation map any
any point we'd care about that state.

Anyways, from the point of view of the bitmap code any write to a sector
sets the bit so the trimming should not be treated differently.

Speicifically libvirt plans to use it on overlay(snapshot) images where
the btimaps are not present so in that case even trimmed sectors need to
mask the data in the backing image so they can technically be considered
as allocated too.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > +
> > +    /* NB: new bitmap is anonymous and enabled */
> > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > +    if (!new_bitmap) {
> > +        return NULL;
> > +    }
> 
> This means if the guest writes to the disk while the job is ongoing, the
> bitmap will be updated to mark that portion of the bitmap as set, even if it
> was not allocated at the time the job started.  But then again, the guest
> writes are causing allocation, so this seems like the right thing to do.

Is the target bitmap active at the same time, i.e. will it get the
correct information only from new_bitmap or are the bits already set in
it anyway?

If it relies on new_bitmap, I think this deserves a comment here.

In the documentation of BitmapPattern and specifically @allocation-top,
we should probably also be more explicit what this means for requests
that the guest makes while the job is running.

> Do we need to worry about the converse case where the job started with
> something allocated but runs in parallel with the guest trimming, such that
> our bitmap marks something as set even though at the conclusion of our job
> it is no longer allocated?

Discard shouldn't make the backing file visible (didn't we fix this for
qcow2 v2 only recently?), so in the sense of bdrv_is_allocated(), they
will still be allocated, even though they are probably zero clusters
now.

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > +
> > > +    /* NB: new bitmap is anonymous and enabled */
> > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > +    if (!new_bitmap) {
> > > +        return NULL;
> > > +    }
> > 
> > This means if the guest writes to the disk while the job is ongoing, the
> > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > was not allocated at the time the job started.  But then again, the guest
> > writes are causing allocation, so this seems like the right thing to do.
> 
> Is the target bitmap active at the same time, i.e. will it get the
> correct information only from new_bitmap or are the bits already set in
> it anyway?

Yes, libvirt plans to use it with an active non-persistent bitmap which
will in subsequent steps be merged into others. The bitmap is added in
the same transaction. The bitmap must be active, because we need to wait
for the block jobs to finish before it becomes usable and thus can't
sequence in other operations until later.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > +
> > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > +    if (!new_bitmap) {
> > > > +        return NULL;
> > > > +    }
> > > 
> > > This means if the guest writes to the disk while the job is ongoing, the
> > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > was not allocated at the time the job started.  But then again, the guest
> > > writes are causing allocation, so this seems like the right thing to do.
> > 
> > Is the target bitmap active at the same time, i.e. will it get the
> > correct information only from new_bitmap or are the bits already set in
> > it anyway?
> 
> Yes, libvirt plans to use it with an active non-persistent bitmap which
> will in subsequent steps be merged into others. The bitmap is added in
> the same transaction. The bitmap must be active, because we need to wait
> for the block jobs to finish before it becomes usable and thus can't
> sequence in other operations until later.

A lot of bitmap merging then, because the block job in this series
already creates a temporary internal bitmap that is merged into the
target bitmap on completion. But if the target bitmap is only libvirt's
temporary bitmap to be merged to yet another bitmap, I wonder if this
process shouldn't be simplified.

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Thu, Jun 04, 2020 at 13:31:45 +0200, Kevin Wolf wrote:
> Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> > On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > > +
> > > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > > +    if (!new_bitmap) {
> > > > > +        return NULL;
> > > > > +    }
> > > > 
> > > > This means if the guest writes to the disk while the job is ongoing, the
> > > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > > was not allocated at the time the job started.  But then again, the guest
> > > > writes are causing allocation, so this seems like the right thing to do.
> > > 
> > > Is the target bitmap active at the same time, i.e. will it get the
> > > correct information only from new_bitmap or are the bits already set in
> > > it anyway?
> > 
> > Yes, libvirt plans to use it with an active non-persistent bitmap which
> > will in subsequent steps be merged into others. The bitmap is added in
> > the same transaction. The bitmap must be active, because we need to wait
> > for the block jobs to finish before it becomes usable and thus can't
> > sequence in other operations until later.
> 
> A lot of bitmap merging then, because the block job in this series
> already creates a temporary internal bitmap that is merged into the
> target bitmap on completion. But if the target bitmap is only libvirt's
> temporary bitmap to be merged to yet another bitmap, I wonder if this
> process shouldn't be simplified.

Possibly yes, but I'll leave that for later. All of this is done when
executin very expensive operations anyways so for our first
implementation it IMO won't matter that much.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 04.06.2020 um 18:22 hat Peter Krempa geschrieben:
> On Thu, Jun 04, 2020 at 13:31:45 +0200, Kevin Wolf wrote:
> > Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> > > On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > > > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > > > +
> > > > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > > > +    if (!new_bitmap) {
> > > > > > +        return NULL;
> > > > > > +    }
> > > > > 
> > > > > This means if the guest writes to the disk while the job is ongoing, the
> > > > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > > > was not allocated at the time the job started.  But then again, the guest
> > > > > writes are causing allocation, so this seems like the right thing to do.
> > > > 
> > > > Is the target bitmap active at the same time, i.e. will it get the
> > > > correct information only from new_bitmap or are the bits already set in
> > > > it anyway?
> > > 
> > > Yes, libvirt plans to use it with an active non-persistent bitmap which
> > > will in subsequent steps be merged into others. The bitmap is added in
> > > the same transaction. The bitmap must be active, because we need to wait
> > > for the block jobs to finish before it becomes usable and thus can't
> > > sequence in other operations until later.
> > 
> > A lot of bitmap merging then, because the block job in this series
> > already creates a temporary internal bitmap that is merged into the
> > target bitmap on completion. But if the target bitmap is only libvirt's
> > temporary bitmap to be merged to yet another bitmap, I wonder if this
> > process shouldn't be simplified.
> 
> Possibly yes, but I'll leave that for later. All of this is done when
> executin very expensive operations anyways so for our first
> implementation it IMO won't matter that much.

I'm not necessarily saying that the change is needed on the libvirt
side. It could also be that the block job should directly work with the
given bitmap instead of having its internal temporary bitmap. Changing
this later would mean changing the semantics of the block job, so it
would be somewhat problematic.

It would be good to have a clear picture of what we want the final
result to look like.

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Fri, Jun 05, 2020 at 11:01:23 +0200, Kevin Wolf wrote:
> Am 04.06.2020 um 18:22 hat Peter Krempa geschrieben:
> > On Thu, Jun 04, 2020 at 13:31:45 +0200, Kevin Wolf wrote:
> > > Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> > > > On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > > > > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > > > > +
> > > > > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > > > > +    if (!new_bitmap) {
> > > > > > > +        return NULL;
> > > > > > > +    }
> > > > > > 
> > > > > > This means if the guest writes to the disk while the job is ongoing, the
> > > > > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > > > > was not allocated at the time the job started.  But then again, the guest
> > > > > > writes are causing allocation, so this seems like the right thing to do.
> > > > > 
> > > > > Is the target bitmap active at the same time, i.e. will it get the
> > > > > correct information only from new_bitmap or are the bits already set in
> > > > > it anyway?
> > > > 
> > > > Yes, libvirt plans to use it with an active non-persistent bitmap which
> > > > will in subsequent steps be merged into others. The bitmap is added in
> > > > the same transaction. The bitmap must be active, because we need to wait
> > > > for the block jobs to finish before it becomes usable and thus can't
> > > > sequence in other operations until later.
> > > 
> > > A lot of bitmap merging then, because the block job in this series
> > > already creates a temporary internal bitmap that is merged into the
> > > target bitmap on completion. But if the target bitmap is only libvirt's
> > > temporary bitmap to be merged to yet another bitmap, I wonder if this
> > > process shouldn't be simplified.
> > 
> > Possibly yes, but I'll leave that for later. All of this is done when
> > executin very expensive operations anyways so for our first
> > implementation it IMO won't matter that much.
> 
> I'm not necessarily saying that the change is needed on the libvirt
> side. It could also be that the block job should directly work with the
> given bitmap instead of having its internal temporary bitmap. Changing
> this later would mean changing the semantics of the block job, so it
> would be somewhat problematic.
> 
> It would be good to have a clear picture of what we want the final
> result to look like.

Well with current semantics of the 'nodename' argument controling both
where the populated bitmap is located and also which node's allocation
bitmap to take I don't think we can optimize it further in libvirt.

Current usage scenario is that we use a temporary bitmap populated with
the job to merge with bitmaps present in nodes which are removed by
blockjobs into the destination node of the block job. This means that
the real destination of the bits populated is in a different node than
it was originally and the above job semantics don't allow that.

Either way I'd strongly prefer to be able to kick off all the populate
jobs at once rather than having to sequence them so any semantic change
towards making it possible to target bitmaps in a different node would
also require that multiple jobs can run in parallel with a single bitmap
as destination. I'm not sure if that doesn't overcomplicate things
though.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 05.06.2020 um 11:24 hat Peter Krempa geschrieben:
> On Fri, Jun 05, 2020 at 11:01:23 +0200, Kevin Wolf wrote:
> > Am 04.06.2020 um 18:22 hat Peter Krempa geschrieben:
> > > On Thu, Jun 04, 2020 at 13:31:45 +0200, Kevin Wolf wrote:
> > > > Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> > > > > On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > > > > > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > > > > > +
> > > > > > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > > > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > > > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > > > > > +    if (!new_bitmap) {
> > > > > > > > +        return NULL;
> > > > > > > > +    }
> > > > > > > 
> > > > > > > This means if the guest writes to the disk while the job is ongoing, the
> > > > > > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > > > > > was not allocated at the time the job started.  But then again, the guest
> > > > > > > writes are causing allocation, so this seems like the right thing to do.
> > > > > > 
> > > > > > Is the target bitmap active at the same time, i.e. will it get the
> > > > > > correct information only from new_bitmap or are the bits already set in
> > > > > > it anyway?
> > > > > 
> > > > > Yes, libvirt plans to use it with an active non-persistent bitmap which
> > > > > will in subsequent steps be merged into others. The bitmap is added in
> > > > > the same transaction. The bitmap must be active, because we need to wait
> > > > > for the block jobs to finish before it becomes usable and thus can't
> > > > > sequence in other operations until later.
> > > > 
> > > > A lot of bitmap merging then, because the block job in this series
> > > > already creates a temporary internal bitmap that is merged into the
> > > > target bitmap on completion. But if the target bitmap is only libvirt's
> > > > temporary bitmap to be merged to yet another bitmap, I wonder if this
> > > > process shouldn't be simplified.
> > > 
> > > Possibly yes, but I'll leave that for later. All of this is done when
> > > executin very expensive operations anyways so for our first
> > > implementation it IMO won't matter that much.
> > 
> > I'm not necessarily saying that the change is needed on the libvirt
> > side. It could also be that the block job should directly work with the
> > given bitmap instead of having its internal temporary bitmap. Changing
> > this later would mean changing the semantics of the block job, so it
> > would be somewhat problematic.
> > 
> > It would be good to have a clear picture of what we want the final
> > result to look like.
> 
> Well with current semantics of the 'nodename' argument controling both
> where the populated bitmap is located and also which node's allocation
> bitmap to take I don't think we can optimize it further in libvirt.
> 
> Current usage scenario is that we use a temporary bitmap populated with
> the job to merge with bitmaps present in nodes which are removed by
> blockjobs into the destination node of the block job. This means that
> the real destination of the bits populated is in a different node than
> it was originally and the above job semantics don't allow that.

So does this mean that a better API wouldn't only take a node-name and
bitmap name (where the node identified by node-name is not only where
the target bitmap is, but also the node whose allocation status is
queried), but that it should take two different node-names for source
(= reading allocation status) and target (= owner of the bitmap)?

> Either way I'd strongly prefer to be able to kick off all the populate
> jobs at once rather than having to sequence them so any semantic change
> towards making it possible to target bitmaps in a different node would
> also require that multiple jobs can run in parallel with a single bitmap
> as destination. I'm not sure if that doesn't overcomplicate things
> though.

Other people are more familiar with the dirty bitmap code, so I may be
wrong, but intuitively, I don't see any problem with multiple jobs
dirtying blocks in the same bitmap. Or, with the internal temporary
bitmap as used in this version of the series, multiple jobs that, one
after another, merge their result to the same bitmap on completion.

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
> Am 05.06.2020 um 11:24 hat Peter Krempa geschrieben:
> > On Fri, Jun 05, 2020 at 11:01:23 +0200, Kevin Wolf wrote:
> > > Am 04.06.2020 um 18:22 hat Peter Krempa geschrieben:
> > > > On Thu, Jun 04, 2020 at 13:31:45 +0200, Kevin Wolf wrote:
> > > > > Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> > > > > > On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > > > > > > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > > > > > > +
> > > > > > > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > > > > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > > > > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > > > > > > +    if (!new_bitmap) {
> > > > > > > > > +        return NULL;
> > > > > > > > > +    }
> > > > > > > > 
> > > > > > > > This means if the guest writes to the disk while the job is ongoing, the
> > > > > > > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > > > > > > was not allocated at the time the job started.  But then again, the guest
> > > > > > > > writes are causing allocation, so this seems like the right thing to do.
> > > > > > > 
> > > > > > > Is the target bitmap active at the same time, i.e. will it get the
> > > > > > > correct information only from new_bitmap or are the bits already set in
> > > > > > > it anyway?
> > > > > > 
> > > > > > Yes, libvirt plans to use it with an active non-persistent bitmap which
> > > > > > will in subsequent steps be merged into others. The bitmap is added in
> > > > > > the same transaction. The bitmap must be active, because we need to wait
> > > > > > for the block jobs to finish before it becomes usable and thus can't
> > > > > > sequence in other operations until later.
> > > > > 
> > > > > A lot of bitmap merging then, because the block job in this series
> > > > > already creates a temporary internal bitmap that is merged into the
> > > > > target bitmap on completion. But if the target bitmap is only libvirt's
> > > > > temporary bitmap to be merged to yet another bitmap, I wonder if this
> > > > > process shouldn't be simplified.
> > > > 
> > > > Possibly yes, but I'll leave that for later. All of this is done when
> > > > executin very expensive operations anyways so for our first
> > > > implementation it IMO won't matter that much.
> > > 
> > > I'm not necessarily saying that the change is needed on the libvirt
> > > side. It could also be that the block job should directly work with the
> > > given bitmap instead of having its internal temporary bitmap. Changing
> > > this later would mean changing the semantics of the block job, so it
> > > would be somewhat problematic.
> > > 
> > > It would be good to have a clear picture of what we want the final
> > > result to look like.
> > 
> > Well with current semantics of the 'nodename' argument controling both
> > where the populated bitmap is located and also which node's allocation
> > bitmap to take I don't think we can optimize it further in libvirt.
> > 
> > Current usage scenario is that we use a temporary bitmap populated with
> > the job to merge with bitmaps present in nodes which are removed by
> > blockjobs into the destination node of the block job. This means that
> > the real destination of the bits populated is in a different node than
> > it was originally and the above job semantics don't allow that.
> 
> So does this mean that a better API wouldn't only take a node-name and
> bitmap name (where the node identified by node-name is not only where
> the target bitmap is, but also the node whose allocation status is
> queried), but that it should take two different node-names for source
> (= reading allocation status) and target (= owner of the bitmap)?

Yes. That way one of the merges would be merged (heh) into the operation
itself preventing us from the need to have an extra temporary bitmap.

> > Either way I'd strongly prefer to be able to kick off all the populate
> > jobs at once rather than having to sequence them so any semantic change
> > towards making it possible to target bitmaps in a different node would
> > also require that multiple jobs can run in parallel with a single bitmap
> > as destination. I'm not sure if that doesn't overcomplicate things
> > though.
> 
> Other people are more familiar with the dirty bitmap code, so I may be
> wrong, but intuitively, I don't see any problem with multiple jobs
> dirtying blocks in the same bitmap. Or, with the internal temporary
> bitmap as used in this version of the series, multiple jobs that, one
> after another, merge their result to the same bitmap on completion.

I don't see a problem with the bitmaps itself since we are only OR-ing
results together. I just wanted to state the desired usage.

The above was actually inspired by a very recent problem I have in my
attempt to use the dirty bitmap populate job to refactor how libvirt
handles bitmaps. I've just figured out that I need to shuffle around
some stuff as I can't run the dirty-bitmap-populate job while an active
layer commit is in synchronised phase and I wanted to do the merging at
that point. That reminded me of a possible gotcha in having to sequence
the blockjobs which certainly would be more painful.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
> On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
> > Am 05.06.2020 um 11:24 hat Peter Krempa geschrieben:
> > > On Fri, Jun 05, 2020 at 11:01:23 +0200, Kevin Wolf wrote:
> > > > Am 04.06.2020 um 18:22 hat Peter Krempa geschrieben:
> > > > > On Thu, Jun 04, 2020 at 13:31:45 +0200, Kevin Wolf wrote:
> > > > > > Am 04.06.2020 um 11:16 hat Peter Krempa geschrieben:
> > > > > > > On Thu, Jun 04, 2020 at 11:12:31 +0200, Kevin Wolf wrote:
> > > > > > > > Am 18.05.2020 um 22:49 hat Eric Blake geschrieben:
> > > > > > > > > > +
> > > > > > > > > > +    /* NB: new bitmap is anonymous and enabled */
> > > > > > > > > > +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> > > > > > > > > > +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> > > > > > > > > > +    if (!new_bitmap) {
> > > > > > > > > > +        return NULL;
> > > > > > > > > > +    }
> > > > > > > > > 
> > > > > > > > > This means if the guest writes to the disk while the job is ongoing, the
> > > > > > > > > bitmap will be updated to mark that portion of the bitmap as set, even if it
> > > > > > > > > was not allocated at the time the job started.  But then again, the guest
> > > > > > > > > writes are causing allocation, so this seems like the right thing to do.
> > > > > > > > 
> > > > > > > > Is the target bitmap active at the same time, i.e. will it get the
> > > > > > > > correct information only from new_bitmap or are the bits already set in
> > > > > > > > it anyway?
> > > > > > > 
> > > > > > > Yes, libvirt plans to use it with an active non-persistent bitmap which
> > > > > > > will in subsequent steps be merged into others. The bitmap is added in
> > > > > > > the same transaction. The bitmap must be active, because we need to wait
> > > > > > > for the block jobs to finish before it becomes usable and thus can't
> > > > > > > sequence in other operations until later.
> > > > > > 
> > > > > > A lot of bitmap merging then, because the block job in this series
> > > > > > already creates a temporary internal bitmap that is merged into the
> > > > > > target bitmap on completion. But if the target bitmap is only libvirt's
> > > > > > temporary bitmap to be merged to yet another bitmap, I wonder if this
> > > > > > process shouldn't be simplified.
> > > > > 
> > > > > Possibly yes, but I'll leave that for later. All of this is done when
> > > > > executin very expensive operations anyways so for our first
> > > > > implementation it IMO won't matter that much.
> > > > 
> > > > I'm not necessarily saying that the change is needed on the libvirt
> > > > side. It could also be that the block job should directly work with the
> > > > given bitmap instead of having its internal temporary bitmap. Changing
> > > > this later would mean changing the semantics of the block job, so it
> > > > would be somewhat problematic.
> > > > 
> > > > It would be good to have a clear picture of what we want the final
> > > > result to look like.
> > > 
> > > Well with current semantics of the 'nodename' argument controling both
> > > where the populated bitmap is located and also which node's allocation
> > > bitmap to take I don't think we can optimize it further in libvirt.
> > > 
> > > Current usage scenario is that we use a temporary bitmap populated with
> > > the job to merge with bitmaps present in nodes which are removed by
> > > blockjobs into the destination node of the block job. This means that
> > > the real destination of the bits populated is in a different node than
> > > it was originally and the above job semantics don't allow that.
> > 
> > So does this mean that a better API wouldn't only take a node-name and
> > bitmap name (where the node identified by node-name is not only where
> > the target bitmap is, but also the node whose allocation status is
> > queried), but that it should take two different node-names for source
> > (= reading allocation status) and target (= owner of the bitmap)?
> 
> Yes. That way one of the merges would be merged (heh) into the operation
> itself preventing us from the need to have an extra temporary bitmap.
> 
> > > Either way I'd strongly prefer to be able to kick off all the populate
> > > jobs at once rather than having to sequence them so any semantic change
> > > towards making it possible to target bitmaps in a different node would
> > > also require that multiple jobs can run in parallel with a single bitmap
> > > as destination. I'm not sure if that doesn't overcomplicate things
> > > though.
> > 
> > Other people are more familiar with the dirty bitmap code, so I may be
> > wrong, but intuitively, I don't see any problem with multiple jobs
> > dirtying blocks in the same bitmap. Or, with the internal temporary
> > bitmap as used in this version of the series, multiple jobs that, one
> > after another, merge their result to the same bitmap on completion.
> 
> I don't see a problem with the bitmaps itself since we are only OR-ing
> results together. I just wanted to state the desired usage.
> 
> The above was actually inspired by a very recent problem I have in my
> attempt to use the dirty bitmap populate job to refactor how libvirt
> handles bitmaps. I've just figured out that I need to shuffle around
> some stuff as I can't run the dirty-bitmap-populate job while an active
> layer commit is in synchronised phase and I wanted to do the merging at
> that point. That reminded me of a possible gotcha in having to sequence
> the blockjobs which certainly would be more painful.

It would probably be good to have not only an iotests case that tests
the low-level functionality of the block job, but also one that
resembles the way libvirt would actually use it in combination with
other jobs.

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Fri, Jun 05, 2020 at 12:07:47 +0200, Kevin Wolf wrote:
> Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
> > On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:

[...]

> > The above was actually inspired by a very recent problem I have in my
> > attempt to use the dirty bitmap populate job to refactor how libvirt
> > handles bitmaps. I've just figured out that I need to shuffle around
> > some stuff as I can't run the dirty-bitmap-populate job while an active
> > layer commit is in synchronised phase and I wanted to do the merging at
> > that point. That reminded me of a possible gotcha in having to sequence
> > the blockjobs which certainly would be more painful.
> 
> It would probably be good to have not only an iotests case that tests
> the low-level functionality of the block job, but also one that
> resembles the way libvirt would actually use it in combination with
> other jobs.

I certainly can document the way we'll use it but that in turn depends
on how the job behaves.

With the current state of the job I plan to use it in two scenarios:

Preface: I'm currently changing libvirt to use one active bitmap per
checkpoint (checkpoint is name for the point in time we want to take
backup from). This means that a layer of the backing chain can have
multiple active bitmaps depending on how many checkpoints were created
in the current top layer. (previously we've tried to optimize things by
having just one bitmap active, but the semantics were getting too crazy
to be maintainable long-term)

Bitmaps are no longer propagated over to upper layers when creating
snapshots as we can use block-dirty-bitmap-populate instead.

1) backup

Prior to doing the backup I'm figuring out the final backup bitmap, this
involves creating a temporary bitmap populated by the job for every
layer of the backing chain above of the one which contains the bitmap we
want to take a backup from and then merge all of them together as a base
for the backup.

2) blockjobs

Note: This is currently an outline how the things should be as I've hit
the snag with attempting to run the population jobs during 'ready' state
of a active-layer block-commit/blockdev-mirror job only an hour ago and
I need to change a few things.

2.1) active layer block-commit/blockdev-mirror

When the job reaches 'ready' state I'll create bitmaps in the
destination/base image of the job for every bitmap in the images
dropped/merged (we use blockdev-mirror in full-sync mode) by the
blockjob. This will capture the writes that happen after 'job-complete'.

The job will then be completed and the 2.2. will be executed as well.

2.2) non-active commit and also continuation of active layer block-commit/blockdev-mirror

After the job is completed succesfully I'll create temporary
non-persistent bitmaps for/in the images removed by the blockjob and
merge them into the destination image's bitmaps depending on their
original location in the backing chain so that the bitmap state still
properly describes which blocks have changed.

After that the original images willbe blockdev-del-eted. The above is
partialy in use today and since the job is already completed also
requires blockdev-reopen to successfuly write to the bitmaps.

----

While writing the above down I've actually realized that controling the
destination of the bitmap might not be as useful as I've thought
originally as in 2.2. step I might need the allocation bitmap merged
into multiple bitmaps, so I'd either need a temporary bitmap anyways or
would have to re-run the job multiple times which seems wasteful. I'm no
longer fully persuaded that adding the 'merge' step to the dirty
populate blockjob will be the best thing since sliced bread.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Vladimir Sementsov-Ogievskiy]

05.06.2020 13:59, Peter Krempa wrote:
> On Fri, Jun 05, 2020 at 12:07:47 +0200, Kevin Wolf wrote:
>> Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
>>> On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
> 
> [...]
> 
>>> The above was actually inspired by a very recent problem I have in my
>>> attempt to use the dirty bitmap populate job to refactor how libvirt
>>> handles bitmaps. I've just figured out that I need to shuffle around
>>> some stuff as I can't run the dirty-bitmap-populate job while an active
>>> layer commit is in synchronised phase and I wanted to do the merging at
>>> that point. That reminded me of a possible gotcha in having to sequence
>>> the blockjobs which certainly would be more painful.
>>
>> It would probably be good to have not only an iotests case that tests
>> the low-level functionality of the block job, but also one that
>> resembles the way libvirt would actually use it in combination with
>> other jobs.
> 

Hi! Sorry me missing the discussion for a long time.

About new job semantics: if you create temporary bitmaps anyway, I do think that we should allow to populate into target bitmap directly, without creating any internal temporary bitmaps. I suggested it when reviewing v1, John argued for more transaction-like semantics to look like other jobs. Still, we can support both modes if we want.

Allowing to use one target for several populating job is an interesting idea. Current series does "bdrv_dirty_bitmap_set_busy(target_bitmap, true);", which forbids it.. Hmm. If we just drop it, nothing prevents user just remove target bitmap during the job. So, we'll need something like collective-busy bitmap..

> I certainly can document the way we'll use it but that in turn depends
> on how the job behaves.
> 
> With the current state of the job I plan to use it in two scenarios:
> 
> Preface: I'm currently changing libvirt to use one active bitmap per
> checkpoint (checkpoint is name for the point in time we want to take
> backup from). This means that a layer of the backing chain can have
> multiple active bitmaps depending on how many checkpoints were created
> in the current top layer. (previously we've tried to optimize things by
> having just one bitmap active, but the semantics were getting too crazy
> to be maintainable long-term)

Hmm. I had a plan of creating "lazy" disabled bitmaps, to optimize scenario with one active bitmap, so that disabled bitmaps are not loaded into RAM on start, but only on demand. But how to do it with "many active bitmaps" scenario? I don't think that's a good idea.. Possibly, we can implement laziness by internally make only one active bitmap and merge it here and there when you request some active bitmap which we actually didn't load yet..

Could you describe, what is the exact problem with "several disabled - one active" scheme, and how is it solved by "several active"?

> 
> Bitmaps are no longer propagated over to upper layers when creating
> snapshots as we can use block-dirty-bitmap-populate instead.

Unexpected turn. When all this topic only started, it was reasoned more like "if user forget to create bitmap at start, let's help him".. But now it becomes the common scenario. Hmm.

What do you think of granularity? We in Virtuozzo use 1M cluster as a default for qcow2 images. But we use 64k granularity (default) for bitmaps, to have smaller incremental backups. So, this is advantage of creating bitmap over relaying on block-status capturing by block-dirty-bitmap-populate: you don't control dirtiness granularity. So, I think that bitmap propagation, or just creating new dirty bitmap to track dirtiness from start of new snapshot is better.


> 
> 1) backup
> 
> Prior to doing the backup I'm figuring out the final backup bitmap, this
> involves creating a temporary bitmap populated by the job for every
> layer of the backing chain above of the one which contains the bitmap we
> want to take a backup from and then merge all of them together as a base
> for the backup.

(just thinking out loud)

So, assume the sequence top -> middle -> base

If we have a backup, which was done when we were in base, than bitmap is stored in base. And  is loaded, and is active, but don't changes really, as base is opened read-only.]
We merge block-status information of top and middle together with this bitmap, and aggregate difference between last backup and current state.

> 
> 2) blockjobs
> 
> Note: This is currently an outline how the things should be as I've hit
> the snag with attempting to run the population jobs during 'ready' state
> of a active-layer block-commit/blockdev-mirror job only an hour ago and
> I need to change a few things.
> 
> 2.1) active layer block-commit/blockdev-mirror
> 
> When the job reaches 'ready' state I'll create bitmaps in the
> destination/base image of the job for every bitmap in the images
> dropped/merged (we use blockdev-mirror in full-sync mode) by the
> blockjob. This will capture the writes that happen after 'job-complete'.
> 
> The job will then be completed and the 2.2. will be executed as well.

So, the aim is not to miss any new writes after switching to new bs, but do not capture into bitmaps writes which are copying the whole disk during mirror.

> 
> 2.2) non-active commit and also continuation of active layer block-commit/blockdev-mirror
> 
> After the job is completed succesfully I'll create temporary
> non-persistent bitmaps for/in the images removed by the blockjob and
> merge them into the destination image's bitmaps depending on their
> original location in the backing chain so that the bitmap state still
> properly describes which blocks have changed.

I don't follow. How do you populate these new temporary bitmaps? They are empty after creation..

> 
> After that the original images willbe blockdev-del-eted. The above is
> partialy in use today and since the job is already completed also
> requires blockdev-reopen to successfuly write to the bitmaps.
> 
> ----
> 
> While writing the above down I've actually realized that controling the
> destination of the bitmap might not be as useful as I've thought
> originally as in 2.2. step I might need the allocation bitmap merged
> into multiple bitmaps, so I'd either need a temporary bitmap anyways or
> would have to re-run the job multiple times which seems wasteful. I'm no
> longer fully persuaded that adding the 'merge' step to the dirty
> populate blockjob will be the best thing since sliced bread.
> 

What is 'merge' step? Do you mean that populating directly into target bitmap is not really needed?

-- 
Best regards,
Vladimir

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 06.06.2020 um 08:55 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 05.06.2020 13:59, Peter Krempa wrote:
> > On Fri, Jun 05, 2020 at 12:07:47 +0200, Kevin Wolf wrote:
> > > Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
> > > > On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
> > 
> > [...]
> > 
> > > > The above was actually inspired by a very recent problem I have in my
> > > > attempt to use the dirty bitmap populate job to refactor how libvirt
> > > > handles bitmaps. I've just figured out that I need to shuffle around
> > > > some stuff as I can't run the dirty-bitmap-populate job while an active
> > > > layer commit is in synchronised phase and I wanted to do the merging at
> > > > that point. That reminded me of a possible gotcha in having to sequence
> > > > the blockjobs which certainly would be more painful.
> > > 
> > > It would probably be good to have not only an iotests case that tests
> > > the low-level functionality of the block job, but also one that
> > > resembles the way libvirt would actually use it in combination with
> > > other jobs.
> > 
> 
> Hi! Sorry me missing the discussion for a long time.
> 
> About new job semantics: if you create temporary bitmaps anyway, I do
> think that we should allow to populate into target bitmap directly,
> without creating any internal temporary bitmaps. I suggested it when
> reviewing v1, John argued for more transaction-like semantics to look
> like other jobs. Still, we can support both modes if we want.

But don't all other jobs allow you to see intermediate states? Like,
when you look at the target node in the middle of a mirror job, you'll
see a half-updated target image.

If we have an actual use case for both modes, we can certainly support
that, but do we have one?

> Allowing to use one target for several populating job is an
> interesting idea. Current series does
> "bdrv_dirty_bitmap_set_busy(target_bitmap, true);", which forbids it..
> Hmm. If we just drop it, nothing prevents user just remove target
> bitmap during the job. So, we'll need something like collective-busy
> bitmap..

I'm not sure for what the busy flag is used in detail, but if this is
the problem, maybe it's possible to just replace it with a counter?

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Vladimir Sementsov-Ogievskiy]

08.06.2020 12:21, Kevin Wolf wrote:
> Am 06.06.2020 um 08:55 hat Vladimir Sementsov-Ogievskiy geschrieben:
>> 05.06.2020 13:59, Peter Krempa wrote:
>>> On Fri, Jun 05, 2020 at 12:07:47 +0200, Kevin Wolf wrote:
>>>> Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
>>>>> On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
>>>
>>> [...]
>>>
>>>>> The above was actually inspired by a very recent problem I have in my
>>>>> attempt to use the dirty bitmap populate job to refactor how libvirt
>>>>> handles bitmaps. I've just figured out that I need to shuffle around
>>>>> some stuff as I can't run the dirty-bitmap-populate job while an active
>>>>> layer commit is in synchronised phase and I wanted to do the merging at
>>>>> that point. That reminded me of a possible gotcha in having to sequence
>>>>> the blockjobs which certainly would be more painful.
>>>>
>>>> It would probably be good to have not only an iotests case that tests
>>>> the low-level functionality of the block job, but also one that
>>>> resembles the way libvirt would actually use it in combination with
>>>> other jobs.
>>>
>>
>> Hi! Sorry me missing the discussion for a long time.
>>
>> About new job semantics: if you create temporary bitmaps anyway, I do
>> think that we should allow to populate into target bitmap directly,
>> without creating any internal temporary bitmaps. I suggested it when
>> reviewing v1, John argued for more transaction-like semantics to look
>> like other jobs. Still, we can support both modes if we want.
> 
> But don't all other jobs allow you to see intermediate states? Like,
> when you look at the target node in the middle of a mirror job, you'll
> see a half-updated target image.

That was my argument on v1 :)

> 
> If we have an actual use case for both modes, we can certainly support
> that, but do we have one?

No. We have only one: without extra temporary bitmap, let's implement it.
We can always add transaction-like case later on demand.

> 
>> Allowing to use one target for several populating job is an
>> interesting idea. Current series does
>> "bdrv_dirty_bitmap_set_busy(target_bitmap, true);", which forbids it..
>> Hmm. If we just drop it, nothing prevents user just remove target
>> bitmap during the job. So, we'll need something like collective-busy
>> bitmap..
> 
> I'm not sure for what the busy flag is used in detail, but if this is
> the problem, maybe it's possible to just replace it with a counter?
> 

busy flag means that bitmap is already in-use by some process (for example
backup, or exported through NBD, or being migrated). User can't change or
use busy bitmaps for another jobs. So multiple jobs writing into one bitmaps
should be an exclusion from this rule (we want to allow several similar
block-jobs, but still don't want to allow migration or NBD-export in the same
time, or bitmap removing). I think we can just hardcode this case, add a new
flag, which says that bitmap is used as target of populating jobs and being
busy still allowed as a target for another populating job.

-- 
Best regards,
Vladimir

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 08.06.2020 um 12:00 hat Vladimir Sementsov-Ogievskiy geschrieben:
> 08.06.2020 12:21, Kevin Wolf wrote:
> > Am 06.06.2020 um 08:55 hat Vladimir Sementsov-Ogievskiy geschrieben:
> > > Allowing to use one target for several populating job is an
> > > interesting idea. Current series does
> > > "bdrv_dirty_bitmap_set_busy(target_bitmap, true);", which forbids it..
> > > Hmm. If we just drop it, nothing prevents user just remove target
> > > bitmap during the job. So, we'll need something like collective-busy
> > > bitmap..
> > 
> > I'm not sure for what the busy flag is used in detail, but if this is
> > the problem, maybe it's possible to just replace it with a counter?
> 
> busy flag means that bitmap is already in-use by some process (for
> example backup, or exported through NBD, or being migrated). User
> can't change or use busy bitmaps for another jobs.

I think this rule is overly restrictive. That you can't delete a bitmap
that is in use is obvious. That you can't have more than one "consumer"
of dirty bits that clears bits after processing them seems at least
reasonable. But why shouldn't you be able to have more than one
"producer" of dirty bits?

> So multiple jobs writing into one bitmaps should be an exclusion from
> this rule (we want to allow several similar block-jobs, but still
> don't want to allow migration or NBD-export in the same time, or
> bitmap removing). I think we can just hardcode this case, add a new
> flag, which says that bitmap is used as target of populating jobs and
> being busy still allowed as a target for another populating job.

I think we need to find a more general pattern. Maybe the distinction I
made above between producer (setting dirty bits) and consumer (clearing
dirty bits) is what we need, or maybe we need something else to
distinguish roles. But I don't think the definition should have any
reference to bitmap-populate.

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Sat, Jun 06, 2020 at 09:55:13 +0300, Vladimir Sementsov-Ogievskiy wrote:
> 05.06.2020 13:59, Peter Krempa wrote:
> > On Fri, Jun 05, 2020 at 12:07:47 +0200, Kevin Wolf wrote:
> > > Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
> > > > On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
> > 
> > [...]
> > 
> > > > The above was actually inspired by a very recent problem I have in my
> > > > attempt to use the dirty bitmap populate job to refactor how libvirt
> > > > handles bitmaps. I've just figured out that I need to shuffle around
> > > > some stuff as I can't run the dirty-bitmap-populate job while an active
> > > > layer commit is in synchronised phase and I wanted to do the merging at
> > > > that point. That reminded me of a possible gotcha in having to sequence
> > > > the blockjobs which certainly would be more painful.
> > > 
> > > It would probably be good to have not only an iotests case that tests
> > > the low-level functionality of the block job, but also one that
> > > resembles the way libvirt would actually use it in combination with
> > > other jobs.
> > 
> 
> Hi! Sorry me missing the discussion for a long time.
> 
> About new job semantics: if you create temporary bitmaps anyway, I do think that we should allow to populate into target bitmap directly, without creating any internal temporary bitmaps. I suggested it when reviewing v1, John argued for more transaction-like semantics to look like other jobs. Still, we can support both modes if we want.
> 
> Allowing to use one target for several populating job is an interesting idea. Current series does "bdrv_dirty_bitmap_set_busy(target_bitmap, true);", which forbids it.. Hmm. If we just drop it, nothing prevents user just remove target bitmap during the job. So, we'll need something like collective-busy bitmap..
> 
> > I certainly can document the way we'll use it but that in turn depends
> > on how the job behaves.
> > 
> > With the current state of the job I plan to use it in two scenarios:
> > 
> > Preface: I'm currently changing libvirt to use one active bitmap per
> > checkpoint (checkpoint is name for the point in time we want to take
> > backup from). This means that a layer of the backing chain can have
> > multiple active bitmaps depending on how many checkpoints were created
> > in the current top layer. (previously we've tried to optimize things by
> > having just one bitmap active, but the semantics were getting too crazy
> > to be maintainable long-term)
> 
> Hmm. I had a plan of creating "lazy" disabled bitmaps, to optimize scenario with one active bitmap, so that disabled bitmaps are not loaded into RAM on start, but only on demand. But how to do it with "many active bitmaps" scenario? I don't think that's a good idea.. Possibly, we can implement laziness by internally make only one active bitmap and merge it here and there when you request some active bitmap which we actually didn't load yet..
> 
> Could you describe, what is the exact problem with "several disabled - one active" scheme, and how is it solved by "several active"?

The 'several disabled one active' semantics _heavily_ depend on metadata
which must be tracked outside of qemu and is more prone to break. If any
of the intermediate bitmaps is broken or missing everything breaks.

Then there's the complexity of the code which handles merging of the
bitmaps during block jobs. Jobs such as blockdev-mirror in full mode and
block-commit squash together the data and we need to do something about
the bitmaps for the backups to work properly afterwards.

Without considering overlays which were created without propagating
bitmaps, the code was already getting hairy especially in the case of
backups where we needed to stitch together bitmaps for all the bitmaps
corresponding to the given point in time where the backup is taken from.

When we add overlays without any bitmaps into the mix the code for
resolving which bitmaps to merge the code is becoming very unpleasant,
hard to understand and maintain and that is the main point for the
switch.

I don't want to add unnecessary complexity to the libvirt code which
will make it more fragile or hard to understand and fix in the future.

Both points which I heard for now (performance, and backup granularity
in case of non-default qcow2 block size) don't seem compelling enough to
me to make my life of implementing the feature in libvirt so much
harder.

Also users really can just remove the point in time they wish to backup
from after a successful backup which will also remove the corresponding
active bitmap.

> > Bitmaps are no longer propagated over to upper layers when creating
> > snapshots as we can use block-dirty-bitmap-populate instead.
> 
> Unexpected turn. When all this topic only started, it was reasoned more like "if user forget to create bitmap at start, let's help him".. But now it becomes the common scenario. Hmm.

It's not only a "user forgot" thing, but more that a systemic change
would be required.

Additionally until _very_ recently it wasn't possible to create bitmaps
using qemu-img, which made it impossible to create overlays for inactive
VMs. Arguably this has changed so we could require it. It still adds a
moving part which can break if the user doesn't add the bitmap or
requires yet another special case handling if we want to compensate for
that.

As of such, in libvirt's tech-preview implementation that is present
currently upstream, if you create a qcow2 overlay without adding the
appropriate bitmaps, the backups simply won't work.

> What do you think of granularity? We in Virtuozzo use 1M cluster as a default for qcow2 images. But we use 64k granularity (default) for bitmaps, to have smaller incremental backups. So, this is advantage of creating bitmap over relaying on block-status capturing by block-dirty-bitmap-populate: you don't control dirtiness granularity. So, I think that bitmap propagation, or just creating new dirty bitmap to track dirtiness from start of new snapshot is better.

This is a valid argument. Backups in this scenario will be bigger. I
still don't feel like the code needs to be made much more complex
because of it though.

> > 1) backup
> > 
> > Prior to doing the backup I'm figuring out the final backup bitmap, this
> > involves creating a temporary bitmap populated by the job for every
> > layer of the backing chain above of the one which contains the bitmap we
> > want to take a backup from and then merge all of them together as a base
> > for the backup.
> 
> (just thinking out loud)
> 
> So, assume the sequence top -> middle -> base
> 
> If we have a backup, which was done when we were in base, than bitmap is stored in base. And  is loaded, and is active, but don't changes really, as base is opened read-only.]
> We merge block-status information of top and middle together with this bitmap, and aggregate difference between last backup and current state.
> 
> > 
> > 2) blockjobs
> > 
> > Note: This is currently an outline how the things should be as I've hit
> > the snag with attempting to run the population jobs during 'ready' state
> > of a active-layer block-commit/blockdev-mirror job only an hour ago and
> > I need to change a few things.
> > 
> > 2.1) active layer block-commit/blockdev-mirror
> > 
> > When the job reaches 'ready' state I'll create bitmaps in the
> > destination/base image of the job for every bitmap in the images
> > dropped/merged (we use blockdev-mirror in full-sync mode) by the
> > blockjob. This will capture the writes that happen after 'job-complete'.
> > 
> > The job will then be completed and the 2.2. will be executed as well.
> 
> So, the aim is not to miss any new writes after switching to new bs, but do not capture into bitmaps writes which are copying the whole disk during mirror.
> 
> > 
> > 2.2) non-active commit and also continuation of active layer block-commit/blockdev-mirror
> > 
> > After the job is completed succesfully I'll create temporary
> > non-persistent bitmaps for/in the images removed by the blockjob and
> > merge them into the destination image's bitmaps depending on their
> > original location in the backing chain so that the bitmap state still
> > properly describes which blocks have changed.
> 
> I don't follow. How do you populate these new temporary bitmaps? They are empty after creation..

With the 'block-dirty-bitmap-populate' block job.

> > After that the original images willbe blockdev-del-eted. The above is
> > partialy in use today and since the job is already completed also
> > requires blockdev-reopen to successfuly write to the bitmaps.
> > 
> > ----
> > 
> > While writing the above down I've actually realized that controling the
> > destination of the bitmap might not be as useful as I've thought
> > originally as in 2.2. step I might need the allocation bitmap merged
> > into multiple bitmaps, so I'd either need a temporary bitmap anyways or
> > would have to re-run the job multiple times which seems wasteful. I'm no
> > longer fully persuaded that adding the 'merge' step to the dirty
> > populate blockjob will be the best thing since sliced bread.
> > 
> 
> What is 'merge' step?

In some previous replies to Kevin, we discussed that it might be worth
optimizing 'block-dirty-bitmap-populate' to directly populate the bits
in the target bitmap rather than after the job is complete, so
efectively directly mering it. I probably described it wrong here.

>Do you mean that populating directly into target bitmap is not really needed?

I need the bitmap populated by 'block-dirty-bitmap-populate' to be
merged into multiple bitmaps in the new semantics. If the job itself
doesn't support that sematics, changing it to just to directly populate
one bitmap doesn't seem to be worth it since I'll be using intermediate
bitmaps anyways.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Vladimir Sementsov-Ogievskiy]

08.06.2020 12:38, Peter Krempa wrote:
> On Sat, Jun 06, 2020 at 09:55:13 +0300, Vladimir Sementsov-Ogievskiy wrote:
>> 05.06.2020 13:59, Peter Krempa wrote:
>>> On Fri, Jun 05, 2020 at 12:07:47 +0200, Kevin Wolf wrote:
>>>> Am 05.06.2020 um 11:58 hat Peter Krempa geschrieben:
>>>>> On Fri, Jun 05, 2020 at 11:44:07 +0200, Kevin Wolf wrote:
>>>
>>> [...]
>>>
>>>>> The above was actually inspired by a very recent problem I have in my
>>>>> attempt to use the dirty bitmap populate job to refactor how libvirt
>>>>> handles bitmaps. I've just figured out that I need to shuffle around
>>>>> some stuff as I can't run the dirty-bitmap-populate job while an active
>>>>> layer commit is in synchronised phase and I wanted to do the merging at
>>>>> that point. That reminded me of a possible gotcha in having to sequence
>>>>> the blockjobs which certainly would be more painful.
>>>>
>>>> It would probably be good to have not only an iotests case that tests
>>>> the low-level functionality of the block job, but also one that
>>>> resembles the way libvirt would actually use it in combination with
>>>> other jobs.
>>>
>>
>> Hi! Sorry me missing the discussion for a long time.
>>
>> About new job semantics: if you create temporary bitmaps anyway, I do think that we should allow to populate into target bitmap directly, without creating any internal temporary bitmaps. I suggested it when reviewing v1, John argued for more transaction-like semantics to look like other jobs. Still, we can support both modes if we want.
>>
>> Allowing to use one target for several populating job is an interesting idea. Current series does "bdrv_dirty_bitmap_set_busy(target_bitmap, true);", which forbids it.. Hmm. If we just drop it, nothing prevents user just remove target bitmap during the job. So, we'll need something like collective-busy bitmap..
>>
>>> I certainly can document the way we'll use it but that in turn depends
>>> on how the job behaves.
>>>
>>> With the current state of the job I plan to use it in two scenarios:
>>>
>>> Preface: I'm currently changing libvirt to use one active bitmap per
>>> checkpoint (checkpoint is name for the point in time we want to take
>>> backup from). This means that a layer of the backing chain can have
>>> multiple active bitmaps depending on how many checkpoints were created
>>> in the current top layer. (previously we've tried to optimize things by
>>> having just one bitmap active, but the semantics were getting too crazy
>>> to be maintainable long-term)
>>
>> Hmm. I had a plan of creating "lazy" disabled bitmaps, to optimize scenario with one active bitmap, so that disabled bitmaps are not loaded into RAM on start, but only on demand. But how to do it with "many active bitmaps" scenario? I don't think that's a good idea.. Possibly, we can implement laziness by internally make only one active bitmap and merge it here and there when you request some active bitmap which we actually didn't load yet..
>>
>> Could you describe, what is the exact problem with "several disabled - one active" scheme, and how is it solved by "several active"?
> 
> The 'several disabled one active' semantics _heavily_ depend on metadata
> which must be tracked outside of qemu and is more prone to break. If any
> of the intermediate bitmaps is broken or missing everything breaks.
> 
> Then there's the complexity of the code which handles merging of the
> bitmaps during block jobs. Jobs such as blockdev-mirror in full mode and
> block-commit squash together the data and we need to do something about
> the bitmaps for the backups to work properly afterwards.
> 
> Without considering overlays which were created without propagating
> bitmaps, the code was already getting hairy especially in the case of
> backups where we needed to stitch together bitmaps for all the bitmaps
> corresponding to the given point in time where the backup is taken from.
> 
> When we add overlays without any bitmaps into the mix the code for
> resolving which bitmaps to merge the code is becoming very unpleasant,
> hard to understand and maintain and that is the main point for the
> switch.
> 
> I don't want to add unnecessary complexity to the libvirt code which
> will make it more fragile or hard to understand and fix in the future.
> 
> Both points which I heard for now (performance, and backup granularity
> in case of non-default qcow2 block size) don't seem compelling enough to
> me to make my life of implementing the feature in libvirt so much
> harder.
> 
> Also users really can just remove the point in time they wish to backup
> from after a successful backup which will also remove the corresponding
> active bitmap.

That's all is reasonable enough..

Than, we really need to refactor the code around bitmap support. Currently we
do keep all active bitmaps in RAM and update them in a loop on each write.
But it's obvious, the we can keep only one (with smallest granularity of existing
dirty bitmaps) to track guest writes.

> 
>>> Bitmaps are no longer propagated over to upper layers when creating
>>> snapshots as we can use block-dirty-bitmap-populate instead.
>>
>> Unexpected turn. When all this topic only started, it was reasoned more like "if user forget to create bitmap at start, let's help him".. But now it becomes the common scenario. Hmm.
> 
> It's not only a "user forgot" thing, but more that a systemic change
> would be required.
> 
> Additionally until _very_ recently it wasn't possible to create bitmaps
> using qemu-img, which made it impossible to create overlays for inactive

Didn't you consider to use qemu started in stopped mode to do block
operations in same manner as for running vm? What's wrong with it?
Also, there is qemu-storage-daemon, which is developed as separated
binary, where block-layer is compiled in together with QMP interface.

> VMs. Arguably this has changed so we could require it. It still adds a
> moving part which can break if the user doesn't add the bitmap or
> requires yet another special case handling if we want to compensate for
> that.
> 
> As of such, in libvirt's tech-preview implementation that is present
> currently upstream, if you create a qcow2 overlay without adding the
> appropriate bitmaps, the backups simply won't work.
> 
>> What do you think of granularity? We in Virtuozzo use 1M cluster as a default for qcow2 images. But we use 64k granularity (default) for bitmaps, to have smaller incremental backups. So, this is advantage of creating bitmap over relaying on block-status capturing by block-dirty-bitmap-populate: you don't control dirtiness granularity. So, I think that bitmap propagation, or just creating new dirty bitmap to track dirtiness from start of new snapshot is better.
> 
> This is a valid argument. Backups in this scenario will be bigger. I
> still don't feel like the code needs to be made much more complex
> because of it though.

May be, there is a simple solution? an option for blockdev-snapshot-sync to create a bitmap in a new image (or if you create image by qemu-img, just create bitmap by qemu-img as well, using new functionality).

Isn't it simpler than to just use existing block-status-bitmap, than run a job?

> 
>>> 1) backup
>>>
>>> Prior to doing the backup I'm figuring out the final backup bitmap, this
>>> involves creating a temporary bitmap populated by the job for every
>>> layer of the backing chain above of the one which contains the bitmap we
>>> want to take a backup from and then merge all of them together as a base
>>> for the backup.
>>
>> (just thinking out loud)
>>
>> So, assume the sequence top -> middle -> base
>>
>> If we have a backup, which was done when we were in base, than bitmap is stored in base. And  is loaded, and is active, but don't changes really, as base is opened read-only.]
>> We merge block-status information of top and middle together with this bitmap, and aggregate difference between last backup and current state.
>>
>>>
>>> 2) blockjobs
>>>
>>> Note: This is currently an outline how the things should be as I've hit
>>> the snag with attempting to run the population jobs during 'ready' state
>>> of a active-layer block-commit/blockdev-mirror job only an hour ago and
>>> I need to change a few things.
>>>
>>> 2.1) active layer block-commit/blockdev-mirror
>>>
>>> When the job reaches 'ready' state I'll create bitmaps in the
>>> destination/base image of the job for every bitmap in the images
>>> dropped/merged (we use blockdev-mirror in full-sync mode) by the
>>> blockjob. This will capture the writes that happen after 'job-complete'.
>>>
>>> The job will then be completed and the 2.2. will be executed as well.
>>
>> So, the aim is not to miss any new writes after switching to new bs, but do not capture into bitmaps writes which are copying the whole disk during mirror.
>>
>>>
>>> 2.2) non-active commit and also continuation of active layer block-commit/blockdev-mirror
>>>
>>> After the job is completed succesfully I'll create temporary
>>> non-persistent bitmaps for/in the images removed by the blockjob and
>>> merge them into the destination image's bitmaps depending on their
>>> original location in the backing chain so that the bitmap state still
>>> properly describes which blocks have changed.
>>
>> I don't follow. How do you populate these new temporary bitmaps? They are empty after creation..
> 
> With the 'block-dirty-bitmap-populate' block job.
> 
>>> After that the original images willbe blockdev-del-eted. The above is
>>> partialy in use today and since the job is already completed also
>>> requires blockdev-reopen to successfuly write to the bitmaps.
>>>
>>> ----
>>>
>>> While writing the above down I've actually realized that controling the
>>> destination of the bitmap might not be as useful as I've thought
>>> originally as in 2.2. step I might need the allocation bitmap merged
>>> into multiple bitmaps, so I'd either need a temporary bitmap anyways or
>>> would have to re-run the job multiple times which seems wasteful. I'm no
>>> longer fully persuaded that adding the 'merge' step to the dirty
>>> populate blockjob will be the best thing since sliced bread.
>>>
>>
>> What is 'merge' step?
> 
> In some previous replies to Kevin, we discussed that it might be worth
> optimizing 'block-dirty-bitmap-populate' to directly populate the bits
> in the target bitmap rather than after the job is complete, so
> efectively directly mering it. I probably described it wrong here.
> 
>> Do you mean that populating directly into target bitmap is not really needed?
> 
> I need the bitmap populated by 'block-dirty-bitmap-populate' to be
> merged into multiple bitmaps in the new semantics. If the job itself
> doesn't support that sematics, changing it to just to directly populate
> one bitmap doesn't seem to be worth it since I'll be using intermediate
> bitmaps anyways.
> 

Hmm, if the main use case of populating job is to merge changes since snapshot to several bitmaps (all active bitmaps?), than I think it's correct to implement exactly this semantics, allowing a list of targets as well as list of source bitmaps. We even can reuse same structure for target-list which we use for source-list. And it's simple to implement in Qemu.

-- 
Best regards,
Vladimir

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Peter Krempa]

On Mon, Jun 08, 2020 at 13:30:48 +0300, Vladimir Sementsov-Ogievskiy wrote:
> 08.06.2020 12:38, Peter Krempa wrote:
> > On Sat, Jun 06, 2020 at 09:55:13 +0300, Vladimir Sementsov-Ogievskiy wrote:
> > > 05.06.2020 13:59, Peter Krempa wrote:

[...]

> > 
> > It's not only a "user forgot" thing, but more that a systemic change
> > would be required.
> > 
> > Additionally until _very_ recently it wasn't possible to create bitmaps
> > using qemu-img, which made it impossible to create overlays for inactive
> 
> Didn't you consider to use qemu started in stopped mode to do block
> operations in same manner as for running vm? What's wrong with it?
> Also, there is qemu-storage-daemon, which is developed as separated
> binary, where block-layer is compiled in together with QMP interface.

It's still considered, but I didn't have time to implement anything
related to it and nobody else implemented it either.

Additionally the problem isn't in libvirt's handling but more in other
apps handling. We still due to historical reasons support if users
create overlays outside of libvirt.

This would mean that either backups break if the overlay is not done
properly or we have to have a fallback to use
'block-dirty-bitmap-populate'. At this point for both my sanity and
actually finishing all the details in regards to incremental backup

> > VMs. Arguably this has changed so we could require it. It still adds a
> > moving part which can break if the user doesn't add the bitmap or
> > requires yet another special case handling if we want to compensate for
> > that.
> > 
> > As of such, in libvirt's tech-preview implementation that is present
> > currently upstream, if you create a qcow2 overlay without adding the
> > appropriate bitmaps, the backups simply won't work.
> > 
> > > What do you think of granularity? We in Virtuozzo use 1M cluster as a default for qcow2 images. But we use 64k granularity (default) for bitmaps, to have smaller incremental backups. So, this is advantage of creating bitmap over relaying on block-status capturing by block-dirty-bitmap-populate: you don't control dirtiness granularity. So, I think that bitmap propagation, or just creating new dirty bitmap to track dirtiness from start of new snapshot is better.
> > 
> > This is a valid argument. Backups in this scenario will be bigger. I
> > still don't feel like the code needs to be made much more complex
> > because of it though.
> 
> May be, there is a simple solution? an option for blockdev-snapshot-sync to create a bitmap in a new image (or if you create image by qemu-img, just create bitmap by qemu-img as well, using new functionality).

You mean like we do now?:

https://gitlab.com/libvirt/libvirt/-/blob/master/src/qemu/qemu_driver.c#L15020

That is slated to be deleted with my patchset though.

Good thing is that we can theoretically re-add it later.

For now I'd go with the simpler option that has fewer side effects.

> Isn't it simpler than to just use existing block-status-bitmap, than run a job?

No. Because we'd still need to support cases where user added an overlay
without the appropriate bitmap. As said I'd prefer to keep the code
simple and then work on optimizations. Good thing is that with 'one
active bitmap per point in time' this is simpler to achieve.

[...]

> > > 
> > > What is 'merge' step?
> > 
> > In some previous replies to Kevin, we discussed that it might be worth
> > optimizing 'block-dirty-bitmap-populate' to directly populate the bits
> > in the target bitmap rather than after the job is complete, so
> > efectively directly mering it. I probably described it wrong here.
> > 
> > > Do you mean that populating directly into target bitmap is not really needed?
> > 
> > I need the bitmap populated by 'block-dirty-bitmap-populate' to be
> > merged into multiple bitmaps in the new semantics. If the job itself
> > doesn't support that sematics, changing it to just to directly populate
> > one bitmap doesn't seem to be worth it since I'll be using intermediate
> > bitmaps anyways.
> > 
> 
> Hmm, if the main use case of populating job is to merge changes since snapshot to several bitmaps (all active bitmaps?), than I think it's correct to implement exactly this semantics, allowing a list of targets as well as list of source bitmaps. We even can reuse same structure for target-list which we use for source-list. And it's simple to implement in Qemu.

I'd mainly like for the design to converge. I actually have almost
complete patches which use the current semantics. I'm okay with
reworking it but at some point there should be a line when it won't
change.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 14.05.2020 um 05:49 hat John Snow geschrieben:
> This job copies the allocation map into a bitmap. It's a job because
> there's no guarantee that allocation interrogation will be quick (or
> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
> 
> It was designed with different possible population patterns in mind,
> but only top layer allocation was implemented for now.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>  qapi/block-core.json      |  48 +++++++++
>  qapi/job.json             |   2 +-
>  include/block/block_int.h |  21 ++++
>  block/bitmap-alloc.c      | 207 ++++++++++++++++++++++++++++++++++++++

bitmap-populate.c to be more consistent with the actual job name?

>  blockjob.c                |   3 +-
>  block/Makefile.objs       |   1 +
>  6 files changed, 280 insertions(+), 2 deletions(-)
>  create mode 100644 block/bitmap-alloc.c

[...]

> +BlockJob *bitpop_job_create(
> +    const char *job_id,
> +    BlockDriverState *bs,
> +    BdrvDirtyBitmap *target_bitmap,
> +    BitmapPattern pattern,
> +    BlockdevOnError on_error,
> +    int creation_flags,
> +    BlockCompletionFunc *cb,
> +    void *opaque,
> +    JobTxn *txn,
> +    Error **errp)
> +{
> +    int64_t len;
> +    BitpopBlockJob *job = NULL;
> +    int64_t cluster_size;
> +    BdrvDirtyBitmap *new_bitmap = NULL;
> +
> +    assert(bs);
> +    assert(target_bitmap);
> +
> +    if (!bdrv_is_inserted(bs)) {
> +        error_setg(errp, "Device is not inserted: %s",
> +                   bdrv_get_device_name(bs));
> +        return NULL;
> +    }
> +
> +    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
> +        return NULL;
> +    }

What does this protect? And why does BACKUP_SOURCE describe acccurately
what this job does?

> +    if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
> +        return NULL;
> +    }
> +
> +    if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
> +        error_setg(errp, "Unrecognized bitmap pattern");
> +        return NULL;
> +    }
> +
> +    len = bdrv_getlength(bs);
> +    if (len < 0) {
> +        error_setg_errno(errp, -len, "unable to get length for '%s'",
> +                         bdrv_get_device_name(bs));

This operates on the node level, so bdrv_get_device_or_node_name() is
necessary to avoid empty strings in the message.

> +        return NULL;
> +    }
> +
> +    /* NB: new bitmap is anonymous and enabled */
> +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
> +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
> +    if (!new_bitmap) {
> +        return NULL;
> +    }
> +
> +    /* Take ownership; we reserve the right to write into this on-commit. */
> +    bdrv_dirty_bitmap_set_busy(target_bitmap, true);
> +
> +    job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
> +                           BLK_PERM_CONSISTENT_READ,

I don't think we actually rely on CONSISTENT_READ, but then, using the
job on inconsistent nodes probably makes little sense and we can always
relax the restriction later if necessary.

> +                           BLK_PERM_ALL & ~BLK_PERM_RESIZE,
> +                           0, creation_flags,
> +                           cb, opaque, errp);
> +    if (!job) {
> +        bdrv_dirty_bitmap_set_busy(target_bitmap, false);
> +        bdrv_release_dirty_bitmap(new_bitmap);
> +        return NULL;
> +    }
> +
> +    job->bs = bs;
> +    job->on_error = on_error;
> +    job->target_bitmap = target_bitmap;
> +    job->new_bitmap = new_bitmap;
> +    job->len = len;
> +    job_progress_set_remaining(&job->common.job, job->len);
> +
> +    return &job->common;
> +}

Kevin

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Eric Blake]

On 6/4/20 4:01 AM, Kevin Wolf wrote:
> Am 14.05.2020 um 05:49 hat John Snow geschrieben:
>> This job copies the allocation map into a bitmap. It's a job because
>> there's no guarantee that allocation interrogation will be quick (or
>> won't hang), so it cannot be retrofit into block-dirty-bitmap-merge.
>>
>> It was designed with different possible population patterns in mind,
>> but only top layer allocation was implemented for now.

Other patterns that might make sense someday:
all-ones (we already have bitmap-clear for all zeroes)
an inverse flag (set bits for all unallocated portions)
compressed (set bits for portions that are compressed)

but yeah, I don't see it worth implementing any of them without a client.

>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>>   qapi/block-core.json      |  48 +++++++++
>>   qapi/job.json             |   2 +-
>>   include/block/block_int.h |  21 ++++
>>   block/bitmap-alloc.c      | 207 ++++++++++++++++++++++++++++++++++++++
> 
> bitmap-populate.c to be more consistent with the actual job name?

Seems reasonable to me.

> 
>>   blockjob.c                |   3 +-
>>   block/Makefile.objs       |   1 +
>>   6 files changed, 280 insertions(+), 2 deletions(-)
>>   create mode 100644 block/bitmap-alloc.c
> 
> [...]
> 
>> +BlockJob *bitpop_job_create(

>> +    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
>> +        return NULL;
>> +    }
> 
> What does this protect? And why does BACKUP_SOURCE describe acccurately
> what this job does?

I'm less certain what the BLOCK_OP_TYPE_* constants are supposed to 
block, or if this is just copy/paste from backup.c. Does BlockOpType in 
block.h need a new entry?

> 
>> +    if (bdrv_dirty_bitmap_check(target_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
>> +        return NULL;
>> +    }
>> +
>> +    if (pattern != BITMAP_PATTERN_ALLOCATION_TOP) {
>> +        error_setg(errp, "Unrecognized bitmap pattern");
>> +        return NULL;
>> +    }
>> +
>> +    len = bdrv_getlength(bs);
>> +    if (len < 0) {
>> +        error_setg_errno(errp, -len, "unable to get length for '%s'",
>> +                         bdrv_get_device_name(bs));
> 
> This operates on the node level, so bdrv_get_device_or_node_name() is
> necessary to avoid empty strings in the message.

Easy to fix.

> 
>> +        return NULL;
>> +    }
>> +
>> +    /* NB: new bitmap is anonymous and enabled */
>> +    cluster_size = bdrv_dirty_bitmap_granularity(target_bitmap);
>> +    new_bitmap = bdrv_create_dirty_bitmap(bs, cluster_size, NULL, errp);
>> +    if (!new_bitmap) {
>> +        return NULL;
>> +    }
>> +
>> +    /* Take ownership; we reserve the right to write into this on-commit. */
>> +    bdrv_dirty_bitmap_set_busy(target_bitmap, true);
>> +
>> +    job = block_job_create(job_id, &bitpop_job_driver, txn, bs,
>> +                           BLK_PERM_CONSISTENT_READ,
> 
> I don't think we actually rely on CONSISTENT_READ, but then, using the
> job on inconsistent nodes probably makes little sense and we can always
> relax the restriction later if necessary.
> 
>> +                           BLK_PERM_ALL & ~BLK_PERM_RESIZE,
>> +                           0, creation_flags,
>> +                           cb, opaque, errp);
>> +    if (!job) {
>> +        bdrv_dirty_bitmap_set_busy(target_bitmap, false);
>> +        bdrv_release_dirty_bitmap(new_bitmap);
>> +        return NULL;
>> +    }
>> +
>> +    job->bs = bs;
>> +    job->on_error = on_error;
>> +    job->target_bitmap = target_bitmap;
>> +    job->new_bitmap = new_bitmap;
>> +    job->len = len;
>> +    job_progress_set_remaining(&job->common.job, job->len);
>> +
>> +    return &job->common;
>> +}
> 
> Kevin
> 

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[John Snow]

On 6/16/20 3:46 PM, Eric Blake wrote:
> I'm less certain what the BLOCK_OP_TYPE_* constants are supposed to
> block, or if this is just copy/paste from backup.c. Does BlockOpType in
> block.h need a new entry?

Copy paste. I wasn't sure myself.

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Eric Blake]

On 6/16/20 2:46 PM, Eric Blake wrote:

>>> +BlockJob *bitpop_job_create(
> 
>>> +    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
>>> +        return NULL;
>>> +    }
>>
>> What does this protect? And why does BACKUP_SOURCE describe acccurately
>> what this job does?
> 
> I'm less certain what the BLOCK_OP_TYPE_* constants are supposed to 
> block, or if this is just copy/paste from backup.c. Does BlockOpType in 
> block.h need a new entry?

As it is, our code base has slowly moved away from op_blockers.  We no 
longer have any explicit bdrv_op_block() except when blocking 
everything, then immediately followed up with unblocking a mere subset 
of all of the defined op types:

block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_COMMIT_TARGET,
block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_STREAM,
block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_SOURCE,
block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_TARGET,
block/replication.c:        bdrv_op_unblock(top_bs, 
BLOCK_OP_TYPE_DATAPLANE, s->blocker);
blockjob.c:    bdrv_op_unblock(bs, BLOCK_OP_TYPE_DATAPLANE, job->blocker);

Are we at the point where we can ditch op_blockers altogether in favor 
of the block permissions system?

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH RFC v2 1/5] block: add bitmap-populate job

[Kevin Wolf]

Am 16.06.2020 um 22:02 hat Eric Blake geschrieben:
> On 6/16/20 2:46 PM, Eric Blake wrote:
> 
> > > > +BlockJob *bitpop_job_create(
> > 
> > > > +    if (bdrv_op_is_blocked(bs, BLOCK_OP_TYPE_BACKUP_SOURCE, errp)) {
> > > > +        return NULL;
> > > > +    }
> > > 
> > > What does this protect? And why does BACKUP_SOURCE describe acccurately
> > > what this job does?
> > 
> > I'm less certain what the BLOCK_OP_TYPE_* constants are supposed to
> > block, or if this is just copy/paste from backup.c. Does BlockOpType in
> > block.h need a new entry?
> 
> As it is, our code base has slowly moved away from op_blockers.

Yes, this is true. We're now trying to express conflicts with the
permission system instead.

> We no longer have any explicit bdrv_op_block() except when blocking
> everything, then immediately followed up with unblocking a mere subset
> of all of the defined op types:

I believe we never had any other pattern because nobody ever could be
bothered to think about specific conflicts. If we had one example of
conflicting uses, we just blocked everything and only unblocked cases
when we needed them (usually not really knowing whether they were really
always safe). So op blocker are almost always overblocking, but
sometimes too permissive, too.

This is why I was asking what we actually protect against here.

> 
> block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_COMMIT_TARGET,
> block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_STREAM,
> block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_SOURCE,
> block.c:    bdrv_op_unblock(backing_hd, BLOCK_OP_TYPE_BACKUP_TARGET,
> block/replication.c:        bdrv_op_unblock(top_bs, BLOCK_OP_TYPE_DATAPLANE,
> s->blocker);
> blockjob.c:    bdrv_op_unblock(bs, BLOCK_OP_TYPE_DATAPLANE, job->blocker);
> 
> Are we at the point where we can ditch op_blockers altogether in favor of
> the block permissions system?

I actually started some patches to remove op blockers the other day. The
tricky part is proving that each type is really unnecessary now. I only
did this for RESIZE and DATAPLANE, which seemed relatively easy to
prove, though I guess I could send at least these patches.

Kevin

[PATCH RFC v2 2/5] blockdev: combine DriveBackupState and BlockdevBackupState

[John Snow]

They have the same fields -- rename it BlockJobState.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 blockdev.c | 30 ++++++++++++------------------
 1 file changed, 12 insertions(+), 18 deletions(-)

diff --git a/blockdev.c b/blockdev.c
index b3c840ec03..d3e8a6ca22 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1702,11 +1702,11 @@ static void external_snapshot_clean(BlkActionState *common)
     aio_context_release(aio_context);
 }
 
-typedef struct DriveBackupState {
+typedef struct BlockJobActionState {
     BlkActionState common;
     BlockDriverState *bs;
     BlockJob *job;
-} DriveBackupState;
+} BlockJobActionState;
 
 static BlockJob *do_backup_common(BackupCommon *backup,
                                   BlockDriverState *bs,
@@ -1716,7 +1716,7 @@ static BlockJob *do_backup_common(BackupCommon *backup,
 
 static void drive_backup_prepare(BlkActionState *common, Error **errp)
 {
-    DriveBackupState *state = DO_UPCAST(DriveBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
     DriveBackup *backup;
     BlockDriverState *bs;
     BlockDriverState *target_bs;
@@ -1853,7 +1853,7 @@ out:
 
 static void drive_backup_commit(BlkActionState *common)
 {
-    DriveBackupState *state = DO_UPCAST(DriveBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
     AioContext *aio_context;
 
     aio_context = bdrv_get_aio_context(state->bs);
@@ -1867,7 +1867,7 @@ static void drive_backup_commit(BlkActionState *common)
 
 static void drive_backup_abort(BlkActionState *common)
 {
-    DriveBackupState *state = DO_UPCAST(DriveBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
 
     if (state->job) {
         AioContext *aio_context;
@@ -1883,7 +1883,7 @@ static void drive_backup_abort(BlkActionState *common)
 
 static void drive_backup_clean(BlkActionState *common)
 {
-    DriveBackupState *state = DO_UPCAST(DriveBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
     AioContext *aio_context;
 
     if (!state->bs) {
@@ -1898,15 +1898,9 @@ static void drive_backup_clean(BlkActionState *common)
     aio_context_release(aio_context);
 }
 
-typedef struct BlockdevBackupState {
-    BlkActionState common;
-    BlockDriverState *bs;
-    BlockJob *job;
-} BlockdevBackupState;
-
 static void blockdev_backup_prepare(BlkActionState *common, Error **errp)
 {
-    BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
     BlockdevBackup *backup;
     BlockDriverState *bs;
     BlockDriverState *target_bs;
@@ -1954,7 +1948,7 @@ static void blockdev_backup_prepare(BlkActionState *common, Error **errp)
 
 static void blockdev_backup_commit(BlkActionState *common)
 {
-    BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
     AioContext *aio_context;
 
     aio_context = bdrv_get_aio_context(state->bs);
@@ -1968,7 +1962,7 @@ static void blockdev_backup_commit(BlkActionState *common)
 
 static void blockdev_backup_abort(BlkActionState *common)
 {
-    BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
 
     if (state->job) {
         AioContext *aio_context;
@@ -1984,7 +1978,7 @@ static void blockdev_backup_abort(BlkActionState *common)
 
 static void blockdev_backup_clean(BlkActionState *common)
 {
-    BlockdevBackupState *state = DO_UPCAST(BlockdevBackupState, common, common);
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
     AioContext *aio_context;
 
     if (!state->bs) {
@@ -2265,14 +2259,14 @@ static const BlkActionOps actions[] = {
         .clean = external_snapshot_clean,
     },
     [TRANSACTION_ACTION_KIND_DRIVE_BACKUP] = {
-        .instance_size = sizeof(DriveBackupState),
+        .instance_size = sizeof(BlockJobActionState),
         .prepare = drive_backup_prepare,
         .commit = drive_backup_commit,
         .abort = drive_backup_abort,
         .clean = drive_backup_clean,
     },
     [TRANSACTION_ACTION_KIND_BLOCKDEV_BACKUP] = {
-        .instance_size = sizeof(BlockdevBackupState),
+        .instance_size = sizeof(BlockJobActionState),
         .prepare = blockdev_backup_prepare,
         .commit = blockdev_backup_commit,
         .abort = blockdev_backup_abort,
-- 
2.21.1

Re: [PATCH RFC v2 2/5] blockdev: combine DriveBackupState and BlockdevBackupState

[Eric Blake]

On 5/13/20 10:49 PM, John Snow wrote:
> They have the same fields -- rename it BlockJobState.

This says BlockJobState...

> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>   blockdev.c | 30 ++++++++++++------------------
>   1 file changed, 12 insertions(+), 18 deletions(-)
> 
> diff --git a/blockdev.c b/blockdev.c
> index b3c840ec03..d3e8a6ca22 100644
> --- a/blockdev.c
> +++ b/blockdev.c
> @@ -1702,11 +1702,11 @@ static void external_snapshot_clean(BlkActionState *common)
>       aio_context_release(aio_context);
>   }
>   
> -typedef struct DriveBackupState {
> +typedef struct BlockJobActionState {

...but this does not.  I'm assuming it is just a typo in the commit message?

Otherwise,
Reviewed-by: Eric Blake <eblake@redhat.com>

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[PATCH RFC v2 3/5] qmp: expose block-dirty-bitmap-populate

[John Snow]

This is a new job-creating command.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 qapi/block-core.json  | 18 +++++++++++
 qapi/transaction.json |  2 ++
 blockdev.c            | 74 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 94 insertions(+)

diff --git a/qapi/block-core.json b/qapi/block-core.json
index 0fb527a9a1..f7cae77fc9 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2250,6 +2250,24 @@
             '*auto-finalize': 'bool',
             '*auto-dismiss': 'bool' } }
 
+##
+# @block-dirty-bitmap-populate:
+#
+# Creates a new job that writes a pattern into a dirty bitmap.
+#
+# Since: 5.0
+#
+# Example:
+#
+# -> { "execute": "block-dirty-bitmap-populate",
+#      "arguments": { "node": "drive0", "target": "bitmap0",
+#                     "job-id": "job0", "pattern": "allocate-top" } }
+# <- { "return": {} }
+#
+##
+{ 'command': 'block-dirty-bitmap-populate', 'boxed': true,
+  'data': 'BlockDirtyBitmapPopulate' }
+
 ##
 # @BlockDirtyBitmapSha256:
 #
diff --git a/qapi/transaction.json b/qapi/transaction.json
index b6c11158f0..44be517de5 100644
--- a/qapi/transaction.json
+++ b/qapi/transaction.json
@@ -50,6 +50,7 @@
 # - @block-dirty-bitmap-enable: since 4.0
 # - @block-dirty-bitmap-disable: since 4.0
 # - @block-dirty-bitmap-merge: since 4.0
+# - @block-dirty-bitmap-populate: since 5.0
 # - @blockdev-backup: since 2.3
 # - @blockdev-snapshot: since 2.5
 # - @blockdev-snapshot-internal-sync: since 1.7
@@ -67,6 +68,7 @@
        'block-dirty-bitmap-enable': 'BlockDirtyBitmap',
        'block-dirty-bitmap-disable': 'BlockDirtyBitmap',
        'block-dirty-bitmap-merge': 'BlockDirtyBitmapMerge',
+       'block-dirty-bitmap-populate': 'BlockDirtyBitmapPopulate',
        'blockdev-backup': 'BlockdevBackup',
        'blockdev-snapshot': 'BlockdevSnapshot',
        'blockdev-snapshot-internal-sync': 'BlockdevSnapshotInternal',
diff --git a/blockdev.c b/blockdev.c
index d3e8a6ca22..08844e9e33 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -2233,6 +2233,63 @@ static void block_dirty_bitmap_remove_commit(BlkActionState *common)
     bdrv_release_dirty_bitmap(state->bitmap);
 }
 
+static void block_dirty_bitmap_populate_prepare(BlkActionState *common,
+                                                Error **errp)
+{
+    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
+    BlockDirtyBitmapPopulate *bitpop;
+    BlockDriverState *bs;
+    AioContext *aio_context;
+    BdrvDirtyBitmap *bmap = NULL;
+    int job_flags = JOB_DEFAULT;
+
+    assert(common->action->type == \
+           TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE);
+    bitpop = common->action->u.block_dirty_bitmap_populate.data;
+
+    bmap = block_dirty_bitmap_lookup(bitpop->node, bitpop->name, &bs, errp);
+    if (!bmap) {
+        return;
+    }
+
+    aio_context = bdrv_get_aio_context(bs);
+    aio_context_acquire(aio_context);
+    state->bs = bs;
+
+    /* Paired with .clean() */
+    bdrv_drained_begin(state->bs);
+
+    if (!bitpop->has_on_error) {
+        bitpop->on_error = BLOCKDEV_ON_ERROR_REPORT;
+    }
+    if (!bitpop->has_auto_finalize) {
+        bitpop->auto_finalize = true;
+    }
+    if (!bitpop->has_auto_dismiss) {
+        bitpop->auto_dismiss = true;
+    }
+
+    if (!bitpop->auto_finalize) {
+        job_flags |= JOB_MANUAL_FINALIZE;
+    }
+    if (!bitpop->auto_dismiss) {
+        job_flags |= JOB_MANUAL_DISMISS;
+    }
+
+    state->job = bitpop_job_create(
+        bitpop->job_id,
+        bs,
+        bmap,
+        bitpop->pattern,
+        bitpop->on_error,
+        job_flags,
+        NULL, NULL,
+        common->block_job_txn,
+        errp);
+
+    aio_context_release(aio_context);
+}
+
 static void abort_prepare(BlkActionState *common, Error **errp)
 {
     error_setg(errp, "Transaction aborted using Abort action");
@@ -2316,6 +2373,13 @@ static const BlkActionOps actions[] = {
         .commit = block_dirty_bitmap_remove_commit,
         .abort = block_dirty_bitmap_remove_abort,
     },
+    [TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE] = {
+        .instance_size = sizeof(BlockJobActionState),
+        .prepare = block_dirty_bitmap_populate_prepare,
+        .commit = blockdev_backup_commit,
+        .abort = blockdev_backup_abort,
+        .clean = blockdev_backup_clean,
+    },
     /* Where are transactions for MIRROR, COMMIT and STREAM?
      * Although these blockjobs use transaction callbacks like the backup job,
      * these jobs do not necessarily adhere to transaction semantics.
@@ -2672,6 +2736,16 @@ void qmp_block_dirty_bitmap_merge(const char *node, const char *target,
     do_block_dirty_bitmap_merge(node, target, bitmaps, NULL, errp);
 }
 
+void qmp_block_dirty_bitmap_populate(BlockDirtyBitmapPopulate *bitpop,
+                                     Error **errp)
+{
+    TransactionAction action = {
+        .type = TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE,
+        .u.block_dirty_bitmap_populate.data = bitpop,
+    };
+    blockdev_do_action(&action, errp);
+}
+
 BlockDirtyBitmapSha256 *qmp_x_debug_block_dirty_bitmap_sha256(const char *node,
                                                               const char *name,
                                                               Error **errp)
-- 
2.21.1

Re: [PATCH RFC v2 3/5] qmp: expose block-dirty-bitmap-populate

[Eric Blake]

On 5/13/20 10:49 PM, John Snow wrote:
> This is a new job-creating command.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>   qapi/block-core.json  | 18 +++++++++++
>   qapi/transaction.json |  2 ++
>   blockdev.c            | 74 +++++++++++++++++++++++++++++++++++++++++++
>   3 files changed, 94 insertions(+)
> 
> diff --git a/qapi/block-core.json b/qapi/block-core.json
> index 0fb527a9a1..f7cae77fc9 100644
> --- a/qapi/block-core.json
> +++ b/qapi/block-core.json
> @@ -2250,6 +2250,24 @@
>               '*auto-finalize': 'bool',
>               '*auto-dismiss': 'bool' } }
>   
> +##
> +# @block-dirty-bitmap-populate:
> +#
> +# Creates a new job that writes a pattern into a dirty bitmap.
> +#
> +# Since: 5.0

5.1

> +++ b/qapi/transaction.json
> @@ -50,6 +50,7 @@
>   # - @block-dirty-bitmap-enable: since 4.0
>   # - @block-dirty-bitmap-disable: since 4.0
>   # - @block-dirty-bitmap-merge: since 4.0
> +# - @block-dirty-bitmap-populate: since 5.0

ditto


> +++ b/blockdev.c
> @@ -2233,6 +2233,63 @@ static void block_dirty_bitmap_remove_commit(BlkActionState *common)
>       bdrv_release_dirty_bitmap(state->bitmap);
>   }
>   
> +static void block_dirty_bitmap_populate_prepare(BlkActionState *common,
> +                                                Error **errp)
> +{
> +    BlockJobActionState *state = DO_UPCAST(BlockJobActionState, common, common);
> +    BlockDirtyBitmapPopulate *bitpop;
> +    BlockDriverState *bs;
> +    AioContext *aio_context;
> +    BdrvDirtyBitmap *bmap = NULL;
> +    int job_flags = JOB_DEFAULT;
> +
> +    assert(common->action->type == \
> +           TRANSACTION_ACTION_KIND_BLOCK_DIRTY_BITMAP_POPULATE);

\ is not necessary here.

> +    bitpop = common->action->u.block_dirty_bitmap_populate.data;
> +
> +    bmap = block_dirty_bitmap_lookup(bitpop->node, bitpop->name, &bs, errp);
> +    if (!bmap) {
> +        return;
> +    }

So the bitmap has to already exist, and we are just merging into it, 
correct?

Otherwise looks good.

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

[PATCH RFC v2 4/5] iotests: move bitmap helpers into their own file

[John Snow]

Signed-off-by: John Snow <jsnow@redhat.com>
---
 tests/qemu-iotests/257        | 110 +---------------------------
 tests/qemu-iotests/bitmaps.py | 131 ++++++++++++++++++++++++++++++++++
 2 files changed, 132 insertions(+), 109 deletions(-)
 create mode 100644 tests/qemu-iotests/bitmaps.py

diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 2933e2670b..adf4cdfe6e 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -24,120 +24,12 @@ import os
 
 import iotests
 from iotests import log, qemu_img
+from bitmaps import EmulatedBitmap, GROUPS
 
 SIZE = 64 * 1024 * 1024
 GRANULARITY = 64 * 1024
 
 
-class Pattern:
-    def __init__(self, byte, offset, size=GRANULARITY):
-        self.byte = byte
-        self.offset = offset
-        self.size = size
-
-    def bits(self, granularity):
-        lower = self.offset // granularity
-        upper = (self.offset + self.size - 1) // granularity
-        return set(range(lower, upper + 1))
-
-
-class PatternGroup:
-    """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
-    def __init__(self, patterns):
-        self.patterns = patterns
-
-    def bits(self, granularity):
-        """Calculate the unique bits dirtied by this pattern grouping"""
-        res = set()
-        for pattern in self.patterns:
-            res |= pattern.bits(granularity)
-        return res
-
-
-GROUPS = [
-    PatternGroup([
-        # Batch 0: 4 clusters
-        Pattern('0x49', 0x0000000),
-        Pattern('0x6c', 0x0100000),   # 1M
-        Pattern('0x6f', 0x2000000),   # 32M
-        Pattern('0x76', 0x3ff0000)]), # 64M - 64K
-    PatternGroup([
-        # Batch 1: 6 clusters (3 new)
-        Pattern('0x65', 0x0000000),   # Full overwrite
-        Pattern('0x77', 0x00f8000),   # Partial-left (1M-32K)
-        Pattern('0x72', 0x2008000),   # Partial-right (32M+32K)
-        Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
-    PatternGroup([
-        # Batch 2: 7 clusters (3 new)
-        Pattern('0x74', 0x0010000),   # Adjacent-right
-        Pattern('0x69', 0x00e8000),   # Partial-left  (1M-96K)
-        Pattern('0x6e', 0x2018000),   # Partial-right (32M+96K)
-        Pattern('0x67', 0x3fe0000,
-                2*GRANULARITY)]),     # Overwrite [(64M-128K)-64M)
-    PatternGroup([
-        # Batch 3: 8 clusters (5 new)
-        # Carefully chosen such that nothing re-dirties the one cluster
-        # that copies out successfully before failure in Group #1.
-        Pattern('0xaa', 0x0010000,
-                3*GRANULARITY),       # Overwrite and 2x Adjacent-right
-        Pattern('0xbb', 0x00d8000),   # Partial-left (1M-160K)
-        Pattern('0xcc', 0x2028000),   # Partial-right (32M+160K)
-        Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
-]
-
-
-class EmulatedBitmap:
-    def __init__(self, granularity=GRANULARITY):
-        self._bits = set()
-        self.granularity = granularity
-
-    def dirty_bits(self, bits):
-        self._bits |= set(bits)
-
-    def dirty_group(self, n):
-        self.dirty_bits(GROUPS[n].bits(self.granularity))
-
-    def clear(self):
-        self._bits = set()
-
-    def clear_bits(self, bits):
-        self._bits -= set(bits)
-
-    def clear_bit(self, bit):
-        self.clear_bits({bit})
-
-    def clear_group(self, n):
-        self.clear_bits(GROUPS[n].bits(self.granularity))
-
-    @property
-    def first_bit(self):
-        return sorted(self.bits)[0]
-
-    @property
-    def bits(self):
-        return self._bits
-
-    @property
-    def count(self):
-        return len(self.bits)
-
-    def compare(self, qmp_bitmap):
-        """
-        Print a nice human-readable message checking that a bitmap as reported
-        by the QMP interface has as many bits set as we expect it to.
-        """
-
-        name = qmp_bitmap.get('name', '(anonymous)')
-        log("= Checking Bitmap {:s} =".format(name))
-
-        want = self.count
-        have = qmp_bitmap['count'] // qmp_bitmap['granularity']
-
-        log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
-            want, have, "OK!" if want == have else "ERROR!"))
-        log('')
-
-
 class Drive:
     """Represents, vaguely, a drive attached to a VM.
     Includes format, graph, and device information."""
diff --git a/tests/qemu-iotests/bitmaps.py b/tests/qemu-iotests/bitmaps.py
new file mode 100644
index 0000000000..522fc25171
--- /dev/null
+++ b/tests/qemu-iotests/bitmaps.py
@@ -0,0 +1,131 @@
+# Bitmap-related helper utilities
+#
+# Copyright (c) 2020 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# owner=jsnow@redhat.com
+
+from iotests import log
+
+GRANULARITY = 64 * 1024
+
+
+class Pattern:
+    def __init__(self, byte, offset, size=GRANULARITY):
+        self.byte = byte
+        self.offset = offset
+        self.size = size
+
+    def bits(self, granularity):
+        lower = self.offset // granularity
+        upper = (self.offset + self.size - 1) // granularity
+        return set(range(lower, upper + 1))
+
+
+class PatternGroup:
+    """Grouping of Pattern objects. Initialize with an iterable of Patterns."""
+    def __init__(self, patterns):
+        self.patterns = patterns
+
+    def bits(self, granularity):
+        """Calculate the unique bits dirtied by this pattern grouping"""
+        res = set()
+        for pattern in self.patterns:
+            res |= pattern.bits(granularity)
+        return res
+
+
+GROUPS = [
+    PatternGroup([
+        # Batch 0: 4 clusters
+        Pattern('0x49', 0x0000000),
+        Pattern('0x6c', 0x0100000),   # 1M
+        Pattern('0x6f', 0x2000000),   # 32M
+        Pattern('0x76', 0x3ff0000)]), # 64M - 64K
+    PatternGroup([
+        # Batch 1: 6 clusters (3 new)
+        Pattern('0x65', 0x0000000),   # Full overwrite
+        Pattern('0x77', 0x00f8000),   # Partial-left (1M-32K)
+        Pattern('0x72', 0x2008000),   # Partial-right (32M+32K)
+        Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
+    PatternGroup([
+        # Batch 2: 7 clusters (3 new)
+        Pattern('0x74', 0x0010000),   # Adjacent-right
+        Pattern('0x69', 0x00e8000),   # Partial-left  (1M-96K)
+        Pattern('0x6e', 0x2018000),   # Partial-right (32M+96K)
+        Pattern('0x67', 0x3fe0000,
+                2*GRANULARITY)]),     # Overwrite [(64M-128K)-64M)
+    PatternGroup([
+        # Batch 3: 8 clusters (5 new)
+        # Carefully chosen such that nothing re-dirties the one cluster
+        # that copies out successfully before failure in Group #1.
+        Pattern('0xaa', 0x0010000,
+                3*GRANULARITY),       # Overwrite and 2x Adjacent-right
+        Pattern('0xbb', 0x00d8000),   # Partial-left (1M-160K)
+        Pattern('0xcc', 0x2028000),   # Partial-right (32M+160K)
+        Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
+]
+
+
+class EmulatedBitmap:
+    def __init__(self, granularity=GRANULARITY):
+        self._bits = set()
+        self.granularity = granularity
+
+    def dirty_bits(self, bits):
+        self._bits |= set(bits)
+
+    def dirty_group(self, n):
+        self.dirty_bits(GROUPS[n].bits(self.granularity))
+
+    def clear(self):
+        self._bits = set()
+
+    def clear_bits(self, bits):
+        self._bits -= set(bits)
+
+    def clear_bit(self, bit):
+        self.clear_bits({bit})
+
+    def clear_group(self, n):
+        self.clear_bits(GROUPS[n].bits(self.granularity))
+
+    @property
+    def first_bit(self):
+        return sorted(self.bits)[0]
+
+    @property
+    def bits(self):
+        return self._bits
+
+    @property
+    def count(self):
+        return len(self.bits)
+
+    def compare(self, qmp_bitmap):
+        """
+        Print a nice human-readable message checking that a bitmap as reported
+        by the QMP interface has as many bits set as we expect it to.
+        """
+
+        name = qmp_bitmap.get('name', '(anonymous)')
+        log("= Checking Bitmap {:s} =".format(name))
+
+        want = self.count
+        have = qmp_bitmap['count'] // qmp_bitmap['granularity']
+
+        log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
+            want, have, "OK!" if want == have else "ERROR!"))
+        log('')
-- 
2.21.1

[PATCH RFC v2 5/5] iotests: add 287 for block-dirty-bitmap-populate

[John Snow]

Give block-dirty-bitmap-populate a workout.

Signed-off-by: John Snow <jsnow@redhat.com>
---
 tests/qemu-iotests/287     |  242 ++
 tests/qemu-iotests/287.out | 4544 ++++++++++++++++++++++++++++++++++++
 tests/qemu-iotests/group   |    1 +
 3 files changed, 4787 insertions(+)
 create mode 100755 tests/qemu-iotests/287
 create mode 100644 tests/qemu-iotests/287.out

diff --git a/tests/qemu-iotests/287 b/tests/qemu-iotests/287
new file mode 100755
index 0000000000..1faff03917
--- /dev/null
+++ b/tests/qemu-iotests/287
@@ -0,0 +1,242 @@
+#!/usr/bin/env python3
+#
+# Test block-dirty-bitmap-populate
+#
+# Copyright (c) 2020 John Snow for Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program.  If not, see <http://www.gnu.org/licenses/>.
+#
+# owner=jsnow@redhat.com
+
+from collections import namedtuple
+import itertools
+import math
+import os
+
+import iotests
+from iotests import log, qemu_img
+from bitmaps import EmulatedBitmap, GROUPS
+
+SIZE = 64 * 1024 * 1024
+GRANULARITY = 64 * 1024
+
+
+class Drive:
+    def __init__(self, path, vm):
+        self.path = path
+        self.vm = vm
+        self.fmt = None
+        self.size = None
+        self.node = None
+
+    def img_create(self, fmt, size):
+        self.fmt = fmt
+        self.size = size
+        iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))
+
+
+def block_dirty_bitmap_populate(vm, node, bitmap, job_id, pattern, **kwargs):
+    # Strip any arguments explicitly nulled by the caller:
+    kwargs = {key: val for key, val in kwargs.items() if val is not None}
+    result = vm.qmp_log('block-dirty-bitmap-populate',
+                        node=node,
+                        name=bitmap,
+                        job_id=job_id,
+                        pattern=pattern,
+                        **kwargs)
+    return result
+
+
+def populate(drive, bitmap, job_id, pattern='allocation-top', **kwargs):
+    kwargs.setdefault('pattern', pattern)
+    kwargs.setdefault('auto-finalize', False)
+    kwargs.setdefault('auto-dismiss', False)
+    ret = block_dirty_bitmap_populate(drive.vm, drive.node,
+                                      bitmap, job_id, **kwargs)
+    return {
+        'id': job_id,
+        'auto-finalize': kwargs['auto-finalize'],
+        'auto-dismiss': kwargs['auto-dismiss'],
+        'return': ret,
+    }
+
+
+def perform_writes(drive, n, filter_node_name=None):
+    log("-- Write #{:d}:".format(n))
+    node_name = filter_node_name or drive.node
+    for pattern in GROUPS[n].patterns:
+        cmd = "write -P{:s} 0x{:07x} 0x{:x}".format(
+            pattern.byte,
+            pattern.offset,
+            pattern.size)
+        drive.vm.hmp_qemu_io(node_name, cmd, use_log=True)
+    log('')
+
+
+TestConfig = namedtuple('TestConfig', [
+    'base_pattern',
+    'disabled',
+    'pre_writes',
+    'mid_writes',
+    'cancel',
+    'post_writes',
+])
+
+
+def test_bitmap_populate(config):
+    """
+    Test bitmap populate.
+
+    :param base_pattern: Write a base pattern?
+    :param disabled:     Disable the target bitmap?
+    :param pre_writes:   Write a pattern after bitmap creation, but before job?
+    :param mid_writes:   Write a pattern before job finalizes?
+    :param cancel:       Cancel the job instead of finalizing it?
+    :param post_writes:  Write a pattern after the job?
+    """
+    with iotests.FilePaths(['img']) as (img_path,), iotests.VM() as vm:
+        log("\n=== Bitmap Populate {:s} ===\n".format(str(config)))
+
+        log('-- Prepare image & VM:')
+        drive0 = Drive(img_path, vm=vm)
+        drive0.img_create(iotests.imgfmt, SIZE)
+        vm.add_device("{},id=scsi0".format(iotests.get_virtio_scsi_device()))
+        vm.launch()
+
+        file_config = {
+            'driver': 'file',
+            'filename': drive0.path
+        }
+
+        drive0.node = 'drive0'
+        vm.qmp_log('blockdev-add',
+                   filters=[iotests.filter_qmp_testfiles],
+                   node_name=drive0.node,
+                   driver=drive0.fmt,
+                   file=file_config)
+        log('')
+
+
+        # Step 0: Prepare & Base Allocation Pattern
+
+        if config.base_pattern:
+            perform_writes(drive0, 0)
+
+
+        # Step 1: Add test bitmap
+
+        log('-- Add Bitmap:')
+        vm.qmp_log('block-dirty-bitmap-add',
+                   node=drive0.node,
+                   name="target",
+                   granularity=GRANULARITY,
+                   disabled=config.disabled)
+        ebitmap = EmulatedBitmap()
+        log('')
+
+        # Step 2: Pre-Writes
+
+        if config.pre_writes:
+            perform_writes(drive0, 1)
+            if not config.disabled:
+                ebitmap.dirty_group(1)
+            bitmap = vm.get_bitmap(drive0.node, 'target')
+            ebitmap.compare(bitmap)
+
+
+        # Step 3: Launch job & Mid-Writes
+
+        log('-- Test block-dirty-bitmap-populate (bitpop0):')
+        def pre_finalize():
+            # Writes issued prior to job finalization:
+            if config.mid_writes:
+                perform_writes(drive0, 2)
+                if not config.disabled:
+                    ebitmap.dirty_group(2)
+
+
+        class TestJobRunner(iotests.JobRunner):
+            def on_pending(self, event):
+                if config.mid_writes:
+                    perform_writes(drive0, 2)
+                    if not config.disabled:
+                        ebitmap.dirty_group(2)
+                super().on_pending(event)
+
+        job = populate(drive0, 'target', 'bitpop0')
+        assert job['return'] == {'return': {}}
+        job_runner = TestJobRunner(vm, job['id'],
+                                   auto_dismiss=job['auto-dismiss'],
+                                   auto_finalize=job['auto-finalize'],
+                                   cancel=config.cancel)
+        job_runner.run()
+        log('')
+
+
+        # Step 4: Post-job verification
+
+        if not config.cancel:
+            # Any writes made prior to the job finishing should now be visible.
+            if config.base_pattern:
+                ebitmap.dirty_group(0)
+            if config.pre_writes:
+                ebitmap.dirty_group(1)
+            if config.mid_writes:
+                ebitmap.dirty_group(2)
+
+        bitmap = vm.get_bitmap(drive0.node, 'target')
+        ebitmap.compare(bitmap)
+
+
+        # Step 5: Post-Writes
+
+        if config.post_writes:
+            perform_writes(drive0, 3)
+            if not config.disabled:
+                ebitmap.dirty_group(3)
+
+
+        # Step 6: Final Verification
+
+        log('-- Verification:')
+        bitmaps = vm.query_bitmaps()
+        log({'bitmaps': bitmaps}, indent=2)
+        log('')
+        bitmap = vm.get_bitmap(drive0.node, 'target', bitmaps=bitmaps)
+        ebitmap.compare(bitmap)
+
+
+        log('-- Cleanup:')
+        vm.qmp_log("block-dirty-bitmap-remove",
+                   node=drive0.node, name="target")
+
+        bitmaps = vm.query_bitmaps()
+        if bitmaps:
+            log("ERROR: bitmaps unaccounted for:")
+            log(bitmaps)
+        else:
+            log('OK: All bitmaps removed')
+        vm.shutdown()
+        log('')
+
+
+def main():
+    for args in itertools.product((True, False), repeat=6):
+        cfg = TestConfig(*args)
+        test_bitmap_populate(cfg)
+
+
+if __name__ == '__main__':
+    iotests.script_main(main, supported_fmts=['qcow2'],
+                        supported_protocols=['file'])
diff --git a/tests/qemu-iotests/287.out b/tests/qemu-iotests/287.out
new file mode 100644
index 0000000000..7c0afc7192
--- /dev/null
+++ b/tests/qemu-iotests/287.out
@@ -0,0 +1,4544 @@
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 262144,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 262144,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 983040,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 983040,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 917504,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 14 dirty sectors; have 14. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 393216,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 983040,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 786432,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 983040,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 524288,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 8 dirty sectors; have 8. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 786432,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=True, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Write #0:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x49 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6c 0x0100000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6f 0x2000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x76 0x3ff0000 0x10000\""}}
+{"return": ""}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 262144,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 4 dirty sectors; have 4. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 393216,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 393216,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=True, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": true, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": false,
+        "status": "disabled"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 983040,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 983040,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 15 dirty sectors; have 15. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 655360,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 10 dirty sectors; have 10. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 917504,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 14 dirty sectors; have 14. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 393216,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 917504,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 14 dirty sectors; have 14. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=True, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Write #1:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x65 0x0000000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x77 0x00f8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x72 0x2008000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x3fe0000 0x10000\""}}
+{"return": ""}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 393216,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 6 dirty sectors; have 6. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 786432,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 786432,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 12 dirty sectors; have 12. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=True, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+-- Write #2:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x74 0x0010000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x69 0x00e8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x6e 0x2018000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0x67 0x3fe0000 0x20000\""}}
+{"return": ""}
+
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 458752,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 7 dirty sectors; have 7. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 524288,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 8 dirty sectors; have 8. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=True, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-cancel", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+Job failed: Operation canceled
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=True) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Write #3:
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xaa 0x0010000 0x30000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xbb 0x00d8000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xcc 0x2028000 0x10000\""}}
+{"return": ""}
+{"execute": "human-monitor-command", "arguments": {"command-line": "qemu-io drive0 \"write -P0xdd 0x3fc0000 0x10000\""}}
+{"return": ""}
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 524288,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 8 dirty sectors; have 8. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
+
+=== Bitmap Populate TestConfig(base_pattern=False, disabled=False, pre_writes=False, mid_writes=False, cancel=False, post_writes=False) ===
+
+-- Prepare image & VM:
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+
+-- Add Bitmap:
+{"execute": "block-dirty-bitmap-add", "arguments": {"disabled": false, "granularity": 65536, "name": "target", "node": "drive0"}}
+{"return": {}}
+
+-- Test block-dirty-bitmap-populate (bitpop0):
+{"execute": "block-dirty-bitmap-populate", "arguments": {"auto-dismiss": false, "auto-finalize": false, "job-id": "bitpop0", "name": "target", "node": "drive0", "pattern": "allocation-top"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+{"data": {"id": "bitpop0", "type": "bitmap-populate"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "bitpop0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "bitmap-populate"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"execute": "job-dismiss", "arguments": {"id": "bitpop0"}}
+{"return": {}}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Verification:
+{
+  "bitmaps": {
+    "drive0": [
+      {
+        "busy": false,
+        "count": 0,
+        "granularity": 65536,
+        "name": "target",
+        "persistent": false,
+        "recording": true,
+        "status": "active"
+      }
+    ]
+  }
+}
+
+= Checking Bitmap target =
+expecting 0 dirty sectors; have 0. OK!
+
+-- Cleanup:
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "target", "node": "drive0"}}
+{"return": {}}
+OK: All bitmaps removed
+
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index fe649c5b73..122ab02d08 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -295,6 +295,7 @@
 283 auto quick
 284 rw
 286 rw quick
+287 rw
 288 quick
 289 rw quick
 290 rw auto quick
-- 
2.21.1

Re: [PATCH RFC v2 5/5] iotests: add 287 for block-dirty-bitmap-populate

[Eric Blake]

On 5/13/20 10:49 PM, John Snow wrote:
> Give block-dirty-bitmap-populate a workout.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
>   tests/qemu-iotests/287     |  242 ++
>   tests/qemu-iotests/287.out | 4544 ++++++++++++++++++++++++++++++++++++

Sheesh, that output file is a workout indeed.

>   tests/qemu-iotests/group   |    1 +
>   3 files changed, 4787 insertions(+)
>   create mode 100755 tests/qemu-iotests/287
>   create mode 100644 tests/qemu-iotests/287.out

287 has been created in the meantime (commit dd488fc1c00); we'll have to 
rebase.  (Vladimir's series to rename iotests to something sensible 
instead of three digits may help...)

> +
> +class Drive:
> +    def __init__(self, path, vm):
> +        self.path = path
> +        self.vm = vm
> +        self.fmt = None
> +        self.size = None
> +        self.node = None
> +
> +    def img_create(self, fmt, size):
> +        self.fmt = fmt
> +        self.size = size
> +        iotests.qemu_img_create('-f', self.fmt, self.path, str(self.size))

Are we creating images with backing files anywhere in this test?  Should 
we be?  (The question of what is allocated in the top layer is more 
interesting to answer when there is a backing layer)

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org

Re: [PATCH RFC v2 5/5] iotests: add 287 for block-dirty-bitmap-populate

[Kevin Wolf]

Am 14.05.2020 um 05:49 hat John Snow geschrieben:
> Give block-dirty-bitmap-populate a workout.
> 
> Signed-off-by: John Snow <jsnow@redhat.com>

I think it would be good to test on-error behaviour, too. Maybe that
would be a different test file, though, considering that the output for
this one is already huge?

Kevin

Re: [PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

[Peter Krempa]

On Wed, May 13, 2020 at 23:49:17 -0400, John Snow wrote:
> Hi,
> 
> This is a new (very small) block job that writes a pattern into a
> bitmap. The only pattern implemented is the top allocation information.
> 
> This can be used to "recover" an incremental bitmap chain if an external
> snapshot was taken without creating a new bitmap first: any writes made
> to the image will be reflected by the allocation status and can be
> written back into a bitmap.
> 
> This is useful for e.g. libvirt managing backup chains if a user creates
> an external snapshot outside of libvirt.

I've dusted-off my patches for using this blockjob for this very
specific case and it works for me.

Tested-by: Peter Krempa <pkrempa@redhat.com>

For now I'll continue the integration with other blockjobs where we
merge bitmaps.

Re: [PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

[Peter Krempa]

On Mon, May 18, 2020 at 16:52:45 +0200, Peter Krempa wrote:
> On Wed, May 13, 2020 at 23:49:17 -0400, John Snow wrote:
> > Hi,
> > 
> > This is a new (very small) block job that writes a pattern into a
> > bitmap. The only pattern implemented is the top allocation information.
> > 
> > This can be used to "recover" an incremental bitmap chain if an external
> > snapshot was taken without creating a new bitmap first: any writes made
> > to the image will be reflected by the allocation status and can be
> > written back into a bitmap.
> > 
> > This is useful for e.g. libvirt managing backup chains if a user creates
> > an external snapshot outside of libvirt.
> 
> I've dusted-off my patches for using this blockjob for this very
> specific case and it works for me.
> 
> Tested-by: Peter Krempa <pkrempa@redhat.com>
> 
> For now I'll continue the integration with other blockjobs where we
> merge bitmaps.

I've posted the libvirt patches which make use of this blockjob as RFC
on the libvir-list:

https://www.redhat.com/archives/libvir-list/2020-June/msg00292.html

I also have a branch with the patchset rebased to master (except for one
of the test commits IIRC) linked from the cover-letter.

Re: [PATCH RFC v2 0/5] block: add block-dirty-bitmap-populate job

[Eric Blake]

On 5/13/20 10:49 PM, John Snow wrote:
> Hi,
> 
> This is a new (very small) block job that writes a pattern into a
> bitmap. The only pattern implemented is the top allocation information.
> 
> This can be used to "recover" an incremental bitmap chain if an external
> snapshot was taken without creating a new bitmap first: any writes made
> to the image will be reflected by the allocation status and can be
> written back into a bitmap.
> 
> This is useful for e.g. libvirt managing backup chains if a user creates
> an external snapshot outside of libvirt.
> 
> v2:
>   - Addressed some, but not all feedback
>   - Rebased on latest 'job-runner' series; but it's not clear if it
>     should be kept.

Message-id for that series? I'm not finding a message with a subject 
containing a literal 'job-runner', but am not sure which subject to look 
for instead.

I also couldn't find an obvious tag or branch at 
https://github.com/jnsnow/qemu/branches where you might have stashed 
this including prerequisites.

>   - This version doesn't address all of the feedback from v1,
>     but I am posting it to the list as an RFC.

I'm happy to try and take over these patches to prepare a v3, but only 
if I can get them to build by finding the prerequisites :)

-- 
Eric Blake, Principal Software Engineer
Red Hat, Inc.           +1-919-301-3226
Virtualization:  qemu.org | libvirt.org