* [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top
@ 2019-07-10 1:05 John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class John Snow
` (7 more replies)
0 siblings, 8 replies; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
Requires: <20190709232550.10724-1-jsnow@redhat.com>
[PATCH v4 00/18] bitmaps: introduce 'bitmap' sync mode
This follows the previous series which adds the 'bitmap' sync mode
and uses it to add interactions with bitmaps to the 'full' and 'top'
modes to blockdev-backup and drive-backup.
Why?
on-success: Can conveniently synchronize a bitmap to a full backup.
Allows for transactionless anchor backups.
Allows us to attempt an anchor backup without damaging
our bitmap until the backup is successful.
Allows for transactional, ungrouped anchor backups.
always: Allows us to resume full/top style backups with a later
invocation to sync=bitmap. Neat!
Summary:
1-3: Refactor iotest 257 to accommodate this;
4-5: Augment 257 to test trivial failure cases
6-7: Implement feature
8: Test new feature
John Snow (8):
iotests/257: add Pattern class
iotests/257: add EmulatedBitmap class
iotests/257: Refactor backup helpers
block/backup: hoist bitmap check into QMP interface
iotests/257: test API failures
block/backup: issue progress updates for skipped regions
block/backup: support bitmap sync modes for non-bitmap backups
iotests/257: test traditional sync modes
block/backup.c | 22 +-
blockdev.c | 32 +
qapi/block-core.json | 6 +-
tests/qemu-iotests/257 | 329 +++-
tests/qemu-iotests/257.out | 3366 +++++++++++++++++++++++++++++++++++-
5 files changed, 3548 insertions(+), 207 deletions(-)
--
2.21.0
^ permalink raw reply [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 15:10 ` Max Reitz
2019-07-10 16:26 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class John Snow
` (6 subsequent siblings)
7 siblings, 2 replies; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
Just kidding, this is easier to manage with a full class instead of a
namedtuple.
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/257 | 58 +++++++++++++++++++++++-------------------
1 file changed, 32 insertions(+), 26 deletions(-)
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 75a651c7c3..f576a35a5c 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -19,7 +19,6 @@
#
# owner=jsnow@redhat.com
-from collections import namedtuple
import math
import os
@@ -29,10 +28,18 @@ from iotests import log, qemu_img
SIZE = 64 * 1024 * 1024
GRANULARITY = 64 * 1024
-Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
-def mkpattern(byte, offset, size=GRANULARITY):
- """Constructor for Pattern() with default size"""
- return Pattern(byte, offset, size)
+
+class Pattern:
+ def __init__(self, byte, offset, size=GRANULARITY):
+ self.byte = byte
+ self.offset = offset
+ self.size = size
+
+ def bits(self, granularity):
+ lower = math.floor(self.offset / granularity)
+ upper = math.floor((self.offset + self.size - 1) / granularity)
+ return set(range(lower, upper + 1))
+
class PatternGroup:
"""Grouping of Pattern objects. Initialize with an iterable of Patterns."""
@@ -43,40 +50,39 @@ class PatternGroup:
"""Calculate the unique bits dirtied by this pattern grouping"""
res = set()
for pattern in self.patterns:
- lower = math.floor(pattern.offset / granularity)
- upper = math.floor((pattern.offset + pattern.size - 1) / granularity)
- res = res | set(range(lower, upper + 1))
+ res = res | pattern.bits(granularity)
return res
+
GROUPS = [
PatternGroup([
# Batch 0: 4 clusters
- mkpattern('0x49', 0x0000000),
- mkpattern('0x6c', 0x0100000), # 1M
- mkpattern('0x6f', 0x2000000), # 32M
- mkpattern('0x76', 0x3ff0000)]), # 64M - 64K
+ Pattern('0x49', 0x0000000),
+ Pattern('0x6c', 0x0100000), # 1M
+ Pattern('0x6f', 0x2000000), # 32M
+ Pattern('0x76', 0x3ff0000)]), # 64M - 64K
PatternGroup([
# Batch 1: 6 clusters (3 new)
- mkpattern('0x65', 0x0000000), # Full overwrite
- mkpattern('0x77', 0x00f8000), # Partial-left (1M-32K)
- mkpattern('0x72', 0x2008000), # Partial-right (32M+32K)
- mkpattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
+ Pattern('0x65', 0x0000000), # Full overwrite
+ Pattern('0x77', 0x00f8000), # Partial-left (1M-32K)
+ Pattern('0x72', 0x2008000), # Partial-right (32M+32K)
+ Pattern('0x69', 0x3fe0000)]), # Adjacent-left (64M - 128K)
PatternGroup([
# Batch 2: 7 clusters (3 new)
- mkpattern('0x74', 0x0010000), # Adjacent-right
- mkpattern('0x69', 0x00e8000), # Partial-left (1M-96K)
- mkpattern('0x6e', 0x2018000), # Partial-right (32M+96K)
- mkpattern('0x67', 0x3fe0000,
- 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
+ Pattern('0x74', 0x0010000), # Adjacent-right
+ Pattern('0x69', 0x00e8000), # Partial-left (1M-96K)
+ Pattern('0x6e', 0x2018000), # Partial-right (32M+96K)
+ Pattern('0x67', 0x3fe0000,
+ 2*GRANULARITY)]), # Overwrite [(64M-128K)-64M)
PatternGroup([
# Batch 3: 8 clusters (5 new)
# Carefully chosen such that nothing re-dirties the one cluster
# that copies out successfully before failure in Group #1.
- mkpattern('0xaa', 0x0010000,
- 3*GRANULARITY), # Overwrite and 2x Adjacent-right
- mkpattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
- mkpattern('0xcc', 0x2028000), # Partial-right (32M+160K)
- mkpattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
+ Pattern('0xaa', 0x0010000,
+ 3*GRANULARITY), # Overwrite and 2x Adjacent-right
+ Pattern('0xbb', 0x00d8000), # Partial-left (1M-160K)
+ Pattern('0xcc', 0x2028000), # Partial-right (32M+160K)
+ Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
]
class Drive:
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 15:47 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers John Snow
` (5 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
Represent a bitmap with an object that we can mark and clear bits in.
This makes it easier to manage partial writes when we don't write a
full group's worth of patterns before an error.
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/257 | 125 +++++++++++++++++++++++++----------------
1 file changed, 76 insertions(+), 49 deletions(-)
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index f576a35a5c..2ff4aa8695 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -85,6 +85,60 @@ GROUPS = [
Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
]
+
+class EmulatedBitmap:
+ def __init__(self, granularity=GRANULARITY):
+ self._bits = set()
+ self.groups = set()
+ self.granularity = granularity
+
+ def dirty_bits(self, bits):
+ self._bits |= set(bits)
+
+ def dirty_group(self, n):
+ self.dirty_bits(GROUPS[n].bits(self.granularity))
+
+ def clear(self):
+ self._bits = set()
+
+ def clear_bits(self, bits):
+ self._bits = self._bits - set(bits)
+
+ def clear_bit(self, bit):
+ self.clear_bits({bit})
+
+ def clear_group(self, n):
+ self.clear_bits(GROUPS[n].bits(self.granularity))
+
+ @property
+ def first_bit(self):
+ return sorted(self.bits)[0]
+
+ @property
+ def bits(self):
+ return self._bits
+
+ @property
+ def count(self):
+ return len(self.bits)
+
+ def compare(self, qmp_bitmap):
+ """
+ Print a nice human-readable message checking that a bitmap as reported
+ by the QMP interface has as many bits set as we expect it to.
+ """
+
+ name = qmp_bitmap.get('name', '(anonymous)')
+ log("= Checking Bitmap {:s} =".format(name))
+
+ want = self.count
+ have = qmp_bitmap['count'] // qmp_bitmap['granularity']
+
+ log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
+ want, have, "OK!" if want == have else "ERROR!"))
+ log('')
+
+
class Drive:
"""Represents, vaguely, a drive attached to a VM.
Includes format, graph, and device information."""
@@ -195,27 +249,6 @@ def perform_writes(drive, n):
log('')
return bitmaps
-def calculate_bits(groups=None):
- """Calculate how many bits we expect to see dirtied."""
- if groups:
- bits = set.union(*(GROUPS[group].bits(GRANULARITY) for group in groups))
- return len(bits)
- return 0
-
-def bitmap_comparison(bitmap, groups=None, want=0):
- """
- Print a nice human-readable message checking that this bitmap has as
- many bits set as we expect it to.
- """
- log("= Checking Bitmap {:s} =".format(bitmap.get('name', '(anonymous)')))
-
- if groups:
- want = calculate_bits(groups)
- have = bitmap['count'] // bitmap['granularity']
-
- log("expecting {:d} dirty sectors; have {:d}. {:s}".format(
- want, have, "OK!" if want == have else "ERROR!"))
- log('')
def compare_images(image, reference, baseimg=None, expected_match=True):
"""
@@ -321,12 +354,13 @@ def test_bitmap_sync(bsync_mode, failure=None):
vm.qmp_log("block-dirty-bitmap-add", node=drive0.name,
name="bitmap0", granularity=GRANULARITY)
log('')
+ ebitmap = EmulatedBitmap()
# 1 - Writes and Reference Backup
bitmaps = perform_writes(drive0, 1)
- dirty_groups = {1}
+ ebitmap.dirty_group(1)
bitmap = get_bitmap(bitmaps, drive0.device, 'bitmap0')
- bitmap_comparison(bitmap, groups=dirty_groups)
+ ebitmap.compare(bitmap)
reference_backup(drive0, 1, fbackup1)
# 1 - Bitmap Backup (Optional induced failure)
@@ -342,54 +376,47 @@ def test_bitmap_sync(bsync_mode, failure=None):
log('')
bitmaps = perform_writes(drive0, 2)
# Named bitmap (static, should be unchanged)
- bitmap_comparison(get_bitmap(bitmaps, drive0.device, 'bitmap0'),
- groups=dirty_groups)
+ ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
# Anonymous bitmap (dynamic, shows new writes)
- bitmap_comparison(get_bitmap(bitmaps, drive0.device, '',
- recording=True), groups={2})
- dirty_groups.add(2)
+ anonymous = EmulatedBitmap()
+ anonymous.dirty_group(2)
+ anonymous.compare(get_bitmap(bitmaps, drive0.device, '',
+ recording=True))
+
+ # Simulate the order in which this will happen:
+ # group 1 gets cleared first, then group two gets written.
+ if ((bsync_mode == 'on-success' and not failure) or
+ (bsync_mode == 'always')):
+ ebitmap.clear_group(1)
+ ebitmap.dirty_group(2)
vm.run_job(job, auto_dismiss=True, auto_finalize=False,
pre_finalize=_callback,
cancel=(failure == 'simulated'))
bitmaps = query_bitmaps(vm)
- bitmap = get_bitmap(bitmaps, drive0.device, 'bitmap0')
log(bitmaps, indent=2)
log('')
- if ((bsync_mode == 'on-success' and not failure) or
- (bsync_mode == 'always' and failure != 'intermediate')):
- dirty_groups.remove(1)
-
if bsync_mode == 'always' and failure == 'intermediate':
# We manage to copy one sector (one bit) before the error.
- bitmap_comparison(bitmap,
- want=calculate_bits(groups=dirty_groups) - 1)
- else:
- bitmap_comparison(bitmap, groups=dirty_groups)
+ ebitmap.clear_bit(ebitmap.first_bit)
+ ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
# 2 - Writes and Reference Backup
bitmaps = perform_writes(drive0, 3)
- dirty_groups.add(3)
- bitmap = get_bitmap(bitmaps, drive0.device, 'bitmap0')
- if bsync_mode == 'always' and failure == 'intermediate':
- # We're one bit short, still.
- bitmap_comparison(bitmap,
- want=calculate_bits(groups=dirty_groups) - 1)
- else:
- bitmap_comparison(bitmap, groups=dirty_groups)
+ ebitmap.dirty_group(3)
+ ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
reference_backup(drive0, 2, fbackup2)
# 2 - Bitmap Backup (In failure modes, this is a recovery.)
job = bitmap_backup(drive0, 2, bsync2, "bitmap0", bsync_mode)
vm.run_job(job, auto_dismiss=True, auto_finalize=False)
bitmaps = query_bitmaps(vm)
- bitmap = get_bitmap(bitmaps, drive0.device, 'bitmap0')
log(bitmaps, indent=2)
log('')
- bitmap_comparison(bitmap, groups={}
- if bsync_mode != 'never'
- else dirty_groups)
+ if bsync_mode != 'never':
+ ebitmap.clear()
+ ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
log('--- Cleanup ---\n')
vm.qmp_log("block-dirty-bitmap-remove",
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 16:04 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface John Snow
` (4 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
This test needs support for non-bitmap backups and missing or
unspecified bitmap sync modes, so rewrite the helpers to be a little
more generic.
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/257 | 46 +++++----
tests/qemu-iotests/257.out | 192 ++++++++++++++++++-------------------
2 files changed, 124 insertions(+), 114 deletions(-)
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 2ff4aa8695..2eb4f26c28 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -208,6 +208,14 @@ def get_bitmap(bitmaps, drivename, name, recording=None):
return bitmap
return None
+def blockdev_backup(vm, device, target, sync, **kwargs):
+ result = vm.qmp_log('blockdev-backup',
+ device=device,
+ target=target,
+ sync=sync,
+ **kwargs)
+ return result
+
def reference_backup(drive, n, filepath):
log("--- Reference Backup #{:d} ---\n".format(n))
target_id = "ref_target_{:d}".format(n)
@@ -215,24 +223,26 @@ def reference_backup(drive, n, filepath):
target_drive = Drive(filepath, vm=drive.vm)
target_drive.create_target(target_id, drive.fmt, drive.size)
- drive.vm.qmp_log("blockdev-backup",
- job_id=job_id, device=drive.name,
- target=target_id, sync="full")
+ blockdev_backup(drive.vm, drive.name, target_id, "full", job_id=job_id)
drive.vm.run_job(job_id, auto_dismiss=True)
log('')
-def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
- log("--- Bitmap Backup #{:d} ---\n".format(n))
- target_id = "bitmap_target_{:d}".format(n)
- job_id = "bitmap_backup_{:d}".format(n)
+def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
+ log("--- Test Backup #{:d} ---\n".format(n))
+ target_id = "backup_target_{:d}".format(n)
+ job_id = "backup_{:d}".format(n)
target_drive = Drive(filepath, vm=drive.vm)
target_drive.create_target(target_id, drive.fmt, drive.size)
- drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
- target=target_id, sync="bitmap",
- bitmap_mode=bitmap_mode,
- bitmap=bitmap,
- auto_finalize=False)
+
+ kwargs = {
+ 'job_id': job_id,
+ 'auto_finalize': False,
+ 'bitmap': bitmap,
+ 'bitmap_mode': bitmap_mode,
+ }
+ kwargs = {key: val for key, val in kwargs.items() if val is not None}
+ blockdev_backup(drive.vm, drive.name, target_id, sync, **kwargs)
return job_id
def perform_writes(drive, n):
@@ -264,7 +274,7 @@ def compare_images(image, reference, baseimg=None, expected_match=True):
"OK!" if ret == expected_ret else "ERROR!"),
filters=[iotests.filter_testfiles])
-def test_bitmap_sync(bsync_mode, failure=None):
+def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
"""
Test bitmap backup routines.
@@ -292,7 +302,7 @@ def test_bitmap_sync(bsync_mode, failure=None):
fbackup0, fbackup1, fbackup2), \
iotests.VM() as vm:
- mode = "Bitmap Sync Mode {:s}".format(bsync_mode)
+ mode = "Mode {:s}; Bitmap Sync {:s}".format(msync_mode, bsync_mode)
preposition = "with" if failure else "without"
cond = "{:s} {:s}".format(preposition,
"{:s} failure".format(failure) if failure
@@ -363,12 +373,12 @@ def test_bitmap_sync(bsync_mode, failure=None):
ebitmap.compare(bitmap)
reference_backup(drive0, 1, fbackup1)
- # 1 - Bitmap Backup (Optional induced failure)
+ # 1 - Test Backup (w/ Optional induced failure)
if failure == 'intermediate':
# Activate blkdebug induced failure for second-to-next read
log(vm.hmp_qemu_io(drive0.name, 'flush'))
log('')
- job = bitmap_backup(drive0, 1, bsync1, "bitmap0", bsync_mode)
+ job = backup(drive0, 1, bsync1, "bitmap0", bsync_mode, sync=msync_mode)
def _callback():
"""Issue writes while the job is open to test bitmap divergence."""
@@ -409,7 +419,7 @@ def test_bitmap_sync(bsync_mode, failure=None):
reference_backup(drive0, 2, fbackup2)
# 2 - Bitmap Backup (In failure modes, this is a recovery.)
- job = bitmap_backup(drive0, 2, bsync2, "bitmap0", bsync_mode)
+ job = backup(drive0, 2, bsync2, "bitmap0", bsync_mode)
vm.run_job(job, auto_dismiss=True, auto_finalize=False)
bitmaps = query_bitmaps(vm)
log(bitmaps, indent=2)
@@ -443,7 +453,7 @@ def test_bitmap_sync(bsync_mode, failure=None):
def main():
for bsync_mode in ("never", "on-success", "always"):
for failure in ("simulated", "intermediate", None):
- test_bitmap_sync(bsync_mode, failure)
+ test_bitmap_sync(bsync_mode, "bitmap", failure)
if __name__ == '__main__':
iotests.script_main(main, supported_fmts=['qcow2'])
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index e0775d4815..0abc96acd3 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -1,5 +1,5 @@
-=== Bitmap Sync Mode never with simulated failure ===
+=== Mode bitmap; Bitmap Sync never with simulated failure ===
--- Preparing image & VM ---
@@ -86,7 +86,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -96,7 +96,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
--- Write #2 ---
@@ -147,10 +147,10 @@ expecting 6 dirty sectors; have 6. OK!
= Checking Bitmap (anonymous) =
expecting 7 dirty sectors; have 7. OK!
-{"execute": "job-cancel", "arguments": {"id": "bitmap_backup_1"}}
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -213,7 +213,7 @@ expecting 15 dirty sectors; have 15. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -223,12 +223,12 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -265,7 +265,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode never with intermediate failure ===
+=== Mode bitmap; Bitmap Sync never with intermediate failure ===
--- Preparing image & VM ---
@@ -354,7 +354,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": ""}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -364,10 +364,10 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
-{"data": {"action": "report", "device": "bitmap_backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -430,7 +430,7 @@ expecting 14 dirty sectors; have 14. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -440,12 +440,12 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -482,7 +482,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode never without failure ===
+=== Mode bitmap; Bitmap Sync never without failure ===
--- Preparing image & VM ---
@@ -569,7 +569,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -579,7 +579,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
--- Write #2 ---
@@ -630,10 +630,10 @@ expecting 6 dirty sectors; have 6. OK!
= Checking Bitmap (anonymous) =
expecting 7 dirty sectors; have 7. OK!
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_1"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -696,7 +696,7 @@ expecting 15 dirty sectors; have 15. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -706,12 +706,12 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -748,7 +748,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode on-success with simulated failure ===
+=== Mode bitmap; Bitmap Sync on-success with simulated failure ===
--- Preparing image & VM ---
@@ -835,7 +835,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -845,7 +845,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
--- Write #2 ---
@@ -896,10 +896,10 @@ expecting 6 dirty sectors; have 6. OK!
= Checking Bitmap (anonymous) =
expecting 7 dirty sectors; have 7. OK!
-{"execute": "job-cancel", "arguments": {"id": "bitmap_backup_1"}}
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -962,7 +962,7 @@ expecting 15 dirty sectors; have 15. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -972,12 +972,12 @@ expecting 15 dirty sectors; have 15. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1014,7 +1014,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode on-success with intermediate failure ===
+=== Mode bitmap; Bitmap Sync on-success with intermediate failure ===
--- Preparing image & VM ---
@@ -1103,7 +1103,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": ""}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1113,10 +1113,10 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
-{"data": {"action": "report", "device": "bitmap_backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1179,7 +1179,7 @@ expecting 14 dirty sectors; have 14. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1189,12 +1189,12 @@ expecting 14 dirty sectors; have 14. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1231,7 +1231,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode on-success without failure ===
+=== Mode bitmap; Bitmap Sync on-success without failure ===
--- Preparing image & VM ---
@@ -1318,7 +1318,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1328,7 +1328,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
--- Write #2 ---
@@ -1379,10 +1379,10 @@ expecting 6 dirty sectors; have 6. OK!
= Checking Bitmap (anonymous) =
expecting 7 dirty sectors; have 7. OK!
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_1"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1445,7 +1445,7 @@ expecting 12 dirty sectors; have 12. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1455,12 +1455,12 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1497,7 +1497,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode always with simulated failure ===
+=== Mode bitmap; Bitmap Sync always with simulated failure ===
--- Preparing image & VM ---
@@ -1584,7 +1584,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1594,7 +1594,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
--- Write #2 ---
@@ -1645,10 +1645,10 @@ expecting 6 dirty sectors; have 6. OK!
= Checking Bitmap (anonymous) =
expecting 7 dirty sectors; have 7. OK!
-{"execute": "job-cancel", "arguments": {"id": "bitmap_backup_1"}}
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1711,7 +1711,7 @@ expecting 12 dirty sectors; have 12. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1721,12 +1721,12 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1763,7 +1763,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode always with intermediate failure ===
+=== Mode bitmap; Bitmap Sync always with intermediate failure ===
--- Preparing image & VM ---
@@ -1852,7 +1852,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": ""}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1862,10 +1862,10 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
-{"data": {"action": "report", "device": "bitmap_backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 66781184, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1928,7 +1928,7 @@ expecting 13 dirty sectors; have 13. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -1938,12 +1938,12 @@ expecting 13 dirty sectors; have 13. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -1980,7 +1980,7 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
-=== Bitmap Sync Mode always without failure ===
+=== Mode bitmap; Bitmap Sync always without failure ===
--- Preparing image & VM ---
@@ -2067,7 +2067,7 @@ expecting 6 dirty sectors; have 6. OK!
{"return": {}}
{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #1 ---
+--- Test Backup #1 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -2077,7 +2077,7 @@ expecting 6 dirty sectors; have 6. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bitmap_backup_1", "sync": "bitmap", "target": "bitmap_target_1"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "bitmap", "target": "backup_target_1"}}
{"return": {}}
--- Write #2 ---
@@ -2128,10 +2128,10 @@ expecting 6 dirty sectors; have 6. OK!
= Checking Bitmap (anonymous) =
expecting 7 dirty sectors; have 7. OK!
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_1"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
@@ -2194,7 +2194,7 @@ expecting 12 dirty sectors; have 12. OK!
{"return": {}}
{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
---- Bitmap Backup #2 ---
+--- Test Backup #2 ---
{}
{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
@@ -2204,12 +2204,12 @@ expecting 12 dirty sectors; have 12. OK!
{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
{"return": {}}
{}
-{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "bitmap_backup_2", "sync": "bitmap", "target": "bitmap_target_2"}}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
{"return": {}}
-{"execute": "job-finalize", "arguments": {"id": "bitmap_backup_2"}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
{"return": {}}
-{"data": {"id": "bitmap_backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
-{"data": {"device": "bitmap_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
{
"bitmaps": {
"device0": [
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
` (2 preceding siblings ...)
2019-07-10 1:05 ` [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 16:11 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 5/8] iotests/257: test API failures John Snow
` (3 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
This is nicer to do in the unified QMP interface that we have now,
because it lets us use the right terminology back at the user.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 13 ++++---------
blockdev.c | 10 ++++++++++
2 files changed, 14 insertions(+), 9 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index e2729cf6fa..a64b768e24 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
assert(bs);
assert(target);
+ /* QMP interface protects us from these cases */
+ assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
+ assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
+
if (bs == target) {
error_setg(errp, "Source and target cannot be the same");
return NULL;
@@ -597,16 +601,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- /* QMP interface should have handled translating this to bitmap mode */
- assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
-
if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
- if (!sync_bitmap) {
- error_setg(errp, "must provide a valid bitmap name for "
- "'%s' sync mode", MirrorSyncMode_str(sync_mode));
- return NULL;
- }
-
/* If we need to write to this bitmap, check that we can: */
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
diff --git a/blockdev.c b/blockdev.c
index 3e30bc2ca7..f0b7da53b0 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3464,6 +3464,16 @@ static BlockJob *do_backup_common(BackupCommon *backup,
return NULL;
}
+ if ((backup->sync == MIRROR_SYNC_MODE_BITMAP) ||
+ (backup->sync == MIRROR_SYNC_MODE_INCREMENTAL)) {
+ /* done before desugaring 'incremental' to print the right message */
+ if (!backup->has_bitmap) {
+ error_setg(errp, "must provide a valid bitmap name for "
+ "'%s' sync mode", MirrorSyncMode_str(backup->sync));
+ return NULL;
+ }
+ }
+
if (backup->sync == MIRROR_SYNC_MODE_INCREMENTAL) {
if (backup->has_bitmap_mode &&
backup->bitmap_mode != BITMAP_SYNC_MODE_ON_SUCCESS) {
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 5/8] iotests/257: test API failures
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
` (3 preceding siblings ...)
2019-07-10 1:05 ` [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 16:22 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions John Snow
` (2 subsequent siblings)
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/257 | 69 +++++++++++++++++++++++++++++++
tests/qemu-iotests/257.out | 85 ++++++++++++++++++++++++++++++++++++++
2 files changed, 154 insertions(+)
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index 2eb4f26c28..de8707cb19 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -450,10 +450,79 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
compare_images(img_path, fbackup2)
log('')
+def test_backup_api():
+ """
+ """
+ with iotests.FilePaths(['img', 'bsync1']) as \
+ (img_path, backup_path), \
+ iotests.VM() as vm:
+
+ log("\n=== API failure tests ===\n")
+ log('--- Preparing image & VM ---\n')
+ drive0 = Drive(img_path, vm=vm)
+ drive0.img_create(iotests.imgfmt, SIZE)
+ vm.add_device("{},id=scsi0".format(iotests.get_virtio_scsi_device()))
+ vm.launch()
+
+ file_config = {
+ 'driver': 'file',
+ 'filename': drive0.path
+ }
+
+ vm.qmp_log('blockdev-add',
+ filters=[iotests.filter_qmp_testfiles],
+ node_name="drive0",
+ driver=drive0.fmt,
+ file=file_config)
+ drive0.node = 'drive0'
+ drive0.device = 'device0'
+ vm.qmp_log("device_add", id=drive0.device,
+ drive=drive0.name, driver="scsi-hd")
+ log('')
+
+ target0 = Drive(backup_path, vm=vm)
+ target0.create_target("backup_target", drive0.fmt, drive0.size)
+ log('')
+
+ vm.qmp_log("block-dirty-bitmap-add", node=drive0.name,
+ name="bitmap0", granularity=GRANULARITY)
+ log('')
+
+ log('-- Testing invalid QMP commands --\n')
+
+ error_cases = {
+ 'incremental': {
+ None: ['on-success', 'always', 'never', None],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['always', 'never']
+ },
+ 'bitmap': {
+ None: ['on-success', 'always', 'never', None],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': [None],
+ },
+ }
+
+ for sync_mode, config in error_cases.items():
+ log("-- Sync mode {:s} tests --\n".format(sync_mode))
+ for bitmap, policies in config.items():
+ for policy in policies:
+ kwargs = { 'job_id': 'api_job' }
+ if bitmap is not None:
+ kwargs['bitmap'] = bitmap
+ if policy is not None:
+ kwargs['bitmap-mode'] = policy
+ blockdev_backup(drive0.vm, drive0.name, "backup_target",
+ sync_mode, **kwargs)
+ log('')
+
+
def main():
for bsync_mode in ("never", "on-success", "always"):
for failure in ("simulated", "intermediate", None):
test_bitmap_sync(bsync_mode, "bitmap", failure)
+ test_backup_api()
+
if __name__ == '__main__':
iotests.script_main(main, supported_fmts=['qcow2'])
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index 0abc96acd3..43f2e0f9c9 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -2245,3 +2245,88 @@ qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+=== API failure tests ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0"}}
+{"return": {}}
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+-- Testing invalid QMP commands --
+
+-- Sync mode incremental tests --
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'incremental' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "incremental", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be 'on-success' when using sync mode 'incremental'"}}
+
+-- Sync mode bitmap tests --
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "must provide a valid bitmap name for 'bitmap' sync mode"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
+
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
` (4 preceding siblings ...)
2019-07-10 1:05 ` [Qemu-devel] [PATCH 5/8] iotests/257: test API failures John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 16:36 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes John Snow
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
The way bitmap backups work is by starting at 75% if it needs
to copy just 25% of the disk.
The way sync=top currently works, however, is to start at 0% and then
never update the progress if it doesn't copy a region. If it needs to
copy 25% of the disk, we'll finish at 25%.
Update the progress when we skip regions.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/block/backup.c b/block/backup.c
index a64b768e24..38c4a688c6 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -417,6 +417,7 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
bdrv_is_unallocated_range(bs, offset, job->cluster_size))
{
+ job_progress_update(&job->common.job, job->cluster_size);
bdrv_reset_dirty_bitmap(job->copy_bitmap, offset,
job->cluster_size);
continue;
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
` (5 preceding siblings ...)
2019-07-10 1:05 ` [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 16:48 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes John Snow
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
Accept bitmaps and sync policies for the other backup modes.
This allows us to do things like create a bitmap synced to a full backup
without a transaction, or start a resumable backup process.
Some combinations don't make sense, though:
- NEVER policy combined with any non-BITMAP mode doesn't do anything,
because the bitmap isn't used for input or output.
It's harmless, but is almost certainly never what the user wanted.
- sync=NONE is more questionable. It can't use on-success because this
job never completes with success anyway, and the resulting artifact
of 'always' is suspect: because we start with a full bitmap and only
copy out segments that get written to, the final output bitmap will
always be ... a fully set bitmap.
Maybe there's contexts in which bitmaps make sense for sync=none,
but not without more severe changes to the current job, and omitting
it here doesn't prevent us from adding it later.
Signed-off-by: John Snow <jsnow@redhat.com>
---
block/backup.c | 8 +-------
blockdev.c | 22 ++++++++++++++++++++++
qapi/block-core.json | 6 ++++--
3 files changed, 27 insertions(+), 9 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 38c4a688c6..b94ed595ba 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -602,7 +602,7 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
return NULL;
}
- if (sync_mode == MIRROR_SYNC_MODE_BITMAP) {
+ if (sync_bitmap) {
/* If we need to write to this bitmap, check that we can: */
if (bitmap_mode != BITMAP_SYNC_MODE_NEVER &&
bdrv_dirty_bitmap_check(sync_bitmap, BDRV_BITMAP_DEFAULT, errp)) {
@@ -613,12 +613,6 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
if (bdrv_dirty_bitmap_create_successor(bs, sync_bitmap, errp) < 0) {
return NULL;
}
- } else if (sync_bitmap) {
- error_setg(errp,
- "a bitmap was given to backup_job_create, "
- "but it received an incompatible sync_mode (%s)",
- MirrorSyncMode_str(sync_mode));
- return NULL;
}
len = bdrv_getlength(bs);
diff --git a/blockdev.c b/blockdev.c
index f0b7da53b0..bc152f8e0d 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -3502,6 +3502,28 @@ static BlockJob *do_backup_common(BackupCommon *backup,
if (bdrv_dirty_bitmap_check(bmap, BDRV_BITMAP_ALLOW_RO, errp)) {
return NULL;
}
+
+ /* This does not produce a useful bitmap artifact: */
+ if (backup->sync == MIRROR_SYNC_MODE_NONE) {
+ error_setg(errp, "sync mode '%s' does not produce meaningful bitmap"
+ " outputs", MirrorSyncMode_str(backup->sync));
+ return NULL;
+ }
+
+ /* If the bitmap isn't used for input or output, this is useless: */
+ if (backup->bitmap_mode == BITMAP_SYNC_MODE_NEVER &&
+ backup->sync != MIRROR_SYNC_MODE_BITMAP) {
+ error_setg(errp, "Bitmap sync mode '%s' has no meaningful effect"
+ " when combined with sync mode '%s'",
+ BitmapSyncMode_str(backup->bitmap_mode),
+ MirrorSyncMode_str(backup->sync));
+ return NULL;
+ }
+ }
+
+ if (!backup->has_bitmap && backup->has_bitmap_mode) {
+ error_setg(errp, "Cannot specify Bitmap sync mode without a bitmap");
+ return NULL;
}
if (!backup->auto_finalize) {
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 5a578806c5..099e4f37b2 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -1352,13 +1352,15 @@
# @speed: the maximum speed, in bytes per second. The default is 0,
# for unlimited.
#
-# @bitmap: the name of a dirty bitmap if sync is "bitmap" or "incremental".
+# @bitmap: The name of a dirty bitmap to use.
# Must be present if sync is "bitmap" or "incremental".
+# Can be present if sync is "full" or "top".
# Must not be present otherwise.
# (Since 2.4 (drive-backup), 3.1 (blockdev-backup))
#
# @bitmap-mode: Specifies the type of data the bitmap should contain after
-# the operation concludes. Must be present if sync is "bitmap".
+# the operation concludes.
+# Must be present if a bitmap was provided,
# Must NOT be present otherwise. (Since 4.2)
#
# @compress: true to compress data, if the target format supports it.
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
` (6 preceding siblings ...)
2019-07-10 1:05 ` [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups John Snow
@ 2019-07-10 1:05 ` John Snow
2019-07-10 17:14 ` Max Reitz
7 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 1:05 UTC (permalink / raw)
To: qemu-block, qemu-devel
Cc: Kevin Wolf, John Snow, Markus Armbruster, Max Reitz
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/257 | 31 +
tests/qemu-iotests/257.out | 3089 ++++++++++++++++++++++++++++++++++++
2 files changed, 3120 insertions(+)
diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
index de8707cb19..8de1c4da19 100755
--- a/tests/qemu-iotests/257
+++ b/tests/qemu-iotests/257
@@ -288,6 +288,12 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
Bitmaps are always synchronized, regardless of failure.
(Partial images must be kept.)
+ :param msync_mode: The mirror sync mode to use for the first backup.
+ Can be any one of:
+ - bitmap: Backups based on bitmap manifest.
+ - full: Full backups.
+ - top: Full backups of the top layer only.
+
:param failure: Is the (optional) failure mode, and can be any of:
- None: No failure. Test the normative path. Default.
- simulated: Cancel the job right before it completes.
@@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
if bsync_mode == 'always' and failure == 'intermediate':
# We manage to copy one sector (one bit) before the error.
ebitmap.clear_bit(ebitmap.first_bit)
+ if msync_mode in ('full', 'top'):
+ # These modes return all bits set except what was copied/skipped
+ fail_bit = ebitmap.first_bit
+ ebitmap.clear()
+ ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
# 2 - Writes and Reference Backup
@@ -501,6 +512,21 @@ def test_backup_api():
'bitmap404': ['on-success', 'always', 'never', None],
'bitmap0': [None],
},
+ 'full': {
+ None: ['on-success', 'always', 'never'],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['never', None],
+ },
+ 'top': {
+ None: ['on-success', 'always', 'never'],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['never', None],
+ },
+ 'none': {
+ None: ['on-success', 'always', 'never'],
+ 'bitmap404': ['on-success', 'always', 'never', None],
+ 'bitmap0': ['on-success', 'always', 'never', None],
+ }
}
for sync_mode, config in error_cases.items():
@@ -522,6 +548,11 @@ def main():
for failure in ("simulated", "intermediate", None):
test_bitmap_sync(bsync_mode, "bitmap", failure)
+ for sync_mode in ('full', 'top'):
+ for bsync_mode in ('on-success', 'always'):
+ for failure in ('simulated', 'intermediate', None):
+ test_bitmap_sync(bsync_mode, sync_mode, failure)
+
test_backup_api()
if __name__ == '__main__':
diff --git a/tests/qemu-iotests/257.out b/tests/qemu-iotests/257.out
index 43f2e0f9c9..304e33ab73 100644
--- a/tests/qemu-iotests/257.out
+++ b/tests/qemu-iotests/257.out
@@ -2246,6 +2246,3002 @@ qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK
qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+=== Mode full; Bitmap Sync on-success with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode full; Bitmap Sync on-success with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 917504,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 14 dirty sectors; have 14. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode full; Bitmap Sync on-success without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 12 dirty sectors; have 12. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode full; Bitmap Sync always with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 12 dirty sectors; have 12. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode full; Bitmap Sync always with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 66125824,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 1009 dirty sectors; have 1009. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 66453504,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 1014 dirty sectors; have 1014. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode full; Bitmap Sync always without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "full", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 12 dirty sectors; have 12. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode top; Bitmap Sync on-success with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 655360,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 10 dirty sectors; have 10. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 983040,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 15 dirty sectors; have 15. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode top; Bitmap Sync on-success with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 917504,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 14 dirty sectors; have 14. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode top; Bitmap Sync on-success without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 12 dirty sectors; have 12. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode top; Bitmap Sync always with simulated failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-cancel", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_CANCELLED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 12 dirty sectors; have 12. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode top; Bitmap Sync always with intermediate failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "blkdebug", "image": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "inject-error": [{"errno": 5, "event": "read_aio", "immediately": false, "once": true, "state": 3}], "set-state": [{"event": "flush_to_disk", "new-state": 2, "state": 1}, {"event": "read_aio", "new-state": 3, "state": 2}]}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+{"return": ""}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"return": {}}
+{"data": {"action": "report", "device": "backup_1", "operation": "read"}, "event": "BLOCK_JOB_ERROR", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "error": "Input/output error", "len": 67108864, "offset": 983040, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 66125824,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 1009 dirty sectors; have 1009. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 66453504,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 1014 dirty sectors; have 1014. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Mismatch, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
+=== Mode top; Bitmap Sync always without failure ===
+
+--- Preparing image & VM ---
+
+{"execute": "blockdev-add", "arguments": {"driver": "qcow2", "file": {"driver": "file", "filename": "TEST_DIR/PID-img"}, "node-name": "drive0"}}
+{"return": {}}
+{"execute": "device_add", "arguments": {"drive": "drive0", "driver": "scsi-hd", "id": "device0", "share-rw": true}}
+{"return": {}}
+
+--- Write #0 ---
+
+write -P0x49 0x0000000 0x10000
+{"return": ""}
+write -P0x6c 0x0100000 0x10000
+{"return": ""}
+write -P0x6f 0x2000000 0x10000
+{"return": ""}
+write -P0x76 0x3ff0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Reference Backup #0 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_0", "sync": "full", "target": "ref_target_0"}}
+{"return": {}}
+{"data": {"device": "ref_backup_0", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Add Bitmap ---
+
+{"execute": "block-dirty-bitmap-add", "arguments": {"granularity": 65536, "name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+
+--- Write #1 ---
+
+write -P0x65 0x0000000 0x10000
+{"return": ""}
+write -P0x77 0x00f8000 0x10000
+{"return": ""}
+write -P0x72 0x2008000 0x10000
+{"return": ""}
+write -P0x69 0x3fe0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+--- Reference Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_1", "sync": "full", "target": "ref_target_1"}}
+{"return": {}}
+{"data": {"device": "ref_backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #1 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_1", "sync": "top", "target": "backup_target_1"}}
+{"return": {}}
+
+--- Write #2 ---
+
+write -P0x74 0x0010000 0x10000
+{"return": ""}
+write -P0x69 0x00e8000 0x10000
+{"return": ""}
+write -P0x6e 0x2018000 0x10000
+{"return": ""}
+write -P0x67 0x3fe0000 0x20000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": false,
+ "status": "disabled"
+ },
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ },
+ {
+ "busy": true,
+ "count": 393216,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "frozen"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 6 dirty sectors; have 6. OK!
+
+= Checking Bitmap (anonymous) =
+expecting 7 dirty sectors; have 7. OK!
+
+{"execute": "job-finalize", "arguments": {"id": "backup_1"}}
+{"return": {}}
+{"data": {"id": "backup_1", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_1", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 458752,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 7 dirty sectors; have 7. OK!
+
+--- Write #3 ---
+
+write -P0xaa 0x0010000 0x30000
+{"return": ""}
+write -P0xbb 0x00d8000 0x10000
+{"return": ""}
+write -P0xcc 0x2028000 0x10000
+{"return": ""}
+write -P0xdd 0x3fc0000 0x10000
+{"return": ""}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 786432,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 12 dirty sectors; have 12. OK!
+
+--- Reference Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"device": "drive0", "job-id": "ref_backup_2", "sync": "full", "target": "ref_target_2"}}
+{"return": {}}
+{"data": {"device": "ref_backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+
+--- Test Backup #2 ---
+
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-file-job"}}
+{"return": {}}
+{}
+{}
+{"execute": "job-dismiss", "arguments": {"id": "bdc-fmt-job"}}
+{"return": {}}
+{}
+{"execute": "blockdev-backup", "arguments": {"auto-finalize": false, "bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "backup_2", "sync": "bitmap", "target": "backup_target_2"}}
+{"return": {}}
+{"execute": "job-finalize", "arguments": {"id": "backup_2"}}
+{"return": {}}
+{"data": {"id": "backup_2", "type": "backup"}, "event": "BLOCK_JOB_PENDING", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{"data": {"device": "backup_2", "len": 67108864, "offset": 67108864, "speed": 0, "type": "backup"}, "event": "BLOCK_JOB_COMPLETED", "timestamp": {"microseconds": "USECS", "seconds": "SECS"}}
+{
+ "bitmaps": {
+ "device0": [
+ {
+ "busy": false,
+ "count": 0,
+ "granularity": 65536,
+ "name": "bitmap0",
+ "persistent": false,
+ "recording": true,
+ "status": "active"
+ }
+ ]
+ }
+}
+
+= Checking Bitmap bitmap0 =
+expecting 0 dirty sectors; have 0. OK!
+
+--- Cleanup ---
+
+{"execute": "block-dirty-bitmap-remove", "arguments": {"name": "bitmap0", "node": "drive0"}}
+{"return": {}}
+{
+ "bitmaps": {
+ "device0": []
+ }
+}
+
+--- Verification ---
+
+qemu_img compare "TEST_DIR/PID-bsync1" "TEST_DIR/PID-fbackup1" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-bsync2" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
+
+
=== API failure tests ===
--- Preparing image & VM ---
@@ -2330,3 +5326,96 @@ qemu_img compare "TEST_DIR/PID-img" "TEST_DIR/PID-fbackup2" ==> Identical, OK!
{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "job-id": "api_job", "sync": "bitmap", "target": "backup_target"}}
{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
+-- Sync mode full tests --
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode 'never' has no meaningful effect when combined with sync mode 'full'"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "job-id": "api_job", "sync": "full", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
+
+-- Sync mode top tests --
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode 'never' has no meaningful effect when combined with sync mode 'top'"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "job-id": "api_job", "sync": "top", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
+
+-- Sync mode none tests --
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Cannot specify Bitmap sync mode without a bitmap"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap404", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap 'bitmap404' could not be found"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "on-success", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "sync mode 'none' does not produce meaningful bitmap outputs"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "always", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "sync mode 'none' does not produce meaningful bitmap outputs"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "bitmap-mode": "never", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "sync mode 'none' does not produce meaningful bitmap outputs"}}
+
+{"execute": "blockdev-backup", "arguments": {"bitmap": "bitmap0", "device": "drive0", "job-id": "api_job", "sync": "none", "target": "backup_target"}}
+{"error": {"class": "GenericError", "desc": "Bitmap sync mode must be given when providing a bitmap"}}
+
--
2.21.0
^ permalink raw reply related [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class
2019-07-10 1:05 ` [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class John Snow
@ 2019-07-10 15:10 ` Max Reitz
2019-07-10 16:26 ` Max Reitz
1 sibling, 0 replies; 35+ messages in thread
From: Max Reitz @ 2019-07-10 15:10 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 360 bytes --]
On 10.07.19 03:05, John Snow wrote:
> Just kidding, this is easier to manage with a full class instead of a
> namedtuple.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 58 +++++++++++++++++++++++-------------------
> 1 file changed, 32 insertions(+), 26 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class
2019-07-10 1:05 ` [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class John Snow
@ 2019-07-10 15:47 ` Max Reitz
2019-07-10 17:36 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 15:47 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1333 bytes --]
On 10.07.19 03:05, John Snow wrote:
> Represent a bitmap with an object that we can mark and clear bits in.
> This makes it easier to manage partial writes when we don't write a
> full group's worth of patterns before an error.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 125 +++++++++++++++++++++++++----------------
> 1 file changed, 76 insertions(+), 49 deletions(-)
>
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> index f576a35a5c..2ff4aa8695 100755
> --- a/tests/qemu-iotests/257
> +++ b/tests/qemu-iotests/257
> @@ -85,6 +85,60 @@ GROUPS = [
> Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
> ]
>
> +
> +class EmulatedBitmap:
> + def __init__(self, granularity=GRANULARITY):
> + self._bits = set()
> + self.groups = set()
> + self.granularity = granularity
> +
> + def dirty_bits(self, bits):
> + self._bits |= set(bits)
> +
> + def dirty_group(self, n):
> + self.dirty_bits(GROUPS[n].bits(self.granularity))
> +
> + def clear(self):
> + self._bits = set()
> +
> + def clear_bits(self, bits):
> + self._bits = self._bits - set(bits)
Does -= not work here?
No real complaints. Sorry.
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers
2019-07-10 1:05 ` [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers John Snow
@ 2019-07-10 16:04 ` Max Reitz
2019-07-10 17:52 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 16:04 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 2209 bytes --]
On 10.07.19 03:05, John Snow wrote:
> This test needs support for non-bitmap backups and missing or
> unspecified bitmap sync modes, so rewrite the helpers to be a little
> more generic.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 46 +++++----
> tests/qemu-iotests/257.out | 192 ++++++++++++++++++-------------------
> 2 files changed, 124 insertions(+), 114 deletions(-)
>
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> index 2ff4aa8695..2eb4f26c28 100755
> --- a/tests/qemu-iotests/257
> +++ b/tests/qemu-iotests/257
[...]
> -def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
> - log("--- Bitmap Backup #{:d} ---\n".format(n))
> - target_id = "bitmap_target_{:d}".format(n)
> - job_id = "bitmap_backup_{:d}".format(n)
> +def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
> + log("--- Test Backup #{:d} ---\n".format(n))
> + target_id = "backup_target_{:d}".format(n)
> + job_id = "backup_{:d}".format(n)
> target_drive = Drive(filepath, vm=drive.vm)
>
> target_drive.create_target(target_id, drive.fmt, drive.size)
> - drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
> - target=target_id, sync="bitmap",
> - bitmap_mode=bitmap_mode,
> - bitmap=bitmap,
> - auto_finalize=False)
> +
> + kwargs = {
> + 'job_id': job_id,
> + 'auto_finalize': False,
> + 'bitmap': bitmap,
> + 'bitmap_mode': bitmap_mode,
> + }
> + kwargs = {key: val for key, val in kwargs.items() if val is not None}
I suppose this is to remove items that are None?
Very cute, but why not just
kwargs = {
'job_id': job_id,
'auto_finalize': False,
}
if bitmap is not None:
kwargs['bitmap'] = bitmap
kwargs['bitmap_mode'] = bitmap_mode
Exactly the same number of lines, but immediately makes it clear what’s
going on. Not as cute, I admit.
(Yes, I am indeed actively trying to train you not to write cute code.)
The rest looks good to me:
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface
2019-07-10 1:05 ` [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface John Snow
@ 2019-07-10 16:11 ` Max Reitz
2019-07-10 17:57 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 16:11 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1073 bytes --]
On 10.07.19 03:05, John Snow wrote:
> This is nicer to do in the unified QMP interface that we have now,
> because it lets us use the right terminology back at the user.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 13 ++++---------
> blockdev.c | 10 ++++++++++
> 2 files changed, 14 insertions(+), 9 deletions(-)
>
> diff --git a/block/backup.c b/block/backup.c
> index e2729cf6fa..a64b768e24 100644
> --- a/block/backup.c
> +++ b/block/backup.c
> @@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
> assert(bs);
> assert(target);
>
> + /* QMP interface protects us from these cases */
> + assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
> + assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
Implication would be a nice operator sometimes.
("assert(sync_mode == MIRROR_SYNC_MODE_BITMAP -> sync_bitmap)")
(Can you do that in C++? No, you can’t overload bool’s operators, right?)
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 5/8] iotests/257: test API failures
2019-07-10 1:05 ` [Qemu-devel] [PATCH 5/8] iotests/257: test API failures John Snow
@ 2019-07-10 16:22 ` Max Reitz
2019-07-10 18:00 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 16:22 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1030 bytes --]
On 10.07.19 03:05, John Snow wrote:
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 69 +++++++++++++++++++++++++++++++
> tests/qemu-iotests/257.out | 85 ++++++++++++++++++++++++++++++++++++++
> 2 files changed, 154 insertions(+)
>
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> index 2eb4f26c28..de8707cb19 100755
> --- a/tests/qemu-iotests/257
> +++ b/tests/qemu-iotests/257
> @@ -450,10 +450,79 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
> compare_images(img_path, fbackup2)
> log('')
>
> +def test_backup_api():
> + """
> + """
Er, OK?
[...]
> + for sync_mode, config in error_cases.items():
> + log("-- Sync mode {:s} tests --\n".format(sync_mode))
> + for bitmap, policies in config.items():
You might be interested in the fact that the iteration order is
different for Python2. Or maybe you aren’t.
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class
2019-07-10 1:05 ` [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class John Snow
2019-07-10 15:10 ` Max Reitz
@ 2019-07-10 16:26 ` Max Reitz
2019-07-10 17:34 ` John Snow
1 sibling, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 16:26 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1646 bytes --]
On 10.07.19 03:05, John Snow wrote:
> Just kidding, this is easier to manage with a full class instead of a
> namedtuple.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 58 +++++++++++++++++++++++-------------------
> 1 file changed, 32 insertions(+), 26 deletions(-)
>
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> index 75a651c7c3..f576a35a5c 100755
> --- a/tests/qemu-iotests/257
> +++ b/tests/qemu-iotests/257
> @@ -19,7 +19,6 @@
> #
> # owner=jsnow@redhat.com
>
> -from collections import namedtuple
> import math
> import os
>
> @@ -29,10 +28,18 @@ from iotests import log, qemu_img
> SIZE = 64 * 1024 * 1024
> GRANULARITY = 64 * 1024
>
> -Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
> -def mkpattern(byte, offset, size=GRANULARITY):
> - """Constructor for Pattern() with default size"""
> - return Pattern(byte, offset, size)
> +
> +class Pattern:
> + def __init__(self, byte, offset, size=GRANULARITY):
> + self.byte = byte
> + self.offset = offset
> + self.size = size
> +
> + def bits(self, granularity):
> + lower = math.floor(self.offset / granularity)
> + upper = math.floor((self.offset + self.size - 1) / granularity)
> + return set(range(lower, upper + 1))
By the way, this doesn’t work with Python2 (pre-existing in your other
series). It complains that these are floats.
Now I don’t know whether you care but there is the fact that the
expressions would be shorter if they were of the form x // y instead of
math.floor(x / y).
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions
2019-07-10 1:05 ` [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions John Snow
@ 2019-07-10 16:36 ` Max Reitz
2019-07-10 18:20 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 16:36 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1272 bytes --]
On 10.07.19 03:05, John Snow wrote:
> The way bitmap backups work is by starting at 75% if it needs
> to copy just 25% of the disk.
Although there is this comment:
> /* TODO job_progress_set_remaining() would make more sense */
So...
> The way sync=top currently works, however, is to start at 0% and then
> never update the progress if it doesn't copy a region. If it needs to
> copy 25% of the disk, we'll finish at 25%.
>
> Update the progress when we skip regions.
Wouldn’t it be more correct to decrease the job length?
Max
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 1 +
> 1 file changed, 1 insertion(+)
>
> diff --git a/block/backup.c b/block/backup.c
> index a64b768e24..38c4a688c6 100644
> --- a/block/backup.c
> +++ b/block/backup.c
> @@ -417,6 +417,7 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
> if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
> bdrv_is_unallocated_range(bs, offset, job->cluster_size))
> {
> + job_progress_update(&job->common.job, job->cluster_size);
> bdrv_reset_dirty_bitmap(job->copy_bitmap, offset,
> job->cluster_size);
> continue;
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups
2019-07-10 1:05 ` [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups John Snow
@ 2019-07-10 16:48 ` Max Reitz
2019-07-10 18:32 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 16:48 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1695 bytes --]
On 10.07.19 03:05, John Snow wrote:
> Accept bitmaps and sync policies for the other backup modes.
> This allows us to do things like create a bitmap synced to a full backup
> without a transaction, or start a resumable backup process.
>
> Some combinations don't make sense, though:
>
> - NEVER policy combined with any non-BITMAP mode doesn't do anything,
> because the bitmap isn't used for input or output.
> It's harmless, but is almost certainly never what the user wanted.
>
> - sync=NONE is more questionable. It can't use on-success because this
> job never completes with success anyway, and the resulting artifact
> of 'always' is suspect: because we start with a full bitmap and only
> copy out segments that get written to, the final output bitmap will
> always be ... a fully set bitmap.
>
> Maybe there's contexts in which bitmaps make sense for sync=none,
> but not without more severe changes to the current job, and omitting
> it here doesn't prevent us from adding it later.
>
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> block/backup.c | 8 +-------
> blockdev.c | 22 ++++++++++++++++++++++
> qapi/block-core.json | 6 ++++--
> 3 files changed, 27 insertions(+), 9 deletions(-)
[...]
> diff --git a/blockdev.c b/blockdev.c
> index f0b7da53b0..bc152f8e0d 100644
> --- a/blockdev.c
> +++ b/blockdev.c
[...]
> + if (!backup->has_bitmap && backup->has_bitmap_mode) {
> + error_setg(errp, "Cannot specify Bitmap sync mode without a bitmap");
Any reason for capitalizing the first “Bitmap”?
With a reason or it fixed:
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes
2019-07-10 1:05 ` [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes John Snow
@ 2019-07-10 17:14 ` Max Reitz
2019-07-10 19:00 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 17:14 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1368 bytes --]
On 10.07.19 03:05, John Snow wrote:
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/257 | 31 +
> tests/qemu-iotests/257.out | 3089 ++++++++++++++++++++++++++++++++++++
> 2 files changed, 3120 insertions(+)
Oof.
> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
> index de8707cb19..8de1c4da19 100755
> --- a/tests/qemu-iotests/257
> +++ b/tests/qemu-iotests/257
[...]
> @@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
> if bsync_mode == 'always' and failure == 'intermediate':
> # We manage to copy one sector (one bit) before the error.
> ebitmap.clear_bit(ebitmap.first_bit)
> + if msync_mode in ('full', 'top'):
> + # These modes return all bits set except what was copied/skipped
Hm. How useful is bitmap support for 'top' then, anyway? That means
that if you want to resume a top backup, you always have to resume it
like it was a full backup. Which sounds kind of useless.
Max
> + fail_bit = ebitmap.first_bit
> + ebitmap.clear()
> + ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
> ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
>
> # 2 - Writes and Reference Backup
[...]
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class
2019-07-10 16:26 ` Max Reitz
@ 2019-07-10 17:34 ` John Snow
0 siblings, 0 replies; 35+ messages in thread
From: John Snow @ 2019-07-10 17:34 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 12:26 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Just kidding, this is easier to manage with a full class instead of a
>> namedtuple.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> tests/qemu-iotests/257 | 58 +++++++++++++++++++++++-------------------
>> 1 file changed, 32 insertions(+), 26 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index 75a651c7c3..f576a35a5c 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>> @@ -19,7 +19,6 @@
>> #
>> # owner=jsnow@redhat.com
>>
>> -from collections import namedtuple
>> import math
>> import os
>>
>> @@ -29,10 +28,18 @@ from iotests import log, qemu_img
>> SIZE = 64 * 1024 * 1024
>> GRANULARITY = 64 * 1024
>>
>> -Pattern = namedtuple('Pattern', ['byte', 'offset', 'size'])
>> -def mkpattern(byte, offset, size=GRANULARITY):
>> - """Constructor for Pattern() with default size"""
>> - return Pattern(byte, offset, size)
>> +
>> +class Pattern:
>> + def __init__(self, byte, offset, size=GRANULARITY):
>> + self.byte = byte
>> + self.offset = offset
>> + self.size = size
>> +
>> + def bits(self, granularity):
>> + lower = math.floor(self.offset / granularity)
>> + upper = math.floor((self.offset + self.size - 1) / granularity)
>> + return set(range(lower, upper + 1))
>
> By the way, this doesn’t work with Python2 (pre-existing in your other
> series). It complains that these are floats.
>
> Now I don’t know whether you care but there is the fact that the
> expressions would be shorter if they were of the form x // y instead of
> math.floor(x / y).
>
> Max
>
Ah, crud; OK -- I'll play nice with python2 for a while longer. Thanks
for pointing this out.
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class
2019-07-10 15:47 ` Max Reitz
@ 2019-07-10 17:36 ` John Snow
0 siblings, 0 replies; 35+ messages in thread
From: John Snow @ 2019-07-10 17:36 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 11:47 AM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Represent a bitmap with an object that we can mark and clear bits in.
>> This makes it easier to manage partial writes when we don't write a
>> full group's worth of patterns before an error.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> tests/qemu-iotests/257 | 125 +++++++++++++++++++++++++----------------
>> 1 file changed, 76 insertions(+), 49 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index f576a35a5c..2ff4aa8695 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>> @@ -85,6 +85,60 @@ GROUPS = [
>> Pattern('0xdd', 0x3fc0000)]), # New; leaving a gap to the right
>> ]
>>
>> +
>> +class EmulatedBitmap:
>> + def __init__(self, granularity=GRANULARITY):
>> + self._bits = set()
>> + self.groups = set()
>> + self.granularity = granularity
>> +
>> + def dirty_bits(self, bits):
>> + self._bits |= set(bits)
>> +
>> + def dirty_group(self, n):
>> + self.dirty_bits(GROUPS[n].bits(self.granularity))
>> +
>> + def clear(self):
>> + self._bits = set()
>> +
>> + def clear_bits(self, bits):
>> + self._bits = self._bits - set(bits)
>
> Does -= not work here?
>
> No real complaints. Sorry.
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
It's okay. I forget which shorthand operators Python has for which types
sometimes.
>>> a = {'a', 'b', 'c'}
>>> a -= {'c'}
>>> a
{'b', 'a'}
Well, apparently it does. I'll change it.
--js
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers
2019-07-10 16:04 ` Max Reitz
@ 2019-07-10 17:52 ` John Snow
2019-07-10 20:17 ` Max Reitz
0 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 17:52 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 12:04 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> This test needs support for non-bitmap backups and missing or
>> unspecified bitmap sync modes, so rewrite the helpers to be a little
>> more generic.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> tests/qemu-iotests/257 | 46 +++++----
>> tests/qemu-iotests/257.out | 192 ++++++++++++++++++-------------------
>> 2 files changed, 124 insertions(+), 114 deletions(-)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index 2ff4aa8695..2eb4f26c28 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>
> [...]
>
>> -def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
>> - log("--- Bitmap Backup #{:d} ---\n".format(n))
>> - target_id = "bitmap_target_{:d}".format(n)
>> - job_id = "bitmap_backup_{:d}".format(n)
>> +def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
>> + log("--- Test Backup #{:d} ---\n".format(n))
>> + target_id = "backup_target_{:d}".format(n)
>> + job_id = "backup_{:d}".format(n)
>> target_drive = Drive(filepath, vm=drive.vm)
>>
>> target_drive.create_target(target_id, drive.fmt, drive.size)
>> - drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
>> - target=target_id, sync="bitmap",
>> - bitmap_mode=bitmap_mode,
>> - bitmap=bitmap,
>> - auto_finalize=False)
>> +
>> + kwargs = {
>> + 'job_id': job_id,
>> + 'auto_finalize': False,
>> + 'bitmap': bitmap,
>> + 'bitmap_mode': bitmap_mode,
>> + }
>> + kwargs = {key: val for key, val in kwargs.items() if val is not None}
>
> I suppose this is to remove items that are None?
>
> Very cute, but why not just
>
> kwargs = {
> 'job_id': job_id,
> 'auto_finalize': False,
> }
> if bitmap is not None:
> kwargs['bitmap'] = bitmap
> kwargs['bitmap_mode'] = bitmap_mode
>
> Exactly the same number of lines, but immediately makes it clear what’s
> going on. Not as cute, I admit.
>
> (Yes, I am indeed actively trying to train you not to write cute code.)
>
It sneaks in. I genuinely struggle with understanding what other people
will find readable; I have an authentically hard time reviewing other
people's patches too. I'm earnestly not sure how I can help improve
this, but I would like to.
I wasn't sure what the easiest way to avoid sending the "None" over the
wire was, so I went with a general thing, but yes: it's because bitmap
and bitmap_mode are set to None sometimes and I need to omit such keys.
In this case, though, I do test bitmap and bitmap_mode separately, so
for the purposes of testing intentionally bad combinations you do need:
if bitmap is not None:
kwargs['bitmap'] = bitmap
if bitmap_mode is not None:
kwargs['bitmap_mode'] = bitmap_mode
And I just looked at this and it did not spark joy; so I went with a
generic filter to remove nulled keys. I admit it's /slightly/ cute and
not immediately obvious why it needs to be done.
This is even cuter, so maybe I am traveling in the wrong direction:
def backup(drive, n, filepath, sync, **kwargs):
log("--- Test Backup #{:d} ---\n".format(n))
target_id = "backup_target_{:d}".format(n)
job_id = "backup_{:d}".format(n)
target_drive = Drive(filepath, vm=drive.vm)
target_drive.create_target(target_id, drive.fmt, drive.size)
kwargs.setdefault('auto_finalize', False)
# Strip any arguments explicitly nulled by the caller:
kwargs = {key: val for key, val in kwargs.items()
if val is not None}
blockdev_backup(drive.vm, drive.name, target_id, sync, **kwargs)
return job_id
It's quite a bit shorter and also makes backup() more flexible by
omitting the bitmap and bitmap_mode arguments entirely, allowing the
caller to override the auto_finalize default, etc. In this permutation,
we don't know the full extent of kwargs so it makes sense to generically
filter it.
Manually conditionally setting arguments is probably also fine.
Do you still have a preference for the more static approach?
> The rest looks good to me:
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
Thanks for reviewing, as always!
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface
2019-07-10 16:11 ` Max Reitz
@ 2019-07-10 17:57 ` John Snow
2019-07-10 20:19 ` Max Reitz
0 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 17:57 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 12:11 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> This is nicer to do in the unified QMP interface that we have now,
>> because it lets us use the right terminology back at the user.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/backup.c | 13 ++++---------
>> blockdev.c | 10 ++++++++++
>> 2 files changed, 14 insertions(+), 9 deletions(-)
>>
>> diff --git a/block/backup.c b/block/backup.c
>> index e2729cf6fa..a64b768e24 100644
>> --- a/block/backup.c
>> +++ b/block/backup.c
>> @@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
>> assert(bs);
>> assert(target);
>>
>> + /* QMP interface protects us from these cases */
>> + assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
>> + assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
>
> Implication would be a nice operator sometimes.
>
> ("assert(sync_mode == MIRROR_SYNC_MODE_BITMAP -> sync_bitmap)")
>
> (Can you do that in C++? No, you can’t overload bool’s operators, right?)
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
Yes, I also find this assertion kind of hard to read personally, but it
feels somewhat clunky to write:
if (antecedent) {
assert(condition);
}
I suppose we can also phrase this as:
assert(sync_mode == MIRROR_SYNC_MODE_BITMAP ? sync_bitmap : true);
Which might honestly be pretty good. Mind if I change it to this?
--js
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 5/8] iotests/257: test API failures
2019-07-10 16:22 ` Max Reitz
@ 2019-07-10 18:00 ` John Snow
0 siblings, 0 replies; 35+ messages in thread
From: John Snow @ 2019-07-10 18:00 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 12:22 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> tests/qemu-iotests/257 | 69 +++++++++++++++++++++++++++++++
>> tests/qemu-iotests/257.out | 85 ++++++++++++++++++++++++++++++++++++++
>> 2 files changed, 154 insertions(+)
>>
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index 2eb4f26c28..de8707cb19 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>> @@ -450,10 +450,79 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
>> compare_images(img_path, fbackup2)
>> log('')
>>
>> +def test_backup_api():
>> + """
>> + """
>
> Er, OK?
>
> [...]
>
Whooooooooops.
>> + for sync_mode, config in error_cases.items():
>> + log("-- Sync mode {:s} tests --\n".format(sync_mode))
>> + for bitmap, policies in config.items():
>
> You might be interested in the fact that the iteration order is
> different for Python2. Or maybe you aren’t.
>
asdf. This is undoubtedly the worst thing about Python. I'll have to fix
this, because the ordering isn't guaranteed until 3.5 or some such and
we're only EOLing Python2.
--js
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions
2019-07-10 16:36 ` Max Reitz
@ 2019-07-10 18:20 ` John Snow
2019-07-10 20:30 ` Max Reitz
0 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 18:20 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 12:36 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> The way bitmap backups work is by starting at 75% if it needs
>> to copy just 25% of the disk.
>
> Although there is this comment:
>
>> /* TODO job_progress_set_remaining() would make more sense */
>
> So...
>
>> The way sync=top currently works, however, is to start at 0% and then
>> never update the progress if it doesn't copy a region. If it needs to
>> copy 25% of the disk, we'll finish at 25%.
>>
>> Update the progress when we skip regions.
>
> Wouldn’t it be more correct to decrease the job length?
>
> Max
>
Admittedly I have precisely no idea. Maybe? As far as I understand it,
we guarantee only:
(1) Progress monotonically increases
(2) Upon completion, progress will equal the total work estimate.
[Trying to fix that to be true here.]
This means we can do stuff like:
- Total work estimate can increase or decrease arbitrarily
- Neither value has to mean anything in particular
Bitmap sync works by artificially increasing progress for NOP regions,
seen in init_copy_bitmap.
Full sync also tends to increase progress regardless of it actually did
a copy or not; offload support also counts as progress here. So if you
full sync an empty image, you'll see it increasing the progress as it
doesn't actually do anything.
Top sync is the odd one out, which just omits progress for regions it skips.
My only motivation here was to make them consistent. Can I do it the
other way? Yeah, probably. Is one way better than the other? I
legitimately have no idea. I guess whoever wrote the last comment felt
that it should all be the other way instead. Why'd they not do that?
¯\_(ツ)_/¯
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/backup.c | 1 +
>> 1 file changed, 1 insertion(+)
>>
>> diff --git a/block/backup.c b/block/backup.c
>> index a64b768e24..38c4a688c6 100644
>> --- a/block/backup.c
>> +++ b/block/backup.c
>> @@ -417,6 +417,7 @@ static int coroutine_fn backup_loop(BackupBlockJob *job)
>> if (job->sync_mode == MIRROR_SYNC_MODE_TOP &&
>> bdrv_is_unallocated_range(bs, offset, job->cluster_size))
>> {
>> + job_progress_update(&job->common.job, job->cluster_size);
>> bdrv_reset_dirty_bitmap(job->copy_bitmap, offset,
>> job->cluster_size);
>> continue;
>>
>
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups
2019-07-10 16:48 ` Max Reitz
@ 2019-07-10 18:32 ` John Snow
2019-07-10 20:39 ` Max Reitz
0 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 18:32 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 12:48 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Accept bitmaps and sync policies for the other backup modes.
>> This allows us to do things like create a bitmap synced to a full backup
>> without a transaction, or start a resumable backup process.
>>
>> Some combinations don't make sense, though:
>>
>> - NEVER policy combined with any non-BITMAP mode doesn't do anything,
>> because the bitmap isn't used for input or output.
>> It's harmless, but is almost certainly never what the user wanted.
>>
>> - sync=NONE is more questionable. It can't use on-success because this
>> job never completes with success anyway, and the resulting artifact
>> of 'always' is suspect: because we start with a full bitmap and only
>> copy out segments that get written to, the final output bitmap will
>> always be ... a fully set bitmap.
>>
>> Maybe there's contexts in which bitmaps make sense for sync=none,
>> but not without more severe changes to the current job, and omitting
>> it here doesn't prevent us from adding it later.
>>
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> block/backup.c | 8 +-------
>> blockdev.c | 22 ++++++++++++++++++++++
>> qapi/block-core.json | 6 ++++--
>> 3 files changed, 27 insertions(+), 9 deletions(-)
>
> [...]
>
>> diff --git a/blockdev.c b/blockdev.c
>> index f0b7da53b0..bc152f8e0d 100644
>> --- a/blockdev.c
>> +++ b/blockdev.c
>
> [...]
>
>> + if (!backup->has_bitmap && backup->has_bitmap_mode) {
>> + error_setg(errp, "Cannot specify Bitmap sync mode without a bitmap");
>
> Any reason for capitalizing the first “Bitmap”?
>
> With a reason or it fixed:
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
Hanging around Germans too much?
Actually, I can explain why: because a "bitmap" is a generic term, but
whenever I capitalize it as "Bitmap" I am referring to a Block Dirty
Bitmap which is a specific sort of thing. I do this unconsciously.
But in that case, I should actually be consistent in the interface (and
docstrings and docs and error strings) and always call it that specific
thing, which I don't.
"bitmap" is probably more correct for now, but I ought to go through all
the interface and make it consistent in some way or another.
(Actually: I'd like to see if I can't rename the "BdrvDirtyBitmap"
object to something more generic and shorter so I can rename many of the
functions we have something shorter.
Because the structure is "BdrvDirtyBitmap", there's some confusion when
we name functions like bdrv_dirty_bitmap_{verb} because it's not clear
if this is a bdrv function that does something with dirty bitmaps, or if
it's a "BdrvDirtyBitmap" function that does something with that object.
I guess that seems like a subtle point, but it's why the naming
conventions in dirty-bitmap.c are all over the place. I think at one
point, the idea was that:
bdrv_{verb}_dirty_bitmap was an action applied to a BDS that happened to
do something with dirty bitmaps. bdrv_dirty_bitmap_{verb} was an action
that applied to a "BdrvDirtyBitmap". Crystal clear and not confusing at
all, right?
It'd be nice to have functions that operate on a node be named
bdrv_dbitmap_foo() and functions that operate on the bitmap structure
itself named just dbitmap_bar().
Would it be okay if I named them such a thing, I wonder?
we have "bitmap" and "hbitmap" already, I could do something like
"dbitmap" for "dirty bitmap" or some such. Kind of an arbitrary change I
admit, but I'm itching to do a big spring cleaning in dirty-bitmap.c
right after this series is done.)
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes
2019-07-10 17:14 ` Max Reitz
@ 2019-07-10 19:00 ` John Snow
2019-07-10 20:46 ` Max Reitz
0 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 19:00 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 1:14 PM, Max Reitz wrote:
> On 10.07.19 03:05, John Snow wrote:
>> Signed-off-by: John Snow <jsnow@redhat.com>
>> ---
>> tests/qemu-iotests/257 | 31 +
>> tests/qemu-iotests/257.out | 3089 ++++++++++++++++++++++++++++++++++++
>> 2 files changed, 3120 insertions(+)
>
> Oof.
>
Yeah, it's... a lot of test output. We probably shouldn't count the
reference test output against any kind of SLOC metrics.
>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>> index de8707cb19..8de1c4da19 100755
>> --- a/tests/qemu-iotests/257
>> +++ b/tests/qemu-iotests/257
>
> [...]
>
>> @@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
>> if bsync_mode == 'always' and failure == 'intermediate':
>> # We manage to copy one sector (one bit) before the error.
>> ebitmap.clear_bit(ebitmap.first_bit)
>> + if msync_mode in ('full', 'top'):
>> + # These modes return all bits set except what was copied/skipped
>
> Hm. How useful is bitmap support for 'top' then, anyway? That means
> that if you want to resume a top backup, you always have to resume it
> like it was a full backup. Which sounds kind of useless.
>
> Max
>
Good point!
I think this can be fixed by doing an initialization pass of the
copy_bitmap when sync=top to set only the allocated regions in the bitmap.
This means that the write notifier won't copy out regions that are
written to that weren't already in the top layer. I believe this is
actually a bugfix; the data we'd copy out in such cases is actually in
the backing layer and shouldn't be copied with sync=top.
So this would have two effects:
(1) sync=top gets a little more judicious about what it copies out on
sync=top, and
(2) the bitmap return value is more meaningful again.
This doesn't touch sync=none at all, which needs more invasive fixes if
we wanted it to have useful bitmap return values (it needs to
differentiate the idea between must-copy and can-copy, and I still don't
know if this is worthwhile to do, so until I hear otherwise, I'm not gonna.)
>> + fail_bit = ebitmap.first_bit
>> + ebitmap.clear()
>> + ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
>> ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
>>
>> # 2 - Writes and Reference Backup
> [...]
>
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers
2019-07-10 17:52 ` John Snow
@ 2019-07-10 20:17 ` Max Reitz
0 siblings, 0 replies; 35+ messages in thread
From: Max Reitz @ 2019-07-10 20:17 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 4845 bytes --]
On 10.07.19 19:52, John Snow wrote:
>
>
> On 7/10/19 12:04 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> This test needs support for non-bitmap backups and missing or
>>> unspecified bitmap sync modes, so rewrite the helpers to be a little
>>> more generic.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> tests/qemu-iotests/257 | 46 +++++----
>>> tests/qemu-iotests/257.out | 192 ++++++++++++++++++-------------------
>>> 2 files changed, 124 insertions(+), 114 deletions(-)
>>>
>>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>>> index 2ff4aa8695..2eb4f26c28 100755
>>> --- a/tests/qemu-iotests/257
>>> +++ b/tests/qemu-iotests/257
>>
>> [...]
>>
>>> -def bitmap_backup(drive, n, filepath, bitmap, bitmap_mode):
>>> - log("--- Bitmap Backup #{:d} ---\n".format(n))
>>> - target_id = "bitmap_target_{:d}".format(n)
>>> - job_id = "bitmap_backup_{:d}".format(n)
>>> +def backup(drive, n, filepath, bitmap, bitmap_mode, sync='bitmap'):
>>> + log("--- Test Backup #{:d} ---\n".format(n))
>>> + target_id = "backup_target_{:d}".format(n)
>>> + job_id = "backup_{:d}".format(n)
>>> target_drive = Drive(filepath, vm=drive.vm)
>>>
>>> target_drive.create_target(target_id, drive.fmt, drive.size)
>>> - drive.vm.qmp_log("blockdev-backup", job_id=job_id, device=drive.name,
>>> - target=target_id, sync="bitmap",
>>> - bitmap_mode=bitmap_mode,
>>> - bitmap=bitmap,
>>> - auto_finalize=False)
>>> +
>>> + kwargs = {
>>> + 'job_id': job_id,
>>> + 'auto_finalize': False,
>>> + 'bitmap': bitmap,
>>> + 'bitmap_mode': bitmap_mode,
>>> + }
>>> + kwargs = {key: val for key, val in kwargs.items() if val is not None}
>>
>> I suppose this is to remove items that are None?
>>
>> Very cute, but why not just
>>
>> kwargs = {
>> 'job_id': job_id,
>> 'auto_finalize': False,
>> }
>> if bitmap is not None:
>> kwargs['bitmap'] = bitmap
>> kwargs['bitmap_mode'] = bitmap_mode
>>
>> Exactly the same number of lines, but immediately makes it clear what’s
>> going on. Not as cute, I admit.
>>
>> (Yes, I am indeed actively trying to train you not to write cute code.)
>>
>
> It sneaks in. I genuinely struggle with understanding what other people
> will find readable; I have an authentically hard time reviewing other
> people's patches too. I'm earnestly not sure how I can help improve
> this, but I would like to.
>
> I wasn't sure what the easiest way to avoid sending the "None" over the
> wire was, so I went with a general thing, but yes: it's because bitmap
> and bitmap_mode are set to None sometimes and I need to omit such keys.
>
> In this case, though, I do test bitmap and bitmap_mode separately, so
> for the purposes of testing intentionally bad combinations you do need:
>
> if bitmap is not None:
> kwargs['bitmap'] = bitmap
> if bitmap_mode is not None:
> kwargs['bitmap_mode'] = bitmap_mode
>
> And I just looked at this and it did not spark joy; so I went with a
> generic filter to remove nulled keys. I admit it's /slightly/ cute and
> not immediately obvious why it needs to be done.
>
>
> This is even cuter, so maybe I am traveling in the wrong direction:
>
> def backup(drive, n, filepath, sync, **kwargs):
> log("--- Test Backup #{:d} ---\n".format(n))
> target_id = "backup_target_{:d}".format(n)
> job_id = "backup_{:d}".format(n)
> target_drive = Drive(filepath, vm=drive.vm)
>
> target_drive.create_target(target_id, drive.fmt, drive.size)
> kwargs.setdefault('auto_finalize', False)
> # Strip any arguments explicitly nulled by the caller:
> kwargs = {key: val for key, val in kwargs.items()
> if val is not None}
> blockdev_backup(drive.vm, drive.name, target_id, sync, **kwargs)
> return job_id
>
> It's quite a bit shorter and also makes backup() more flexible by
> omitting the bitmap and bitmap_mode arguments entirely, allowing the
> caller to override the auto_finalize default, etc. In this permutation,
> we don't know the full extent of kwargs so it makes sense to generically
> filter it.
>
> Manually conditionally setting arguments is probably also fine.
> Do you still have a preference for the more static approach?
It’s OK with the comment.
Although it needs a kwargs.setdefault('job_id', job_id), too.
(Hm. Shouldn’t 'auto-finalize' and 'job-id' work just fine? It’s not
like the QMP scripts swap - and _. They just replace _ by -.)
Max
>> The rest looks good to me:
>>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>
>
> Thanks for reviewing, as always!
>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface
2019-07-10 17:57 ` John Snow
@ 2019-07-10 20:19 ` Max Reitz
0 siblings, 0 replies; 35+ messages in thread
From: Max Reitz @ 2019-07-10 20:19 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 1643 bytes --]
On 10.07.19 19:57, John Snow wrote:
>
>
> On 7/10/19 12:11 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> This is nicer to do in the unified QMP interface that we have now,
>>> because it lets us use the right terminology back at the user.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> block/backup.c | 13 ++++---------
>>> blockdev.c | 10 ++++++++++
>>> 2 files changed, 14 insertions(+), 9 deletions(-)
>>>
>>> diff --git a/block/backup.c b/block/backup.c
>>> index e2729cf6fa..a64b768e24 100644
>>> --- a/block/backup.c
>>> +++ b/block/backup.c
>>> @@ -566,6 +566,10 @@ BlockJob *backup_job_create(const char *job_id, BlockDriverState *bs,
>>> assert(bs);
>>> assert(target);
>>>
>>> + /* QMP interface protects us from these cases */
>>> + assert(sync_mode != MIRROR_SYNC_MODE_INCREMENTAL);
>>> + assert(sync_bitmap || sync_mode != MIRROR_SYNC_MODE_BITMAP);
>>
>> Implication would be a nice operator sometimes.
>>
>> ("assert(sync_mode == MIRROR_SYNC_MODE_BITMAP -> sync_bitmap)")
>>
>> (Can you do that in C++? No, you can’t overload bool’s operators, right?)
>>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>
>
> Yes, I also find this assertion kind of hard to read personally, but it
> feels somewhat clunky to write:
>
> if (antecedent) {
> assert(condition);
> }
>
> I suppose we can also phrase this as:
>
> assert(sync_mode == MIRROR_SYNC_MODE_BITMAP ? sync_bitmap : true);
>
> Which might honestly be pretty good. Mind if I change it to this?
Looks weird (mostly unfamiliar), but I do not.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions
2019-07-10 18:20 ` John Snow
@ 2019-07-10 20:30 ` Max Reitz
2019-07-10 20:47 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 20:30 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 2225 bytes --]
On 10.07.19 20:20, John Snow wrote:
>
>
> On 7/10/19 12:36 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> The way bitmap backups work is by starting at 75% if it needs
>>> to copy just 25% of the disk.
>>
>> Although there is this comment:
>>
>>> /* TODO job_progress_set_remaining() would make more sense */
>>
>> So...
>>
>>> The way sync=top currently works, however, is to start at 0% and then
>>> never update the progress if it doesn't copy a region. If it needs to
>>> copy 25% of the disk, we'll finish at 25%.
>>>
>>> Update the progress when we skip regions.
>>
>> Wouldn’t it be more correct to decrease the job length?
>>
>> Max
>>
>
> Admittedly I have precisely no idea. Maybe? As far as I understand it,
> we guarantee only:
>
> (1) Progress monotonically increases
> (2) Upon completion, progress will equal the total work estimate.
> [Trying to fix that to be true here.]
>
> This means we can do stuff like:
>
> - Total work estimate can increase or decrease arbitrarily
> - Neither value has to mean anything in particular
>
>
> Bitmap sync works by artificially increasing progress for NOP regions,
> seen in init_copy_bitmap.
Yes, and it has a TODO comment that says it should be done differently.
> Full sync also tends to increase progress regardless of it actually did
> a copy or not; offload support also counts as progress here. So if you
> full sync an empty image, you'll see it increasing the progress as it
> doesn't actually do anything.
>
> Top sync is the odd one out, which just omits progress for regions it skips.
>
> My only motivation here was to make them consistent. Can I do it the
> other way? Yeah, probably. Is one way better than the other? I
> legitimately have no idea. I guess whoever wrote the last comment felt
> that it should all be the other way instead. Why'd they not do that?
If you look at the commit (05df8a6a2b4), I suppose it was because that
commit simply did not intend to change behavior. It just touched that
piece of code and noted that maybe there should be a follow-up commit to
change it.
But yeah, whatever.
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups
2019-07-10 18:32 ` John Snow
@ 2019-07-10 20:39 ` Max Reitz
0 siblings, 0 replies; 35+ messages in thread
From: Max Reitz @ 2019-07-10 20:39 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 4405 bytes --]
On 10.07.19 20:32, John Snow wrote:
>
>
> On 7/10/19 12:48 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> Accept bitmaps and sync policies for the other backup modes.
>>> This allows us to do things like create a bitmap synced to a full backup
>>> without a transaction, or start a resumable backup process.
>>>
>>> Some combinations don't make sense, though:
>>>
>>> - NEVER policy combined with any non-BITMAP mode doesn't do anything,
>>> because the bitmap isn't used for input or output.
>>> It's harmless, but is almost certainly never what the user wanted.
>>>
>>> - sync=NONE is more questionable. It can't use on-success because this
>>> job never completes with success anyway, and the resulting artifact
>>> of 'always' is suspect: because we start with a full bitmap and only
>>> copy out segments that get written to, the final output bitmap will
>>> always be ... a fully set bitmap.
>>>
>>> Maybe there's contexts in which bitmaps make sense for sync=none,
>>> but not without more severe changes to the current job, and omitting
>>> it here doesn't prevent us from adding it later.
>>>
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> block/backup.c | 8 +-------
>>> blockdev.c | 22 ++++++++++++++++++++++
>>> qapi/block-core.json | 6 ++++--
>>> 3 files changed, 27 insertions(+), 9 deletions(-)
>>
>> [...]
>>
>>> diff --git a/blockdev.c b/blockdev.c
>>> index f0b7da53b0..bc152f8e0d 100644
>>> --- a/blockdev.c
>>> +++ b/blockdev.c
>>
>> [...]
>>
>>> + if (!backup->has_bitmap && backup->has_bitmap_mode) {
>>> + error_setg(errp, "Cannot specify Bitmap sync mode without a bitmap");
>>
>> Any reason for capitalizing the first “Bitmap”?
>>
>> With a reason or it fixed:
>>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>
>
> Hanging around Germans too much?
You should know then that the korrekt way to write it would be:
„Specifying Binarydigitmapsynchronizationmode without a Binarydigitmap
is absolutely verboten!“
> Actually, I can explain why: because a "bitmap" is a generic term, but
> whenever I capitalize it as "Bitmap" I am referring to a Block Dirty
> Bitmap which is a specific sort of thing. I do this unconsciously.
>
> But in that case, I should actually be consistent in the interface (and
> docstrings and docs and error strings) and always call it that specific
> thing, which I don't.
>
> "bitmap" is probably more correct for now, but I ought to go through all
> the interface and make it consistent in some way or another.
>
>
> (Actually: I'd like to see if I can't rename the "BdrvDirtyBitmap"
> object to something more generic and shorter so I can rename many of the
> functions we have something shorter.
Well, BDB is free. That path has worked fine for BlockDriverStates.
Or what I said on IRC, but you know.
> Because the structure is "BdrvDirtyBitmap", there's some confusion when
> we name functions like bdrv_dirty_bitmap_{verb} because it's not clear
> if this is a bdrv function that does something with dirty bitmaps, or if
> it's a "BdrvDirtyBitmap" function that does something with that object.
>
> I guess that seems like a subtle point, but it's why the naming
> conventions in dirty-bitmap.c are all over the place. I think at one
> point, the idea was that:
>
> bdrv_{verb}_dirty_bitmap was an action applied to a BDS that happened to
> do something with dirty bitmaps. bdrv_dirty_bitmap_{verb} was an action
> that applied to a "BdrvDirtyBitmap". Crystal clear and not confusing at
> all, right?
I just thought people named their functions whatever they felt like at
the time.
> It'd be nice to have functions that operate on a node be named
> bdrv_dbitmap_foo() and functions that operate on the bitmap structure
> itself named just dbitmap_bar().
>
> Would it be okay if I named them such a thing, I wonder?
>
> we have "bitmap" and "hbitmap" already, I could do something like
> "dbitmap" for "dirty bitmap" or some such. Kind of an arbitrary change I
> admit, but I'm itching to do a big spring cleaning in dirty-bitmap.c
> right after this series is done.)
HBitmaps are generally used to track dirty areas, so I’d find this a
misnomer. BDBitmap would be OK. The “block” part should be in there
somewhere.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes
2019-07-10 19:00 ` John Snow
@ 2019-07-10 20:46 ` Max Reitz
[not found] ` <2f221513-f173-8d9f-a3b2-d790ef6f6f51@redhat.com>
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-10 20:46 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 2731 bytes --]
On 10.07.19 21:00, John Snow wrote:
>
>
> On 7/10/19 1:14 PM, Max Reitz wrote:
>> On 10.07.19 03:05, John Snow wrote:
>>> Signed-off-by: John Snow <jsnow@redhat.com>
>>> ---
>>> tests/qemu-iotests/257 | 31 +
>>> tests/qemu-iotests/257.out | 3089 ++++++++++++++++++++++++++++++++++++
>>> 2 files changed, 3120 insertions(+)
>>
>> Oof.
>>
>
> Yeah, it's... a lot of test output. We probably shouldn't count the
> reference test output against any kind of SLOC metrics.
>
>>> diff --git a/tests/qemu-iotests/257 b/tests/qemu-iotests/257
>>> index de8707cb19..8de1c4da19 100755
>>> --- a/tests/qemu-iotests/257
>>> +++ b/tests/qemu-iotests/257
>>
>> [...]
>>
>>> @@ -410,6 +416,11 @@ def test_bitmap_sync(bsync_mode, msync_mode='bitmap', failure=None):
>>> if bsync_mode == 'always' and failure == 'intermediate':
>>> # We manage to copy one sector (one bit) before the error.
>>> ebitmap.clear_bit(ebitmap.first_bit)
>>> + if msync_mode in ('full', 'top'):
>>> + # These modes return all bits set except what was copied/skipped
>>
>> Hm. How useful is bitmap support for 'top' then, anyway? That means
>> that if you want to resume a top backup, you always have to resume it
>> like it was a full backup. Which sounds kind of useless.
>>
>> Max
>>
>
> Good point!
>
> I think this can be fixed by doing an initialization pass of the
> copy_bitmap when sync=top to set only the allocated regions in the bitmap.
>
> This means that the write notifier won't copy out regions that are
> written to that weren't already in the top layer. I believe this is
> actually a bugfix; the data we'd copy out in such cases is actually in
> the backing layer and shouldn't be copied with sync=top.
Now that you mention it... I didn’t realize that. Yes, you’re right.
> So this would have two effects:
> (1) sync=top gets a little more judicious about what it copies out on
> sync=top, and
> (2) the bitmap return value is more meaningful again.
>
> This doesn't touch sync=none at all, which needs more invasive fixes if
> we wanted it to have useful bitmap return values (it needs to
> differentiate the idea between must-copy and can-copy, and I still don't
> know if this is worthwhile to do, so until I hear otherwise, I'm not gonna.)
No, I’m with you on that one.
Max
>>> + fail_bit = ebitmap.first_bit
>>> + ebitmap.clear()
>>> + ebitmap.dirty_bits(range(fail_bit, SIZE // GRANULARITY))
>>> ebitmap.compare(get_bitmap(bitmaps, drive0.device, 'bitmap0'))
>>>
>>> # 2 - Writes and Reference Backup
>> [...]
>>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions
2019-07-10 20:30 ` Max Reitz
@ 2019-07-10 20:47 ` John Snow
2019-07-10 20:53 ` Max Reitz
0 siblings, 1 reply; 35+ messages in thread
From: John Snow @ 2019-07-10 20:47 UTC (permalink / raw)
To: Max Reitz, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
On 7/10/19 4:30 PM, Max Reitz wrote:
> On 10.07.19 20:20, John Snow wrote:
>>
>>
>> On 7/10/19 12:36 PM, Max Reitz wrote:
>>> On 10.07.19 03:05, John Snow wrote:
>>>> The way bitmap backups work is by starting at 75% if it needs
>>>> to copy just 25% of the disk.
>>>
>>> Although there is this comment:
>>>
>>>> /* TODO job_progress_set_remaining() would make more sense */
>>>
>>> So...
>>>
>>>> The way sync=top currently works, however, is to start at 0% and then
>>>> never update the progress if it doesn't copy a region. If it needs to
>>>> copy 25% of the disk, we'll finish at 25%.
>>>>
>>>> Update the progress when we skip regions.
>>>
>>> Wouldn’t it be more correct to decrease the job length?
>>>
>>> Max
>>>
>>
>> Admittedly I have precisely no idea. Maybe? As far as I understand it,
>> we guarantee only:
>>
>> (1) Progress monotonically increases
>> (2) Upon completion, progress will equal the total work estimate.
>> [Trying to fix that to be true here.]
>>
>> This means we can do stuff like:
>>
>> - Total work estimate can increase or decrease arbitrarily
>> - Neither value has to mean anything in particular
>>
>>
>> Bitmap sync works by artificially increasing progress for NOP regions,
>> seen in init_copy_bitmap.
>
> Yes, and it has a TODO comment that says it should be done differently.
>
>> Full sync also tends to increase progress regardless of it actually did
>> a copy or not; offload support also counts as progress here. So if you
>> full sync an empty image, you'll see it increasing the progress as it
>> doesn't actually do anything.
>>
>> Top sync is the odd one out, which just omits progress for regions it skips.
>>
>> My only motivation here was to make them consistent. Can I do it the
>> other way? Yeah, probably. Is one way better than the other? I
>> legitimately have no idea. I guess whoever wrote the last comment felt
>> that it should all be the other way instead. Why'd they not do that?
>
> If you look at the commit (05df8a6a2b4), I suppose it was because that
> commit simply did not intend to change behavior. It just touched that
> piece of code and noted that maybe there should be a follow-up commit to
> change it.
>
> But yeah, whatever.
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
>
I mean. I'll make it consistent either way, but I actually don't know
which way I should make it go. I just think that all the modes should
work the same if we can help it.
Flip a coin?
--js
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions
2019-07-10 20:47 ` John Snow
@ 2019-07-10 20:53 ` Max Reitz
0 siblings, 0 replies; 35+ messages in thread
From: Max Reitz @ 2019-07-10 20:53 UTC (permalink / raw)
To: John Snow, qemu-block, qemu-devel; +Cc: Kevin Wolf, Markus Armbruster
[-- Attachment #1.1: Type: text/plain, Size: 3209 bytes --]
On 10.07.19 22:47, John Snow wrote:
>
>
> On 7/10/19 4:30 PM, Max Reitz wrote:
>> On 10.07.19 20:20, John Snow wrote:
>>>
>>>
>>> On 7/10/19 12:36 PM, Max Reitz wrote:
>>>> On 10.07.19 03:05, John Snow wrote:
>>>>> The way bitmap backups work is by starting at 75% if it needs
>>>>> to copy just 25% of the disk.
>>>>
>>>> Although there is this comment:
>>>>
>>>>> /* TODO job_progress_set_remaining() would make more sense */
>>>>
>>>> So...
>>>>
>>>>> The way sync=top currently works, however, is to start at 0% and then
>>>>> never update the progress if it doesn't copy a region. If it needs to
>>>>> copy 25% of the disk, we'll finish at 25%.
>>>>>
>>>>> Update the progress when we skip regions.
>>>>
>>>> Wouldn’t it be more correct to decrease the job length?
>>>>
>>>> Max
>>>>
>>>
>>> Admittedly I have precisely no idea. Maybe? As far as I understand it,
>>> we guarantee only:
>>>
>>> (1) Progress monotonically increases
>>> (2) Upon completion, progress will equal the total work estimate.
>>> [Trying to fix that to be true here.]
>>>
>>> This means we can do stuff like:
>>>
>>> - Total work estimate can increase or decrease arbitrarily
>>> - Neither value has to mean anything in particular
>>>
>>>
>>> Bitmap sync works by artificially increasing progress for NOP regions,
>>> seen in init_copy_bitmap.
>>
>> Yes, and it has a TODO comment that says it should be done differently.
>>
>>> Full sync also tends to increase progress regardless of it actually did
>>> a copy or not; offload support also counts as progress here. So if you
>>> full sync an empty image, you'll see it increasing the progress as it
>>> doesn't actually do anything.
>>>
>>> Top sync is the odd one out, which just omits progress for regions it skips.
>>>
>>> My only motivation here was to make them consistent. Can I do it the
>>> other way? Yeah, probably. Is one way better than the other? I
>>> legitimately have no idea. I guess whoever wrote the last comment felt
>>> that it should all be the other way instead. Why'd they not do that?
>>
>> If you look at the commit (05df8a6a2b4), I suppose it was because that
>> commit simply did not intend to change behavior. It just touched that
>> piece of code and noted that maybe there should be a follow-up commit to
>> change it.
>>
>> But yeah, whatever.
>>
>> Reviewed-by: Max Reitz <mreitz@redhat.com>
>>
>
> I mean. I'll make it consistent either way, but I actually don't know
> which way I should make it go. I just think that all the modes should
> work the same if we can help it.
>
> Flip a coin?
If you’d flip a coin, I can say that I’d find it a bit more meaningful
to reduce the length. Especially if you change sync=top to calculate
beforehand how much we need to copy, so the length isn’t even reduced
while the job is running.
And it’s easier to remove the TODO comment this way. If we decide
against it, you’d have to remove the TODO comment aso (because we
decided against it), but you’d need to justify it in the commit message.
And we all know that writing explanations and documentation is the
hardest thing of all.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes
[not found] ` <2f221513-f173-8d9f-a3b2-d790ef6f6f51@redhat.com>
@ 2019-07-11 12:37 ` Max Reitz
2019-07-11 17:58 ` John Snow
0 siblings, 1 reply; 35+ messages in thread
From: Max Reitz @ 2019-07-11 12:37 UTC (permalink / raw)
To: John Snow, Qemu-block, qemu-devel
[-- Attachment #1.1: Type: text/plain, Size: 3713 bytes --]
On 11.07.19 05:21, John Snow wrote:
>
> On 7/10/19 4:46 PM, Max Reitz wrote:
>> On 10.07.19 21:00, John Snow wrote:
>>> On 7/10/19 1:14 PM, Max Reitz wrote:
>>>> On 10.07.19 03:05, John Snow wrote:
>>>>
>>>> Hm. How useful is bitmap support for 'top' then, anyway? That means
>>>> that if you want to resume a top backup, you always have to resume it
>>>> like it was a full backup. Which sounds kind of useless.
>>>>
>>>> Max
>>>>
>>>
>>> Good point!
>>>
>>> I think this can be fixed by doing an initialization pass of the
>>> copy_bitmap when sync=top to set only the allocated regions in the bitmap.
>>>
>>> This means that the write notifier won't copy out regions that are
>>> written to that weren't already in the top layer. I believe this is
>>> actually a bugfix; the data we'd copy out in such cases is actually in
>>> the backing layer and shouldn't be copied with sync=top.
>>
>> Now that you mention it... I didn’t realize that. Yes, you’re right.
>>
>>> So this would have two effects:
>>> (1) sync=top gets a little more judicious about what it copies out on
>>> sync=top, and
>>> (2) the bitmap return value is more meaningful again.
>>>
>
> This might be harder than I first thought.
>
> initializing the copy_bitmap generally happens before we install the
> write notifier, which means that it occurs before the first yield.
>
> However, checking the allocation status can potentially be very slow,
> can't it? I can't just hog the thread while I check.
I was thinking about that myself. It isn’t that bad, because you aren’t
doing the full block_status dance but just checking allocation status,
which is reasonably quick (it just needs to look at the image format
metadata, it doesn’t go down to the protocol layer).
But it’s probably not so good to halt the monitor for this, yes.
> There are ways to cooperatively process write notifier interrupts and
> continue to check allocated status once we enter the main loop, but the
> problem there becomes: if we fail early, how can we tell if the backup
> is worth resuming?
>
> We might not have reached a convergence point for the copy_bitmap before
> we failed, and still have a lot of extra bits set.
Is that so bad?
> I suppose at least in the case where we aren't trying to save the
> copy_bitmap and need it to mean something specific, this is a reasonable
> approach to fixing sync=TOP.
>
> As far as resume is concerned, I don't think I have good ideas. I could
> emit an event or something if you're using sync=top with a bitmap for
> output, but that feels *so* specialized for a niche(?) command that I
> don't know if it's worth pursuing.
>
> (Plus, even then, what do you do if it fails before you see that event?
> You just have to give up on what we copied out? That seems like a waste
> and not the point of this exercise.)
Before that event, the bitmap can still be usable, as long as all
“unknown” areas are set to dirty. Sure, your resumed backup will then
copy too much data. But who cares.
So I don’t think you even need an event.
> The only way I can think of at all to get a retry on sync=top is to take
> an always policy, and to allow a special invocation with something like
> mode=bitmap+top:
Yes, that was my first idea, too. But I didn’t even write about it,
because of...
> "Assume we need to copy anything set in the bitmap, unless it's not in
> the top layer, and then skip it."
>
> Which seems awful, because it would be a specialty mode for the
> exclusive purpose of re-trying sync=top backups.
...exactly this.
> Meh.
I don’t think it’s all that bad.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 35+ messages in thread
* Re: [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes
2019-07-11 12:37 ` Max Reitz
@ 2019-07-11 17:58 ` John Snow
0 siblings, 0 replies; 35+ messages in thread
From: John Snow @ 2019-07-11 17:58 UTC (permalink / raw)
To: Max Reitz, Qemu-block, qemu-devel
On 7/11/19 8:37 AM, Max Reitz wrote:
> On 11.07.19 05:21, John Snow wrote:
>>
>> On 7/10/19 4:46 PM, Max Reitz wrote:
>>> On 10.07.19 21:00, John Snow wrote:
>>>> On 7/10/19 1:14 PM, Max Reitz wrote:
>>>>> On 10.07.19 03:05, John Snow wrote:
>>>>>
>>>>> Hm. How useful is bitmap support for 'top' then, anyway? That means
>>>>> that if you want to resume a top backup, you always have to resume it
>>>>> like it was a full backup. Which sounds kind of useless.
>>>>>
>>>>> Max
>>>>>
>>>>
>>>> Good point!
>>>>
>>>> I think this can be fixed by doing an initialization pass of the
>>>> copy_bitmap when sync=top to set only the allocated regions in the bitmap.
>>>>
>>>> This means that the write notifier won't copy out regions that are
>>>> written to that weren't already in the top layer. I believe this is
>>>> actually a bugfix; the data we'd copy out in such cases is actually in
>>>> the backing layer and shouldn't be copied with sync=top.
>>>
>>> Now that you mention it... I didn’t realize that. Yes, you’re right.
>>>
>>>> So this would have two effects:
>>>> (1) sync=top gets a little more judicious about what it copies out on
>>>> sync=top, and
>>>> (2) the bitmap return value is more meaningful again.
>>>>
>>
>> This might be harder than I first thought.
>>
>> initializing the copy_bitmap generally happens before we install the
>> write notifier, which means that it occurs before the first yield.
>>
>> However, checking the allocation status can potentially be very slow,
>> can't it? I can't just hog the thread while I check.
>
> I was thinking about that myself. It isn’t that bad, because you aren’t
> doing the full block_status dance but just checking allocation status,
> which is reasonably quick (it just needs to look at the image format
> metadata, it doesn’t go down to the protocol layer).
>
> But it’s probably not so good to halt the monitor for this, yes.
>
>> There are ways to cooperatively process write notifier interrupts and
>> continue to check allocated status once we enter the main loop, but the
>> problem there becomes: if we fail early, how can we tell if the backup
>> is worth resuming?
>>
>> We might not have reached a convergence point for the copy_bitmap before
>> we failed, and still have a lot of extra bits set.
>
> Is that so bad?
>
>> I suppose at least in the case where we aren't trying to save the
>> copy_bitmap and need it to mean something specific, this is a reasonable
>> approach to fixing sync=TOP.
>>
>> As far as resume is concerned, I don't think I have good ideas. I could
>> emit an event or something if you're using sync=top with a bitmap for
>> output, but that feels *so* specialized for a niche(?) command that I
>> don't know if it's worth pursuing.
>>
>> (Plus, even then, what do you do if it fails before you see that event?
>> You just have to give up on what we copied out? That seems like a waste
>> and not the point of this exercise.)
>
> Before that event, the bitmap can still be usable, as long as all
> “unknown” areas are set to dirty. Sure, your resumed backup will then
> copy too much data. But who cares.
>
> So I don’t think you even need an event.
>
>> The only way I can think of at all to get a retry on sync=top is to take
>> an always policy, and to allow a special invocation with something like
>> mode=bitmap+top:
>
> Yes, that was my first idea, too. But I didn’t even write about it,
> because of...
>
>> "Assume we need to copy anything set in the bitmap, unless it's not in
>> the top layer, and then skip it."
>>
>> Which seems awful, because it would be a specialty mode for the
>> exclusive purpose of re-trying sync=top backups.
>
> ...exactly this.
>
>> Meh.
>
> I don’t think it’s all that bad.
>
No, it's just not ideal and it's something I'd have to defend in a
patch. It's a caveat that would need documenting.
"Hey, depending on how far the job got before it failed, you might not
want to resume it because it may not have finished determining which
segments held allocated data. There's no way to tell if this happened or
not."
It makes the decision making process by e.g. libvirt harder, though
there are still some heuristics you could use, like:
- Is the bitmap count less than the size of the top image?
- Is it bigger?
And that might be good enough when deciding how to proceed. I suppose if
we want to give more precise mechanisms for this we'd always be within
our right to continue refining it and just document that it MIGHT have
extra bits set.
--js
^ permalink raw reply [flat|nested] 35+ messages in thread
end of thread, other threads:[~2019-07-11 17:59 UTC | newest]
Thread overview: 35+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-10 1:05 [Qemu-devel] [PATCH 0/8] bitmaps: allow bitmaps to be used with full and top John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 1/8] iotests/257: add Pattern class John Snow
2019-07-10 15:10 ` Max Reitz
2019-07-10 16:26 ` Max Reitz
2019-07-10 17:34 ` John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 2/8] iotests/257: add EmulatedBitmap class John Snow
2019-07-10 15:47 ` Max Reitz
2019-07-10 17:36 ` John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 3/8] iotests/257: Refactor backup helpers John Snow
2019-07-10 16:04 ` Max Reitz
2019-07-10 17:52 ` John Snow
2019-07-10 20:17 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 4/8] block/backup: hoist bitmap check into QMP interface John Snow
2019-07-10 16:11 ` Max Reitz
2019-07-10 17:57 ` John Snow
2019-07-10 20:19 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 5/8] iotests/257: test API failures John Snow
2019-07-10 16:22 ` Max Reitz
2019-07-10 18:00 ` John Snow
2019-07-10 1:05 ` [Qemu-devel] [PATCH 6/8] block/backup: issue progress updates for skipped regions John Snow
2019-07-10 16:36 ` Max Reitz
2019-07-10 18:20 ` John Snow
2019-07-10 20:30 ` Max Reitz
2019-07-10 20:47 ` John Snow
2019-07-10 20:53 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 7/8] block/backup: support bitmap sync modes for non-bitmap backups John Snow
2019-07-10 16:48 ` Max Reitz
2019-07-10 18:32 ` John Snow
2019-07-10 20:39 ` Max Reitz
2019-07-10 1:05 ` [Qemu-devel] [PATCH 8/8] iotests/257: test traditional sync modes John Snow
2019-07-10 17:14 ` Max Reitz
2019-07-10 19:00 ` John Snow
2019-07-10 20:46 ` Max Reitz
[not found] ` <2f221513-f173-8d9f-a3b2-d790ef6f6f51@redhat.com>
2019-07-11 12:37 ` Max Reitz
2019-07-11 17:58 ` John Snow
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.