* [Qemu-devel] [PULL 00/23] Block layer patches
@ 2016-10-24 17:01 Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 01/23] block: failed qemu-img command should return non-zero exit code Kevin Wolf
` (23 more replies)
0 siblings, 24 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit a3ae21ec3fe036f536dc94cad735931777143103:
Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2016-10-24 15:03:09 +0100)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 25493dc012e7c10dba51ee893b634a1dbfeed126:
Merge remote-tracking branch 'mreitz/tags/pull-block-2016-10-24' into queue-block (2016-10-24 18:02:26 +0200)
----------------------------------------------------------------
Block layer patches
----------------------------------------------------------------
Alberto Garcia (2):
throttle: Correct access to wrong BlockBackendPublic structures
qemu-iotests: Test I/O in a single drive from a throttling group
Changlong Xie (1):
block/replication: Clarify 'top-id' parameter usage
Fam Zheng (9):
qcow2: Support BDRV_REQ_MAY_UNMAP
block: Hide HBitmap in block dirty bitmap interface
HBitmap: Introduce "meta" bitmap to track bit changes
tests: Add test code for meta bitmap
block: Support meta dirty bitmap
block: Add two dirty bitmap getters
block: Assert that bdrv_release_dirty_bitmap succeeded
tests: Add test code for hbitmap serialization
block: More operations for meta dirty bitmap
Halil Pasic (1):
block: improve error handling in raw_open
Kevin Wolf (2):
block: Remove "options" indirection from blockdev-add
Merge remote-tracking branch 'mreitz/tags/pull-block-2016-10-24' into queue-block
Max Reitz (3):
qemu-nbd: Add --fork option
iotests: Remove raciness from 162
iotests: Do not rely on unavailable domains in 162
Paolo Bonzini (2):
quorum: change child_iter to children_read
quorum: do not allocate multiple iovecs for FIFO strategy
Pino Toscano (1):
qapi: fix memory leak in bdrv_image_info_specific_dump
Vladimir Sementsov-Ogievskiy (2):
hbitmap: serialization
block: BdrvDirtyBitmap serialization interface
Xu Tian (1):
block: failed qemu-img command should return non-zero exit code
block/backup.c | 14 ++-
block/dirty-bitmap.c | 160 ++++++++++++++++++++++++-
block/mirror.c | 24 ++--
block/qapi.c | 1 +
block/qcow2-cluster.c | 9 +-
block/qcow2.c | 3 +-
block/qcow2.h | 3 +-
block/quorum.c | 93 +++++++--------
block/raw-posix.c | 1 +
block/raw-win32.c | 1 +
block/replication.c | 5 +
block/throttle-groups.c | 27 ++++-
docs/qmp-commands.txt | 84 +++++++------
include/block/dirty-bitmap.h | 35 +++++-
include/qemu/hbitmap.h | 100 ++++++++++++++++
include/qemu/typedefs.h | 1 +
qapi/block-core.json | 7 +-
qemu-img.c | 2 +
qemu-nbd.c | 17 ++-
qemu-nbd.texi | 2 +
tests/qemu-iotests/041 | 11 +-
tests/qemu-iotests/067 | 12 +-
tests/qemu-iotests/071 | 118 ++++++++-----------
tests/qemu-iotests/081 | 52 ++++-----
tests/qemu-iotests/085 | 9 +-
tests/qemu-iotests/087 | 76 +++++-------
tests/qemu-iotests/093 | 33 +++++-
tests/qemu-iotests/093.out | 4 +-
tests/qemu-iotests/117 | 12 +-
tests/qemu-iotests/118 | 42 +++----
tests/qemu-iotests/124 | 20 ++--
tests/qemu-iotests/139 | 10 +-
tests/qemu-iotests/141 | 13 +--
tests/qemu-iotests/155 | 10 +-
tests/qemu-iotests/162 | 22 +++-
tests/qemu-iotests/162.out | 2 +-
tests/test-hbitmap.c | 272 +++++++++++++++++++++++++++++++++++++++++++
util/hbitmap.c | 206 +++++++++++++++++++++++++++++---
38 files changed, 1140 insertions(+), 373 deletions(-)
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 01/23] block: failed qemu-img command should return non-zero exit code
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 02/23] qcow2: Support BDRV_REQ_MAY_UNMAP Kevin Wolf
` (22 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Xu Tian <xutian@redhat.com>
If the backing file cannot be opened when doing qemu-img rebase, the
variable 'ret' was not assigned a non-zero value, and the qemu-img
process terminated with exit code zero. Fix this.
Signed-off-by: Xu Tian <xutian@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qemu-img.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/qemu-img.c b/qemu-img.c
index 67e8512..ab395a9 100644
--- a/qemu-img.c
+++ b/qemu-img.c
@@ -2956,6 +2956,7 @@ static int img_rebase(int argc, char **argv)
error_reportf_err(local_err,
"Could not open old backing file '%s': ",
backing_name);
+ ret = -1;
goto out;
}
@@ -2973,6 +2974,7 @@ static int img_rebase(int argc, char **argv)
error_reportf_err(local_err,
"Could not open new backing file '%s': ",
out_baseimg);
+ ret = -1;
goto out;
}
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 02/23] qcow2: Support BDRV_REQ_MAY_UNMAP
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 01/23] block: failed qemu-img command should return non-zero exit code Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 03/23] block: Remove "options" indirection from blockdev-add Kevin Wolf
` (21 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
Handling this is similar to what is done to the L2 entry in the case of
compressed clusters.
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/qcow2-cluster.c | 9 +++++----
block/qcow2.c | 3 ++-
block/qcow2.h | 3 ++-
3 files changed, 9 insertions(+), 6 deletions(-)
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 61d1ffd..928c1e2 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1558,7 +1558,7 @@ fail:
* clusters.
*/
static int zero_single_l2(BlockDriverState *bs, uint64_t offset,
- uint64_t nb_clusters)
+ uint64_t nb_clusters, int flags)
{
BDRVQcow2State *s = bs->opaque;
uint64_t *l2_table;
@@ -1582,7 +1582,7 @@ static int zero_single_l2(BlockDriverState *bs, uint64_t offset,
/* Update L2 entries */
qcow2_cache_entry_mark_dirty(bs, s->l2_table_cache, l2_table);
- if (old_offset & QCOW_OFLAG_COMPRESSED) {
+ if (old_offset & QCOW_OFLAG_COMPRESSED || flags & BDRV_REQ_MAY_UNMAP) {
l2_table[l2_index + i] = cpu_to_be64(QCOW_OFLAG_ZERO);
qcow2_free_any_clusters(bs, old_offset, 1, QCOW2_DISCARD_REQUEST);
} else {
@@ -1595,7 +1595,8 @@ static int zero_single_l2(BlockDriverState *bs, uint64_t offset,
return nb_clusters;
}
-int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors)
+int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors,
+ int flags)
{
BDRVQcow2State *s = bs->opaque;
uint64_t nb_clusters;
@@ -1612,7 +1613,7 @@ int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors)
s->cache_discards = true;
while (nb_clusters > 0) {
- ret = zero_single_l2(bs, offset, nb_clusters);
+ ret = zero_single_l2(bs, offset, nb_clusters, flags);
if (ret < 0) {
goto fail;
}
diff --git a/block/qcow2.c b/block/qcow2.c
index e11c7c9..6d5689a 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -1155,6 +1155,7 @@ static int qcow2_open(BlockDriverState *bs, QDict *options, int flags,
/* Initialise locks */
qemu_co_mutex_init(&s->lock);
+ bs->supported_zero_flags = BDRV_REQ_MAY_UNMAP;
/* Repair image if dirty */
if (!(flags & (BDRV_O_CHECK | BDRV_O_INACTIVE)) && !bs->read_only &&
@@ -2477,7 +2478,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
trace_qcow2_pwrite_zeroes(qemu_coroutine_self(), offset, count);
/* Whatever is left can use real zero clusters */
- ret = qcow2_zero_clusters(bs, offset, count >> BDRV_SECTOR_BITS);
+ ret = qcow2_zero_clusters(bs, offset, count >> BDRV_SECTOR_BITS, flags);
qemu_co_mutex_unlock(&s->lock);
return ret;
diff --git a/block/qcow2.h b/block/qcow2.h
index 9ce5a37..92203a8 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -547,7 +547,8 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m);
int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
int nb_sectors, enum qcow2_discard_type type, bool full_discard);
-int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors);
+int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors,
+ int flags);
int qcow2_expand_zero_clusters(BlockDriverState *bs,
BlockDriverAmendStatusCB *status_cb,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 03/23] block: Remove "options" indirection from blockdev-add
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 01/23] block: failed qemu-img command should return non-zero exit code Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 02/23] qcow2: Support BDRV_REQ_MAY_UNMAP Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 04/23] block: improve error handling in raw_open Kevin Wolf
` (20 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
Now that QAPI supports boxed types, we can have unions at the top level
of a command, so let's put our real options directly there for
blockdev-add instead of having a single "options" dict that contains the
real arguments.
blockdev-add is still experimental and we already made substantial
changes to the API recently, so we're free to make changes like this
one, too.
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Markus Armbruster <armbru@redhat.com>
Reviewed-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
docs/qmp-commands.txt | 84 ++++++++++++++++-------------------
qapi/block-core.json | 4 +-
tests/qemu-iotests/041 | 11 +++--
tests/qemu-iotests/067 | 12 +++--
tests/qemu-iotests/071 | 118 +++++++++++++++++++++----------------------------
tests/qemu-iotests/081 | 52 ++++++++++------------
tests/qemu-iotests/085 | 9 ++--
tests/qemu-iotests/087 | 76 +++++++++++++------------------
tests/qemu-iotests/117 | 12 ++---
tests/qemu-iotests/118 | 42 +++++++++---------
tests/qemu-iotests/124 | 20 ++++-----
tests/qemu-iotests/139 | 10 ++---
tests/qemu-iotests/141 | 13 +++---
tests/qemu-iotests/155 | 10 ++---
14 files changed, 214 insertions(+), 259 deletions(-)
diff --git a/docs/qmp-commands.txt b/docs/qmp-commands.txt
index 3220fb1..284576d 100644
--- a/docs/qmp-commands.txt
+++ b/docs/qmp-commands.txt
@@ -1090,11 +1090,11 @@ Arguments:
Example:
-> { "execute": "blockdev-add",
- "arguments": { "options": { "driver": "qcow2",
- "node-name": "node1534",
- "file": { "driver": "file",
- "filename": "hd1.qcow2" },
- "backing": "" } } }
+ "arguments": { "driver": "qcow2",
+ "node-name": "node1534",
+ "file": { "driver": "file",
+ "filename": "hd1.qcow2" },
+ "backing": "" } }
<- { "return": {} }
@@ -3130,41 +3130,37 @@ This command is still a work in progress. It doesn't support all
block drivers among other things. Stay away from it unless you want
to help with its development.
-Arguments:
-
-- "options": block driver options
+For the arguments, see the QAPI schema documentation of BlockdevOptions.
Example (1):
-> { "execute": "blockdev-add",
- "arguments": { "options" : { "driver": "qcow2",
- "file": { "driver": "file",
- "filename": "test.qcow2" } } } }
+ "arguments": { "driver": "qcow2",
+ "file": { "driver": "file",
+ "filename": "test.qcow2" } } }
<- { "return": {} }
Example (2):
-> { "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "qcow2",
- "node-name": "my_disk",
- "discard": "unmap",
- "cache": {
- "direct": true,
- "writeback": true
- },
- "file": {
- "driver": "file",
- "filename": "/tmp/test.qcow2"
- },
- "backing": {
- "driver": "raw",
- "file": {
- "driver": "file",
- "filename": "/dev/fdset/4"
- }
- }
+ "driver": "qcow2",
+ "node-name": "my_disk",
+ "discard": "unmap",
+ "cache": {
+ "direct": true,
+ "writeback": true
+ },
+ "file": {
+ "driver": "file",
+ "filename": "/tmp/test.qcow2"
+ },
+ "backing": {
+ "driver": "raw",
+ "file": {
+ "driver": "file",
+ "filename": "/dev/fdset/4"
+ }
}
}
}
@@ -3191,13 +3187,11 @@ Example:
-> { "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "qcow2",
- "node-name": "node0",
- "file": {
- "driver": "file",
- "filename": "test.qcow2"
- }
+ "driver": "qcow2",
+ "node-name": "node0",
+ "file": {
+ "driver": "file",
+ "filename": "test.qcow2"
}
}
}
@@ -3342,10 +3336,10 @@ Arguments:
Example:
-> { "execute": "blockdev-add",
- "arguments": { "options": { "node-name": "node0",
- "driver": "raw",
- "file": { "driver": "file",
- "filename": "fedora.iso" } } } }
+ "arguments": { { "node-name": "node0",
+ "driver": "raw",
+ "file": { "driver": "file",
+ "filename": "fedora.iso" } } }
<- { "return": {} }
@@ -3383,10 +3377,10 @@ Example:
Add a new node to a quorum
-> { "execute": "blockdev-add",
- "arguments": { "options": { "driver": "raw",
- "node-name": "new_node",
- "file": { "driver": "file",
- "filename": "test.raw" } } } }
+ "arguments": { "driver": "raw",
+ "node-name": "new_node",
+ "file": { "driver": "file",
+ "filename": "test.raw" } } }
<- { "return": {} }
-> { "execute": "x-blockdev-change",
"arguments": { "parent": "disk1",
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 1b7aa1b..c59047b 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2312,11 +2312,11 @@
# block drivers among other things. Stay away from it unless you want
# to help with its development.
#
-# @options: block device options for the new device
+# For the arguments, see the documentation of BlockdevOptions.
#
# Since: 1.7
##
-{ 'command': 'blockdev-add', 'data': { 'options': 'BlockdevOptions' } }
+{ 'command': 'blockdev-add', 'data': 'BlockdevOptions', 'boxed': true }
##
# @x-blockdev-del:
diff --git a/tests/qemu-iotests/041 b/tests/qemu-iotests/041
index d1e1ad8..30e628f 100755
--- a/tests/qemu-iotests/041
+++ b/tests/qemu-iotests/041
@@ -194,10 +194,9 @@ class TestSingleBlockdev(TestSingleDrive):
def setUp(self):
TestSingleDrive.setUp(self)
qemu_img('create', '-f', iotests.imgfmt, '-o', 'backing_file=%s' % backing_img, target_img)
- args = {'options':
- {'driver': iotests.imgfmt,
- 'node-name': self.qmp_target,
- 'file': { 'filename': target_img, 'driver': 'file' } } }
+ args = {'driver': iotests.imgfmt,
+ 'node-name': self.qmp_target,
+ 'file': { 'filename': target_img, 'driver': 'file' } }
result = self.vm.qmp("blockdev-add", **args)
self.assert_qmp(result, 'return', {})
@@ -782,8 +781,8 @@ class TestRepairQuorum(iotests.QMPTestCase):
self.vm.launch()
#assemble the quorum block device from the individual files
- args = { "options" : { "driver": "quorum", "node-name": "quorum0",
- "vote-threshold": 2, "children": [ "img0", "img1", "img2" ] } }
+ args = { "driver": "quorum", "node-name": "quorum0",
+ "vote-threshold": 2, "children": [ "img0", "img1", "img2" ] }
if self.has_quorum():
result = self.vm.qmp("blockdev-add", **args)
self.assert_qmp(result, 'return', {})
diff --git a/tests/qemu-iotests/067 b/tests/qemu-iotests/067
index a12125b..38d23fc 100755
--- a/tests/qemu-iotests/067
+++ b/tests/qemu-iotests/067
@@ -119,13 +119,11 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "disk",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "driver": "$IMGFMT",
+ "node-name": "disk",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
diff --git a/tests/qemu-iotests/071 b/tests/qemu-iotests/071
index 6d0864c..48b4955 100755
--- a/tests/qemu-iotests/071
+++ b/tests/qemu-iotests/071
@@ -107,25 +107,21 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "node-name": "drive0",
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "node-name": "drive0",
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "drive0-debug",
- "file": {
- "driver": "blkdebug",
- "image": "drive0",
- "inject-error": [{
- "event": "l2_load"
- }]
- }
+ "driver": "$IMGFMT",
+ "node-name": "drive0-debug",
+ "file": {
+ "driver": "blkdebug",
+ "image": "drive0",
+ "inject-error": [{
+ "event": "l2_load"
+ }]
}
}
}
@@ -145,26 +141,22 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "node-name": "drive0",
- "driver": "$IMGFMT",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "node-name": "drive0",
+ "driver": "$IMGFMT",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "blkverify",
- "node-name": "drive0-verify",
- "test": "drive0",
- "raw": {
- "driver": "file",
- "filename": "$TEST_IMG.base"
- }
+ "driver": "blkverify",
+ "node-name": "drive0-verify",
+ "test": "drive0",
+ "raw": {
+ "driver": "file",
+ "filename": "$TEST_IMG.base"
}
}
}
@@ -184,27 +176,23 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "node-name": "drive0",
- "driver": "file",
- "filename": "$TEST_IMG.base"
- }
+ "node-name": "drive0",
+ "driver": "file",
+ "filename": "$TEST_IMG.base"
}
}
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "blkverify",
- "node-name": "drive0-verify",
- "test": {
- "driver": "$IMGFMT",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
- },
- "raw": "drive0"
- }
+ "driver": "blkverify",
+ "node-name": "drive0-verify",
+ "test": {
+ "driver": "$IMGFMT",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
+ }
+ },
+ "raw": "drive0"
}
}
{ "execute": "human-monitor-command",
@@ -223,30 +211,26 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "node-name": "drive0",
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "node-name": "drive0",
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "drive0-debug",
- "file": {
- "driver": "blkdebug",
- "image": "drive0",
- "inject-error": [{
- "event": "read_aio",
- "state": 42
- }],
- "set-state": [{
- "event": "write_aio",
- "new_state": 42
- }]
- }
+ "driver": "$IMGFMT",
+ "node-name": "drive0-debug",
+ "file": {
+ "driver": "blkdebug",
+ "image": "drive0",
+ "inject-error": [{
+ "event": "read_aio",
+ "state": 42
+ }],
+ "set-state": [{
+ "event": "write_aio",
+ "new_state": 42
+ }]
}
}
}
diff --git a/tests/qemu-iotests/081 b/tests/qemu-iotests/081
index 0a809f3..da3fb09 100755
--- a/tests/qemu-iotests/081
+++ b/tests/qemu-iotests/081
@@ -105,40 +105,36 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "node-name": "drive2",
- "driver": "$IMGFMT",
- "file": {
- "driver": "file",
- "filename": "$TEST_DIR/2.raw"
- }
+ "node-name": "drive2",
+ "driver": "$IMGFMT",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_DIR/2.raw"
}
}
}
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "quorum",
- "node-name": "drive0-quorum",
- "vote-threshold": 2,
- "children": [
- {
- "driver": "$IMGFMT",
- "file": {
- "driver": "file",
- "filename": "$TEST_DIR/1.raw"
- }
- },
- "drive2",
- {
- "driver": "$IMGFMT",
- "file": {
- "driver": "file",
- "filename": "$TEST_DIR/3.raw"
- }
+ "driver": "quorum",
+ "node-name": "drive0-quorum",
+ "vote-threshold": 2,
+ "children": [
+ {
+ "driver": "$IMGFMT",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_DIR/1.raw"
}
- ]
- }
+ },
+ "drive2",
+ {
+ "driver": "$IMGFMT",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_DIR/3.raw"
+ }
+ }
+ ]
}
}
{ "execute": "human-monitor-command",
diff --git a/tests/qemu-iotests/085 b/tests/qemu-iotests/085
index aa77eca..c53e97f 100755
--- a/tests/qemu-iotests/085
+++ b/tests/qemu-iotests/085
@@ -100,11 +100,10 @@ function add_snapshot_image()
_make_test_img -b "${base_image}" "$size"
mv "${TEST_IMG}" "${snapshot_file}"
cmd="{ 'execute': 'blockdev-add', 'arguments':
- { 'options':
- { 'driver': 'qcow2', 'node-name': 'snap_${1}', ${extra_params}
- 'file':
- { 'driver': 'file', 'filename': '${snapshot_file}',
- 'node-name': 'file_${1}' } } } }"
+ { 'driver': 'qcow2', 'node-name': 'snap_${1}', ${extra_params}
+ 'file':
+ { 'driver': 'file', 'filename': '${snapshot_file}',
+ 'node-name': 'file_${1}' } } }"
_send_qemu_cmd $h "${cmd}" "return"
}
diff --git a/tests/qemu-iotests/087 b/tests/qemu-iotests/087
index b1ac71f..9de57dd 100755
--- a/tests/qemu-iotests/087
+++ b/tests/qemu-iotests/087
@@ -61,12 +61,10 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "driver": "$IMGFMT",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
@@ -81,25 +79,21 @@ run_qemu -drive driver=$IMGFMT,id=disk,node-name=test-node,file="$TEST_IMG" <<EO
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "disk",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "driver": "$IMGFMT",
+ "node-name": "disk",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "test-node",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "driver": "$IMGFMT",
+ "node-name": "test-node",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
@@ -114,14 +108,12 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "disk",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG",
- "aio": "native"
- }
+ "driver": "$IMGFMT",
+ "node-name": "disk",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG",
+ "aio": "native"
}
}
}
@@ -137,13 +129,11 @@ run_qemu -S <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "disk",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "driver": "$IMGFMT",
+ "node-name": "disk",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
@@ -154,13 +144,11 @@ run_qemu <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "driver": "$IMGFMT",
- "node-name": "disk",
- "file": {
- "driver": "file",
- "filename": "$TEST_IMG"
- }
+ "driver": "$IMGFMT",
+ "node-name": "disk",
+ "file": {
+ "driver": "file",
+ "filename": "$TEST_IMG"
}
}
}
@@ -176,9 +164,7 @@ run_qemu -S <<EOF
{ "execute": "qmp_capabilities" }
{ "execute": "blockdev-add",
"arguments": {
- "options": {
- "node-name": "disk"
- }
+ "node-name": "disk"
}
}
{ "execute": "quit" }
diff --git a/tests/qemu-iotests/117 b/tests/qemu-iotests/117
index 5b28039..e955d52 100755
--- a/tests/qemu-iotests/117
+++ b/tests/qemu-iotests/117
@@ -52,16 +52,16 @@ _send_qemu_cmd $QEMU_HANDLE \
_send_qemu_cmd $QEMU_HANDLE \
"{ 'execute': 'blockdev-add',
- 'arguments': { 'options': { 'node-name': 'protocol',
- 'driver': 'file',
- 'filename': '$TEST_IMG' } } }" \
+ 'arguments': { 'node-name': 'protocol',
+ 'driver': 'file',
+ 'filename': '$TEST_IMG' } }" \
'return'
_send_qemu_cmd $QEMU_HANDLE \
"{ 'execute': 'blockdev-add',
- 'arguments': { 'options': { 'node-name': 'format',
- 'driver': '$IMGFMT',
- 'file': 'protocol' } } }" \
+ 'arguments': { 'node-name': 'format',
+ 'driver': '$IMGFMT',
+ 'file': 'protocol' } }" \
'return'
_send_qemu_cmd $QEMU_HANDLE \
diff --git a/tests/qemu-iotests/118 b/tests/qemu-iotests/118
index e63a40f..8a9e838 100755
--- a/tests/qemu-iotests/118
+++ b/tests/qemu-iotests/118
@@ -229,10 +229,10 @@ class GeneralChangeTestsBaseClass(ChangeBaseClass):
def test_cycle(self):
result = self.vm.qmp('blockdev-add',
- options={'node-name': 'new',
- 'driver': iotests.imgfmt,
- 'file': {'filename': new_img,
- 'driver': 'file'}})
+ node_name='new',
+ driver=iotests.imgfmt,
+ file={'filename': new_img,
+ 'driver': 'file'})
self.assert_qmp(result, 'return', {})
if self.device_name is not None:
@@ -309,10 +309,10 @@ class GeneralChangeTestsBaseClass(ChangeBaseClass):
return
result = self.vm.qmp('blockdev-add',
- options={'node-name': 'new',
- 'driver': iotests.imgfmt,
- 'file': {'filename': new_img,
- 'driver': 'file'}})
+ node_name='new',
+ driver=iotests.imgfmt,
+ file={'filename': new_img,
+ 'driver': 'file'})
self.assert_qmp(result, 'return', {})
result = self.vm.qmp('x-blockdev-insert-medium', device='drive0',
@@ -341,10 +341,10 @@ class TestInitiallyFilled(GeneralChangeTestsBaseClass):
def test_insert_on_filled(self):
result = self.vm.qmp('blockdev-add',
- options={'node-name': 'new',
- 'driver': iotests.imgfmt,
- 'file': {'filename': new_img,
- 'driver': 'file'}})
+ node_name='new',
+ driver=iotests.imgfmt,
+ file={'filename': new_img,
+ 'driver': 'file'})
self.assert_qmp(result, 'return', {})
result = self.vm.qmp('blockdev-open-tray', device='drive0')
@@ -609,11 +609,11 @@ class TestChangeReadOnly(ChangeBaseClass):
self.assert_qmp(result, 'return[0]/inserted/image/filename', old_img)
result = self.vm.qmp('blockdev-add',
- options={'node-name': 'new',
- 'driver': iotests.imgfmt,
- 'read-only': True,
- 'file': {'filename': new_img,
- 'driver': 'file'}})
+ node_name='new',
+ driver=iotests.imgfmt,
+ read_only=True,
+ file={'filename': new_img,
+ 'driver': 'file'})
self.assert_qmp(result, 'return', {})
result = self.vm.qmp('query-block')
@@ -663,10 +663,10 @@ class TestBlockJobsAfterCycle(ChangeBaseClass):
self.assert_qmp_absent(result, 'return[0]/inserted')
result = self.vm.qmp('blockdev-add',
- options={'node-name': 'node0',
- 'driver': iotests.imgfmt,
- 'file': {'filename': old_img,
- 'driver': 'file'}})
+ node_name='node0',
+ driver=iotests.imgfmt,
+ file={'filename': old_img,
+ 'driver': 'file'})
self.assert_qmp(result, 'return', {})
result = self.vm.qmp('x-blockdev-insert-medium', device='drive0',
diff --git a/tests/qemu-iotests/124 b/tests/qemu-iotests/124
index 2f0bc24..f06938e 100644
--- a/tests/qemu-iotests/124
+++ b/tests/qemu-iotests/124
@@ -416,10 +416,10 @@ class TestIncrementalBackup(TestIncrementalBackupBase):
('0xcd', '32M', '124k')))
# Create a blkdebug interface to this img as 'drive1'
- result = self.vm.qmp('blockdev-add', options={
- 'node-name': drive1['id'],
- 'driver': drive1['fmt'],
- 'file': {
+ result = self.vm.qmp('blockdev-add',
+ node_name=drive1['id'],
+ driver=drive1['fmt'],
+ file={
'driver': 'blkdebug',
'image': {
'driver': 'file',
@@ -438,7 +438,7 @@ class TestIncrementalBackup(TestIncrementalBackupBase):
'once': True
}],
}
- })
+ )
self.assert_qmp(result, 'return', {})
# Create bitmaps and full backups for both drives
@@ -560,10 +560,10 @@ class TestIncrementalBackupBlkdebug(TestIncrementalBackupBase):
'''
drive0 = self.drives[0]
- result = self.vm.qmp('blockdev-add', options={
- 'node-name': drive0['id'],
- 'driver': drive0['fmt'],
- 'file': {
+ result = self.vm.qmp('blockdev-add',
+ node_name=drive0['id'],
+ driver=drive0['fmt'],
+ file={
'driver': 'blkdebug',
'image': {
'driver': 'file',
@@ -582,7 +582,7 @@ class TestIncrementalBackupBlkdebug(TestIncrementalBackupBase):
'once': True
}],
}
- })
+ )
self.assert_qmp(result, 'return', {})
self.create_anchor_backup(drive0)
diff --git a/tests/qemu-iotests/139 b/tests/qemu-iotests/139
index 47a4c26..6a0f6ca 100644
--- a/tests/qemu-iotests/139
+++ b/tests/qemu-iotests/139
@@ -57,7 +57,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
'file': {'driver': 'file',
'node-name': file_node,
'filename': base_img}}
- result = self.vm.qmp('blockdev-add', conv_keys = False, options = opts)
+ result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
self.assert_qmp(result, 'return', {})
self.checkBlockDriverState(node)
self.checkBlockDriverState(file_node)
@@ -72,7 +72,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
'backing': '',
'file': {'driver': 'file',
'filename': new_img}}
- result = self.vm.qmp('blockdev-add', conv_keys = False, options = opts)
+ result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
self.assert_qmp(result, 'return', {})
self.checkBlockDriverState(node)
@@ -185,7 +185,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
opts = {'driver': 'blkdebug',
'node-name': debug,
'image': image}
- result = self.vm.qmp('blockdev-add', conv_keys = False, options = opts)
+ result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
self.assert_qmp(result, 'return', {})
self.checkBlockDriverState(node)
self.checkBlockDriverState(debug)
@@ -210,7 +210,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
'node-name': blkverify,
'test': node_0,
'raw': node_1}
- result = self.vm.qmp('blockdev-add', conv_keys = False, options = opts)
+ result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
self.assert_qmp(result, 'return', {})
self.checkBlockDriverState(test)
self.checkBlockDriverState(raw)
@@ -234,7 +234,7 @@ class TestBlockdevDel(iotests.QMPTestCase):
'node-name': quorum,
'vote-threshold': 1,
'children': [ child_0, child_1 ]}
- result = self.vm.qmp('blockdev-add', conv_keys = False, options = opts)
+ result = self.vm.qmp('blockdev-add', conv_keys = False, **opts)
self.assert_qmp(result, 'return', {})
self.checkBlockDriverState(child0)
self.checkBlockDriverState(child1)
diff --git a/tests/qemu-iotests/141 b/tests/qemu-iotests/141
index c092d87..3ba79f0 100755
--- a/tests/qemu-iotests/141
+++ b/tests/qemu-iotests/141
@@ -50,13 +50,12 @@ test_blockjob()
_send_qemu_cmd $QEMU_HANDLE \
"{'execute': 'blockdev-add',
'arguments': {
- 'options': {
- 'node-name': 'drv0',
- 'driver': '$IMGFMT',
- 'file': {
- 'driver': 'file',
- 'filename': '$TEST_IMG'
- }}}}" \
+ 'node-name': 'drv0',
+ 'driver': '$IMGFMT',
+ 'file': {
+ 'driver': 'file',
+ 'filename': '$TEST_IMG'
+ }}}" \
'return'
_send_qemu_cmd $QEMU_HANDLE \
diff --git a/tests/qemu-iotests/155 b/tests/qemu-iotests/155
index 4057b5e..0b86ea4 100755
--- a/tests/qemu-iotests/155
+++ b/tests/qemu-iotests/155
@@ -63,10 +63,10 @@ class BaseClass(iotests.QMPTestCase):
# Add the BDS via blockdev-add so it stays around after the mirror block
# job has been completed
result = self.vm.qmp('blockdev-add',
- options={'node-name': 'source',
- 'driver': iotests.imgfmt,
- 'file': {'driver': 'file',
- 'filename': source_img}})
+ node_name='source',
+ driver=iotests.imgfmt,
+ file={'driver': 'file',
+ 'filename': source_img})
self.assert_qmp(result, 'return', {})
result = self.vm.qmp('x-blockdev-insert-medium',
@@ -90,7 +90,7 @@ class BaseClass(iotests.QMPTestCase):
if self.target_blockdev_backing:
options['backing'] = self.target_blockdev_backing
- result = self.vm.qmp('blockdev-add', options=options)
+ result = self.vm.qmp('blockdev-add', **options)
self.assert_qmp(result, 'return', {})
def tearDown(self):
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 04/23] block: improve error handling in raw_open
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (2 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 03/23] block: Remove "options" indirection from blockdev-add Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 05/23] qapi: fix memory leak in bdrv_image_info_specific_dump Kevin Wolf
` (19 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Halil Pasic <pasic@linux.vnet.ibm.com>
Make raw_open for POSIX more consistent in handling errors by setting
the error object also when qemu_open fails. The error object was set
generally set in case of errors, but I guess this case was overlooked.
Do the same for win32.
Signed-off-by: Halil Pasic <pasic@linux.vnet.ibm.com>
Reviewed-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Tested-by: Marc Hartmayer <mhartmay@linux.vnet.ibm.com> (POSIX only)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/raw-posix.c | 1 +
block/raw-win32.c | 1 +
2 files changed, 2 insertions(+)
diff --git a/block/raw-posix.c b/block/raw-posix.c
index 166e9d1..f481e57 100644
--- a/block/raw-posix.c
+++ b/block/raw-posix.c
@@ -443,6 +443,7 @@ static int raw_open_common(BlockDriverState *bs, QDict *options,
fd = qemu_open(filename, s->open_flags, 0644);
if (fd < 0) {
ret = -errno;
+ error_setg_errno(errp, errno, "Could not open '%s'", filename);
if (ret == -EROFS) {
ret = -EACCES;
}
diff --git a/block/raw-win32.c b/block/raw-win32.c
index 734bb10..800fabd 100644
--- a/block/raw-win32.c
+++ b/block/raw-win32.c
@@ -373,6 +373,7 @@ static int raw_open(BlockDriverState *bs, QDict *options, int flags,
if (s->hfile == INVALID_HANDLE_VALUE) {
int err = GetLastError();
+ error_setg_win32(errp, err, "Could not open '%s'", filename);
if (err == ERROR_ACCESS_DENIED) {
ret = -EACCES;
} else {
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 05/23] qapi: fix memory leak in bdrv_image_info_specific_dump
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (3 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 04/23] block: improve error handling in raw_open Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 06/23] throttle: Correct access to wrong BlockBackendPublic structures Kevin Wolf
` (18 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Pino Toscano <ptoscano@redhat.com>
The 'obj' result of the visitor was not properly freed, like done in
other places doing a similar job.
Signed-off-by: Pino Toscano <ptoscano@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/qapi.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/block/qapi.c b/block/qapi.c
index 6f947e3..50d3090 100644
--- a/block/qapi.c
+++ b/block/qapi.c
@@ -698,6 +698,7 @@ void bdrv_image_info_specific_dump(fprintf_function func_fprintf, void *f,
assert(qobject_type(obj) == QTYPE_QDICT);
data = qdict_get(qobject_to_qdict(obj), "data");
dump_qobject(func_fprintf, f, 1, data);
+ qobject_decref(obj);
visit_free(v);
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 06/23] throttle: Correct access to wrong BlockBackendPublic structures
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (4 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 05/23] qapi: fix memory leak in bdrv_image_info_specific_dump Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 07/23] qemu-iotests: Test I/O in a single drive from a throttling group Kevin Wolf
` (17 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Alberto Garcia <berto@igalia.com>
In 27ccdd52598290f0f8b58be56e235aff7aebfaf3 the throttling fields were
moved from BlockDriverState to BlockBackend. However in a few cases
the code started using throttling fields from the active BlockBackend
instead of the round-robin token, making the algorithm behave
incorrectly.
This can cause starvation if there's a throttling group with several
drives but only one of them has I/O.
Cc: qemu-stable@nongnu.org
Reported-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Alberto Garcia <berto@igalia.com>
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/throttle-groups.c | 27 +++++++++++++++++++++++----
1 file changed, 23 insertions(+), 4 deletions(-)
diff --git a/block/throttle-groups.c b/block/throttle-groups.c
index 59545e2..17b2efb 100644
--- a/block/throttle-groups.c
+++ b/block/throttle-groups.c
@@ -168,6 +168,22 @@ static BlockBackend *throttle_group_next_blk(BlockBackend *blk)
return blk_by_public(next);
}
+/*
+ * Return whether a BlockBackend has pending requests.
+ *
+ * This assumes that tg->lock is held.
+ *
+ * @blk: the BlockBackend
+ * @is_write: the type of operation (read/write)
+ * @ret: whether the BlockBackend has pending requests.
+ */
+static inline bool blk_has_pending_reqs(BlockBackend *blk,
+ bool is_write)
+{
+ const BlockBackendPublic *blkp = blk_get_public(blk);
+ return blkp->pending_reqs[is_write];
+}
+
/* Return the next BlockBackend in the round-robin sequence with pending I/O
* requests.
*
@@ -188,7 +204,7 @@ static BlockBackend *next_throttle_token(BlockBackend *blk, bool is_write)
/* get next bs round in round robin style */
token = throttle_group_next_blk(token);
- while (token != start && !blkp->pending_reqs[is_write]) {
+ while (token != start && !blk_has_pending_reqs(token, is_write)) {
token = throttle_group_next_blk(token);
}
@@ -196,10 +212,13 @@ static BlockBackend *next_throttle_token(BlockBackend *blk, bool is_write)
* then decide the token is the current bs because chances are
* the current bs get the current request queued.
*/
- if (token == start && !blkp->pending_reqs[is_write]) {
+ if (token == start && !blk_has_pending_reqs(token, is_write)) {
token = blk;
}
+ /* Either we return the original BB, or one with pending requests */
+ assert(token == blk || blk_has_pending_reqs(token, is_write));
+
return token;
}
@@ -257,7 +276,7 @@ static void schedule_next_request(BlockBackend *blk, bool is_write)
/* Check if there's any pending request to schedule next */
token = next_throttle_token(blk, is_write);
- if (!blkp->pending_reqs[is_write]) {
+ if (!blk_has_pending_reqs(token, is_write)) {
return;
}
@@ -271,7 +290,7 @@ static void schedule_next_request(BlockBackend *blk, bool is_write)
qemu_co_queue_next(&blkp->throttled_reqs[is_write])) {
token = blk;
} else {
- ThrottleTimers *tt = &blkp->throttle_timers;
+ ThrottleTimers *tt = &blk_get_public(token)->throttle_timers;
int64_t now = qemu_clock_get_ns(tt->clock_type);
timer_mod(tt->timers[is_write], now + 1);
tg->any_timer_armed[is_write] = true;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 07/23] qemu-iotests: Test I/O in a single drive from a throttling group
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (5 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 06/23] throttle: Correct access to wrong BlockBackendPublic structures Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 08/23] qemu-nbd: Add --fork option Kevin Wolf
` (16 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Alberto Garcia <berto@igalia.com>
iotest 093 contains a test that creates a throttling group with
several drives and performs I/O in all of them. This patch adds a new
test that creates a similar setup but only performs I/O in one of the
drives at the same time.
This is useful to test that the round robin algorithm is behaving
properly in these scenarios, and is specifically written using the
regression introduced in 27ccdd52598290f0f8b58be56e as an example.
Signed-off-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/093 | 33 ++++++++++++++++++++++++++++-----
tests/qemu-iotests/093.out | 4 ++--
2 files changed, 30 insertions(+), 7 deletions(-)
diff --git a/tests/qemu-iotests/093 b/tests/qemu-iotests/093
index ffcb271..2ed393a 100755
--- a/tests/qemu-iotests/093
+++ b/tests/qemu-iotests/093
@@ -53,7 +53,7 @@ class ThrottleTestCase(iotests.QMPTestCase):
result = self.vm.qmp("block_set_io_throttle", conv_keys=False, **params)
self.assert_qmp(result, 'return', {})
- def do_test_throttle(self, ndrives, seconds, params):
+ def do_test_throttle(self, ndrives, seconds, params, first_drive = 0):
def check_limit(limit, num):
# IO throttling algorithm is discrete, allow 10% error so the test
# is more robust
@@ -85,12 +85,14 @@ class ThrottleTestCase(iotests.QMPTestCase):
# Send I/O requests to all drives
for i in range(rd_nr):
for drive in range(0, ndrives):
- self.vm.hmp_qemu_io("drive%d" % drive, "aio_read %d %d" %
+ idx = first_drive + drive
+ self.vm.hmp_qemu_io("drive%d" % idx, "aio_read %d %d" %
(i * rq_size, rq_size))
for i in range(wr_nr):
for drive in range(0, ndrives):
- self.vm.hmp_qemu_io("drive%d" % drive, "aio_write %d %d" %
+ idx = first_drive + drive
+ self.vm.hmp_qemu_io("drive%d" % idx, "aio_write %d %d" %
(i * rq_size, rq_size))
# We'll store the I/O stats for each drive in these arrays
@@ -105,15 +107,17 @@ class ThrottleTestCase(iotests.QMPTestCase):
# Read the stats before advancing the clock
for i in range(0, ndrives):
+ idx = first_drive + i
start_rd_bytes[i], start_rd_iops[i], start_wr_bytes[i], \
- start_wr_iops[i] = self.blockstats('drive%d' % i)
+ start_wr_iops[i] = self.blockstats('drive%d' % idx)
self.vm.qtest("clock_step %d" % ns)
# Read the stats after advancing the clock
for i in range(0, ndrives):
+ idx = first_drive + i
end_rd_bytes[i], end_rd_iops[i], end_wr_bytes[i], \
- end_wr_iops[i] = self.blockstats('drive%d' % i)
+ end_wr_iops[i] = self.blockstats('drive%d' % idx)
# Check that the I/O is within the limits and evenly distributed
for i in range(0, ndrives):
@@ -129,6 +133,7 @@ class ThrottleTestCase(iotests.QMPTestCase):
self.assertTrue(check_limit(params['iops_rd'], rd_iops))
self.assertTrue(check_limit(params['iops_wr'], wr_iops))
+ # Connect N drives to a VM and test I/O in all of them
def test_all(self):
params = {"bps": 4096,
"bps_rd": 4096,
@@ -146,6 +151,24 @@ class ThrottleTestCase(iotests.QMPTestCase):
self.configure_throttle(ndrives, limits)
self.do_test_throttle(ndrives, 5, limits)
+ # Connect N drives to a VM and test I/O in just one of them a time
+ def test_one(self):
+ params = {"bps": 4096,
+ "bps_rd": 4096,
+ "bps_wr": 4096,
+ "iops": 10,
+ "iops_rd": 10,
+ "iops_wr": 10,
+ }
+ # Repeat the test for each one of the drives
+ for drive in range(0, self.max_drives):
+ # Pick each out of all possible params and test
+ for tk in params:
+ limits = dict([(k, 0) for k in params])
+ limits[tk] = params[tk] * self.max_drives
+ self.configure_throttle(self.max_drives, limits)
+ self.do_test_throttle(1, 5, limits, drive)
+
def test_burst(self):
params = {"bps": 4096,
"bps_rd": 4096,
diff --git a/tests/qemu-iotests/093.out b/tests/qemu-iotests/093.out
index 914e373..2f7d390 100644
--- a/tests/qemu-iotests/093.out
+++ b/tests/qemu-iotests/093.out
@@ -1,5 +1,5 @@
-.....
+.......
----------------------------------------------------------------------
-Ran 5 tests
+Ran 7 tests
OK
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 08/23] qemu-nbd: Add --fork option
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (6 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 07/23] qemu-iotests: Test I/O in a single drive from a throttling group Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 09/23] iotests: Remove raciness from 162 Kevin Wolf
` (15 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Max Reitz <mreitz@redhat.com>
Using the --fork option, one can make qemu-nbd fork the worker process.
The original process will exit on error of the worker or once the worker
enters the main loop.
Suggested-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
qemu-nbd.c | 17 ++++++++++++++++-
qemu-nbd.texi | 2 ++
2 files changed, 18 insertions(+), 1 deletion(-)
diff --git a/qemu-nbd.c b/qemu-nbd.c
index cca4a98..b757dc7 100644
--- a/qemu-nbd.c
+++ b/qemu-nbd.c
@@ -48,6 +48,7 @@
#define QEMU_NBD_OPT_OBJECT 260
#define QEMU_NBD_OPT_TLSCREDS 261
#define QEMU_NBD_OPT_IMAGE_OPTS 262
+#define QEMU_NBD_OPT_FORK 263
#define MBR_SIZE 512
@@ -92,6 +93,8 @@ static void usage(const char *name)
" passwords and/or encryption keys\n"
" -T, --trace [[enable=]<pattern>][,events=<file>][,file=<file>]\n"
" specify tracing options\n"
+" --fork fork off the server process and exit the parent\n"
+" once the server is running\n"
#ifdef __linux__
"Kernel NBD client support:\n"
" -c, --connect=DEV connect FILE to the local NBD device DEV\n"
@@ -503,6 +506,7 @@ int main(int argc, char **argv)
{ "tls-creds", required_argument, NULL, QEMU_NBD_OPT_TLSCREDS },
{ "image-opts", no_argument, NULL, QEMU_NBD_OPT_IMAGE_OPTS },
{ "trace", required_argument, NULL, 'T' },
+ { "fork", no_argument, NULL, QEMU_NBD_OPT_FORK },
{ NULL, 0, NULL, 0 }
};
int ch;
@@ -524,6 +528,8 @@ int main(int argc, char **argv)
bool imageOpts = false;
bool writethrough = true;
char *trace_file = NULL;
+ bool fork_process = false;
+ int old_stderr = -1;
/* The client thread uses SIGTERM to interrupt the server. A signal
* handler ensures that "qemu-nbd -v -c" exits with a nice status code.
@@ -715,6 +721,9 @@ int main(int argc, char **argv)
g_free(trace_file);
trace_file = trace_opt_parse(optarg);
break;
+ case QEMU_NBD_OPT_FORK:
+ fork_process = true;
+ break;
}
}
@@ -774,7 +783,7 @@ int main(int argc, char **argv)
return 0;
}
- if (device && !verbose) {
+ if ((device && !verbose) || fork_process) {
int stderr_fd[2];
pid_t pid;
int ret;
@@ -797,6 +806,7 @@ int main(int argc, char **argv)
ret = qemu_daemon(1, 0);
/* Temporarily redirect stderr to the parent's pipe... */
+ old_stderr = dup(STDERR_FILENO);
dup2(stderr_fd[1], STDERR_FILENO);
if (ret < 0) {
error_report("Failed to daemonize: %s", strerror(errno));
@@ -960,6 +970,11 @@ int main(int argc, char **argv)
exit(EXIT_FAILURE);
}
+ if (fork_process) {
+ dup2(old_stderr, STDERR_FILENO);
+ close(old_stderr);
+ }
+
state = RUNNING;
do {
main_loop_wait(false);
diff --git a/qemu-nbd.texi b/qemu-nbd.texi
index 91ebf04..b7a9c6d 100644
--- a/qemu-nbd.texi
+++ b/qemu-nbd.texi
@@ -86,6 +86,8 @@ the new style NBD protocol negotiation
Enable mandatory TLS encryption for the server by setting the ID
of the TLS credentials object previously created with the --object
option.
+@item --fork
+Fork off the server process and exit the parent once the server is running.
@item -v, --verbose
Display extra debugging information
@item -h, --help
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 09/23] iotests: Remove raciness from 162
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (7 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 08/23] qemu-nbd: Add --fork option Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 10/23] iotests: Do not rely on unavailable domains in 162 Kevin Wolf
` (14 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Max Reitz <mreitz@redhat.com>
With qemu-nbd's new --fork option, we no longer need to launch it the
hacky way.
Suggested-by: Sascha Silbe <silbe@linux.vnet.ibm.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/162 | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index 0b43ea3..c7e6593 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -51,8 +51,7 @@ $QEMU_IMG info 'json:{"driver": "nbd", "host": "does.not.exist.example.com", "po
# This is a test for NBD's bdrv_refresh_filename() implementation: It expects
# either host or path to be set, but it must not assume that they are set to
# strings in the options QDict
-$QEMU_NBD -k "$PWD/42" -f raw null-co:// &
-sleep 0.5
+$QEMU_NBD -k "$PWD/42" -f raw --fork null-co://
$QEMU_IMG info 'json:{"driver": "nbd", "path": 42}' | grep '^image'
rm -f 42
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 10/23] iotests: Do not rely on unavailable domains in 162
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (8 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 09/23] iotests: Remove raciness from 162 Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 11/23] quorum: change child_iter to children_read Kevin Wolf
` (13 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Max Reitz <mreitz@redhat.com>
There are some (mostly ISP-specific) name servers who will redirect
non-existing domains to special hosts. In this case, we will get a
different error message when trying to connect to such a host, which
breaks test 162.
162 needed this specific error message so it can confirm that qemu was
indeed trying to connect to the user-specified port. However, we can
also confirm this by setting up a local NBD server on exactly that port;
so we can fix the issue by doing just that.
Reported-by: Stefan Hajnoczi <stefanha@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/162 | 19 +++++++++++++++----
tests/qemu-iotests/162.out | 2 +-
2 files changed, 16 insertions(+), 5 deletions(-)
diff --git a/tests/qemu-iotests/162 b/tests/qemu-iotests/162
index c7e6593..f8eecb3 100755
--- a/tests/qemu-iotests/162
+++ b/tests/qemu-iotests/162
@@ -43,10 +43,21 @@ echo '=== NBD ==='
$QEMU_IMG info 'json:{"driver": "nbd", "host": 42}'
# And this should not treat @port as if it had not been specified
-# (We cannot use localhost with an invalid port here, but we need to use a
-# non-existing domain, because otherwise the error message will not contain
-# the port)
-$QEMU_IMG info 'json:{"driver": "nbd", "host": "does.not.exist.example.com", "port": 42}'
+# (We need to set up a server here, because the error message for "Connection
+# refused" does not contain the destination port)
+
+# Launching qemu-nbd is done in a loop: We try to set up an NBD server on some
+# random port and continue until success, i.e. until we have found a port that
+# is not in use yet.
+while true; do
+ port=$((RANDOM + 32768))
+ if $QEMU_NBD -p $port -f raw --fork null-co:// 2> /dev/null; then
+ break
+ fi
+done
+
+$QEMU_IMG info "json:{'driver': 'nbd', 'host': 'localhost', 'port': $port}" \
+ | grep '^image' | sed -e "s/$port/PORT/"
# This is a test for NBD's bdrv_refresh_filename() implementation: It expects
# either host or path to be set, but it must not assume that they are set to
diff --git a/tests/qemu-iotests/162.out b/tests/qemu-iotests/162.out
index 9bba723..3c5be2c 100644
--- a/tests/qemu-iotests/162.out
+++ b/tests/qemu-iotests/162.out
@@ -2,7 +2,7 @@ QA output created by 162
=== NBD ===
qemu-img: Could not open 'json:{"driver": "nbd", "host": 42}': Failed to connect socket: Invalid argument
-qemu-img: Could not open 'json:{"driver": "nbd", "host": "does.not.exist.example.com", "port": 42}': address resolution failed for does.not.exist.example.com:42: Name or service not known
+image: nbd://localhost:PORT
image: nbd+unix://?socket=42
=== SSH ===
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 11/23] quorum: change child_iter to children_read
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (9 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 10/23] iotests: Do not rely on unavailable domains in 162 Kevin Wolf
@ 2016-10-24 17:01 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 12/23] quorum: do not allocate multiple iovecs for FIFO strategy Kevin Wolf
` (12 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:01 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Paolo Bonzini <pbonzini@redhat.com>
This simplifies a bit the code by using the usual C "inclusive start,
exclusive end" pattern for ranges.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1475685327-22767-2-git-send-email-pbonzini@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/quorum.c | 30 +++++++++++++-----------------
1 file changed, 13 insertions(+), 17 deletions(-)
diff --git a/block/quorum.c b/block/quorum.c
index 9cf876f..435296e 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -130,7 +130,7 @@ struct QuorumAIOCB {
bool is_read;
int vote_ret;
- int child_iter; /* which child to read in fifo pattern */
+ int children_read; /* how many children have been read from */
};
static bool quorum_vote(QuorumAIOCB *acb);
@@ -165,8 +165,7 @@ static void quorum_aio_finalize(QuorumAIOCB *acb)
acb->common.cb(acb->common.opaque, ret);
if (acb->is_read) {
- /* on the quorum case acb->child_iter == s->num_children - 1 */
- for (i = 0; i <= acb->child_iter; i++) {
+ for (i = 0; i < acb->children_read; i++) {
qemu_vfree(acb->qcrs[i].buf);
qemu_iovec_destroy(&acb->qcrs[i].qiov);
}
@@ -301,14 +300,13 @@ static void quorum_aio_cb(void *opaque, int ret)
if (acb->is_read && s->read_pattern == QUORUM_READ_PATTERN_FIFO) {
/* We try to read next child in FIFO order if we fail to read */
- if (ret < 0 && (acb->child_iter + 1) < s->num_children) {
- acb->child_iter++;
+ if (ret < 0 && acb->children_read < s->num_children) {
read_fifo_child(acb);
return;
}
if (ret == 0) {
- quorum_copy_qiov(acb->qiov, &acb->qcrs[acb->child_iter].qiov);
+ quorum_copy_qiov(acb->qiov, &acb->qcrs[acb->children_read - 1].qiov);
}
acb->vote_ret = ret;
quorum_aio_finalize(acb);
@@ -653,6 +651,7 @@ static BlockAIOCB *read_quorum_children(QuorumAIOCB *acb)
BDRVQuorumState *s = acb->common.bs->opaque;
int i;
+ acb->children_read = s->num_children;
for (i = 0; i < s->num_children; i++) {
acb->qcrs[i].buf = qemu_blockalign(s->children[i]->bs, acb->qiov->size);
qemu_iovec_init(&acb->qcrs[i].qiov, acb->qiov->niov);
@@ -671,16 +670,14 @@ static BlockAIOCB *read_quorum_children(QuorumAIOCB *acb)
static BlockAIOCB *read_fifo_child(QuorumAIOCB *acb)
{
BDRVQuorumState *s = acb->common.bs->opaque;
+ int n = acb->children_read++;
- acb->qcrs[acb->child_iter].buf =
- qemu_blockalign(s->children[acb->child_iter]->bs, acb->qiov->size);
- qemu_iovec_init(&acb->qcrs[acb->child_iter].qiov, acb->qiov->niov);
- qemu_iovec_clone(&acb->qcrs[acb->child_iter].qiov, acb->qiov,
- acb->qcrs[acb->child_iter].buf);
- acb->qcrs[acb->child_iter].aiocb =
- bdrv_aio_readv(s->children[acb->child_iter], acb->sector_num,
- &acb->qcrs[acb->child_iter].qiov, acb->nb_sectors,
- quorum_aio_cb, &acb->qcrs[acb->child_iter]);
+ acb->qcrs[n].buf = qemu_blockalign(s->children[n]->bs, acb->qiov->size);
+ qemu_iovec_init(&acb->qcrs[n].qiov, acb->qiov->niov);
+ qemu_iovec_clone(&acb->qcrs[n].qiov, acb->qiov, acb->qcrs[n].buf);
+ acb->qcrs[n].aiocb = bdrv_aio_readv(s->children[n], acb->sector_num,
+ &acb->qcrs[n].qiov, acb->nb_sectors,
+ quorum_aio_cb, &acb->qcrs[n]);
return &acb->common;
}
@@ -696,13 +693,12 @@ static BlockAIOCB *quorum_aio_readv(BlockDriverState *bs,
QuorumAIOCB *acb = quorum_aio_get(s, bs, qiov, sector_num,
nb_sectors, cb, opaque);
acb->is_read = true;
+ acb->children_read = 0;
if (s->read_pattern == QUORUM_READ_PATTERN_QUORUM) {
- acb->child_iter = s->num_children - 1;
return read_quorum_children(acb);
}
- acb->child_iter = 0;
return read_fifo_child(acb);
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 12/23] quorum: do not allocate multiple iovecs for FIFO strategy
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (10 preceding siblings ...)
2016-10-24 17:01 ` [Qemu-devel] [PULL 11/23] quorum: change child_iter to children_read Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 13/23] block: Hide HBitmap in block dirty bitmap interface Kevin Wolf
` (11 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Paolo Bonzini <pbonzini@redhat.com>
In FIFO mode there are no parallel reads, hence there is no need to
allocate separate buffers and clone the iovecs.
The two cases of quorum_aio_cb are now even more different, and
most of quorum_aio_finalize is only needed in one of them, so split
them in separate functions.
Signed-off-by: Paolo Bonzini <pbonzini@redhat.com>
Message-id: 1475685327-22767-3-git-send-email-pbonzini@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Alberto Garcia <berto@igalia.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/quorum.c | 79 +++++++++++++++++++++++++++++-----------------------------
1 file changed, 40 insertions(+), 39 deletions(-)
diff --git a/block/quorum.c b/block/quorum.c
index 435296e..d122299 100644
--- a/block/quorum.c
+++ b/block/quorum.c
@@ -156,21 +156,7 @@ static AIOCBInfo quorum_aiocb_info = {
static void quorum_aio_finalize(QuorumAIOCB *acb)
{
- int i, ret = 0;
-
- if (acb->vote_ret) {
- ret = acb->vote_ret;
- }
-
- acb->common.cb(acb->common.opaque, ret);
-
- if (acb->is_read) {
- for (i = 0; i < acb->children_read; i++) {
- qemu_vfree(acb->qcrs[i].buf);
- qemu_iovec_destroy(&acb->qcrs[i].qiov);
- }
- }
-
+ acb->common.cb(acb->common.opaque, acb->vote_ret);
g_free(acb->qcrs);
qemu_aio_unref(acb);
}
@@ -282,38 +268,52 @@ static void quorum_copy_qiov(QEMUIOVector *dest, QEMUIOVector *source)
}
}
-static void quorum_aio_cb(void *opaque, int ret)
+static void quorum_report_bad_acb(QuorumChildRequest *sacb, int ret)
+{
+ QuorumAIOCB *acb = sacb->parent;
+ QuorumOpType type = acb->is_read ? QUORUM_OP_TYPE_READ : QUORUM_OP_TYPE_WRITE;
+ quorum_report_bad(type, acb->sector_num, acb->nb_sectors,
+ sacb->aiocb->bs->node_name, ret);
+}
+
+static void quorum_fifo_aio_cb(void *opaque, int ret)
{
QuorumChildRequest *sacb = opaque;
QuorumAIOCB *acb = sacb->parent;
BDRVQuorumState *s = acb->common.bs->opaque;
- bool rewrite = false;
- if (ret == 0) {
- acb->success_count++;
- } else {
- QuorumOpType type;
- type = acb->is_read ? QUORUM_OP_TYPE_READ : QUORUM_OP_TYPE_WRITE;
- quorum_report_bad(type, acb->sector_num, acb->nb_sectors,
- sacb->aiocb->bs->node_name, ret);
- }
+ assert(acb->is_read && s->read_pattern == QUORUM_READ_PATTERN_FIFO);
+
+ if (ret < 0) {
+ quorum_report_bad_acb(sacb, ret);
- if (acb->is_read && s->read_pattern == QUORUM_READ_PATTERN_FIFO) {
/* We try to read next child in FIFO order if we fail to read */
- if (ret < 0 && acb->children_read < s->num_children) {
+ if (acb->children_read < s->num_children) {
read_fifo_child(acb);
return;
}
-
- if (ret == 0) {
- quorum_copy_qiov(acb->qiov, &acb->qcrs[acb->children_read - 1].qiov);
- }
- acb->vote_ret = ret;
- quorum_aio_finalize(acb);
- return;
}
+ acb->vote_ret = ret;
+
+ /* FIXME: rewrite failed children if acb->children_read > 1? */
+ quorum_aio_finalize(acb);
+}
+
+static void quorum_aio_cb(void *opaque, int ret)
+{
+ QuorumChildRequest *sacb = opaque;
+ QuorumAIOCB *acb = sacb->parent;
+ BDRVQuorumState *s = acb->common.bs->opaque;
+ bool rewrite = false;
+ int i;
+
sacb->ret = ret;
+ if (ret == 0) {
+ acb->success_count++;
+ } else {
+ quorum_report_bad_acb(sacb, ret);
+ }
acb->count++;
assert(acb->count <= s->num_children);
assert(acb->success_count <= s->num_children);
@@ -324,6 +324,10 @@ static void quorum_aio_cb(void *opaque, int ret)
/* Do the vote on read */
if (acb->is_read) {
rewrite = quorum_vote(acb);
+ for (i = 0; i < s->num_children; i++) {
+ qemu_vfree(acb->qcrs[i].buf);
+ qemu_iovec_destroy(&acb->qcrs[i].qiov);
+ }
} else {
quorum_has_too_much_io_failed(acb);
}
@@ -672,12 +676,9 @@ static BlockAIOCB *read_fifo_child(QuorumAIOCB *acb)
BDRVQuorumState *s = acb->common.bs->opaque;
int n = acb->children_read++;
- acb->qcrs[n].buf = qemu_blockalign(s->children[n]->bs, acb->qiov->size);
- qemu_iovec_init(&acb->qcrs[n].qiov, acb->qiov->niov);
- qemu_iovec_clone(&acb->qcrs[n].qiov, acb->qiov, acb->qcrs[n].buf);
acb->qcrs[n].aiocb = bdrv_aio_readv(s->children[n], acb->sector_num,
- &acb->qcrs[n].qiov, acb->nb_sectors,
- quorum_aio_cb, &acb->qcrs[n]);
+ acb->qiov, acb->nb_sectors,
+ quorum_fifo_aio_cb, &acb->qcrs[n]);
return &acb->common;
}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 13/23] block: Hide HBitmap in block dirty bitmap interface
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (11 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 12/23] quorum: do not allocate multiple iovecs for FIFO strategy Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 14/23] HBitmap: Introduce "meta" bitmap to track bit changes Kevin Wolf
` (10 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
HBitmap is an implementation detail of block dirty bitmap that should be hidden
from users. Introduce a BdrvDirtyBitmapIter to encapsulate the underlying
HBitmapIter.
A small difference in the interface is, before, an HBitmapIter is initialized
in place, now the new BdrvDirtyBitmapIter must be dynamically allocated because
the structure definition is in block/dirty-bitmap.c.
Two current users are converted too.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-2-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/backup.c | 14 ++++++++------
block/dirty-bitmap.c | 39 +++++++++++++++++++++++++++++++++------
block/mirror.c | 24 +++++++++++++-----------
include/block/dirty-bitmap.h | 7 +++++--
include/qemu/typedefs.h | 1 +
5 files changed, 60 insertions(+), 25 deletions(-)
diff --git a/block/backup.c b/block/backup.c
index 582bd0f..02dbe48 100644
--- a/block/backup.c
+++ b/block/backup.c
@@ -372,14 +372,14 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
int64_t end;
int64_t last_cluster = -1;
int64_t sectors_per_cluster = cluster_size_sectors(job);
- HBitmapIter hbi;
+ BdrvDirtyBitmapIter *dbi;
granularity = bdrv_dirty_bitmap_granularity(job->sync_bitmap);
clusters_per_iter = MAX((granularity / job->cluster_size), 1);
- bdrv_dirty_iter_init(job->sync_bitmap, &hbi);
+ dbi = bdrv_dirty_iter_new(job->sync_bitmap, 0);
/* Find the next dirty sector(s) */
- while ((sector = hbitmap_iter_next(&hbi)) != -1) {
+ while ((sector = bdrv_dirty_iter_next(dbi)) != -1) {
cluster = sector / sectors_per_cluster;
/* Fake progress updates for any clusters we skipped */
@@ -391,7 +391,7 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
for (end = cluster + clusters_per_iter; cluster < end; cluster++) {
do {
if (yield_and_check(job)) {
- return ret;
+ goto out;
}
ret = backup_do_cow(job, cluster * sectors_per_cluster,
sectors_per_cluster, &error_is_read,
@@ -399,7 +399,7 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
if ((ret < 0) &&
backup_error_action(job, error_is_read, -ret) ==
BLOCK_ERROR_ACTION_REPORT) {
- return ret;
+ goto out;
}
} while (ret < 0);
}
@@ -407,7 +407,7 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
/* If the bitmap granularity is smaller than the backup granularity,
* we need to advance the iterator pointer to the next cluster. */
if (granularity < job->cluster_size) {
- bdrv_set_dirty_iter(&hbi, cluster * sectors_per_cluster);
+ bdrv_set_dirty_iter(dbi, cluster * sectors_per_cluster);
}
last_cluster = cluster - 1;
@@ -419,6 +419,8 @@ static int coroutine_fn backup_run_incremental(BackupBlockJob *job)
job->common.offset += ((end - last_cluster - 1) * job->cluster_size);
}
+out:
+ bdrv_dirty_iter_free(dbi);
return ret;
}
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index f2bfdcf..c572dfa 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -42,9 +42,15 @@ struct BdrvDirtyBitmap {
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap (Number of sectors) */
bool disabled; /* Bitmap is read-only */
+ int active_iterators; /* How many iterators are active */
QLIST_ENTRY(BdrvDirtyBitmap) list;
};
+struct BdrvDirtyBitmapIter {
+ HBitmapIter hbi;
+ BdrvDirtyBitmap *bitmap;
+};
+
BdrvDirtyBitmap *bdrv_find_dirty_bitmap(BlockDriverState *bs, const char *name)
{
BdrvDirtyBitmap *bm;
@@ -212,6 +218,7 @@ void bdrv_dirty_bitmap_truncate(BlockDriverState *bs)
QLIST_FOREACH(bitmap, &bs->dirty_bitmaps, list) {
assert(!bdrv_dirty_bitmap_frozen(bitmap));
+ assert(!bitmap->active_iterators);
hbitmap_truncate(bitmap->bitmap, size);
bitmap->size = size;
}
@@ -224,6 +231,7 @@ static void bdrv_do_release_matching_dirty_bitmap(BlockDriverState *bs,
BdrvDirtyBitmap *bm, *next;
QLIST_FOREACH_SAFE(bm, &bs->dirty_bitmaps, list, next) {
if ((!bitmap || bm == bitmap) && (!only_named || bm->name)) {
+ assert(!bm->active_iterators);
assert(!bdrv_dirty_bitmap_frozen(bm));
QLIST_REMOVE(bm, list);
hbitmap_free(bm->bitmap);
@@ -320,9 +328,29 @@ uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap)
return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
}
-void bdrv_dirty_iter_init(BdrvDirtyBitmap *bitmap, HBitmapIter *hbi)
+BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap,
+ uint64_t first_sector)
+{
+ BdrvDirtyBitmapIter *iter = g_new(BdrvDirtyBitmapIter, 1);
+ hbitmap_iter_init(&iter->hbi, bitmap->bitmap, first_sector);
+ iter->bitmap = bitmap;
+ bitmap->active_iterators++;
+ return iter;
+}
+
+void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter)
+{
+ if (!iter) {
+ return;
+ }
+ assert(iter->bitmap->active_iterators > 0);
+ iter->bitmap->active_iterators--;
+ g_free(iter);
+}
+
+int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter)
{
- hbitmap_iter_init(hbi, bitmap->bitmap, 0);
+ return hbitmap_iter_next(&iter->hbi);
}
void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
@@ -373,12 +401,11 @@ void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
}
/**
- * Advance an HBitmapIter to an arbitrary offset.
+ * Advance a BdrvDirtyBitmapIter to an arbitrary offset.
*/
-void bdrv_set_dirty_iter(HBitmapIter *hbi, int64_t offset)
+void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *iter, int64_t sector_num)
{
- assert(hbi->hb);
- hbitmap_iter_init(hbi, hbi->hb, offset);
+ hbitmap_iter_init(&iter->hbi, iter->hbi.hb, sector_num);
}
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
diff --git a/block/mirror.c b/block/mirror.c
index f9d1fec..a433e68 100644
--- a/block/mirror.c
+++ b/block/mirror.c
@@ -55,7 +55,7 @@ typedef struct MirrorBlockJob {
int64_t bdev_length;
unsigned long *cow_bitmap;
BdrvDirtyBitmap *dirty_bitmap;
- HBitmapIter hbi;
+ BdrvDirtyBitmapIter *dbi;
uint8_t *buf;
QSIMPLEQ_HEAD(, MirrorBuffer) buf_free;
int buf_free_count;
@@ -330,10 +330,10 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
int max_io_sectors = MAX((s->buf_size >> BDRV_SECTOR_BITS) / MAX_IN_FLIGHT,
MAX_IO_SECTORS);
- sector_num = hbitmap_iter_next(&s->hbi);
+ sector_num = bdrv_dirty_iter_next(s->dbi);
if (sector_num < 0) {
- bdrv_dirty_iter_init(s->dirty_bitmap, &s->hbi);
- sector_num = hbitmap_iter_next(&s->hbi);
+ bdrv_set_dirty_iter(s->dbi, 0);
+ sector_num = bdrv_dirty_iter_next(s->dbi);
trace_mirror_restart_iter(s, bdrv_get_dirty_count(s->dirty_bitmap));
assert(sector_num >= 0);
}
@@ -349,7 +349,7 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
/* Find the number of consective dirty chunks following the first dirty
* one, and wait for in flight requests in them. */
while (nb_chunks * sectors_per_chunk < (s->buf_size >> BDRV_SECTOR_BITS)) {
- int64_t hbitmap_next;
+ int64_t next_dirty;
int64_t next_sector = sector_num + nb_chunks * sectors_per_chunk;
int64_t next_chunk = next_sector / sectors_per_chunk;
if (next_sector >= end ||
@@ -360,13 +360,13 @@ static uint64_t coroutine_fn mirror_iteration(MirrorBlockJob *s)
break;
}
- hbitmap_next = hbitmap_iter_next(&s->hbi);
- if (hbitmap_next > next_sector || hbitmap_next < 0) {
+ next_dirty = bdrv_dirty_iter_next(s->dbi);
+ if (next_dirty > next_sector || next_dirty < 0) {
/* The bitmap iterator's cache is stale, refresh it */
- bdrv_set_dirty_iter(&s->hbi, next_sector);
- hbitmap_next = hbitmap_iter_next(&s->hbi);
+ bdrv_set_dirty_iter(s->dbi, next_sector);
+ next_dirty = bdrv_dirty_iter_next(s->dbi);
}
- assert(hbitmap_next == next_sector);
+ assert(next_dirty == next_sector);
nb_chunks++;
}
@@ -679,7 +679,8 @@ static void coroutine_fn mirror_run(void *opaque)
}
}
- bdrv_dirty_iter_init(s->dirty_bitmap, &s->hbi);
+ assert(!s->dbi);
+ s->dbi = bdrv_dirty_iter_new(s->dirty_bitmap, 0);
for (;;) {
uint64_t delay_ns = 0;
int64_t cnt, delta;
@@ -793,6 +794,7 @@ immediate_exit:
qemu_vfree(s->buf);
g_free(s->cow_bitmap);
g_free(s->in_flight_bitmap);
+ bdrv_dirty_iter_free(s->dbi);
bdrv_release_dirty_bitmap(bs, s->dirty_bitmap);
data = g_malloc(sizeof(*data));
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index ee3388f..0ef927d 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -36,8 +36,11 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int64_t nr_sectors);
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int64_t nr_sectors);
-void bdrv_dirty_iter_init(BdrvDirtyBitmap *bitmap, struct HBitmapIter *hbi);
-void bdrv_set_dirty_iter(struct HBitmapIter *hbi, int64_t offset);
+BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap,
+ uint64_t first_sector);
+void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter);
+int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter);
+void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_bitmap_truncate(BlockDriverState *bs);
diff --git a/include/qemu/typedefs.h b/include/qemu/typedefs.h
index b113fcf..1b8c30a 100644
--- a/include/qemu/typedefs.h
+++ b/include/qemu/typedefs.h
@@ -11,6 +11,7 @@ typedef struct AioContext AioContext;
typedef struct AllwinnerAHCIState AllwinnerAHCIState;
typedef struct AudioState AudioState;
typedef struct BdrvDirtyBitmap BdrvDirtyBitmap;
+typedef struct BdrvDirtyBitmapIter BdrvDirtyBitmapIter;
typedef struct BlockBackend BlockBackend;
typedef struct BlockBackendRootState BlockBackendRootState;
typedef struct BlockDriverState BlockDriverState;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 14/23] HBitmap: Introduce "meta" bitmap to track bit changes
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (12 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 13/23] block: Hide HBitmap in block dirty bitmap interface Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 15/23] tests: Add test code for meta bitmap Kevin Wolf
` (9 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
Upon each bit toggle, the corresponding bit in the meta bitmap will be
set.
Signed-off-by: Fam Zheng <famz@redhat.com>
[Amended text inline. --js]
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-3-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/qemu/hbitmap.h | 21 +++++++++++++++
util/hbitmap.c | 69 +++++++++++++++++++++++++++++++++++++++-----------
2 files changed, 75 insertions(+), 15 deletions(-)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 8ab721e..1725919 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -178,6 +178,27 @@ void hbitmap_iter_init(HBitmapIter *hbi, const HBitmap *hb, uint64_t first);
*/
unsigned long hbitmap_iter_skip_words(HBitmapIter *hbi);
+/* hbitmap_create_meta:
+ * Create a "meta" hbitmap to track dirtiness of the bits in this HBitmap.
+ * The caller owns the created bitmap and must call hbitmap_free_meta(hb) to
+ * free it.
+ *
+ * Currently, we only guarantee that if a bit in the hbitmap is changed it
+ * will be reflected in the meta bitmap, but we do not yet guarantee the
+ * opposite.
+ *
+ * @hb: The HBitmap to operate on.
+ * @chunk_size: How many bits in @hb does one bit in the meta track.
+ */
+HBitmap *hbitmap_create_meta(HBitmap *hb, int chunk_size);
+
+/* hbitmap_free_meta:
+ * Free the meta bitmap of @hb.
+ *
+ * @hb: The HBitmap whose meta bitmap should be freed.
+ */
+void hbitmap_free_meta(HBitmap *hb);
+
/**
* hbitmap_iter_next:
* @hbi: HBitmapIter to operate on.
diff --git a/util/hbitmap.c b/util/hbitmap.c
index 99fd2ba..f303975 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -78,6 +78,9 @@ struct HBitmap {
*/
int granularity;
+ /* A meta dirty bitmap to track the dirtiness of bits in this HBitmap. */
+ HBitmap *meta;
+
/* A number of progressively less coarse bitmaps (i.e. level 0 is the
* coarsest). Each bit in level N represents a word in level N+1 that
* has a set bit, except the last level where each bit represents the
@@ -209,25 +212,27 @@ static uint64_t hb_count_between(HBitmap *hb, uint64_t start, uint64_t last)
}
/* Setting starts at the last layer and propagates up if an element
- * changes from zero to non-zero.
+ * changes.
*/
static inline bool hb_set_elem(unsigned long *elem, uint64_t start, uint64_t last)
{
unsigned long mask;
- bool changed;
+ unsigned long old;
assert((last >> BITS_PER_LEVEL) == (start >> BITS_PER_LEVEL));
assert(start <= last);
mask = 2UL << (last & (BITS_PER_LONG - 1));
mask -= 1UL << (start & (BITS_PER_LONG - 1));
- changed = (*elem == 0);
+ old = *elem;
*elem |= mask;
- return changed;
+ return old != *elem;
}
-/* The recursive workhorse (the depth is limited to HBITMAP_LEVELS)... */
-static void hb_set_between(HBitmap *hb, int level, uint64_t start, uint64_t last)
+/* The recursive workhorse (the depth is limited to HBITMAP_LEVELS)...
+ * Returns true if at least one bit is changed. */
+static bool hb_set_between(HBitmap *hb, int level, uint64_t start,
+ uint64_t last)
{
size_t pos = start >> BITS_PER_LEVEL;
size_t lastpos = last >> BITS_PER_LEVEL;
@@ -256,23 +261,28 @@ static void hb_set_between(HBitmap *hb, int level, uint64_t start, uint64_t last
if (level > 0 && changed) {
hb_set_between(hb, level - 1, pos, lastpos);
}
+ return changed;
}
void hbitmap_set(HBitmap *hb, uint64_t start, uint64_t count)
{
/* Compute range in the last layer. */
+ uint64_t first, n;
uint64_t last = start + count - 1;
trace_hbitmap_set(hb, start, count,
start >> hb->granularity, last >> hb->granularity);
- start >>= hb->granularity;
+ first = start >> hb->granularity;
last >>= hb->granularity;
- count = last - start + 1;
assert(last < hb->size);
+ n = last - first + 1;
- hb->count += count - hb_count_between(hb, start, last);
- hb_set_between(hb, HBITMAP_LEVELS - 1, start, last);
+ hb->count += n - hb_count_between(hb, first, last);
+ if (hb_set_between(hb, HBITMAP_LEVELS - 1, first, last) &&
+ hb->meta) {
+ hbitmap_set(hb->meta, start, count);
+ }
}
/* Resetting works the other way round: propagate up if the new
@@ -293,8 +303,10 @@ static inline bool hb_reset_elem(unsigned long *elem, uint64_t start, uint64_t l
return blanked;
}
-/* The recursive workhorse (the depth is limited to HBITMAP_LEVELS)... */
-static void hb_reset_between(HBitmap *hb, int level, uint64_t start, uint64_t last)
+/* The recursive workhorse (the depth is limited to HBITMAP_LEVELS)...
+ * Returns true if at least one bit is changed. */
+static bool hb_reset_between(HBitmap *hb, int level, uint64_t start,
+ uint64_t last)
{
size_t pos = start >> BITS_PER_LEVEL;
size_t lastpos = last >> BITS_PER_LEVEL;
@@ -337,22 +349,29 @@ static void hb_reset_between(HBitmap *hb, int level, uint64_t start, uint64_t la
if (level > 0 && changed) {
hb_reset_between(hb, level - 1, pos, lastpos);
}
+
+ return changed;
+
}
void hbitmap_reset(HBitmap *hb, uint64_t start, uint64_t count)
{
/* Compute range in the last layer. */
+ uint64_t first;
uint64_t last = start + count - 1;
trace_hbitmap_reset(hb, start, count,
start >> hb->granularity, last >> hb->granularity);
- start >>= hb->granularity;
+ first = start >> hb->granularity;
last >>= hb->granularity;
assert(last < hb->size);
- hb->count -= hb_count_between(hb, start, last);
- hb_reset_between(hb, HBITMAP_LEVELS - 1, start, last);
+ hb->count -= hb_count_between(hb, first, last);
+ if (hb_reset_between(hb, HBITMAP_LEVELS - 1, first, last) &&
+ hb->meta) {
+ hbitmap_set(hb->meta, start, count);
+ }
}
void hbitmap_reset_all(HBitmap *hb)
@@ -381,6 +400,7 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
void hbitmap_free(HBitmap *hb)
{
unsigned i;
+ assert(!hb->meta);
for (i = HBITMAP_LEVELS; i-- > 0; ) {
g_free(hb->levels[i]);
}
@@ -458,6 +478,9 @@ void hbitmap_truncate(HBitmap *hb, uint64_t size)
(size - old) * sizeof(*hb->levels[i]));
}
}
+ if (hb->meta) {
+ hbitmap_truncate(hb->meta, hb->size << hb->granularity);
+ }
}
@@ -493,3 +516,19 @@ bool hbitmap_merge(HBitmap *a, const HBitmap *b)
return true;
}
+
+HBitmap *hbitmap_create_meta(HBitmap *hb, int chunk_size)
+{
+ assert(!(chunk_size & (chunk_size - 1)));
+ assert(!hb->meta);
+ hb->meta = hbitmap_alloc(hb->size << hb->granularity,
+ hb->granularity + ctz32(chunk_size));
+ return hb->meta;
+}
+
+void hbitmap_free_meta(HBitmap *hb)
+{
+ assert(hb->meta);
+ hbitmap_free(hb->meta);
+ hb->meta = NULL;
+}
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 15/23] tests: Add test code for meta bitmap
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (13 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 14/23] HBitmap: Introduce "meta" bitmap to track bit changes Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 16/23] block: Support meta dirty bitmap Kevin Wolf
` (8 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-4-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/test-hbitmap.c | 116 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 116 insertions(+)
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index c0e9895..e3abde1 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -11,6 +11,7 @@
#include "qemu/osdep.h"
#include "qemu/hbitmap.h"
+#include "block/block.h"
#define LOG_BITS_PER_LONG (BITS_PER_LONG == 32 ? 5 : 6)
@@ -20,6 +21,7 @@
typedef struct TestHBitmapData {
HBitmap *hb;
+ HBitmap *meta;
unsigned long *bits;
size_t size;
size_t old_size;
@@ -91,6 +93,14 @@ static void hbitmap_test_init(TestHBitmapData *data,
}
}
+static void hbitmap_test_init_meta(TestHBitmapData *data,
+ uint64_t size, int granularity,
+ int meta_chunk)
+{
+ hbitmap_test_init(data, size, granularity);
+ data->meta = hbitmap_create_meta(data->hb, meta_chunk);
+}
+
static inline size_t hbitmap_test_array_size(size_t bits)
{
size_t n = DIV_ROUND_UP(bits, BITS_PER_LONG);
@@ -133,6 +143,9 @@ static void hbitmap_test_teardown(TestHBitmapData *data,
const void *unused)
{
if (data->hb) {
+ if (data->meta) {
+ hbitmap_free_meta(data->hb);
+ }
hbitmap_free(data->hb);
data->hb = NULL;
}
@@ -634,6 +647,103 @@ static void test_hbitmap_truncate_shrink_large(TestHBitmapData *data,
hbitmap_test_truncate(data, size, -diff, 0);
}
+static void hbitmap_check_meta(TestHBitmapData *data,
+ int64_t start, int count)
+{
+ int64_t i;
+
+ for (i = 0; i < data->size; i++) {
+ if (i >= start && i < start + count) {
+ g_assert(hbitmap_get(data->meta, i));
+ } else {
+ g_assert(!hbitmap_get(data->meta, i));
+ }
+ }
+}
+
+static void hbitmap_test_meta(TestHBitmapData *data,
+ int64_t start, int count,
+ int64_t check_start, int check_count)
+{
+ hbitmap_reset_all(data->hb);
+ hbitmap_reset_all(data->meta);
+
+ /* Test "unset" -> "unset" will not update meta. */
+ hbitmap_reset(data->hb, start, count);
+ hbitmap_check_meta(data, 0, 0);
+
+ /* Test "unset" -> "set" will update meta */
+ hbitmap_set(data->hb, start, count);
+ hbitmap_check_meta(data, check_start, check_count);
+
+ /* Test "set" -> "set" will not update meta */
+ hbitmap_reset_all(data->meta);
+ hbitmap_set(data->hb, start, count);
+ hbitmap_check_meta(data, 0, 0);
+
+ /* Test "set" -> "unset" will update meta */
+ hbitmap_reset_all(data->meta);
+ hbitmap_reset(data->hb, start, count);
+ hbitmap_check_meta(data, check_start, check_count);
+}
+
+static void hbitmap_test_meta_do(TestHBitmapData *data, int chunk_size)
+{
+ uint64_t size = chunk_size * 100;
+ hbitmap_test_init_meta(data, size, 0, chunk_size);
+
+ hbitmap_test_meta(data, 0, 1, 0, chunk_size);
+ hbitmap_test_meta(data, 0, chunk_size, 0, chunk_size);
+ hbitmap_test_meta(data, chunk_size - 1, 1, 0, chunk_size);
+ hbitmap_test_meta(data, chunk_size - 1, 2, 0, chunk_size * 2);
+ hbitmap_test_meta(data, chunk_size - 1, chunk_size + 1, 0, chunk_size * 2);
+ hbitmap_test_meta(data, chunk_size - 1, chunk_size + 2, 0, chunk_size * 3);
+ hbitmap_test_meta(data, 7 * chunk_size - 1, chunk_size + 2,
+ 6 * chunk_size, chunk_size * 3);
+ hbitmap_test_meta(data, size - 1, 1, size - chunk_size, chunk_size);
+ hbitmap_test_meta(data, 0, size, 0, size);
+}
+
+static void test_hbitmap_meta_byte(TestHBitmapData *data, const void *unused)
+{
+ hbitmap_test_meta_do(data, BITS_PER_BYTE);
+}
+
+static void test_hbitmap_meta_word(TestHBitmapData *data, const void *unused)
+{
+ hbitmap_test_meta_do(data, BITS_PER_LONG);
+}
+
+static void test_hbitmap_meta_sector(TestHBitmapData *data, const void *unused)
+{
+ hbitmap_test_meta_do(data, BDRV_SECTOR_SIZE * BITS_PER_BYTE);
+}
+
+/**
+ * Create an HBitmap and test set/unset.
+ */
+static void test_hbitmap_meta_one(TestHBitmapData *data, const void *unused)
+{
+ int i;
+ int64_t offsets[] = {
+ 0, 1, L1 - 1, L1, L1 + 1, L2 - 1, L2, L2 + 1, L3 - 1, L3, L3 + 1
+ };
+
+ hbitmap_test_init_meta(data, L3 * 2, 0, 1);
+ for (i = 0; i < ARRAY_SIZE(offsets); i++) {
+ hbitmap_test_meta(data, offsets[i], 1, offsets[i], 1);
+ hbitmap_test_meta(data, offsets[i], L1, offsets[i], L1);
+ hbitmap_test_meta(data, offsets[i], L2, offsets[i], L2);
+ }
+}
+
+static void test_hbitmap_meta_zero(TestHBitmapData *data, const void *unused)
+{
+ hbitmap_test_init_meta(data, 0, 0, 1);
+
+ hbitmap_check_meta(data, 0, 0);
+}
+
static void hbitmap_test_add(const char *testpath,
void (*test_func)(TestHBitmapData *data, const void *user_data))
{
@@ -683,6 +793,12 @@ int main(int argc, char **argv)
test_hbitmap_truncate_grow_large);
hbitmap_test_add("/hbitmap/truncate/shrink/large",
test_hbitmap_truncate_shrink_large);
+
+ hbitmap_test_add("/hbitmap/meta/zero", test_hbitmap_meta_zero);
+ hbitmap_test_add("/hbitmap/meta/one", test_hbitmap_meta_one);
+ hbitmap_test_add("/hbitmap/meta/byte", test_hbitmap_meta_byte);
+ hbitmap_test_add("/hbitmap/meta/word", test_hbitmap_meta_word);
+ hbitmap_test_add("/hbitmap/meta/sector", test_hbitmap_meta_sector);
g_test_run();
return 0;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 16/23] block: Support meta dirty bitmap
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (14 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 15/23] tests: Add test code for meta bitmap Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 17/23] block: Add two dirty bitmap getters Kevin Wolf
` (7 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
The added group of operations enables tracking of the changed bits in
the dirty bitmap.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-5-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/dirty-bitmap.c | 52 ++++++++++++++++++++++++++++++++++++++++++++
include/block/dirty-bitmap.h | 9 ++++++++
2 files changed, 61 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index c572dfa..9c6febb 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -38,6 +38,7 @@
*/
struct BdrvDirtyBitmap {
HBitmap *bitmap; /* Dirty sector bitmap implementation */
+ HBitmap *meta; /* Meta dirty bitmap */
BdrvDirtyBitmap *successor; /* Anonymous child; implies frozen status */
char *name; /* Optional non-empty unique ID */
int64_t size; /* Size of the bitmap (Number of sectors) */
@@ -103,6 +104,56 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
return bitmap;
}
+/* bdrv_create_meta_dirty_bitmap
+ *
+ * Create a meta dirty bitmap that tracks the changes of bits in @bitmap. I.e.
+ * when a dirty status bit in @bitmap is changed (either from reset to set or
+ * the other way around), its respective meta dirty bitmap bit will be marked
+ * dirty as well.
+ *
+ * @bitmap: the block dirty bitmap for which to create a meta dirty bitmap.
+ * @chunk_size: how many bytes of bitmap data does each bit in the meta bitmap
+ * track.
+ */
+void bdrv_create_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap,
+ int chunk_size)
+{
+ assert(!bitmap->meta);
+ bitmap->meta = hbitmap_create_meta(bitmap->bitmap,
+ chunk_size * BITS_PER_BYTE);
+}
+
+void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap)
+{
+ assert(bitmap->meta);
+ hbitmap_free_meta(bitmap->bitmap);
+ bitmap->meta = NULL;
+}
+
+int bdrv_dirty_bitmap_get_meta(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap, int64_t sector,
+ int nb_sectors)
+{
+ uint64_t i;
+ int sectors_per_bit = 1 << hbitmap_granularity(bitmap->meta);
+
+ /* To optimize: we can make hbitmap to internally check the range in a
+ * coarse level, or at least do it word by word. */
+ for (i = sector; i < sector + nb_sectors; i += sectors_per_bit) {
+ if (hbitmap_get(bitmap->meta, i)) {
+ return true;
+ }
+ }
+ return false;
+}
+
+void bdrv_dirty_bitmap_reset_meta(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap, int64_t sector,
+ int nb_sectors)
+{
+ hbitmap_reset(bitmap->meta, sector, nb_sectors);
+}
+
bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap)
{
return bitmap->successor;
@@ -233,6 +284,7 @@ static void bdrv_do_release_matching_dirty_bitmap(BlockDriverState *bs,
if ((!bitmap || bm == bitmap) && (!only_named || bm->name)) {
assert(!bm->active_iterators);
assert(!bdrv_dirty_bitmap_frozen(bm));
+ assert(!bm->meta);
QLIST_REMOVE(bm, list);
hbitmap_free(bm->bitmap);
g_free(bm->name);
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 0ef927d..69c500b 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -8,6 +8,9 @@ BdrvDirtyBitmap *bdrv_create_dirty_bitmap(BlockDriverState *bs,
uint32_t granularity,
const char *name,
Error **errp);
+void bdrv_create_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap,
+ int chunk_size);
+void bdrv_release_meta_dirty_bitmap(BdrvDirtyBitmap *bitmap);
int bdrv_dirty_bitmap_create_successor(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap,
Error **errp);
@@ -36,6 +39,12 @@ void bdrv_set_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int64_t nr_sectors);
void bdrv_reset_dirty_bitmap(BdrvDirtyBitmap *bitmap,
int64_t cur_sector, int64_t nr_sectors);
+int bdrv_dirty_bitmap_get_meta(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap, int64_t sector,
+ int nb_sectors);
+void bdrv_dirty_bitmap_reset_meta(BlockDriverState *bs,
+ BdrvDirtyBitmap *bitmap, int64_t sector,
+ int nb_sectors);
BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap,
uint64_t first_sector);
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 17/23] block: Add two dirty bitmap getters
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (15 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 16/23] block: Support meta dirty bitmap Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 18/23] block: Assert that bdrv_release_dirty_bitmap succeeded Kevin Wolf
` (6 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
For dirty bitmap users to get the size and the name of a
BdrvDirtyBitmap.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-6-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/dirty-bitmap.c | 10 ++++++++++
include/block/dirty-bitmap.h | 2 ++
2 files changed, 12 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 9c6febb..860acc9 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -154,6 +154,16 @@ void bdrv_dirty_bitmap_reset_meta(BlockDriverState *bs,
hbitmap_reset(bitmap->meta, sector, nb_sectors);
}
+int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap)
+{
+ return bitmap->size;
+}
+
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap)
+{
+ return bitmap->name;
+}
+
bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap)
{
return bitmap->successor;
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index 69c500b..c4e7858 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -32,6 +32,8 @@ uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap);
+const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
+int64_t bdrv_dirty_bitmap_size(const BdrvDirtyBitmap *bitmap);
DirtyBitmapStatus bdrv_dirty_bitmap_status(BdrvDirtyBitmap *bitmap);
int bdrv_get_dirty(BlockDriverState *bs, BdrvDirtyBitmap *bitmap,
int64_t sector);
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 18/23] block: Assert that bdrv_release_dirty_bitmap succeeded
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (16 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 17/23] block: Add two dirty bitmap getters Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 19/23] hbitmap: serialization Kevin Wolf
` (5 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
We use a loop over bs->dirty_bitmaps to make sure the caller is
only releasing a bitmap owned by bs. Let's also assert that in this case
the caller is releasing a bitmap that does exist.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-7-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/dirty-bitmap.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 860acc9..31d5296 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -305,6 +305,9 @@ static void bdrv_do_release_matching_dirty_bitmap(BlockDriverState *bs,
}
}
}
+ if (bitmap) {
+ abort();
+ }
}
void bdrv_release_dirty_bitmap(BlockDriverState *bs, BdrvDirtyBitmap *bitmap)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 19/23] hbitmap: serialization
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (17 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 18/23] block: Assert that bdrv_release_dirty_bitmap succeeded Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 20/23] block: BdrvDirtyBitmap serialization interface Kevin Wolf
` (4 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Functions to serialize / deserialize(restore) HBitmap. HBitmap should be
saved to linear sequence of bits independently of endianness and bitmap
array element (unsigned long) size. Therefore Little Endian is chosen.
These functions are appropriate for dirty bitmap migration, restoring
the bitmap in several steps is available. To save performance, every
step writes only the last level of the bitmap. All other levels are
restored by hbitmap_deserialize_finish() as a last step of restoring.
So, HBitmap is inconsistent while restoring.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[Fix left shift operand to 1UL; add "finish" parameter. - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-8-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
include/qemu/hbitmap.h | 79 ++++++++++++++++++++++++++++
util/hbitmap.c | 137 +++++++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 216 insertions(+)
diff --git a/include/qemu/hbitmap.h b/include/qemu/hbitmap.h
index 1725919..eb46475 100644
--- a/include/qemu/hbitmap.h
+++ b/include/qemu/hbitmap.h
@@ -146,6 +146,85 @@ void hbitmap_reset_all(HBitmap *hb);
bool hbitmap_get(const HBitmap *hb, uint64_t item);
/**
+ * hbitmap_serialization_granularity:
+ * @hb: HBitmap to operate on.
+ *
+ * Granularity of serialization chunks, used by other serialization functions.
+ * For every chunk:
+ * 1. Chunk start should be aligned to this granularity.
+ * 2. Chunk size should be aligned too, except for last chunk (for which
+ * start + count == hb->size)
+ */
+uint64_t hbitmap_serialization_granularity(const HBitmap *hb);
+
+/**
+ * hbitmap_serialization_size:
+ * @hb: HBitmap to operate on.
+ * @start: Starting bit
+ * @count: Number of bits
+ *
+ * Return number of bytes hbitmap_(de)serialize_part needs
+ */
+uint64_t hbitmap_serialization_size(const HBitmap *hb,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_serialize_part
+ * @hb: HBitmap to operate on.
+ * @buf: Buffer to store serialized bitmap.
+ * @start: First bit to store.
+ * @count: Number of bits to store.
+ *
+ * Stores HBitmap data corresponding to given region. The format of saved data
+ * is linear sequence of bits, so it can be used by hbitmap_deserialize_part
+ * independently of endianness and size of HBitmap level array elements
+ */
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count);
+
+/**
+ * hbitmap_deserialize_part
+ * @hb: HBitmap to operate on.
+ * @buf: Buffer to restore bitmap data from.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ * @finish: Whether to call hbitmap_deserialize_finish automatically.
+ *
+ * Restores HBitmap data corresponding to given region. The format is the same
+ * as for hbitmap_serialize_part.
+ *
+ * If @finish is false, caller must call hbitmap_serialize_finish before using
+ * the bitmap.
+ */
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count,
+ bool finish);
+
+/**
+ * hbitmap_deserialize_zeroes
+ * @hb: HBitmap to operate on.
+ * @start: First bit to restore.
+ * @count: Number of bits to restore.
+ * @finish: Whether to call hbitmap_deserialize_finish automatically.
+ *
+ * Fills the bitmap with zeroes.
+ *
+ * If @finish is false, caller must call hbitmap_serialize_finish before using
+ * the bitmap.
+ */
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count,
+ bool finish);
+
+/**
+ * hbitmap_deserialize_finish
+ * @hb: HBitmap to operate on.
+ *
+ * Repair HBitmap after calling hbitmap_deserialize_data. Actually, all HBitmap
+ * layers are restored here.
+ */
+void hbitmap_deserialize_finish(HBitmap *hb);
+
+/**
* hbitmap_free:
* @hb: HBitmap to operate on.
*
diff --git a/util/hbitmap.c b/util/hbitmap.c
index f303975..5d1a21c 100644
--- a/util/hbitmap.c
+++ b/util/hbitmap.c
@@ -397,6 +397,143 @@ bool hbitmap_get(const HBitmap *hb, uint64_t item)
return (hb->levels[HBITMAP_LEVELS - 1][pos >> BITS_PER_LEVEL] & bit) != 0;
}
+uint64_t hbitmap_serialization_granularity(const HBitmap *hb)
+{
+ /* Require at least 64 bit granularity to be safe on both 64 bit and 32 bit
+ * hosts. */
+ return 64 << hb->granularity;
+}
+
+/* Start should be aligned to serialization granularity, chunk size should be
+ * aligned to serialization granularity too, except for last chunk.
+ */
+static void serialization_chunk(const HBitmap *hb,
+ uint64_t start, uint64_t count,
+ unsigned long **first_el, uint64_t *el_count)
+{
+ uint64_t last = start + count - 1;
+ uint64_t gran = hbitmap_serialization_granularity(hb);
+
+ assert((start & (gran - 1)) == 0);
+ assert((last >> hb->granularity) < hb->size);
+ if ((last >> hb->granularity) != hb->size - 1) {
+ assert((count & (gran - 1)) == 0);
+ }
+
+ start = (start >> hb->granularity) >> BITS_PER_LEVEL;
+ last = (last >> hb->granularity) >> BITS_PER_LEVEL;
+
+ *first_el = &hb->levels[HBITMAP_LEVELS - 1][start];
+ *el_count = last - start + 1;
+}
+
+uint64_t hbitmap_serialization_size(const HBitmap *hb,
+ uint64_t start, uint64_t count)
+{
+ uint64_t el_count;
+ unsigned long *cur;
+
+ if (!count) {
+ return 0;
+ }
+ serialization_chunk(hb, start, count, &cur, &el_count);
+
+ return el_count * sizeof(unsigned long);
+}
+
+void hbitmap_serialize_part(const HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count)
+{
+ uint64_t el_count;
+ unsigned long *cur, *end;
+
+ if (!count) {
+ return;
+ }
+ serialization_chunk(hb, start, count, &cur, &el_count);
+ end = cur + el_count;
+
+ while (cur != end) {
+ unsigned long el =
+ (BITS_PER_LONG == 32 ? cpu_to_le32(*cur) : cpu_to_le64(*cur));
+
+ memcpy(buf, &el, sizeof(el));
+ buf += sizeof(el);
+ cur++;
+ }
+}
+
+void hbitmap_deserialize_part(HBitmap *hb, uint8_t *buf,
+ uint64_t start, uint64_t count,
+ bool finish)
+{
+ uint64_t el_count;
+ unsigned long *cur, *end;
+
+ if (!count) {
+ return;
+ }
+ serialization_chunk(hb, start, count, &cur, &el_count);
+ end = cur + el_count;
+
+ while (cur != end) {
+ memcpy(cur, buf, sizeof(*cur));
+
+ if (BITS_PER_LONG == 32) {
+ le32_to_cpus((uint32_t *)cur);
+ } else {
+ le64_to_cpus((uint64_t *)cur);
+ }
+
+ buf += sizeof(unsigned long);
+ cur++;
+ }
+ if (finish) {
+ hbitmap_deserialize_finish(hb);
+ }
+}
+
+void hbitmap_deserialize_zeroes(HBitmap *hb, uint64_t start, uint64_t count,
+ bool finish)
+{
+ uint64_t el_count;
+ unsigned long *first;
+
+ if (!count) {
+ return;
+ }
+ serialization_chunk(hb, start, count, &first, &el_count);
+
+ memset(first, 0, el_count * sizeof(unsigned long));
+ if (finish) {
+ hbitmap_deserialize_finish(hb);
+ }
+}
+
+void hbitmap_deserialize_finish(HBitmap *bitmap)
+{
+ int64_t i, size, prev_size;
+ int lev;
+
+ /* restore levels starting from penultimate to zero level, assuming
+ * that the last level is ok */
+ size = MAX((bitmap->size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+ for (lev = HBITMAP_LEVELS - 1; lev-- > 0; ) {
+ prev_size = size;
+ size = MAX((size + BITS_PER_LONG - 1) >> BITS_PER_LEVEL, 1);
+ memset(bitmap->levels[lev], 0, size * sizeof(unsigned long));
+
+ for (i = 0; i < prev_size; ++i) {
+ if (bitmap->levels[lev + 1][i]) {
+ bitmap->levels[lev][i >> BITS_PER_LEVEL] |=
+ 1UL << (i & (BITS_PER_LONG - 1));
+ }
+ }
+ }
+
+ bitmap->levels[0][0] |= 1UL << (BITS_PER_LONG - 1);
+}
+
void hbitmap_free(HBitmap *hb)
{
unsigned i;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 20/23] block: BdrvDirtyBitmap serialization interface
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (18 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 19/23] hbitmap: serialization Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 21/23] tests: Add test code for hbitmap serialization Kevin Wolf
` (3 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Several functions to provide necessary access to BdrvDirtyBitmap for
block-migration.c
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
[Add the "finish" parameters. - Fam]
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-9-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/dirty-bitmap.c | 37 +++++++++++++++++++++++++++++++++++++
include/block/dirty-bitmap.h | 14 ++++++++++++++
2 files changed, 51 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 31d5296..384146b 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -453,6 +453,43 @@ void bdrv_undo_clear_dirty_bitmap(BdrvDirtyBitmap *bitmap, HBitmap *in)
hbitmap_free(tmp);
}
+uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count)
+{
+ return hbitmap_serialization_size(bitmap->bitmap, start, count);
+}
+
+uint64_t bdrv_dirty_bitmap_serialization_align(const BdrvDirtyBitmap *bitmap)
+{
+ return hbitmap_serialization_granularity(bitmap->bitmap);
+}
+
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count)
+{
+ hbitmap_serialize_part(bitmap->bitmap, buf, start, count);
+}
+
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count, bool finish)
+{
+ hbitmap_deserialize_part(bitmap->bitmap, buf, start, count, finish);
+}
+
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count,
+ bool finish)
+{
+ hbitmap_deserialize_zeroes(bitmap->bitmap, start, count, finish);
+}
+
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap)
+{
+ hbitmap_deserialize_finish(bitmap->bitmap);
+}
+
void bdrv_set_dirty(BlockDriverState *bs, int64_t cur_sector,
int64_t nr_sectors)
{
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index c4e7858..efc2965 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -55,4 +55,18 @@ void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_bitmap_truncate(BlockDriverState *bs);
+uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count);
+uint64_t bdrv_dirty_bitmap_serialization_align(const BdrvDirtyBitmap *bitmap);
+void bdrv_dirty_bitmap_serialize_part(const BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count);
+void bdrv_dirty_bitmap_deserialize_part(BdrvDirtyBitmap *bitmap,
+ uint8_t *buf, uint64_t start,
+ uint64_t count, bool finish);
+void bdrv_dirty_bitmap_deserialize_zeroes(BdrvDirtyBitmap *bitmap,
+ uint64_t start, uint64_t count,
+ bool finish);
+void bdrv_dirty_bitmap_deserialize_finish(BdrvDirtyBitmap *bitmap);
+
#endif
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 21/23] tests: Add test code for hbitmap serialization
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (19 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 20/23] block: BdrvDirtyBitmap serialization interface Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 22/23] block: More operations for meta dirty bitmap Kevin Wolf
` (2 subsequent siblings)
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
Signed-off-by: Fam Zheng <famz@redhat.com>
[Fixed minor constant issue. --js]
Signed-off-by: John Snow <jsnow@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-10-git-send-email-jsnow@redhat.com
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/test-hbitmap.c | 156 +++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 156 insertions(+)
diff --git a/tests/test-hbitmap.c b/tests/test-hbitmap.c
index e3abde1..9b7495c 100644
--- a/tests/test-hbitmap.c
+++ b/tests/test-hbitmap.c
@@ -11,6 +11,7 @@
#include "qemu/osdep.h"
#include "qemu/hbitmap.h"
+#include "qemu/bitmap.h"
#include "block/block.h"
#define LOG_BITS_PER_LONG (BITS_PER_LONG == 32 ? 5 : 6)
@@ -737,6 +738,16 @@ static void test_hbitmap_meta_one(TestHBitmapData *data, const void *unused)
}
}
+static void test_hbitmap_serialize_granularity(TestHBitmapData *data,
+ const void *unused)
+{
+ int r;
+
+ hbitmap_test_init(data, L3 * 2, 3);
+ r = hbitmap_serialization_granularity(data->hb);
+ g_assert_cmpint(r, ==, 64 << 3);
+}
+
static void test_hbitmap_meta_zero(TestHBitmapData *data, const void *unused)
{
hbitmap_test_init_meta(data, 0, 0, 1);
@@ -744,6 +755,142 @@ static void test_hbitmap_meta_zero(TestHBitmapData *data, const void *unused)
hbitmap_check_meta(data, 0, 0);
}
+static void hbitmap_test_serialize_range(TestHBitmapData *data,
+ uint8_t *buf, size_t buf_size,
+ uint64_t pos, uint64_t count)
+{
+ size_t i;
+ unsigned long *el = (unsigned long *)buf;
+
+ assert(hbitmap_granularity(data->hb) == 0);
+ hbitmap_reset_all(data->hb);
+ memset(buf, 0, buf_size);
+ if (count) {
+ hbitmap_set(data->hb, pos, count);
+ }
+ hbitmap_serialize_part(data->hb, buf, 0, data->size);
+
+ /* Serialized buffer is inherently LE, convert it back manually to test */
+ for (i = 0; i < buf_size / sizeof(unsigned long); i++) {
+ el[i] = (BITS_PER_LONG == 32 ? le32_to_cpu(el[i]) : le64_to_cpu(el[i]));
+ }
+
+ for (i = 0; i < data->size; i++) {
+ int is_set = test_bit(i, (unsigned long *)buf);
+ if (i >= pos && i < pos + count) {
+ g_assert(is_set);
+ } else {
+ g_assert(!is_set);
+ }
+ }
+
+ /* Re-serialize for deserialization testing */
+ memset(buf, 0, buf_size);
+ hbitmap_serialize_part(data->hb, buf, 0, data->size);
+ hbitmap_reset_all(data->hb);
+ hbitmap_deserialize_part(data->hb, buf, 0, data->size, true);
+
+ for (i = 0; i < data->size; i++) {
+ int is_set = hbitmap_get(data->hb, i);
+ if (i >= pos && i < pos + count) {
+ g_assert(is_set);
+ } else {
+ g_assert(!is_set);
+ }
+ }
+}
+
+static void test_hbitmap_serialize_basic(TestHBitmapData *data,
+ const void *unused)
+{
+ int i, j;
+ size_t buf_size;
+ uint8_t *buf;
+ uint64_t positions[] = { 0, 1, L1 - 1, L1, L2 - 1, L2, L2 + 1, L3 - 1 };
+ int num_positions = sizeof(positions) / sizeof(positions[0]);
+
+ hbitmap_test_init(data, L3, 0);
+ buf_size = hbitmap_serialization_size(data->hb, 0, data->size);
+ buf = g_malloc0(buf_size);
+
+ for (i = 0; i < num_positions; i++) {
+ for (j = 0; j < num_positions; j++) {
+ hbitmap_test_serialize_range(data, buf, buf_size,
+ positions[i],
+ MIN(positions[j], L3 - positions[i]));
+ }
+ }
+
+ g_free(buf);
+}
+
+static void test_hbitmap_serialize_part(TestHBitmapData *data,
+ const void *unused)
+{
+ int i, j, k;
+ size_t buf_size;
+ uint8_t *buf;
+ uint64_t positions[] = { 0, 1, L1 - 1, L1, L2 - 1, L2, L2 + 1, L3 - 1 };
+ int num_positions = sizeof(positions) / sizeof(positions[0]);
+
+ hbitmap_test_init(data, L3, 0);
+ buf_size = L2;
+ buf = g_malloc0(buf_size);
+
+ for (i = 0; i < num_positions; i++) {
+ hbitmap_set(data->hb, positions[i], 1);
+ }
+
+ for (i = 0; i < data->size; i += buf_size) {
+ unsigned long *el = (unsigned long *)buf;
+ hbitmap_serialize_part(data->hb, buf, i, buf_size);
+ for (j = 0; j < buf_size / sizeof(unsigned long); j++) {
+ el[j] = (BITS_PER_LONG == 32 ? le32_to_cpu(el[j]) : le64_to_cpu(el[j]));
+ }
+
+ for (j = 0; j < buf_size; j++) {
+ bool should_set = false;
+ for (k = 0; k < num_positions; k++) {
+ if (positions[k] == j + i) {
+ should_set = true;
+ break;
+ }
+ }
+ g_assert_cmpint(should_set, ==, test_bit(j, (unsigned long *)buf));
+ }
+ }
+
+ g_free(buf);
+}
+
+static void test_hbitmap_serialize_zeroes(TestHBitmapData *data,
+ const void *unused)
+{
+ int i;
+ HBitmapIter iter;
+ int64_t next;
+ uint64_t min_l1 = MAX(L1, 64);
+ uint64_t positions[] = { 0, min_l1, L2, L3 - min_l1};
+ int num_positions = sizeof(positions) / sizeof(positions[0]);
+
+ hbitmap_test_init(data, L3, 0);
+
+ for (i = 0; i < num_positions; i++) {
+ hbitmap_set(data->hb, positions[i], L1);
+ }
+
+ for (i = 0; i < num_positions; i++) {
+ hbitmap_deserialize_zeroes(data->hb, positions[i], min_l1, true);
+ hbitmap_iter_init(&iter, data->hb, 0);
+ next = hbitmap_iter_next(&iter);
+ if (i == num_positions - 1) {
+ g_assert_cmpint(next, ==, -1);
+ } else {
+ g_assert_cmpint(next, ==, positions[i + 1]);
+ }
+ }
+}
+
static void hbitmap_test_add(const char *testpath,
void (*test_func)(TestHBitmapData *data, const void *user_data))
{
@@ -799,6 +946,15 @@ int main(int argc, char **argv)
hbitmap_test_add("/hbitmap/meta/byte", test_hbitmap_meta_byte);
hbitmap_test_add("/hbitmap/meta/word", test_hbitmap_meta_word);
hbitmap_test_add("/hbitmap/meta/sector", test_hbitmap_meta_sector);
+
+ hbitmap_test_add("/hbitmap/serialize/granularity",
+ test_hbitmap_serialize_granularity);
+ hbitmap_test_add("/hbitmap/serialize/basic",
+ test_hbitmap_serialize_basic);
+ hbitmap_test_add("/hbitmap/serialize/part",
+ test_hbitmap_serialize_part);
+ hbitmap_test_add("/hbitmap/serialize/zeroes",
+ test_hbitmap_serialize_zeroes);
g_test_run();
return 0;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 22/23] block: More operations for meta dirty bitmap
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (20 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 21/23] tests: Add test code for hbitmap serialization Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 23/23] block/replication: Clarify 'top-id' parameter usage Kevin Wolf
2016-10-24 18:36 ` [Qemu-devel] [PULL 00/23] Block layer patches Peter Maydell
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Fam Zheng <famz@redhat.com>
Callers can create an iterator of meta bitmap with
bdrv_dirty_meta_iter_new(), then use the bdrv_dirty_iter_* operations on
it. Meta iterators are also counted by bitmap->active_iterators.
Also add a couple of functions to retrieve granularity and count.
Signed-off-by: Fam Zheng <famz@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
Message-id: 1476395910-8697-11-git-send-email-jsnow@redhat.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/dirty-bitmap.c | 19 +++++++++++++++++++
include/block/dirty-bitmap.h | 3 +++
2 files changed, 22 insertions(+)
diff --git a/block/dirty-bitmap.c b/block/dirty-bitmap.c
index 384146b..519737c 100644
--- a/block/dirty-bitmap.c
+++ b/block/dirty-bitmap.c
@@ -393,6 +393,11 @@ uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap)
return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->bitmap);
}
+uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap)
+{
+ return BDRV_SECTOR_SIZE << hbitmap_granularity(bitmap->meta);
+}
+
BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap,
uint64_t first_sector)
{
@@ -403,6 +408,15 @@ BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap,
return iter;
}
+BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap)
+{
+ BdrvDirtyBitmapIter *iter = g_new(BdrvDirtyBitmapIter, 1);
+ hbitmap_iter_init(&iter->hbi, bitmap->meta, 0);
+ iter->bitmap = bitmap;
+ bitmap->active_iterators++;
+ return iter;
+}
+
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter)
{
if (!iter) {
@@ -514,3 +528,8 @@ int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap)
{
return hbitmap_count(bitmap->bitmap);
}
+
+int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap)
+{
+ return hbitmap_count(bitmap->meta);
+}
diff --git a/include/block/dirty-bitmap.h b/include/block/dirty-bitmap.h
index efc2965..9dea14b 100644
--- a/include/block/dirty-bitmap.h
+++ b/include/block/dirty-bitmap.h
@@ -30,6 +30,7 @@ void bdrv_enable_dirty_bitmap(BdrvDirtyBitmap *bitmap);
BlockDirtyInfoList *bdrv_query_dirty_bitmaps(BlockDriverState *bs);
uint32_t bdrv_get_default_bitmap_granularity(BlockDriverState *bs);
uint32_t bdrv_dirty_bitmap_granularity(BdrvDirtyBitmap *bitmap);
+uint32_t bdrv_dirty_bitmap_meta_granularity(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_enabled(BdrvDirtyBitmap *bitmap);
bool bdrv_dirty_bitmap_frozen(BdrvDirtyBitmap *bitmap);
const char *bdrv_dirty_bitmap_name(const BdrvDirtyBitmap *bitmap);
@@ -47,12 +48,14 @@ int bdrv_dirty_bitmap_get_meta(BlockDriverState *bs,
void bdrv_dirty_bitmap_reset_meta(BlockDriverState *bs,
BdrvDirtyBitmap *bitmap, int64_t sector,
int nb_sectors);
+BdrvDirtyBitmapIter *bdrv_dirty_meta_iter_new(BdrvDirtyBitmap *bitmap);
BdrvDirtyBitmapIter *bdrv_dirty_iter_new(BdrvDirtyBitmap *bitmap,
uint64_t first_sector);
void bdrv_dirty_iter_free(BdrvDirtyBitmapIter *iter);
int64_t bdrv_dirty_iter_next(BdrvDirtyBitmapIter *iter);
void bdrv_set_dirty_iter(BdrvDirtyBitmapIter *hbi, int64_t sector_num);
int64_t bdrv_get_dirty_count(BdrvDirtyBitmap *bitmap);
+int64_t bdrv_get_meta_dirty_count(BdrvDirtyBitmap *bitmap);
void bdrv_dirty_bitmap_truncate(BlockDriverState *bs);
uint64_t bdrv_dirty_bitmap_serialization_size(const BdrvDirtyBitmap *bitmap,
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 23/23] block/replication: Clarify 'top-id' parameter usage
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (21 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 22/23] block: More operations for meta dirty bitmap Kevin Wolf
@ 2016-10-24 17:02 ` Kevin Wolf
2016-10-24 18:36 ` [Qemu-devel] [PULL 00/23] Block layer patches Peter Maydell
23 siblings, 0 replies; 34+ messages in thread
From: Kevin Wolf @ 2016-10-24 17:02 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
From: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
The replication driver only supports the 'top-id' parameter for the
secondary side; it must not be supplied for the primary side.
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Changlong Xie <xiecl.fnst@cn.fujitsu.com>
Message-id: 1476247808-15646-1-git-send-email-xiecl.fnst@cn.fujitsu.com
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/replication.c | 5 +++++
qapi/block-core.json | 3 ++-
2 files changed, 7 insertions(+), 1 deletion(-)
diff --git a/block/replication.c b/block/replication.c
index 3bd1cf1..8bbfc8f 100644
--- a/block/replication.c
+++ b/block/replication.c
@@ -101,6 +101,11 @@ static int replication_open(BlockDriverState *bs, QDict *options,
if (!strcmp(mode, "primary")) {
s->mode = REPLICATION_MODE_PRIMARY;
+ top_id = qemu_opt_get(opts, REPLICATION_TOP_ID);
+ if (top_id) {
+ error_setg(&local_err, "The primary side does not support option top-id");
+ goto fail;
+ }
} else if (!strcmp(mode, "secondary")) {
s->mode = REPLICATION_MODE_SECONDARY;
top_id = qemu_opt_get(opts, REPLICATION_TOP_ID);
diff --git a/qapi/block-core.json b/qapi/block-core.json
index c59047b..97b1205 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2197,7 +2197,8 @@
# @mode: the replication mode
#
# @top-id: #optional In secondary mode, node name or device ID of the root
-# node who owns the replication node chain. Ignored in primary mode.
+# node who owns the replication node chain. Must not be given in
+# primary mode.
#
# Since: 2.8
##
--
1.8.3.1
^ permalink raw reply related [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
` (22 preceding siblings ...)
2016-10-24 17:02 ` [Qemu-devel] [PULL 23/23] block/replication: Clarify 'top-id' parameter usage Kevin Wolf
@ 2016-10-24 18:36 ` Peter Maydell
23 siblings, 0 replies; 34+ messages in thread
From: Peter Maydell @ 2016-10-24 18:36 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Qemu-block, QEMU Developers
On 24 October 2016 at 18:01, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit a3ae21ec3fe036f536dc94cad735931777143103:
>
> Merge remote-tracking branch 'remotes/bonzini/tags/for-upstream' into staging (2016-10-24 15:03:09 +0100)
>
> are available in the git repository at:
>
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 25493dc012e7c10dba51ee893b634a1dbfeed126:
>
> Merge remote-tracking branch 'mreitz/tags/pull-block-2016-10-24' into queue-block (2016-10-24 18:02:26 +0200)
>
> ----------------------------------------------------------------
>
> Block layer patches
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2018-10-02 8:06 ` Peter Maydell
@ 2018-10-03 15:46 ` Peter Maydell
0 siblings, 0 replies; 34+ messages in thread
From: Peter Maydell @ 2018-10-03 15:46 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Qemu-block, QEMU Developers, Paolo Bonzini
On 2 October 2018 at 09:06, Peter Maydell <peter.maydell@linaro.org> wrote:
> I still got a hang on OSX on test-bdrv-drain, but I've applied
> this anyway, since hopefully it fixes the other intermittent
> failure and may reduce the likelihood with the test-bdrv-drain.
OSX seems to fail test-bdrv-drain fairly frequently. Here's
a back trace from a debug build. When run under the debugger
it seems to stop with a NULL pointer failure in notifier_list_notify();
when not run under the debugger it seems to hang eating CPU...
/bdrv-drain/iothread/drain_subtree: Process 77283 stopped
* thread #12, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
frame #0: 0x0000000000000000
error: memory read failed for 0x0
Target 1: (test-bdrv-drain) stopped.
(lldb) bt
* thread #12, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000000000000
frame #1: 0x000000010016524f
test-bdrv-drain`notifier_list_notify(list=0x0000700008501e50,
data=0x0000000000000000) at notify.c:40
frame #2: 0x0000000100150c92
test-bdrv-drain`qemu_thread_atexit_run(arg=0x0000000100b24f88) at
qemu-thread-posix.c:473
frame #3: 0x00007fff5a0e1163
libsystem_pthread.dylib`_pthread_tsd_cleanup + 463
frame #4: 0x00007fff5a0e0ee9 libsystem_pthread.dylib`_pthread_exit + 79
frame #5: 0x00007fff5a0df66c libsystem_pthread.dylib`_pthread_body + 351
frame #6: 0x00007fff5a0df50d libsystem_pthread.dylib`_pthread_start + 377
frame #7: 0x00007fff5a0debf9 libsystem_pthread.dylib`thread_start + 13
(lldb) info thread
error: 'info' is not a valid command.
error: Unrecognized command 'info'.
(lldb) thread backtrace all
thread #1, queue = 'com.apple.main-thread'
frame #0: 0x00007fff59f17d82 libsystem_kernel.dylib`__semwait_signal + 10
frame #1: 0x00007fff5a0e3824 libsystem_pthread.dylib`_pthread_join + 626
frame #2: 0x0000000100150f2a
test-bdrv-drain`qemu_thread_join(thread=0x0000000103001058) at
qemu-thread-posix.c:565
frame #3: 0x00000001000f6d70
test-bdrv-drain`iothread_join(iothread=0x0000000103001050) at
iothread.c:62
frame #4: 0x000000010000a9a0
test-bdrv-drain`test_iothread_common(drain_type=BDRV_SUBTREE_DRAIN,
drain_thread=1) at test-bdrv-drain.c:762
frame #5: 0x000000010000789f
test-bdrv-drain`test_iothread_drain_subtree at test-bdrv-drain.c:781
frame #6: 0x00000001003aea47
libglib-2.0.0.dylib`g_test_run_suite_internal + 697
frame #7: 0x00000001003aec0a
libglib-2.0.0.dylib`g_test_run_suite_internal + 1148
frame #8: 0x00000001003aec0a
libglib-2.0.0.dylib`g_test_run_suite_internal + 1148
frame #9: 0x00000001003ae020 libglib-2.0.0.dylib`g_test_run_suite + 121
frame #10: 0x00000001003adf73 libglib-2.0.0.dylib`g_test_run + 17
frame #11: 0x0000000100001dd0 test-bdrv-drain`main(argc=1,
argv=0x00007ffeefbffa70) at test-bdrv-drain.c:1606
frame #12: 0x00007fff59dc7015 libdyld.dylib`start + 1
thread #2
frame #0: 0x00007fff59f17a16 libsystem_kernel.dylib`__psynch_cvwait + 10
frame #1: 0x00007fff5a0e0589
libsystem_pthread.dylib`_pthread_cond_wait + 732
frame #2: 0x0000000100150b5e
test-bdrv-drain`qemu_futex_wait(ev=0x00000001001bbad8, val=4294967295)
at qemu-thread-posix.c:347
frame #3: 0x0000000100150acd
test-bdrv-drain`qemu_event_wait(ev=0x00000001001bbad8) at
qemu-thread-posix.c:442
frame #4: 0x000000010016ca82
test-bdrv-drain`call_rcu_thread(opaque=0x0000000000000000) at
rcu.c:261
frame #5: 0x0000000100150e76
test-bdrv-drain`qemu_thread_start(args=0x0000000100b1dfb0) at
qemu-thread-posix.c:504
frame #6: 0x00007fff5a0df661 libsystem_pthread.dylib`_pthread_body + 340
frame #7: 0x00007fff5a0df50d libsystem_pthread.dylib`_pthread_start + 377
frame #8: 0x00007fff5a0debf9 libsystem_pthread.dylib`thread_start + 13
thread #3
frame #0: 0x00007fff59f1803a libsystem_kernel.dylib`__sigwait + 10
frame #1: 0x00007fff5a0e1ad9 libsystem_pthread.dylib`sigwait + 61
frame #2: 0x000000010014d781
test-bdrv-drain`sigwait_compat(opaque=0x0000000100b027d0) at
compatfd.c:36
frame #3: 0x0000000100150e76
test-bdrv-drain`qemu_thread_start(args=0x0000000100b1e560) at
qemu-thread-posix.c:504
frame #4: 0x00007fff5a0df661 libsystem_pthread.dylib`_pthread_body + 340
frame #5: 0x00007fff5a0df50d libsystem_pthread.dylib`_pthread_start + 377
frame #6: 0x00007fff5a0debf9 libsystem_pthread.dylib`thread_start + 13
* thread #12, stop reason = EXC_BAD_ACCESS (code=1, address=0x0)
* frame #0: 0x0000000000000000
frame #1: 0x000000010016524f
test-bdrv-drain`notifier_list_notify(list=0x0000700008501e50,
data=0x0000000000000000) at notify.c:40
frame #2: 0x0000000100150c92
test-bdrv-drain`qemu_thread_atexit_run(arg=0x0000000100b24f88) at
qemu-thread-posix.c:473
frame #3: 0x00007fff5a0e1163
libsystem_pthread.dylib`_pthread_tsd_cleanup + 463
frame #4: 0x00007fff5a0e0ee9 libsystem_pthread.dylib`_pthread_exit + 79
frame #5: 0x00007fff5a0df66c libsystem_pthread.dylib`_pthread_body + 351
frame #6: 0x00007fff5a0df50d libsystem_pthread.dylib`_pthread_start + 377
frame #7: 0x00007fff5a0debf9 libsystem_pthread.dylib`thread_start + 13
thread #13
frame #0: 0x00007fff59f17cf2 libsystem_kernel.dylib`__select + 10
frame #1: 0x000000010039bb60 libglib-2.0.0.dylib`g_poll + 430
frame #2: 0x0000000100149d7b
test-bdrv-drain`qemu_poll_ns(fds=0x0000000100b25570, nfds=1,
timeout=-1) at qemu-timer.c:337
frame #3: 0x000000010014c609
test-bdrv-drain`aio_poll(ctx=0x0000000100b26330, blocking=true) at
aio-posix.c:645
frame #4: 0x00000001000f700f
test-bdrv-drain`iothread_run(opaque=0x0000000100a03620) at
iothread.c:51
frame #5: 0x0000000100150e76
test-bdrv-drain`qemu_thread_start(args=0x0000000100a05240) at
qemu-thread-posix.c:504
frame #6: 0x00007fff5a0df661 libsystem_pthread.dylib`_pthread_body + 340
frame #7: 0x00007fff5a0df50d libsystem_pthread.dylib`_pthread_start + 377
frame #8: 0x00007fff5a0debf9 libsystem_pthread.dylib`thread_start + 13
As far as I can tell it always fails with
/bdrv-drain/iothread/drain_subtree, but this test
doesn't fail if we just run it alone, so something
earlier in the test is setting it up to go wrong.
I don't understand entirely what's going on with the
union in qemu_thread_atexit_run() (this seems to be
Paolo's code from a few years back), but the pointer
passed to qemu_thread_atexit_run() is a pointer to
zeroed memory:
(lldb) memory read -c 32 arg
0x100a25558: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
0x100a25568: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................
which when interpreted as a list_head means that
the iteration through the list gets a node with
NULLs in all its fields, and we try to call NULL.
thanks
-- PMM
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2018-10-01 17:18 Kevin Wolf
@ 2018-10-02 8:06 ` Peter Maydell
2018-10-03 15:46 ` Peter Maydell
0 siblings, 1 reply; 34+ messages in thread
From: Peter Maydell @ 2018-10-02 8:06 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Qemu-block, QEMU Developers
On 1 October 2018 at 18:18, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit 07f426c35eddd79388a23d11cb278600d7e3831d:
>
> Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20180926' into staging (2018-09-28 18:56:09 +0100)
>
> are available in the git repository at:
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to dd353157942a59c21da07da5ac8749a871f7c3ed:
>
> tests/test-bdrv-drain: Fix too late qemu_event_reset() (2018-10-01 19:13:55 +0200)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - qcow2 cache option default changes (Linux: 32 MB maximum, limited by
> whatever cache size can be made use of with the specific image;
> default cache-clean-interval of 10 minutes)
> - reopen: Allow specifying unchanged child node references, and changing
> a few generic options (discard, detect-zeroes)
> - Fix werror/rerror defaults for -device drive=<node-name>
> - Test case fixes
I still got a hang on OSX on test-bdrv-drain, but I've applied
this anyway, since hopefully it fixes the other intermittent
failure and may reduce the likelihood with the test-bdrv-drain.
thanks
-- PMM
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 00/23] Block layer patches
@ 2018-10-01 17:18 Kevin Wolf
2018-10-02 8:06 ` Peter Maydell
0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2018-10-01 17:18 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, peter.maydell, qemu-devel
The following changes since commit 07f426c35eddd79388a23d11cb278600d7e3831d:
Merge remote-tracking branch 'remotes/rth/tags/pull-tcg-20180926' into staging (2018-09-28 18:56:09 +0100)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to dd353157942a59c21da07da5ac8749a871f7c3ed:
tests/test-bdrv-drain: Fix too late qemu_event_reset() (2018-10-01 19:13:55 +0200)
----------------------------------------------------------------
Block layer patches:
- qcow2 cache option default changes (Linux: 32 MB maximum, limited by
whatever cache size can be made use of with the specific image;
default cache-clean-interval of 10 minutes)
- reopen: Allow specifying unchanged child node references, and changing
a few generic options (discard, detect-zeroes)
- Fix werror/rerror defaults for -device drive=<node-name>
- Test case fixes
----------------------------------------------------------------
Alberto Garcia (9):
qemu-io: Fix writethrough check in reopen
file-posix: x-check-cache-dropped should default to false on reopen
block: Remove child references from bs->{options,explicit_options}
block: Don't look for child references in append_open_options()
block: Allow child references on reopen
block: Forbid trying to change unsupported options during reopen
file-posix: Forbid trying to change unsupported options during reopen
block: Allow changing 'discard' on reopen
block: Allow changing 'detect-zeroes' on reopen
Fam Zheng (1):
file-posix: Include filename in locking error message
Kevin Wolf (3):
block-backend: Set werror/rerror defaults in blk_new()
test-replication: Lock AioContext around blk_unref()
tests/test-bdrv-drain: Fix too late qemu_event_reset()
Leonid Bloch (10):
qcow2: Options' documentation fixes
include: Add a lookup table of sizes
qcow2: Make sizes more humanly readable
qcow2: Avoid duplication in setting the refcount cache size
qcow2: Assign the L2 cache relatively to the image size
qcow2: Increase the default upper limit on the L2 cache size
qcow2: Resize the cache upon image resizing
qcow2: Set the default cache-clean-interval to 10 minutes
qcow2: Explicit number replaced by a constant
qcow2: Fix cache-clean-interval documentation
qapi/block-core.json | 4 +-
docs/qcow2-cache.txt | 59 ++++++++++++--------
block/qcow2.h | 19 ++++---
include/block/block.h | 1 +
include/qemu/units.h | 55 ++++++++++++++++++
block.c | 135 +++++++++++++++++++++++++++++----------------
block/block-backend.c | 3 +
block/file-posix.c | 19 +++++--
block/qcow2.c | 43 +++++++++------
qemu-io-cmds.c | 2 +-
tests/test-bdrv-drain.c | 4 +-
tests/test-replication.c | 11 ++++
qemu-options.hx | 12 ++--
tests/qemu-iotests/067.out | 1 +
tests/qemu-iotests/137 | 8 ++-
tests/qemu-iotests/137.out | 4 +-
tests/qemu-iotests/153.out | 76 ++++++++++++-------------
tests/qemu-iotests/182.out | 2 +-
18 files changed, 307 insertions(+), 151 deletions(-)
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2016-10-27 18:08 Kevin Wolf
@ 2016-10-28 13:29 ` Peter Maydell
0 siblings, 0 replies; 34+ messages in thread
From: Peter Maydell @ 2016-10-28 13:29 UTC (permalink / raw)
To: Kevin Wolf; +Cc: Qemu-block, QEMU Developers
On 27 October 2016 at 19:08, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit 5929d7e8a0e1f43333bc3528b50397ae8dd0fd6b:
>
> Merge remote-tracking branch 'remotes/rth/tags/pull-atomic-20161026' into staging (2016-10-27 14:06:34 +0100)
>
> are available in the git repository at:
>
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to b74fc7f78e0dd54fbae67d46552cebf81b59ae9f:
>
> iotests: Add test for NBD's blockdev-add interface (2016-10-27 19:05:23 +0200)
>
> ----------------------------------------------------------------
> Block layer patches
>
Applied, thanks.
-- PMM
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 00/23] Block layer patches
@ 2016-10-27 18:08 Kevin Wolf
2016-10-28 13:29 ` Peter Maydell
0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2016-10-27 18:08 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit 5929d7e8a0e1f43333bc3528b50397ae8dd0fd6b:
Merge remote-tracking branch 'remotes/rth/tags/pull-atomic-20161026' into staging (2016-10-27 14:06:34 +0100)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to b74fc7f78e0dd54fbae67d46552cebf81b59ae9f:
iotests: Add test for NBD's blockdev-add interface (2016-10-27 19:05:23 +0200)
----------------------------------------------------------------
Block layer patches
----------------------------------------------------------------
Kevin Wolf (10):
block: Use blk_co_flush() for all BB level flushes
block: Use blk_co_pdiscard() for all BB level discard
block: Remove bdrv_aio_pdiscard()
block: Use blk_co_ioctl() for all BB level ioctls
raw-posix: Don't use bdrv_ioctl()
block: Remove bdrv_ioctl()
block: Introduce .bdrv_co_ioctl() driver callback
raw: Implement .bdrv_co_ioctl instead of .bdrv_aio_ioctl
block: Remove bdrv_aio_ioctl()
qemu-iotests: Fix typo for NFS with IMGOPTSSYNTAX
Max Reitz (13):
block/nbd: Drop trailing "." in error messages
block/nbd: Reject port parameter without host
block/nbd: Default port in nbd_refresh_filename()
block/nbd: Use qdict_put()
block/nbd: Add nbd_has_filename_options_conflict()
block/nbd: Accept SocketAddress
block/nbd: Use SocketAddress options
qapi: Allow blockdev-add for NBD
iotests.py: Add qemu_nbd function
iotests.py: Allow concurrent qemu instances
socket_scm_helper: Accept fd directly
iotests: Add assert_json_filename_equal() method
iotests: Add test for NBD's blockdev-add interface
block/block-backend.c | 94 +++++++++----
block/io.c | 111 ++--------------
block/nbd.c | 234 +++++++++++++++++++++------------
block/raw-posix.c | 16 ++-
block/raw_bsd.c | 9 +-
block/trace-events | 1 -
include/block/block.h | 8 +-
include/block/block_int.h | 2 +
include/sysemu/block-backend.h | 1 +
qapi/block-core.json | 27 +++-
tests/qemu-iotests/051.out | 4 +-
tests/qemu-iotests/051.pc.out | 4 +-
tests/qemu-iotests/147 | 195 +++++++++++++++++++++++++++
tests/qemu-iotests/147.out | 5 +
tests/qemu-iotests/common.rc | 2 +-
tests/qemu-iotests/group | 1 +
tests/qemu-iotests/iotests.py | 34 ++++-
tests/qemu-iotests/socket_scm_helper.c | 29 ++--
18 files changed, 527 insertions(+), 250 deletions(-)
create mode 100755 tests/qemu-iotests/147
create mode 100644 tests/qemu-iotests/147.out
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2015-09-14 9:57 ` Kevin Wolf
@ 2015-09-14 14:36 ` Max Reitz
0 siblings, 0 replies; 34+ messages in thread
From: Max Reitz @ 2015-09-14 14:36 UTC (permalink / raw)
To: Kevin Wolf, Peter Maydell; +Cc: QEMU Developers, Qemu-block
[-- Attachment #1: Type: text/plain, Size: 1788 bytes --]
On 14.09.2015 11:57, Kevin Wolf wrote:
> Am 14.09.2015 um 11:46 hat Peter Maydell geschrieben:
>> On 11 September 2015 at 20:40, Kevin Wolf <kwolf@redhat.com> wrote:
>>> The following changes since commit 30c38c90bd3f1bb105ebc069ac1821067c980b7c:
>>>
>>> scripts/qemu-gdb: Add brief comment describing usage (2015-09-11 17:14:50 +0100)
>>>
>>> are available in the git repository at:
>>>
>>> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>>>
>>> for you to fetch changes up to 1fcbcc93872953d08cd35830d1169fed19196290:
>>>
>>> qcow2: Make qcow2_alloc_bytes() more explicit (2015-09-11 20:03:02 +0200)
>>>
>>> ----------------------------------------------------------------
>>> Block layer patches
>>>
>>> ----------------------------------------------------------------
>>
>> Hi. I'm afraid this fails to build on 32-bit:
>>
>> /home/pm215/qemu/block/qcow2-refcount.c: In function ‘realloc_refcount_array’:
>> /home/pm215/qemu/block/qcow2-refcount.c:1294:16: error: cast to
>> pointer from integer of different size [-Werror=int-to-pointer-cast]
>> memset((void *)((uintptr_t)new_ptr + old_byte_size), 0,
>> ^
>> cc1: all warnings being treated as errors
>> /home/pm215/qemu/rules.mak:57: recipe for target 'block/qcow2-refcount.o' failed
>> make: *** [block/qcow2-refcount.o] Error 1
>>
>> (old_byte_size is int64_t, so (uintptr_t)new_ptr + old_byte_size
>> becomes a 64-bit addition, and then you cast it to a 32-bit
>> pointer.)
>
> Max, I think this is yours.
Indeed, I'll send a v3 for "qcow2: Make size_to_clusters() return
uint64_t" and "iotests: Add test for checking large image files".
You decide whether you are going to include them in a new pull request
or just drop them for now.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 473 bytes --]
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2015-09-14 9:46 ` Peter Maydell
@ 2015-09-14 9:57 ` Kevin Wolf
2015-09-14 14:36 ` Max Reitz
0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2015-09-14 9:57 UTC (permalink / raw)
To: Peter Maydell; +Cc: QEMU Developers, Qemu-block, mreitz
Am 14.09.2015 um 11:46 hat Peter Maydell geschrieben:
> On 11 September 2015 at 20:40, Kevin Wolf <kwolf@redhat.com> wrote:
> > The following changes since commit 30c38c90bd3f1bb105ebc069ac1821067c980b7c:
> >
> > scripts/qemu-gdb: Add brief comment describing usage (2015-09-11 17:14:50 +0100)
> >
> > are available in the git repository at:
> >
> > git://repo.or.cz/qemu/kevin.git tags/for-upstream
> >
> > for you to fetch changes up to 1fcbcc93872953d08cd35830d1169fed19196290:
> >
> > qcow2: Make qcow2_alloc_bytes() more explicit (2015-09-11 20:03:02 +0200)
> >
> > ----------------------------------------------------------------
> > Block layer patches
> >
> > ----------------------------------------------------------------
>
> Hi. I'm afraid this fails to build on 32-bit:
>
> /home/pm215/qemu/block/qcow2-refcount.c: In function ‘realloc_refcount_array’:
> /home/pm215/qemu/block/qcow2-refcount.c:1294:16: error: cast to
> pointer from integer of different size [-Werror=int-to-pointer-cast]
> memset((void *)((uintptr_t)new_ptr + old_byte_size), 0,
> ^
> cc1: all warnings being treated as errors
> /home/pm215/qemu/rules.mak:57: recipe for target 'block/qcow2-refcount.o' failed
> make: *** [block/qcow2-refcount.o] Error 1
>
> (old_byte_size is int64_t, so (uintptr_t)new_ptr + old_byte_size
> becomes a 64-bit addition, and then you cast it to a 32-bit
> pointer.)
Max, I think this is yours.
Kevin
^ permalink raw reply [flat|nested] 34+ messages in thread
* Re: [Qemu-devel] [PULL 00/23] Block layer patches
2015-09-11 19:40 Kevin Wolf
@ 2015-09-14 9:46 ` Peter Maydell
2015-09-14 9:57 ` Kevin Wolf
0 siblings, 1 reply; 34+ messages in thread
From: Peter Maydell @ 2015-09-14 9:46 UTC (permalink / raw)
To: Kevin Wolf; +Cc: QEMU Developers, Qemu-block
On 11 September 2015 at 20:40, Kevin Wolf <kwolf@redhat.com> wrote:
> The following changes since commit 30c38c90bd3f1bb105ebc069ac1821067c980b7c:
>
> scripts/qemu-gdb: Add brief comment describing usage (2015-09-11 17:14:50 +0100)
>
> are available in the git repository at:
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 1fcbcc93872953d08cd35830d1169fed19196290:
>
> qcow2: Make qcow2_alloc_bytes() more explicit (2015-09-11 20:03:02 +0200)
>
> ----------------------------------------------------------------
> Block layer patches
>
> ----------------------------------------------------------------
Hi. I'm afraid this fails to build on 32-bit:
/home/pm215/qemu/block/qcow2-refcount.c: In function ‘realloc_refcount_array’:
/home/pm215/qemu/block/qcow2-refcount.c:1294:16: error: cast to
pointer from integer of different size [-Werror=int-to-pointer-cast]
memset((void *)((uintptr_t)new_ptr + old_byte_size), 0,
^
cc1: all warnings being treated as errors
/home/pm215/qemu/rules.mak:57: recipe for target 'block/qcow2-refcount.o' failed
make: *** [block/qcow2-refcount.o] Error 1
(old_byte_size is int64_t, so (uintptr_t)new_ptr + old_byte_size
becomes a 64-bit addition, and then you cast it to a 32-bit
pointer.)
thanks
-- PMM
^ permalink raw reply [flat|nested] 34+ messages in thread
* [Qemu-devel] [PULL 00/23] Block layer patches
@ 2015-09-11 19:40 Kevin Wolf
2015-09-14 9:46 ` Peter Maydell
0 siblings, 1 reply; 34+ messages in thread
From: Kevin Wolf @ 2015-09-11 19:40 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, qemu-devel
The following changes since commit 30c38c90bd3f1bb105ebc069ac1821067c980b7c:
scripts/qemu-gdb: Add brief comment describing usage (2015-09-11 17:14:50 +0100)
are available in the git repository at:
git://repo.or.cz/qemu/kevin.git tags/for-upstream
for you to fetch changes up to 1fcbcc93872953d08cd35830d1169fed19196290:
qcow2: Make qcow2_alloc_bytes() more explicit (2015-09-11 20:03:02 +0200)
----------------------------------------------------------------
Block layer patches
----------------------------------------------------------------
Kevin Wolf (14):
qcow2: Rename BDRVQcowState to BDRVQcow2State
block: Allow specifying driver-specific options to reopen
qemu-io: Remove duplicate 'open' error message
qemu-io: Add command 'reopen'
qcow2: Improve error message
qcow2: Factor out qcow2_update_options()
qcow2: Move qcow2_update_options() call up
qcow2: Move rest of option handling to qcow2_update_options()
qcow2: Leave s unchanged on qcow2_update_options() failure
qcow2: Fix memory leak in qcow2_update_options() error path
qcow2: Make qcow2_update_options() suitable for transactions
qcow2: Support updating driver-specific options in reopen
qemu-iotests: Reopen qcow2 with lazy-refcounts change
qemu-iotests: More qcow2 reopen tests
Max Reitz (8):
block: Always pass NULL as drv for bdrv_open()
block: Drop drv parameter from bdrv_open()
block: Drop drv parameter from bdrv_open_inherit()
block: Drop drv parameter from bdrv_fill_options()
block: Drop bdrv_find_whitelisted_format()
qcow2: Make size_to_clusters() return uint64_t
iotests: Add test for checking large image files
qcow2: Make qcow2_alloc_bytes() more explicit
Radoslav Gerganov (1):
vmdk: Fix next_cluster_sector for compressed write
block.c | 150 +++++++-------
block/block-backend.c | 2 +-
block/commit.c | 4 +-
block/parallels.c | 2 +-
block/qcow.c | 2 +-
block/qcow2-cache.c | 14 +-
block/qcow2-cluster.c | 76 +++----
block/qcow2-refcount.c | 74 ++++---
block/qcow2-snapshot.c | 20 +-
block/qcow2.c | 486 +++++++++++++++++++++++++++++----------------
block/qcow2.h | 26 +--
block/qed.c | 2 +-
block/sheepdog.c | 5 +-
block/vdi.c | 2 +-
block/vhdx.c | 2 +-
block/vmdk.c | 15 +-
block/vpc.c | 2 +-
block/vvfat.c | 8 +-
blockdev.c | 72 +++----
include/block/block.h | 9 +-
qemu-io-cmds.c | 90 +++++++++
qemu-io.c | 1 -
tests/qemu-iotests/039 | 27 +++
tests/qemu-iotests/039.out | 18 ++
tests/qemu-iotests/137 | 145 ++++++++++++++
tests/qemu-iotests/137.out | 42 ++++
tests/qemu-iotests/138 | 73 +++++++
tests/qemu-iotests/138.out | 9 +
tests/qemu-iotests/group | 2 +
29 files changed, 972 insertions(+), 408 deletions(-)
create mode 100755 tests/qemu-iotests/137
create mode 100644 tests/qemu-iotests/137.out
create mode 100755 tests/qemu-iotests/138
create mode 100644 tests/qemu-iotests/138.out
^ permalink raw reply [flat|nested] 34+ messages in thread
end of thread, other threads:[~2018-10-03 15:46 UTC | newest]
Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-10-24 17:01 [Qemu-devel] [PULL 00/23] Block layer patches Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 01/23] block: failed qemu-img command should return non-zero exit code Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 02/23] qcow2: Support BDRV_REQ_MAY_UNMAP Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 03/23] block: Remove "options" indirection from blockdev-add Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 04/23] block: improve error handling in raw_open Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 05/23] qapi: fix memory leak in bdrv_image_info_specific_dump Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 06/23] throttle: Correct access to wrong BlockBackendPublic structures Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 07/23] qemu-iotests: Test I/O in a single drive from a throttling group Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 08/23] qemu-nbd: Add --fork option Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 09/23] iotests: Remove raciness from 162 Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 10/23] iotests: Do not rely on unavailable domains in 162 Kevin Wolf
2016-10-24 17:01 ` [Qemu-devel] [PULL 11/23] quorum: change child_iter to children_read Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 12/23] quorum: do not allocate multiple iovecs for FIFO strategy Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 13/23] block: Hide HBitmap in block dirty bitmap interface Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 14/23] HBitmap: Introduce "meta" bitmap to track bit changes Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 15/23] tests: Add test code for meta bitmap Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 16/23] block: Support meta dirty bitmap Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 17/23] block: Add two dirty bitmap getters Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 18/23] block: Assert that bdrv_release_dirty_bitmap succeeded Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 19/23] hbitmap: serialization Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 20/23] block: BdrvDirtyBitmap serialization interface Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 21/23] tests: Add test code for hbitmap serialization Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 22/23] block: More operations for meta dirty bitmap Kevin Wolf
2016-10-24 17:02 ` [Qemu-devel] [PULL 23/23] block/replication: Clarify 'top-id' parameter usage Kevin Wolf
2016-10-24 18:36 ` [Qemu-devel] [PULL 00/23] Block layer patches Peter Maydell
-- strict thread matches above, loose matches on Subject: below --
2018-10-01 17:18 Kevin Wolf
2018-10-02 8:06 ` Peter Maydell
2018-10-03 15:46 ` Peter Maydell
2016-10-27 18:08 Kevin Wolf
2016-10-28 13:29 ` Peter Maydell
2015-09-11 19:40 Kevin Wolf
2015-09-14 9:46 ` Peter Maydell
2015-09-14 9:57 ` Kevin Wolf
2015-09-14 14:36 ` Max Reitz
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.