* [Qemu-devel] [PATCH v4 0/2] block: formalize and test fleecing
@ 2018-07-02 19:46 John Snow
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 1/2] blockdev: enable non-root nodes for backup source John Snow
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing John Snow
0 siblings, 2 replies; 6+ messages in thread
From: John Snow @ 2018-07-02 19:46 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Max Reitz, Markus Armbruster, eblake, John Snow
This two patch series formalizes a workflow we expect to work for the
purposes of image fleecing. There are some remaining questions on
serialization correctness that we intend to address at a later time,
possibly during the 3.0 soft freeze window.
V4:
- Replace my 1/2 with Vladimir's, which was more exhaustive.
- Formatting improvements to 2/2 (Vladimir)
- Specify the image format on write (Vladimir)
V3:
- Added explicit zero checking as per Eblake's suggestion
- Remove stale python code
John Snow (1):
iotests: add 222 to test basic fleecing
Vladimir Sementsov-Ogievskiy (1):
blockdev: enable non-root nodes for backup source
blockdev.c | 4 +-
tests/qemu-iotests/222 | 155 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/222.out | 67 ++++++++++++++++++++
tests/qemu-iotests/group | 1 +
4 files changed, 225 insertions(+), 2 deletions(-)
create mode 100644 tests/qemu-iotests/222
create mode 100644 tests/qemu-iotests/222.out
--
2.14.4
^ permalink raw reply [flat|nested] 6+ messages in thread
* [Qemu-devel] [PATCH v4 1/2] blockdev: enable non-root nodes for backup source
2018-07-02 19:46 [Qemu-devel] [PATCH v4 0/2] block: formalize and test fleecing John Snow
@ 2018-07-02 19:46 ` John Snow
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing John Snow
1 sibling, 0 replies; 6+ messages in thread
From: John Snow @ 2018-07-02 19:46 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Max Reitz, Markus Armbruster, eblake,
Vladimir Sementsov-Ogievskiy, John Snow
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
This is needed to implement the image-fleecing workflow where we
create a temporary node backed by an active node, then start
backupdev-backup sync=none from the active node to the temp node.
In this case, the active node is now a root node AND a backing node,
so it no longer qualifies as a root node, so we loosen the restriction
on which nodes can be considered as the source for a backup.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: John Snow <jsnow@redhat.com>
---
blockdev.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/blockdev.c b/blockdev.c
index 58d7570932..72f5347df5 100644
--- a/blockdev.c
+++ b/blockdev.c
@@ -1859,7 +1859,7 @@ static void blockdev_backup_prepare(BlkActionState *common, Error **errp)
assert(common->action->type == TRANSACTION_ACTION_KIND_BLOCKDEV_BACKUP);
backup = common->action->u.blockdev_backup.data;
- bs = qmp_get_root_bs(backup->device, errp);
+ bs = bdrv_lookup_bs(backup->device, backup->device, errp);
if (!bs) {
return;
}
@@ -3517,7 +3517,7 @@ BlockJob *do_blockdev_backup(BlockdevBackup *backup, JobTxn *txn,
backup->compress = false;
}
- bs = qmp_get_root_bs(backup->device, errp);
+ bs = bdrv_lookup_bs(backup->device, backup->device, errp);
if (!bs) {
return NULL;
}
--
2.14.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing
2018-07-02 19:46 [Qemu-devel] [PATCH v4 0/2] block: formalize and test fleecing John Snow
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 1/2] blockdev: enable non-root nodes for backup source John Snow
@ 2018-07-02 19:46 ` John Snow
2018-07-02 20:06 ` Eric Blake
2018-07-03 9:15 ` Vladimir Sementsov-Ogievskiy
1 sibling, 2 replies; 6+ messages in thread
From: John Snow @ 2018-07-02 19:46 UTC (permalink / raw)
To: qemu-devel, qemu-block
Cc: Kevin Wolf, Max Reitz, Markus Armbruster, eblake, John Snow
Signed-off-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/222 | 155 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/222.out | 67 ++++++++++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 223 insertions(+)
create mode 100644 tests/qemu-iotests/222
create mode 100644 tests/qemu-iotests/222.out
diff --git a/tests/qemu-iotests/222 b/tests/qemu-iotests/222
new file mode 100644
index 0000000000..ff3bfc1470
--- /dev/null
+++ b/tests/qemu-iotests/222
@@ -0,0 +1,155 @@
+#!/usr/bin/env python
+#
+# This test covers the basic fleecing workflow, which provides a
+# point-in-time snapshot of a node that can be queried over NBD.
+#
+# Copyright (C) 2018 Red Hat, Inc.
+# John helped, too.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+# Creator/Owner: John Snow <jsnow@redhat.com>
+
+import iotests
+from iotests import log, qemu_img, qemu_io, qemu_io_silent
+
+iotests.verify_platform(['linux'])
+
+patterns = [("0x5d", "0", "64k"),
+ ("0xd5", "1M", "64k"),
+ ("0xdc", "32M", "64k"),
+ ("0xcd", "0x3ff0000", "64k")] # 64M - 64K
+
+overwrite = [("0xab", "0", "64k"), # Full overwrite
+ ("0xad", "0x00f8000", "64k"), # Partial-left (1M-32K)
+ ("0x1d", "0x2008000", "64k"), # Partial-right (32M+32K)
+ ("0xea", "0x3fe0000", "64k")] # Adjacent-left (64M - 128K)
+
+zeroes = [("0", "0x00f8000", "32k"), # Left-end of partial-left (1M-32K)
+ ("0", "0x2010000", "32k"), # Right-end of partial-right (32M+64K)
+ ("0", "0x3fe0000", "64k")] # overwrite[3]
+
+remainder = [("0xd5", "0x108000", "32k"), # Right-end of partial-left [1]
+ ("0xdc", "32M", "32k"), # Left-end of partial-right [2]
+ ("0xcd", "0x3ff0000", "64k")] # patterns[3]
+
+with iotests.FilePath('base.img') as base_img_path, \
+ iotests.FilePath('fleece.img') as fleece_img_path, \
+ iotests.FilePath('nbd.sock') as nbd_sock_path, \
+ iotests.VM() as vm:
+
+ log('--- Setting up images ---')
+ log('')
+
+ assert qemu_img('create', '-f', iotests.imgfmt, base_img_path, '64M') == 0
+ assert qemu_img('create', '-f', "qcow2", fleece_img_path, '64M') == 0
+
+ for p in patterns:
+ qemu_io('-f', iotests.imgfmt,
+ '-c', 'write -P%s %s %s' % p, base_img_path)
+
+ log('Done')
+
+ log('')
+ log('--- Launching VM ---')
+ log('')
+
+ vm.add_drive(base_img_path)
+ vm.launch()
+ log('Done')
+
+ log('')
+ log('--- Setting up Fleecing Graph ---')
+ log('')
+
+ src_node = "drive0"
+ tgt_node = "fleeceNode"
+
+ # create tgt_node backed by src_node
+ log(vm.qmp("blockdev-add", **{
+ "driver": "qcow2",
+ "node-name": tgt_node,
+ "file": {
+ "driver": "file",
+ "filename": fleece_img_path,
+ },
+ "backing": src_node,
+ }))
+
+ # Establish COW from source to fleecing node
+ log(vm.qmp("blockdev-backup",
+ device=src_node,
+ target=tgt_node,
+ sync="none"))
+
+ log('')
+ log('--- Setting up NBD Export ---')
+ log('')
+
+ nbd_uri = 'nbd+unix:///%s?socket=%s' % (tgt_node, nbd_sock_path)
+ log(vm.qmp("nbd-server-start",
+ **{"addr": { "type": "unix",
+ "data": { "path": nbd_sock_path } } }))
+
+ log(vm.qmp("nbd-server-add", device=tgt_node))
+
+ log('')
+ log('--- Sanity Check ---')
+ log('')
+
+ for p in (patterns + zeroes):
+ cmd = "read -P%s %s %s" % p
+ log(cmd)
+ assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0
+
+ log('')
+ log('--- Testing COW ---')
+ log('')
+
+ for p in overwrite:
+ cmd = "write -P%s %s %s" % p
+ log(cmd)
+ log(vm.hmp_qemu_io(src_node, cmd))
+
+ log('')
+ log('--- Verifying Data ---')
+ log('')
+
+ for p in (patterns + zeroes):
+ cmd = "read -P%s %s %s" % p
+ log(cmd)
+ assert qemu_io_silent('-r', '-f', 'raw', '-c', cmd, nbd_uri) == 0
+
+ log('')
+ log('--- Cleanup ---')
+ log('')
+
+ log(vm.qmp('block-job-cancel', device=src_node))
+ log(vm.event_wait('BLOCK_JOB_CANCELLED'),
+ filters=[iotests.filter_qmp_event])
+ log(vm.qmp('nbd-server-stop'))
+ log(vm.qmp('blockdev-del', node_name=tgt_node))
+ vm.shutdown()
+
+ log('')
+ log('--- Confirming writes ---')
+ log('')
+
+ for p in (overwrite + remainder):
+ cmd = "read -P%s %s %s" % p
+ log(cmd)
+ assert qemu_io_silent(base_img_path, '-c', cmd) == 0
+
+ log('')
+ log('Done')
diff --git a/tests/qemu-iotests/222.out b/tests/qemu-iotests/222.out
new file mode 100644
index 0000000000..48f336a02b
--- /dev/null
+++ b/tests/qemu-iotests/222.out
@@ -0,0 +1,67 @@
+--- Setting up images ---
+
+Done
+
+--- Launching VM ---
+
+Done
+
+--- Setting up Fleecing Graph ---
+
+{u'return': {}}
+{u'return': {}}
+
+--- Setting up NBD Export ---
+
+{u'return': {}}
+{u'return': {}}
+
+--- Sanity Check ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Testing COW ---
+
+write -P0xab 0 64k
+{u'return': u''}
+write -P0xad 0x00f8000 64k
+{u'return': u''}
+write -P0x1d 0x2008000 64k
+{u'return': u''}
+write -P0xea 0x3fe0000 64k
+{u'return': u''}
+
+--- Verifying Data ---
+
+read -P0x5d 0 64k
+read -P0xd5 1M 64k
+read -P0xdc 32M 64k
+read -P0xcd 0x3ff0000 64k
+read -P0 0x00f8000 32k
+read -P0 0x2010000 32k
+read -P0 0x3fe0000 64k
+
+--- Cleanup ---
+
+{u'return': {}}
+{u'timestamp': {u'seconds': 'SECS', u'microseconds': 'USECS'}, u'data': {u'device': u'drive0', u'type': u'backup', u'speed': 0, u'len': 67108864, u'offset': 393216}, u'event': u'BLOCK_JOB_CANCELLED'}
+{u'return': {}}
+{u'return': {}}
+
+--- Confirming writes ---
+
+read -P0xab 0 64k
+read -P0xad 0x00f8000 64k
+read -P0x1d 0x2008000 64k
+read -P0xea 0x3fe0000 64k
+read -P0xd5 0x108000 32k
+read -P0xdc 32M 32k
+read -P0xcd 0x3ff0000 64k
+
+Done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index eea75819d2..8019a9f721 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -220,3 +220,4 @@
218 rw auto quick
219 rw auto
221 rw auto quick
+222 rw auto quick
--
2.14.4
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing John Snow
@ 2018-07-02 20:06 ` Eric Blake
2018-07-03 9:15 ` Vladimir Sementsov-Ogievskiy
1 sibling, 0 replies; 6+ messages in thread
From: Eric Blake @ 2018-07-02 20:06 UTC (permalink / raw)
To: John Snow, qemu-devel, qemu-block
Cc: Kevin Wolf, Max Reitz, Markus Armbruster
On 07/02/2018 02:46 PM, John Snow wrote:
> Signed-off-by: John Snow <jsnow@redhat.com>
> ---
> tests/qemu-iotests/222 | 155 +++++++++++++++++++++++++++++++++++++++++++++
> tests/qemu-iotests/222.out | 67 ++++++++++++++++++++
> tests/qemu-iotests/group | 1 +
> 3 files changed, 223 insertions(+)
> create mode 100644 tests/qemu-iotests/222
> create mode 100644 tests/qemu-iotests/222.out
Reviewed-by: Eric Blake <eblake@redhat.com>
Since this iotest involves NBD, I'm fine including this series in my NBD
queue; pull request to come before freeze.
--
Eric Blake, Principal Software Engineer
Red Hat, Inc. +1-919-301-3266
Virtualization: qemu.org | libvirt.org
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing John Snow
2018-07-02 20:06 ` Eric Blake
@ 2018-07-03 9:15 ` Vladimir Sementsov-Ogievskiy
2018-07-03 15:23 ` John Snow
1 sibling, 1 reply; 6+ messages in thread
From: Vladimir Sementsov-Ogievskiy @ 2018-07-03 9:15 UTC (permalink / raw)
To: John Snow, qemu-devel, qemu-block
Cc: Kevin Wolf, Markus Armbruster, Max Reitz
02.07.2018 22:46, John Snow wrote:
> Signed-off-by: John Snow <jsnow@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
--
Best regards,
Vladimir
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing
2018-07-03 9:15 ` Vladimir Sementsov-Ogievskiy
@ 2018-07-03 15:23 ` John Snow
0 siblings, 0 replies; 6+ messages in thread
From: John Snow @ 2018-07-03 15:23 UTC (permalink / raw)
To: Vladimir Sementsov-Ogievskiy, qemu-devel, qemu-block
Cc: Kevin Wolf, Markus Armbruster, Max Reitz
On 07/03/2018 05:15 AM, Vladimir Sementsov-Ogievskiy wrote:
> 02.07.2018 22:46, John Snow wrote:
>> Signed-off-by: John Snow <jsnow@redhat.com>
>
>
> Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
>
Thanks, and sorry for the 3.0 freeze rush.
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2018-07-03 15:23 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-07-02 19:46 [Qemu-devel] [PATCH v4 0/2] block: formalize and test fleecing John Snow
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 1/2] blockdev: enable non-root nodes for backup source John Snow
2018-07-02 19:46 ` [Qemu-devel] [PATCH v4 2/2] iotests: add 222 to test basic fleecing John Snow
2018-07-02 20:06 ` Eric Blake
2018-07-03 9:15 ` Vladimir Sementsov-Ogievskiy
2018-07-03 15:23 ` John Snow
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.