* [Qemu-devel] [PATCH v2 1/7] iotests: Fix _filter_img_create()
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 2/7] vmdk: Use bdrv_dirname() for relative extent paths Max Reitz
` (7 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
fe646693acc changed qemu-img create's output so that it no longer prints
single quotes around parameter values. The subformat and adapter_type
filters in _filter_img_create() have never been adapted to that change.
Fixes: fe646693acc13ac48b98435d14149ab04dc597bc
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/059.out | 16 ++++++++--------
tests/qemu-iotests/common.filter | 4 ++--
2 files changed, 10 insertions(+), 10 deletions(-)
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index 4fab42a28c..77d8984428 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -13,17 +13,17 @@ Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
qemu-io: can't open device TEST_DIR/t.vmdk: L1 size too big
=== Testing monolithicFlat creation and opening ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
image: TEST_DIR/t.IMGFMT
file format: IMGFMT
virtual size: 2 GiB (2147483648 bytes)
=== Testing monolithicFlat with zeroed_grain ===
qemu-img: TEST_DIR/t.IMGFMT: Flat image can't enable zeroed grain
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2147483648
=== Testing big twoGbMaxExtentFlat ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000 subformat=twoGbMaxExtentFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1073741824000
image: TEST_DIR/t.vmdk
file format: vmdk
virtual size: 0.977 TiB (1073741824000 bytes)
@@ -2038,7 +2038,7 @@ Format specific information:
qemu-img: Could not open 'TEST_DIR/t.IMGFMT': Invalid extent line: RW 12582912 VMFS "dummy.IMGFMT" 1
=== Testing truncated sparse ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=107374182400
qemu-img: Could not open 'TEST_DIR/t.IMGFMT': File truncated, expecting at least 13172736 bytes
=== Converting to streamOptimized from image with small cluster size===
@@ -2049,7 +2049,7 @@ wrote 512/512 bytes at offset 10240
512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing monolithicFlat with internally generated JSON file name ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
=== Testing version 3 ===
@@ -2259,7 +2259,7 @@ read 512/512 bytes at offset 64931328
512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing 4TB monolithicFlat creation and IO ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104 subformat=monolithicFlat
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4398046511104
image: TEST_DIR/t.IMGFMT
file format: IMGFMT
virtual size: 4 TiB (4398046511104 bytes)
@@ -2333,7 +2333,7 @@ read 1024/1024 bytes at offset 966367641600
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing qemu-img map on extents ===
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=monolithicSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
wrote 1024/1024 bytes at offset 65024
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 1024/1024 bytes at offset 2147483136
@@ -2344,7 +2344,7 @@ Offset Length Mapped to File
0 0x20000 0x3f0000 TEST_DIR/t.vmdk
0x7fff0000 0x20000 0x410000 TEST_DIR/t.vmdk
0x140000000 0x10000 0x430000 TEST_DIR/t.vmdk
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544 subformat=twoGbMaxExtentSparse
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=33285996544
wrote 1024/1024 bytes at offset 65024
1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 1024/1024 bytes at offset 2147483136
diff --git a/tests/qemu-iotests/common.filter b/tests/qemu-iotests/common.filter
index 35fddc746f..64dd9200f3 100644
--- a/tests/qemu-iotests/common.filter
+++ b/tests/qemu-iotests/common.filter
@@ -130,8 +130,8 @@ _filter_img_create()
-e "s# compat6=\\(on\\|off\\)##g" \
-e "s# static=\\(on\\|off\\)##g" \
-e "s# zeroed_grain=\\(on\\|off\\)##g" \
- -e "s# subformat='[^']*'##g" \
- -e "s# adapter_type='[^']*'##g" \
+ -e "s# subformat=[^ ]*##g" \
+ -e "s# adapter_type=[^ ]*##g" \
-e "s# hwversion=[^ ]*##g" \
-e "s# lazy_refcounts=\\(on\\|off\\)##g" \
-e "s# block_size=[0-9]\\+##g" \
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [Qemu-devel] [PATCH v2 2/7] vmdk: Use bdrv_dirname() for relative extent paths
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 1/7] iotests: Fix _filter_img_create() Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 3/7] iotests: Keep testing broken " Max Reitz
` (6 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
This makes iotest 033 pass with e.g. subformat=monolithicFlat. It also
turns a former error in 059 into success.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
block/vmdk.c | 54 ++++++++++++++++++++++++--------------
tests/qemu-iotests/059 | 7 +++--
tests/qemu-iotests/059.out | 4 ++-
3 files changed, 42 insertions(+), 23 deletions(-)
diff --git a/block/vmdk.c b/block/vmdk.c
index bd36ece125..825222de55 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -1076,8 +1076,7 @@ static const char *next_line(const char *s)
}
static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
- const char *desc_file_path, QDict *options,
- Error **errp)
+ QDict *options, Error **errp)
{
int ret;
int matches;
@@ -1087,6 +1086,7 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
const char *p, *np;
int64_t sectors = 0;
int64_t flat_offset;
+ char *desc_file_dir = NULL;
char *extent_path;
BdrvChild *extent_file;
BDRVVmdkState *s = bs->opaque;
@@ -1130,16 +1130,23 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
continue;
}
- if (!path_is_absolute(fname) && !path_has_protocol(fname) &&
- !desc_file_path[0])
- {
- bdrv_refresh_filename(bs->file->bs);
- error_setg(errp, "Cannot use relative extent paths with VMDK "
- "descriptor file '%s'", bs->file->bs->filename);
- return -EINVAL;
- }
+ if (path_is_absolute(fname)) {
+ extent_path = g_strdup(fname);
+ } else {
+ if (!desc_file_dir) {
+ desc_file_dir = bdrv_dirname(bs->file->bs, errp);
+ if (!desc_file_dir) {
+ bdrv_refresh_filename(bs->file->bs);
+ error_prepend(errp, "Cannot use relative paths with VMDK "
+ "descriptor file '%s': ",
+ bs->file->bs->filename);
+ ret = -EINVAL;
+ goto out;
+ }
+ }
- extent_path = path_combine(desc_file_path, fname);
+ extent_path = g_strconcat(desc_file_dir, fname, NULL);
+ }
ret = snprintf(extent_opt_prefix, 32, "extents.%d", s->num_extents);
assert(ret < 32);
@@ -1149,7 +1156,8 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
g_free(extent_path);
if (local_err) {
error_propagate(errp, local_err);
- return -EINVAL;
+ ret = -EINVAL;
+ goto out;
}
/* save to extents array */
@@ -1160,7 +1168,7 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
0, 0, 0, 0, 0, &extent, errp);
if (ret < 0) {
bdrv_unref_child(bs, extent_file);
- return ret;
+ goto out;
}
extent->flat_start_offset = flat_offset << 9;
} else if (!strcmp(type, "SPARSE") || !strcmp(type, "VMFSSPARSE")) {
@@ -1175,24 +1183,27 @@ static int vmdk_parse_extents(const char *desc, BlockDriverState *bs,
g_free(buf);
if (ret) {
bdrv_unref_child(bs, extent_file);
- return ret;
+ goto out;
}
extent = &s->extents[s->num_extents - 1];
} else if (!strcmp(type, "SESPARSE")) {
ret = vmdk_open_se_sparse(bs, extent_file, bs->open_flags, errp);
if (ret) {
bdrv_unref_child(bs, extent_file);
- return ret;
+ goto out;
}
extent = &s->extents[s->num_extents - 1];
} else {
error_setg(errp, "Unsupported extent type '%s'", type);
bdrv_unref_child(bs, extent_file);
- return -ENOTSUP;
+ ret = -ENOTSUP;
+ goto out;
}
extent->type = g_strdup(type);
}
- return 0;
+
+ ret = 0;
+ goto out;
invalid:
np = next_line(p);
@@ -1201,7 +1212,11 @@ invalid:
np--;
}
error_setg(errp, "Invalid extent line: %.*s", (int)(np - p), p);
- return -EINVAL;
+ ret = -EINVAL;
+
+out:
+ g_free(desc_file_dir);
+ return ret;
}
static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
@@ -1228,8 +1243,7 @@ static int vmdk_open_desc_file(BlockDriverState *bs, int flags, char *buf,
}
s->create_type = g_strdup(ct);
s->desc_offset = 0;
- ret = vmdk_parse_extents(buf, bs, bs->file->bs->exact_filename, options,
- errp);
+ ret = vmdk_parse_extents(buf, bs, options, errp);
exit:
return ret;
}
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index 279aee6815..fbed5f9483 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -114,9 +114,12 @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
echo
echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+# Should work, because bdrv_dirname() works fine with blkdebug
IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
-$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" 2>&1 \
- | _filter_testdir | _filter_imgfmt
+$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
+ -c info \
+ 2>&1 \
+ | _filter_testdir | _filter_imgfmt | _filter_img_info
_cleanup_test_img
echo
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index 77d8984428..120cddd207 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -2050,7 +2050,9 @@ wrote 512/512 bytes at offset 10240
=== Testing monolithicFlat with internally generated JSON file name ===
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
-qemu-io: can't open: Cannot use relative extent paths with VMDK descriptor file 'json:{"image": {"driver": "file", "filename": "TEST_DIR/t.IMGFMT"}, "driver": "blkdebug", "inject-error.0.event": "read_aio"}'
+format name: IMGFMT
+cluster size: 0 bytes
+vm state offset: 0 bytes
=== Testing version 3 ===
image: TEST_DIR/iotest-version3.IMGFMT
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [Qemu-devel] [PATCH v2 3/7] iotests: Keep testing broken relative extent paths
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 1/7] iotests: Fix _filter_img_create() Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 2/7] vmdk: Use bdrv_dirname() for relative extent paths Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 4/7] vmdk: Reject invalid compressed writes Max Reitz
` (5 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
We had a test for a case where relative extent paths did not work, but
unfortunately we just fixed the underlying problem, so it works now.
This patch adds a new test case that still fails.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/059 | 27 +++++++++++++++++++++++++++
tests/qemu-iotests/059.out | 4 ++++
2 files changed, 31 insertions(+)
diff --git a/tests/qemu-iotests/059 b/tests/qemu-iotests/059
index fbed5f9483..10bfbaecec 100755
--- a/tests/qemu-iotests/059
+++ b/tests/qemu-iotests/059
@@ -114,6 +114,8 @@ $QEMU_IMG convert -f qcow2 -O vmdk -o subformat=streamOptimized "$TEST_IMG.qcow2
echo
echo "=== Testing monolithicFlat with internally generated JSON file name ==="
+
+echo '--- blkdebug ---'
# Should work, because bdrv_dirname() works fine with blkdebug
IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
$QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TEST_IMG,file.inject-error.0.event=read_aio" \
@@ -122,6 +124,31 @@ $QEMU_IO -c "open -o driver=$IMGFMT,file.driver=blkdebug,file.image.filename=$TE
| _filter_testdir | _filter_imgfmt | _filter_img_info
_cleanup_test_img
+echo '--- quorum ---'
+# Should not work, because bdrv_dirname() does not work with quorum
+IMGOPTS="subformat=monolithicFlat" _make_test_img 64M
+cp "$TEST_IMG" "$TEST_IMG.orig"
+
+filename="json:{
+ \"driver\": \"$IMGFMT\",
+ \"file\": {
+ \"driver\": \"quorum\",
+ \"children\": [ {
+ \"driver\": \"file\",
+ \"filename\": \"$TEST_IMG\"
+ }, {
+ \"driver\": \"file\",
+ \"filename\": \"$TEST_IMG.orig\"
+ } ],
+ \"vote-threshold\": 1
+ } }"
+
+filename=$(echo "$filename" | tr '\n' ' ' | sed -e 's/\s\+/ /g')
+$QEMU_IMG info "$filename" 2>&1 \
+ | sed -e "s/'json:[^']*'/\$QUORUM_FILE/g" \
+ | _filter_testdir | _filter_imgfmt | _filter_img_info
+
+
echo
echo "=== Testing version 3 ==="
_use_sample_img iotest-version3.vmdk.bz2
diff --git a/tests/qemu-iotests/059.out b/tests/qemu-iotests/059.out
index 120cddd207..f8895ba434 100644
--- a/tests/qemu-iotests/059.out
+++ b/tests/qemu-iotests/059.out
@@ -2049,10 +2049,14 @@ wrote 512/512 bytes at offset 10240
512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
=== Testing monolithicFlat with internally generated JSON file name ===
+--- blkdebug ---
Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
format name: IMGFMT
cluster size: 0 bytes
vm state offset: 0 bytes
+--- quorum ---
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+qemu-img: Could not open $QUORUM_FILE: Cannot use relative paths with VMDK descriptor file $QUORUM_FILE: Cannot generate a base directory for quorum nodes
=== Testing version 3 ===
image: TEST_DIR/iotest-version3.IMGFMT
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [Qemu-devel] [PATCH v2 4/7] vmdk: Reject invalid compressed writes
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
` (2 preceding siblings ...)
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 3/7] iotests: Keep testing broken " Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 5/7] iotests: Disable broken streamOptimized tests Max Reitz
` (4 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
Compressed writes generally have to write full clusters, not just in
theory but also in practice when it comes to vmdk's streamOptimized
subformat. It currently is just silently broken for writes with
non-zero in-cluster offsets:
$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 4k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 4096
4 KiB, 1 ops; 00.01 sec (443.724 KiB/sec and 110.9309 ops/sec)
read failed: Invalid argument
(The technical reason is that vmdk_write_extent() just writes the
incomplete compressed data actually to offset 4k. When reading the
data, vmdk_read_extent() looks at offset 0 and finds the compressed data
size to be 0, because that is what it reads from there. This yields an
error.)
For incomplete writes with zero in-cluster offsets, the error path when
reading the rest of the cluster is a bit different, but the result is
the same:
$ qemu-img create -f vmdk -o subformat=streamOptimized foo.vmdk 1M
$ qemu-io -c 'write 0k 4k' -c 'read 4k 4k' foo.vmdk
wrote 4096/4096 bytes at offset 0
4 KiB, 1 ops; 00.01 sec (362.641 KiB/sec and 90.6603 ops/sec)
read failed: Invalid argument
(Here, vmdk_read_extent() finds the data and then sees that the
uncompressed data is short.)
It is better to reject invalid writes than to make the user believe they
might have succeeded and then fail when trying to read it back.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
block/vmdk.c | 10 ++++++++++
1 file changed, 10 insertions(+)
diff --git a/block/vmdk.c b/block/vmdk.c
index 825222de55..94a7021fcc 100644
--- a/block/vmdk.c
+++ b/block/vmdk.c
@@ -1731,6 +1731,16 @@ static int vmdk_write_extent(VmdkExtent *extent, int64_t cluster_offset,
if (extent->compressed) {
void *compressed_data;
+ /* Only whole clusters */
+ if (offset_in_cluster ||
+ n_bytes > (extent->cluster_sectors * SECTOR_SIZE) ||
+ (n_bytes < (extent->cluster_sectors * SECTOR_SIZE) &&
+ offset + n_bytes != extent->end_sector * SECTOR_SIZE))
+ {
+ ret = -EINVAL;
+ goto out;
+ }
+
if (!extent->has_marker) {
ret = -EINVAL;
goto out;
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [Qemu-devel] [PATCH v2 5/7] iotests: Disable broken streamOptimized tests
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
` (3 preceding siblings ...)
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 4/7] vmdk: Reject invalid compressed writes Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 6/7] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse Max Reitz
` (3 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
streamOptimized does not support writes that do not span exactly one
cluster. Furthermore, it cannot rewrite already allocated clusters.
As such, many iotests do not work with it. Disable them.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/002 | 1 +
tests/qemu-iotests/003 | 1 +
tests/qemu-iotests/005 | 3 ++-
tests/qemu-iotests/009 | 1 +
tests/qemu-iotests/010 | 1 +
tests/qemu-iotests/011 | 1 +
tests/qemu-iotests/017 | 3 ++-
tests/qemu-iotests/018 | 3 ++-
tests/qemu-iotests/019 | 3 ++-
tests/qemu-iotests/020 | 3 ++-
tests/qemu-iotests/027 | 1 +
tests/qemu-iotests/032 | 1 +
tests/qemu-iotests/033 | 1 +
tests/qemu-iotests/034 | 3 ++-
tests/qemu-iotests/037 | 3 ++-
tests/qemu-iotests/063 | 3 ++-
tests/qemu-iotests/072 | 1 +
tests/qemu-iotests/105 | 3 ++-
tests/qemu-iotests/197 | 1 +
tests/qemu-iotests/215 | 1 +
tests/qemu-iotests/251 | 1 +
21 files changed, 30 insertions(+), 9 deletions(-)
diff --git a/tests/qemu-iotests/002 b/tests/qemu-iotests/002
index fd413bce48..1a0d411df5 100755
--- a/tests/qemu-iotests/002
+++ b/tests/qemu-iotests/002
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=128M
diff --git a/tests/qemu-iotests/003 b/tests/qemu-iotests/003
index ccd3a39dfb..33eeade0de 100755
--- a/tests/qemu-iotests/003
+++ b/tests/qemu-iotests/003
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=128M
offset=67M
diff --git a/tests/qemu-iotests/005 b/tests/qemu-iotests/005
index 9c7681c19b..58442762fe 100755
--- a/tests/qemu-iotests/005
+++ b/tests/qemu-iotests/005
@@ -43,7 +43,8 @@ _supported_fmt generic
_supported_proto generic
_supported_os Linux
_unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
# vpc is limited to 127GB, so we can't test it here
if [ "$IMGFMT" = "vpc" ]; then
diff --git a/tests/qemu-iotests/009 b/tests/qemu-iotests/009
index 51b200db1d..4dc7d210f9 100755
--- a/tests/qemu-iotests/009
+++ b/tests/qemu-iotests/009
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=6G
diff --git a/tests/qemu-iotests/010 b/tests/qemu-iotests/010
index 48c533f632..df809b3088 100755
--- a/tests/qemu-iotests/010
+++ b/tests/qemu-iotests/010
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=6G
diff --git a/tests/qemu-iotests/011 b/tests/qemu-iotests/011
index 56f704b5b9..57b99ae4a9 100755
--- a/tests/qemu-iotests/011
+++ b/tests/qemu-iotests/011
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=6G
diff --git a/tests/qemu-iotests/017 b/tests/qemu-iotests/017
index 79875de454..0a4b854e65 100755
--- a/tests/qemu-iotests/017
+++ b/tests/qemu-iotests/017
@@ -41,7 +41,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow qcow2 vmdk qed
_supported_proto generic
_unsupported_proto vxhs
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+ "subformat=streamOptimized"
TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/018 b/tests/qemu-iotests/018
index 78169838ba..c69ce09209 100755
--- a/tests/qemu-iotests/018
+++ b/tests/qemu-iotests/018
@@ -41,7 +41,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow qcow2 vmdk qed
_supported_proto file
_supported_os Linux
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+ "streamOptimized"
TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/019 b/tests/qemu-iotests/019
index a56dd30bed..b4f5234609 100755
--- a/tests/qemu-iotests/019
+++ b/tests/qemu-iotests/019
@@ -47,7 +47,8 @@ _supported_proto file
_supported_os Linux
_unsupported_imgopts "subformat=monolithicFlat" \
"subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
TEST_OFFSETS="0 4294967296"
CLUSTER_SIZE=65536
diff --git a/tests/qemu-iotests/020 b/tests/qemu-iotests/020
index 6b0ebb37d2..f41b92f35f 100755
--- a/tests/qemu-iotests/020
+++ b/tests/qemu-iotests/020
@@ -44,7 +44,8 @@ _supported_fmt qcow qcow2 vmdk qed
_supported_proto file
_unsupported_imgopts "subformat=monolithicFlat" \
"subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
TEST_OFFSETS="0 4294967296"
diff --git a/tests/qemu-iotests/027 b/tests/qemu-iotests/027
index 4cb638022a..494be0921f 100755
--- a/tests/qemu-iotests/027
+++ b/tests/qemu-iotests/027
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt vmdk qcow qcow2 qed
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=128M
diff --git a/tests/qemu-iotests/032 b/tests/qemu-iotests/032
index 988a8c5d8f..8337a4d825 100755
--- a/tests/qemu-iotests/032
+++ b/tests/qemu-iotests/032
@@ -42,6 +42,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
# This works for any image format (though unlikely to segfault for raw)
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
echo
echo === Prepare image ===
diff --git a/tests/qemu-iotests/033 b/tests/qemu-iotests/033
index 362a48c0a0..8b40991d55 100755
--- a/tests/qemu-iotests/033
+++ b/tests/qemu-iotests/033
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto generic
+_unsupported_imgopts "subformat=streamOptimized"
size=128M
diff --git a/tests/qemu-iotests/034 b/tests/qemu-iotests/034
index 324bed28c6..da4cea1571 100755
--- a/tests/qemu-iotests/034
+++ b/tests/qemu-iotests/034
@@ -41,7 +41,8 @@ _supported_proto file
_supported_os Linux
_unsupported_imgopts "subformat=monolithicFlat" \
"subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
CLUSTER_SIZE=4k
size=128M
diff --git a/tests/qemu-iotests/037 b/tests/qemu-iotests/037
index 4946b9be92..e6517acbd4 100755
--- a/tests/qemu-iotests/037
+++ b/tests/qemu-iotests/037
@@ -40,7 +40,8 @@ _supported_fmt qcow qcow2 vmdk qed
_supported_proto file
_unsupported_imgopts "subformat=monolithicFlat" \
"subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
CLUSTER_SIZE=4k
size=128M
diff --git a/tests/qemu-iotests/063 b/tests/qemu-iotests/063
index fe4892e467..7cf0427af4 100755
--- a/tests/qemu-iotests/063
+++ b/tests/qemu-iotests/063
@@ -43,7 +43,8 @@ _supported_fmt qcow qcow2 vmdk qed raw
_supported_proto file
_unsupported_imgopts "subformat=monolithicFlat" \
"subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
_make_test_img 4M
diff --git a/tests/qemu-iotests/072 b/tests/qemu-iotests/072
index 661b36da2d..f0b73e7e65 100755
--- a/tests/qemu-iotests/072
+++ b/tests/qemu-iotests/072
@@ -38,6 +38,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt vpc vmdk vhdx vdi qed qcow2 qcow
_supported_proto file
+_unsupported_imgopts "subformat=streamOptimized"
IMG_SIZE=64M
diff --git a/tests/qemu-iotests/105 b/tests/qemu-iotests/105
index 3346e8cb25..4d55a2d3ef 100755
--- a/tests/qemu-iotests/105
+++ b/tests/qemu-iotests/105
@@ -39,7 +39,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt qcow2 vmdk vhdx qed
_supported_proto generic
_unsupported_imgopts "subformat=twoGbMaxExtentFlat" \
- "subformat=twoGbMaxExtentSparse"
+ "subformat=twoGbMaxExtentSparse" \
+ "subformat=streamOptimized"
echo
echo "creating large image"
diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index 383d7d7f61..1d4f6786db 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -53,6 +53,7 @@ _supported_fmt generic
_supported_proto generic
# LUKS support may be possible, but it complicates things.
_unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
echo
echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/215 b/tests/qemu-iotests/215
index 958c14f5a0..2eb377d682 100755
--- a/tests/qemu-iotests/215
+++ b/tests/qemu-iotests/215
@@ -50,6 +50,7 @@ _supported_fmt generic
_supported_proto generic
# LUKS support may be possible, but it complicates things.
_unsupported_fmt luks
+_unsupported_imgopts "subformat=streamOptimized"
echo
echo '=== Copy-on-read ==='
diff --git a/tests/qemu-iotests/251 b/tests/qemu-iotests/251
index 13f85de9cd..7918ba3559 100755
--- a/tests/qemu-iotests/251
+++ b/tests/qemu-iotests/251
@@ -40,6 +40,7 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
_supported_fmt generic
_supported_proto file
_supported_os Linux
+_unsupported_imgopts "subformat=streamOptimized"
if [ "$IMGOPTSSYNTAX" = "true" ]; then
# We use json:{} filenames here, so we cannot work with additional options.
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [Qemu-devel] [PATCH v2 6/7] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
` (4 preceding siblings ...)
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 5/7] iotests: Disable broken streamOptimized tests Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 7/7] iotests: Disable 126 for flat vmdk subformats Max Reitz
` (2 subsequent siblings)
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
The error message for the test case where we have a quorum node for
which no directory name can be generated is different: For
twoGbMaxExtentSparse, it complains that it cannot open the extent file.
For other (sub)formats, it just notes that it cannot determine the
backing file path. Both are fine, but just disable twoGbMaxExtentSparse
for simplicity's sake.
Signed-off-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: John Snow <jsnow@redhat.com>
---
tests/qemu-iotests/110 | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/tests/qemu-iotests/110 b/tests/qemu-iotests/110
index 2cdc7c8a72..2ef516baf1 100755
--- a/tests/qemu-iotests/110
+++ b/tests/qemu-iotests/110
@@ -40,7 +40,8 @@ trap "_cleanup; exit \$status" 0 1 2 3 15
# Any format supporting backing files
_supported_fmt qed qcow qcow2 vmdk
_supported_proto file
-_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat"
+_unsupported_imgopts "subformat=monolithicFlat" "subformat=twoGbMaxExtentFlat" \
+ "subformat=twoGbMaxExtentSparse"
TEST_IMG_REL=$(basename "$TEST_IMG")
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* [Qemu-devel] [PATCH v2 7/7] iotests: Disable 126 for flat vmdk subformats
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
` (5 preceding siblings ...)
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 6/7] iotests: Disable 110 for vmdk.twoGbMaxExtentSparse Max Reitz
@ 2019-08-15 15:36 ` Max Reitz
2019-08-15 21:20 ` [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes John Snow
2019-08-27 17:16 ` Max Reitz
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-15 15:36 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel, Max Reitz
iotest 126 requires backing file support, which flat vmdks cannot offer.
Skip this test for such subformats.
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
tests/qemu-iotests/126 | 2 ++
1 file changed, 2 insertions(+)
diff --git a/tests/qemu-iotests/126 b/tests/qemu-iotests/126
index 9b0dcf9255..b7fce1e59d 100755
--- a/tests/qemu-iotests/126
+++ b/tests/qemu-iotests/126
@@ -33,6 +33,8 @@ status=1 # failure is the default!
# Needs backing file support
_supported_fmt qcow qcow2 qed vmdk
+_unsupported_imgopts "subformat=monolithicFlat" \
+ "subformat=twoGbMaxExtentFlat"
# This is the default protocol (and we want to test the difference between
# colons which separate a protocol prefix from the rest and colons which are
# just part of the filename, so we cannot test protocols which require a prefix)
--
2.21.0
^ permalink raw reply related [flat|nested] 10+ messages in thread* Re: [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
` (6 preceding siblings ...)
2019-08-15 15:36 ` [Qemu-devel] [PATCH v2 7/7] iotests: Disable 126 for flat vmdk subformats Max Reitz
@ 2019-08-15 21:20 ` John Snow
2019-08-27 17:16 ` Max Reitz
8 siblings, 0 replies; 10+ messages in thread
From: John Snow @ 2019-08-15 21:20 UTC (permalink / raw)
To: Max Reitz, qemu-block; +Cc: Kevin Wolf, qemu-devel
On 8/15/19 11:36 AM, Max Reitz wrote:
> I made the mistake of trying to run the iotests with all non-default
> subformats our vmdk driver has to offer:
> - monolithicFlat
> - twoGbMaxExtentSparse
> - twoGbMaxExtentFlat
> - streamOptimized
>
> Many things broke, so this series fixes what I found. It’s mostly just
> iotest fixes, but there are actually two real fixes in here.
>
>
> v2:
> - Patch 2: Don’t treat extent filenames with protocol prefixes as
> absolute filenames – this may be the right thing to do, but:
> (1) path_combine() doesn’t (it just ignores whether the supposed
> relative filename has a potential protocol prefix), so this is how
> we handled it so far,
> (2) It would break other cases (when a filename contains a colon for
> no particular reason), as seen in iotest 126.
> That means you cannot have an extent file e.g. on an http server while
> the descriptor is on a local filesystem, but I hope nobody would ever
> want to do that.
>
I guess we'll fix it if it comes up.
> - Patch 3: Fix paste-o [John]
>
> - Patch 7: twoGbMaxExtentSparse works now with the change to patch 2, so
> we no longer have to mark it unsupported [Thanks for the insistent
> inquiry, John :-)]
>
Thanks for fixing the "weird" formats!
Reviewed-by: John Snow <jsnow@redhat.com>
>
> git-backport-diff against v1:
>
> Key:
> [----] : patches are identical
> [####] : number of functional differences between upstream/downstream patch
> [down] : patch is downstream-only
> The flags [FC] indicate (F)unctional and (C)ontextual differences, respectively
>
> 001/7:[----] [--] 'iotests: Fix _filter_img_create()'
> 002/7:[0002] [FC] 'vmdk: Use bdrv_dirname() for relative extent paths'
> 003/7:[0002] [FC] 'iotests: Keep testing broken relative extent paths'
> 004/7:[----] [--] 'vmdk: Reject invalid compressed writes'
> 005/7:[----] [--] 'iotests: Disable broken streamOptimized tests'
> 006/7:[----] [--] 'iotests: Disable 110 for vmdk.twoGbMaxExtentSparse'
> 007/7:[0006] [FC] 'iotests: Disable 126 for some vmdk subformats'
>
>
> Max Reitz (7):
> iotests: Fix _filter_img_create()
> vmdk: Use bdrv_dirname() for relative extent paths
> iotests: Keep testing broken relative extent paths
> vmdk: Reject invalid compressed writes
> iotests: Disable broken streamOptimized tests
> iotests: Disable 110 for vmdk.twoGbMaxExtentSparse
> iotests: Disable 126 for flat vmdk subformats
>
> block/vmdk.c | 64 ++++++++++++++++++++++----------
> tests/qemu-iotests/002 | 1 +
> tests/qemu-iotests/003 | 1 +
> tests/qemu-iotests/005 | 3 +-
> tests/qemu-iotests/009 | 1 +
> tests/qemu-iotests/010 | 1 +
> tests/qemu-iotests/011 | 1 +
> tests/qemu-iotests/017 | 3 +-
> tests/qemu-iotests/018 | 3 +-
> tests/qemu-iotests/019 | 3 +-
> tests/qemu-iotests/020 | 3 +-
> tests/qemu-iotests/027 | 1 +
> tests/qemu-iotests/032 | 1 +
> tests/qemu-iotests/033 | 1 +
> tests/qemu-iotests/034 | 3 +-
> tests/qemu-iotests/037 | 3 +-
> tests/qemu-iotests/059 | 34 ++++++++++++++++-
> tests/qemu-iotests/059.out | 24 +++++++-----
> tests/qemu-iotests/063 | 3 +-
> tests/qemu-iotests/072 | 1 +
> tests/qemu-iotests/105 | 3 +-
> tests/qemu-iotests/110 | 3 +-
> tests/qemu-iotests/126 | 2 +
> tests/qemu-iotests/197 | 1 +
> tests/qemu-iotests/215 | 1 +
> tests/qemu-iotests/251 | 1 +
> tests/qemu-iotests/common.filter | 4 +-
> 27 files changed, 127 insertions(+), 43 deletions(-)
>
--
—js
^ permalink raw reply [flat|nested] 10+ messages in thread* Re: [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes
2019-08-15 15:36 [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes Max Reitz
` (7 preceding siblings ...)
2019-08-15 21:20 ` [Qemu-devel] [PATCH v2 0/7] vmdk: Misc fixes John Snow
@ 2019-08-27 17:16 ` Max Reitz
8 siblings, 0 replies; 10+ messages in thread
From: Max Reitz @ 2019-08-27 17:16 UTC (permalink / raw)
To: qemu-block; +Cc: Kevin Wolf, John Snow, qemu-devel
[-- Attachment #1.1: Type: text/plain, Size: 452 bytes --]
On 15.08.19 17:36, Max Reitz wrote:
> I made the mistake of trying to run the iotests with all non-default
> subformats our vmdk driver has to offer:
> - monolithicFlat
> - twoGbMaxExtentSparse
> - twoGbMaxExtentFlat
> - streamOptimized
>
> Many things broke, so this series fixes what I found. It’s mostly just
> iotest fixes, but there are actually two real fixes in here.
Thanks for the review, applied to my block branch.
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 10+ messages in thread