* [Qemu-devel] [PATCH v6 00/10] add blkdebug tests
@ 2017-03-08 2:54 Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 01/10] iotests: fix 097 when run with qcow Eric Blake
` (10 more replies)
0 siblings, 11 replies; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Available as a tag at:
git fetch git://repo.or.cz/qemu/ericb.git nbd-blkdebug-v6
v5 was:
https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg03042.html
Since then:
- Rebase to master
- Pull in Dan's patch that splits test 97 for qcow (patch 1)
- Address comments from Kevin (includes creating new patch 2, 6)
- Update patches to reflect that this is now 2.10 material, since it
missed soft freeze and is not fixing a bug
001/10:[down] 'iotests: fix 097 when run with qcow'
002/10:[down] 'iotests: Improve image-clear tests on non-aligned image'
003/10:[0007] [FC] 'qcow2: Assert that cluster operations are aligned'
004/10:[0002] [FC] 'qcow2: Discard/zero clusters by byte count'
005/10:[----] [--] 'blkdebug: Sanity check block layer guarantees'
006/10:[down] 'blkdebug: Refactor error injection'
007/10:[0046] [FC] 'blkdebug: Add pass-through write_zero and discard support'
008/10:[0003] [FC] 'blkdebug: Simplify override logic'
009/10:[0045] [FC] 'blkdebug: Add ability to override unmap geometries'
010/10:[0004] [FC] 'tests: Add coverage for recent block geometry fixes'
Daniel P. Berrange (1):
iotests: fix 097 when run with qcow
Eric Blake (9):
iotests: Improve image-clear tests on non-aligned image
qcow2: Assert that cluster operations are aligned
qcow2: Discard/zero clusters by byte count
blkdebug: Sanity check block layer guarantees
blkdebug: Refactor error injection
blkdebug: Add pass-through write_zero and discard support
blkdebug: Simplify override logic
blkdebug: Add ability to override unmap geometries
tests: Add coverage for recent block geometry fixes
qapi/block-core.json | 31 +++++-
block/qcow2.h | 9 +-
block/blkdebug.c | 264 +++++++++++++++++++++++++++++++++++++--------
block/qcow2-cluster.c | 51 +++++----
block/qcow2-snapshot.c | 7 +-
block/qcow2.c | 21 ++--
tests/qemu-iotests/097 | 17 +--
tests/qemu-iotests/097.out | 149 +++----------------------
tests/qemu-iotests/176 | 127 ++++++++++++++++++++++
tests/qemu-iotests/176.out | 119 ++++++++++++++++++++
tests/qemu-iotests/177 | 114 ++++++++++++++++++++
tests/qemu-iotests/177.out | 49 +++++++++
tests/qemu-iotests/group | 2 +
13 files changed, 724 insertions(+), 236 deletions(-)
create mode 100755 tests/qemu-iotests/176
create mode 100644 tests/qemu-iotests/176.out
create mode 100755 tests/qemu-iotests/177
create mode 100644 tests/qemu-iotests/177.out
--
2.9.3
^ permalink raw reply [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 01/10] iotests: fix 097 when run with qcow
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image Eric Blake
` (9 subsequent siblings)
10 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz, Daniel P. Berrange
From: "Daniel P. Berrange" <berrange@redhat.com>
The previous commit:
commit a3e1505daec31ef56f0489f8c8fff1b8e4ca92bd
Author: Eric Blake <eblake@redhat.com>
Date: Mon Dec 5 09:49:34 2016 -0600
qcow2: Don't strand clusters near 2G intervals during commit
extended the 097 test case so that it did two passes, once
with an internal snapshot, once without.
qcow (v1) does not support internal snapshots, so this change
broke test 097 when run against qcow.
This splits 097 in two, creating a new 176 that tests the
internal snapshot codepath, effectively putting 097 back
to its content before the above commit.
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Daniel P. Berrange <berrange@redhat.com>
Message-Id: <20170221115512.21918-8-berrange@redhat.com>
[eblake: test collisions: s/173/176/g]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/097 | 10 +---
tests/qemu-iotests/097.out | 125 ++------------------------------------------
tests/qemu-iotests/176 | 126 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/176.out | 119 ++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/group | 1 +
5 files changed, 251 insertions(+), 130 deletions(-)
create mode 100755 tests/qemu-iotests/176
create mode 100644 tests/qemu-iotests/176.out
diff --git a/tests/qemu-iotests/097 b/tests/qemu-iotests/097
index 4c33e80..1d28aff 100755
--- a/tests/qemu-iotests/097
+++ b/tests/qemu-iotests/097
@@ -56,26 +56,19 @@ _supported_os Linux
# 3: Two-layer backing chain, commit to lower backing file
# (in this case, the top image will implicitly stay unchanged)
#
-# Each pass is run twice, since qcow2 has different code paths for cleaning
-# an image depending on whether it has a snapshot.
-#
# 020 already tests committing, so this only tests whether image chains are
# working properly and that all images above the base are emptied; therefore,
# no complicated patterns are necessary. Check near the 2G mark, as qcow2
# has been buggy at that boundary in the past.
for i in 0 1 2 3; do
-for j in 0 1; do
echo
-echo "=== Test pass $i.$j ==="
+echo "=== Test pass $i ==="
echo
TEST_IMG="$TEST_IMG.base" _make_test_img 2100M
TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" 2100M
_make_test_img -b "$TEST_IMG.itmd" 2100M
-if [ $j -eq 0 ]; then
- $QEMU_IMG snapshot -c snap "$TEST_IMG"
-fi
$QEMU_IO -c 'write -P 1 0x7ffd0000 192k' "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c 'write -P 2 0x7ffe0000 128k' "$TEST_IMG.itmd" | _filter_qemu_io
@@ -121,7 +114,6 @@ $QEMU_IMG map "$TEST_IMG.itmd" | _filter_qemu_img_map
$QEMU_IMG map "$TEST_IMG" | _filter_qemu_img_map
done
-done
# success, all done
diff --git a/tests/qemu-iotests/097.out b/tests/qemu-iotests/097.out
index 8106cc9..81fc225 100644
--- a/tests/qemu-iotests/097.out
+++ b/tests/qemu-iotests/097.out
@@ -1,6 +1,6 @@
QA output created by 097
-=== Test pass 0.0 ===
+=== Test pass 0 ===
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
@@ -29,66 +29,7 @@ Offset Length File
0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
-=== Test pass 0.1 ===
-
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
-wrote 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 131072/131072 bytes at offset 2147352576
-128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Image committed.
-read 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147287040
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147352576
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Offset Length File
-0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
-
-=== Test pass 1.0 ===
-
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
-wrote 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 131072/131072 bytes at offset 2147352576
-128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Image committed.
-read 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147287040
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147352576
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Offset Length File
-0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
-0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
-
-=== Test pass 1.1 ===
+=== Test pass 1 ===
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
@@ -118,7 +59,7 @@ Offset Length File
0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
-=== Test pass 2.0 ===
+=== Test pass 2 ===
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
@@ -148,65 +89,7 @@ Offset Length File
0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
-=== Test pass 2.1 ===
-
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
-wrote 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 131072/131072 bytes at offset 2147352576
-128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Image committed.
-read 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147287040
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147352576
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Offset Length File
-0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
-0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
-
-=== Test pass 3.0 ===
-
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
-wrote 196608/196608 bytes at offset 2147287040
-192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 131072/131072 bytes at offset 2147352576
-128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-wrote 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Image committed.
-read 65536/65536 bytes at offset 2147287040
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147352576
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-read 65536/65536 bytes at offset 2147418112
-64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
-Offset Length File
-0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
-Offset Length File
-0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
-0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
-0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
-
-=== Test pass 3.1 ===
+=== Test pass 3 ===
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
diff --git a/tests/qemu-iotests/176 b/tests/qemu-iotests/176
new file mode 100755
index 0000000..9a70a87
--- /dev/null
+++ b/tests/qemu-iotests/176
@@ -0,0 +1,126 @@
+#!/bin/bash
+#
+# Commit changes into backing chains and empty the top image if the
+# backing image is not explicitly specified.
+#
+# Variant of 097, which includes snapshots to test different codepath
+# in qcow2
+#
+# Copyright (C) 2014 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=mreitz@redhat.com
+
+seq="$(basename $0)"
+echo "QA output created by $seq"
+
+here="$PWD"
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+ _rm_test_img "$TEST_IMG.itmd"
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+. ./common.pattern
+
+# Any format supporting backing files and bdrv_make_empty
+_supported_fmt qcow2
+_supported_proto file
+_supported_os Linux
+
+
+# Four passes:
+# 0: Two-layer backing chain, commit to upper backing file (implicitly)
+# (in this case, the top image will be emptied)
+# 1: Two-layer backing chain, commit to upper backing file (explicitly)
+# (in this case, the top image will implicitly stay unchanged)
+# 2: Two-layer backing chain, commit to upper backing file (implicitly with -d)
+# (in this case, the top image will explicitly stay unchanged)
+# 3: Two-layer backing chain, commit to lower backing file
+# (in this case, the top image will implicitly stay unchanged)
+#
+# 020 already tests committing, so this only tests whether image chains are
+# working properly and that all images above the base are emptied; therefore,
+# no complicated patterns are necessary. Check near the 2G mark, as qcow2
+# has been buggy at that boundary in the past.
+for i in 0 1 2 3; do
+
+echo
+echo "=== Test pass $i ==="
+echo
+
+TEST_IMG="$TEST_IMG.base" _make_test_img 2100M
+TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" 2100M
+_make_test_img -b "$TEST_IMG.itmd" 2100M
+$QEMU_IMG snapshot -c snap "$TEST_IMG"
+
+$QEMU_IO -c 'write -P 1 0x7ffd0000 192k' "$TEST_IMG.base" | _filter_qemu_io
+$QEMU_IO -c 'write -P 2 0x7ffe0000 128k' "$TEST_IMG.itmd" | _filter_qemu_io
+$QEMU_IO -c 'write -P 3 0x7fff0000 64k' "$TEST_IMG" | _filter_qemu_io
+
+if [ $i -lt 3 ]; then
+ if [ $i == 0 ]; then
+ # -b "$TEST_IMG.itmd" should be the default (that is, committing to the
+ # first backing file in the chain)
+ $QEMU_IMG commit "$TEST_IMG"
+ elif [ $i == 1 ]; then
+ # explicitly specify the commit target (this should imply -d)
+ $QEMU_IMG commit -b "$TEST_IMG.itmd" "$TEST_IMG"
+ else
+ # do not explicitly specify the commit target, but use -d to leave the
+ # top image unchanged
+ $QEMU_IMG commit -d "$TEST_IMG"
+ fi
+
+ # Bottom should be unchanged
+ $QEMU_IO -c 'read -P 1 0x7ffd0000 192k' "$TEST_IMG.base" | _filter_qemu_io
+
+ # Intermediate should contain changes from top
+ $QEMU_IO -c 'read -P 1 0x7ffd0000 64k' "$TEST_IMG.itmd" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 2 0x7ffe0000 64k' "$TEST_IMG.itmd" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 3 0x7fff0000 64k' "$TEST_IMG.itmd" | _filter_qemu_io
+
+ # And in pass 0, the top image should be empty, whereas in both other passes
+ # it should be unchanged (which is both checked by qemu-img map)
+else
+ $QEMU_IMG commit -b "$TEST_IMG.base" "$TEST_IMG"
+
+ # Bottom should contain all changes
+ $QEMU_IO -c 'read -P 1 0x7ffd0000 64k' "$TEST_IMG.base" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 2 0x7ffe0000 64k' "$TEST_IMG.base" | _filter_qemu_io
+ $QEMU_IO -c 'read -P 3 0x7fff0000 64k' "$TEST_IMG.base" | _filter_qemu_io
+
+ # Both top and intermediate should be unchanged
+fi
+
+$QEMU_IMG map "$TEST_IMG.base" | _filter_qemu_img_map
+$QEMU_IMG map "$TEST_IMG.itmd" | _filter_qemu_img_map
+$QEMU_IMG map "$TEST_IMG" | _filter_qemu_img_map
+
+done
+
+
+# success, all done
+echo "*** done"
+rm -f $seq.full
+status=0
diff --git a/tests/qemu-iotests/176.out b/tests/qemu-iotests/176.out
new file mode 100644
index 0000000..fc63653
--- /dev/null
+++ b/tests/qemu-iotests/176.out
@@ -0,0 +1,119 @@
+QA output created by 176
+
+=== Test pass 0 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+wrote 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 2147352576
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147287040
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147352576
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
+
+=== Test pass 1 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+wrote 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 2147352576
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147287040
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147352576
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
+0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
+
+=== Test pass 2 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+wrote 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 2147352576
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147287040
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147352576
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
+0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
+
+=== Test pass 3 ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+wrote 196608/196608 bytes at offset 2147287040
+192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 131072/131072 bytes at offset 2147352576
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+wrote 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Image committed.
+read 65536/65536 bytes at offset 2147287040
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147352576
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 65536/65536 bytes at offset 2147418112
+64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Offset Length File
+0x7ffd0000 0x30000 TEST_DIR/t.IMGFMT.base
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x20000 TEST_DIR/t.IMGFMT.itmd
+Offset Length File
+0x7ffd0000 0x10000 TEST_DIR/t.IMGFMT.base
+0x7ffe0000 0x10000 TEST_DIR/t.IMGFMT.itmd
+0x7fff0000 0x10000 TEST_DIR/t.IMGFMT
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 1f4bf03..43142dd 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -168,3 +168,4 @@
173 rw auto
174 auto
175 auto quick
+176 rw auto backing
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 01/10] iotests: fix 097 when run with qcow Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-13 22:00 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 03/10] qcow2: Assert that cluster operations are aligned Eric Blake
` (8 subsequent siblings)
10 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Tweak 097 and 176 to operate on an image that is not cluster-aligned,
to give further coverage of clearing out an entire image.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v6: new patch
---
tests/qemu-iotests/097 | 7 ++++---
tests/qemu-iotests/097.out | 24 ++++++++++++------------
tests/qemu-iotests/176 | 7 ++++---
tests/qemu-iotests/176.out | 24 ++++++++++++------------
4 files changed, 32 insertions(+), 30 deletions(-)
diff --git a/tests/qemu-iotests/097 b/tests/qemu-iotests/097
index 1d28aff..2e3a886 100755
--- a/tests/qemu-iotests/097
+++ b/tests/qemu-iotests/097
@@ -66,9 +66,10 @@ echo
echo "=== Test pass $i ==="
echo
-TEST_IMG="$TEST_IMG.base" _make_test_img 2100M
-TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" 2100M
-_make_test_img -b "$TEST_IMG.itmd" 2100M
+len=$((2100 * 1024 * 1024 + 512)) # larger than 2G, and not cluster aligned
+TEST_IMG="$TEST_IMG.base" _make_test_img $len
+TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" $len
+_make_test_img -b "$TEST_IMG.itmd" $len
$QEMU_IO -c 'write -P 1 0x7ffd0000 192k' "$TEST_IMG.base" | _filter_qemu_io
$QEMU_IO -c 'write -P 2 0x7ffe0000 128k' "$TEST_IMG.itmd" | _filter_qemu_io
diff --git a/tests/qemu-iotests/097.out b/tests/qemu-iotests/097.out
index 81fc225..30a64ba 100644
--- a/tests/qemu-iotests/097.out
+++ b/tests/qemu-iotests/097.out
@@ -2,9 +2,9 @@ QA output created by 097
=== Test pass 0 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
@@ -31,9 +31,9 @@ Offset Length File
=== Test pass 1 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
@@ -61,9 +61,9 @@ Offset Length File
=== Test pass 2 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
@@ -91,9 +91,9 @@ Offset Length File
=== Test pass 3 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
diff --git a/tests/qemu-iotests/176 b/tests/qemu-iotests/176
index 9a70a87..39fc6fb 100755
--- a/tests/qemu-iotests/176
+++ b/tests/qemu-iotests/176
@@ -69,9 +69,10 @@ echo
echo "=== Test pass $i ==="
echo
-TEST_IMG="$TEST_IMG.base" _make_test_img 2100M
-TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" 2100M
-_make_test_img -b "$TEST_IMG.itmd" 2100M
+len=$((2100 * 1024 * 1024 + 512)) # larger than 2G, and not cluster aligned
+TEST_IMG="$TEST_IMG.base" _make_test_img $len
+TEST_IMG="$TEST_IMG.itmd" _make_test_img -b "$TEST_IMG.base" $len
+_make_test_img -b "$TEST_IMG.itmd" $len
$QEMU_IMG snapshot -c snap "$TEST_IMG"
$QEMU_IO -c 'write -P 1 0x7ffd0000 192k' "$TEST_IMG.base" | _filter_qemu_io
diff --git a/tests/qemu-iotests/176.out b/tests/qemu-iotests/176.out
index fc63653..db27b31 100644
--- a/tests/qemu-iotests/176.out
+++ b/tests/qemu-iotests/176.out
@@ -2,9 +2,9 @@ QA output created by 176
=== Test pass 0 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
@@ -31,9 +31,9 @@ Offset Length File
=== Test pass 1 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
@@ -61,9 +61,9 @@ Offset Length File
=== Test pass 2 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
@@ -91,9 +91,9 @@ Offset Length File
=== Test pass 3 ===
-Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202009600
-Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.base
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202009600 backing_file=TEST_DIR/t.IMGFMT.itmd
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=2202010112
+Formatting 'TEST_DIR/t.IMGFMT.itmd', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.base
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=2202010112 backing_file=TEST_DIR/t.IMGFMT.itmd
wrote 196608/196608 bytes at offset 2147287040
192 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
wrote 131072/131072 bytes at offset 2147352576
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 03/10] qcow2: Assert that cluster operations are aligned
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 01/10] iotests: fix 097 when run with qcow Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-13 22:07 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 04/10] qcow2: Discard/zero clusters by byte count Eric Blake
` (7 subsequent siblings)
10 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
qcow2_discard_clusters() is set up to silently ignore sub-cluster
head or tail on unaligned requests. However, it is easy to audit
the various callers: qcow2_snapshot_create() has always passed
aligned data since the call was introduced in 1ebf561;
qcow2_co_pdiscard() has passed aligned clusters since commit
ecdbead taught the block layer the preferred discard alignment (the
block layer can still pass sub-cluster values, but those are
handled directly in qcow2_co_pdiscard()); and qcow2_make_empty()
was fixed to pass aligned clusters in commit a3e1505. Replace
rounding with assertions to hold us to the tighter contract,
eliminating the now-impossible case of an early exit for a
sub-cluster request.
qcow2_zero_clusters() has always been called with cluster-aligned
arguments from its lone caller qcow2_co_pwrite_zeroes() (like
qcow2_co_pdiscard(), the caller takes care of sub-cluster requests
from the block layer; and qcow2_zero_clusters() would have
misbehaved on unaligned requests), but it deserves the same
assertion for symmetry.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v6: avoid assertion on non-cluster-aligned image, use s->cluster_sectors
to avoid a shift, drop R-b
v5: no change
v4: new patch
---
block/qcow2-cluster.c | 15 ++++++++-------
1 file changed, 8 insertions(+), 7 deletions(-)
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 78c11d4..046fbd8 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1519,13 +1519,10 @@ int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
end_offset = offset + (nb_sectors << BDRV_SECTOR_BITS);
- /* Round start up and end down */
- offset = align_offset(offset, s->cluster_size);
- end_offset = start_of_cluster(s, end_offset);
-
- if (offset > end_offset) {
- return 0;
- }
+ /* Caller must pass aligned values, except at image end */
+ assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
+ assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) ||
+ end_offset == bs->total_sectors << BDRV_SECTOR_BITS);
nb_clusters = size_to_clusters(s, end_offset - offset);
@@ -1600,6 +1597,10 @@ int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors,
uint64_t nb_clusters;
int ret;
+ /* Caller must pass aligned values */
+ assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
+ assert(QEMU_IS_ALIGNED(nb_sectors, s->cluster_sectors));
+
/* The zero flag is only supported by version 3 and newer */
if (s->qcow_version < 3) {
return -ENOTSUP;
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 04/10] qcow2: Discard/zero clusters by byte count
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (2 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 03/10] qcow2: Assert that cluster operations are aligned Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 05/10] blkdebug: Sanity check block layer guarantees Eric Blake
` (6 subsequent siblings)
10 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Passing a byte offset, but sector count, when we ultimately
want to operate on cluster granularity, is madness. Clean up
the external interfaces to take both offset and count as bytes,
while still keeping the assertion added previously that the
caller must align the values to a cluster. Then rename things
to make sure backports don't get confused by changed units:
instead of qcow2_discard_clusters() and qcow2_zero_clusters(),
we now have qcow2_cluster_discard() and qcow2_cluster_zeroize().
The internal functions still operate on clusters at a time, and
return an int for number of cleared clusters; but on an image
with 2M clusters, a single L2 table holds 256k entries that each
represent a 2M cluster, totalling well over INT_MAX bytes if we
ever had a request for that many bytes at once. All our callers
currently limit themselves to 32-bit bytes (and therefore fewer
clusters), but by making this function 64-bit clean, we have one
less place to clean up if we later improve the block layer to
support 64-bit bytes through all operations (with the block layer
auto-fragmenting on behalf of more-limited drivers), rather than
the current state where some interfaces are artificially limited
to INT_MAX at a time.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
---
v6: rebase due to context
v5: s/count/byte/ to make the units obvious, and rework the math
to ensure no 32-bit integer overflow on large clusters
v4: improve function names, split assertion additions into earlier patch
[no v3 or v2]
v1: https://lists.gnu.org/archive/html/qemu-devel/2016-12/msg00339.html
---
block/qcow2.h | 9 +++++----
block/qcow2-cluster.c | 38 +++++++++++++++++++++-----------------
block/qcow2-snapshot.c | 7 +++----
block/qcow2.c | 21 +++++++++------------
4 files changed, 38 insertions(+), 37 deletions(-)
diff --git a/block/qcow2.h b/block/qcow2.h
index f8aeb08..808104c 100644
--- a/block/qcow2.h
+++ b/block/qcow2.h
@@ -544,10 +544,11 @@ uint64_t qcow2_alloc_compressed_cluster_offset(BlockDriverState *bs,
int compressed_size);
int qcow2_alloc_cluster_link_l2(BlockDriverState *bs, QCowL2Meta *m);
-int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
- int nb_sectors, enum qcow2_discard_type type, bool full_discard);
-int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors,
- int flags);
+int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset,
+ uint64_t bytes, enum qcow2_discard_type type,
+ bool full_discard);
+int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
+ uint64_t bytes, int flags);
int qcow2_expand_zero_clusters(BlockDriverState *bs,
BlockDriverAmendStatusCB *status_cb,
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 046fbd8..ca9e7e6 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1509,16 +1509,16 @@ static int discard_single_l2(BlockDriverState *bs, uint64_t offset,
return nb_clusters;
}
-int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
- int nb_sectors, enum qcow2_discard_type type, bool full_discard)
+int qcow2_cluster_discard(BlockDriverState *bs, uint64_t offset,
+ uint64_t bytes, enum qcow2_discard_type type,
+ bool full_discard)
{
BDRVQcow2State *s = bs->opaque;
- uint64_t end_offset;
+ uint64_t end_offset = offset + bytes;
uint64_t nb_clusters;
+ int64_t cleared;
int ret;
- end_offset = offset + (nb_sectors << BDRV_SECTOR_BITS);
-
/* Caller must pass aligned values, except at image end */
assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) ||
@@ -1530,13 +1530,15 @@ int qcow2_discard_clusters(BlockDriverState *bs, uint64_t offset,
/* Each L2 table is handled by its own loop iteration */
while (nb_clusters > 0) {
- ret = discard_single_l2(bs, offset, nb_clusters, type, full_discard);
- if (ret < 0) {
+ cleared = discard_single_l2(bs, offset, nb_clusters, type,
+ full_discard);
+ if (cleared < 0) {
+ ret = cleared;
goto fail;
}
- nb_clusters -= ret;
- offset += (ret * s->cluster_size);
+ nb_clusters -= cleared;
+ offset += (cleared * s->cluster_size);
}
ret = 0;
@@ -1590,16 +1592,17 @@ static int zero_single_l2(BlockDriverState *bs, uint64_t offset,
return nb_clusters;
}
-int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors,
- int flags)
+int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
+ uint64_t bytes, int flags)
{
BDRVQcow2State *s = bs->opaque;
uint64_t nb_clusters;
+ int64_t cleared;
int ret;
/* Caller must pass aligned values */
assert(QEMU_IS_ALIGNED(offset, s->cluster_size));
- assert(QEMU_IS_ALIGNED(nb_sectors, s->cluster_sectors));
+ assert(QEMU_IS_ALIGNED(bytes, s->cluster_size));
/* The zero flag is only supported by version 3 and newer */
if (s->qcow_version < 3) {
@@ -1607,18 +1610,19 @@ int qcow2_zero_clusters(BlockDriverState *bs, uint64_t offset, int nb_sectors,
}
/* Each L2 table is handled by its own loop iteration */
- nb_clusters = size_to_clusters(s, nb_sectors << BDRV_SECTOR_BITS);
+ nb_clusters = size_to_clusters(s, bytes);
s->cache_discards = true;
while (nb_clusters > 0) {
- ret = zero_single_l2(bs, offset, nb_clusters, flags);
- if (ret < 0) {
+ cleared = zero_single_l2(bs, offset, nb_clusters, flags);
+ if (cleared < 0) {
+ ret = cleared;
goto fail;
}
- nb_clusters -= ret;
- offset += (ret * s->cluster_size);
+ nb_clusters -= cleared;
+ offset += (cleared * s->cluster_size);
}
ret = 0;
diff --git a/block/qcow2-snapshot.c b/block/qcow2-snapshot.c
index 0324243..44243e0 100644
--- a/block/qcow2-snapshot.c
+++ b/block/qcow2-snapshot.c
@@ -440,10 +440,9 @@ int qcow2_snapshot_create(BlockDriverState *bs, QEMUSnapshotInfo *sn_info)
/* The VM state isn't needed any more in the active L1 table; in fact, it
* hurts by causing expensive COW for the next snapshot. */
- qcow2_discard_clusters(bs, qcow2_vm_state_offset(s),
- align_offset(sn->vm_state_size, s->cluster_size)
- >> BDRV_SECTOR_BITS,
- QCOW2_DISCARD_NEVER, false);
+ qcow2_cluster_discard(bs, qcow2_vm_state_offset(s),
+ align_offset(sn->vm_state_size, s->cluster_size),
+ QCOW2_DISCARD_NEVER, false);
#ifdef DEBUG_ALLOC
{
diff --git a/block/qcow2.c b/block/qcow2.c
index 6a92d2e..03215e2 100644
--- a/block/qcow2.c
+++ b/block/qcow2.c
@@ -2501,7 +2501,7 @@ static coroutine_fn int qcow2_co_pwrite_zeroes(BlockDriverState *bs,
trace_qcow2_pwrite_zeroes(qemu_coroutine_self(), offset, count);
/* Whatever is left can use real zero clusters */
- ret = qcow2_zero_clusters(bs, offset, count >> BDRV_SECTOR_BITS, flags);
+ ret = qcow2_cluster_zeroize(bs, offset, count, flags);
qemu_co_mutex_unlock(&s->lock);
return ret;
@@ -2519,8 +2519,8 @@ static coroutine_fn int qcow2_co_pdiscard(BlockDriverState *bs,
}
qemu_co_mutex_lock(&s->lock);
- ret = qcow2_discard_clusters(bs, offset, count >> BDRV_SECTOR_BITS,
- QCOW2_DISCARD_REQUEST, false);
+ ret = qcow2_cluster_discard(bs, offset, count, QCOW2_DISCARD_REQUEST,
+ false);
qemu_co_mutex_unlock(&s->lock);
return ret;
}
@@ -2822,9 +2822,8 @@ fail:
static int qcow2_make_empty(BlockDriverState *bs)
{
BDRVQcow2State *s = bs->opaque;
- uint64_t start_sector;
- int sector_step = (QEMU_ALIGN_DOWN(INT_MAX, s->cluster_size) /
- BDRV_SECTOR_SIZE);
+ uint64_t offset, end_offset;
+ int step = QEMU_ALIGN_DOWN(INT_MAX, s->cluster_size);
int l1_clusters, ret = 0;
l1_clusters = DIV_ROUND_UP(s->l1_size, s->cluster_size / sizeof(uint64_t));
@@ -2841,18 +2840,16 @@ static int qcow2_make_empty(BlockDriverState *bs)
/* This fallback code simply discards every active cluster; this is slow,
* but works in all cases */
- for (start_sector = 0; start_sector < bs->total_sectors;
- start_sector += sector_step)
+ end_offset = bs->total_sectors * BDRV_SECTOR_SIZE;
+ for (offset = 0; offset < end_offset; offset += step)
{
/* As this function is generally used after committing an external
* snapshot, QCOW2_DISCARD_SNAPSHOT seems appropriate. Also, the
* default action for this kind of discard is to pass the discard,
* which will ideally result in an actually smaller image file, as
* is probably desired. */
- ret = qcow2_discard_clusters(bs, start_sector * BDRV_SECTOR_SIZE,
- MIN(sector_step,
- bs->total_sectors - start_sector),
- QCOW2_DISCARD_SNAPSHOT, true);
+ ret = qcow2_cluster_discard(bs, offset, MIN(step, end_offset - offset),
+ QCOW2_DISCARD_SNAPSHOT, true);
if (ret < 0) {
break;
}
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 05/10] blkdebug: Sanity check block layer guarantees
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (3 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 04/10] qcow2: Discard/zero clusters by byte count Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 06/10] blkdebug: Refactor error injection Eric Blake
` (5 subsequent siblings)
10 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Commits 04ed95f4 and 1a62d0ac updated the block layer to auto-fragment
any I/O to fit within device boundaries. Additionally, when using a
minimum alignment of 4k, we want to ensure the block layer does proper
read-modify-write rather than requesting I/O on a slice of a sector.
Let's enforce that the contract is obeyed when using blkdebug. For
now, blkdebug only allows alignment overrides, and just inherits other
limits from whatever device it is wrapping, but a future patch will
further enhance things.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
v5: no change
v4: no change
v3: rebase to byte-based interfaces
v2: new patch
---
block/blkdebug.c | 14 ++++++++++++++
1 file changed, 14 insertions(+)
diff --git a/block/blkdebug.c b/block/blkdebug.c
index 67e8024..7c804b2 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -431,6 +431,13 @@ blkdebug_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
BDRVBlkdebugState *s = bs->opaque;
BlkdebugRule *rule = NULL;
+ /* Sanity check block layer guarantees */
+ assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
+ assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment));
+ if (bs->bl.max_transfer) {
+ assert(bytes <= bs->bl.max_transfer);
+ }
+
QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
uint64_t inject_offset = rule->options.inject.offset;
@@ -455,6 +462,13 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
BDRVBlkdebugState *s = bs->opaque;
BlkdebugRule *rule = NULL;
+ /* Sanity check block layer guarantees */
+ assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
+ assert(QEMU_IS_ALIGNED(bytes, bs->bl.request_alignment));
+ if (bs->bl.max_transfer) {
+ assert(bytes <= bs->bl.max_transfer);
+ }
+
QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
uint64_t inject_offset = rule->options.inject.offset;
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 06/10] blkdebug: Refactor error injection
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (4 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 05/10] blkdebug: Sanity check block layer guarantees Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-13 22:16 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 07/10] blkdebug: Add pass-through write_zero and discard support Eric Blake
` (4 subsequent siblings)
10 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Rather than repeat the logic at each caller of checking if a Rule
exists that warrants an error injection, fold that logic into
inject_error(); and rename it to rule_check() for legibility.
This will help the next patch, which adds two more callers that
need to check rules for the potential of injecting errors.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v6: new patch
---
block/blkdebug.c | 74 +++++++++++++++++++++++++-------------------------------
1 file changed, 33 insertions(+), 41 deletions(-)
diff --git a/block/blkdebug.c b/block/blkdebug.c
index 7c804b2..f721bcb 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -405,11 +405,30 @@ out:
return ret;
}
-static int inject_error(BlockDriverState *bs, BlkdebugRule *rule)
+static int rule_check(BlockDriverState *bs, uint64_t offset, uint64_t bytes)
{
BDRVBlkdebugState *s = bs->opaque;
- int error = rule->options.inject.error;
- bool immediately = rule->options.inject.immediately;
+ BlkdebugRule *rule = NULL;
+ int error;
+ bool immediately;
+
+ QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
+ uint64_t inject_offset = rule->options.inject.offset;
+
+ if (inject_offset == -1 ||
+ (bytes && inject_offset >= offset &&
+ inject_offset < offset + bytes))
+ {
+ break;
+ }
+ }
+
+ if (!rule || !rule->options.inject.error) {
+ return 0;
+ }
+
+ immediately = rule->options.inject.immediately;
+ error = rule->options.inject.error;
if (rule->options.inject.once) {
QSIMPLEQ_REMOVE(&s->active_rules, rule, BlkdebugRule, active_next);
@@ -428,8 +447,7 @@ static int coroutine_fn
blkdebug_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
QEMUIOVector *qiov, int flags)
{
- BDRVBlkdebugState *s = bs->opaque;
- BlkdebugRule *rule = NULL;
+ int err;
/* Sanity check block layer guarantees */
assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
@@ -438,18 +456,9 @@ blkdebug_co_preadv(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
assert(bytes <= bs->bl.max_transfer);
}
- QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
- uint64_t inject_offset = rule->options.inject.offset;
-
- if (inject_offset == -1 ||
- (inject_offset >= offset && inject_offset < offset + bytes))
- {
- break;
- }
- }
-
- if (rule && rule->options.inject.error) {
- return inject_error(bs, rule);
+ err = rule_check(bs, offset, bytes);
+ if (err) {
+ return err;
}
return bdrv_co_preadv(bs->file, offset, bytes, qiov, flags);
@@ -459,8 +468,7 @@ static int coroutine_fn
blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
QEMUIOVector *qiov, int flags)
{
- BDRVBlkdebugState *s = bs->opaque;
- BlkdebugRule *rule = NULL;
+ int err;
/* Sanity check block layer guarantees */
assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
@@ -469,18 +477,9 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
assert(bytes <= bs->bl.max_transfer);
}
- QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
- uint64_t inject_offset = rule->options.inject.offset;
-
- if (inject_offset == -1 ||
- (inject_offset >= offset && inject_offset < offset + bytes))
- {
- break;
- }
- }
-
- if (rule && rule->options.inject.error) {
- return inject_error(bs, rule);
+ err = rule_check(bs, offset, bytes);
+ if (err) {
+ return err;
}
return bdrv_co_pwritev(bs->file, offset, bytes, qiov, flags);
@@ -488,17 +487,10 @@ blkdebug_co_pwritev(BlockDriverState *bs, uint64_t offset, uint64_t bytes,
static int blkdebug_co_flush(BlockDriverState *bs)
{
- BDRVBlkdebugState *s = bs->opaque;
- BlkdebugRule *rule = NULL;
+ int err = rule_check(bs, 0, 0);
- QSIMPLEQ_FOREACH(rule, &s->active_rules, active_next) {
- if (rule->options.inject.offset == -1) {
- break;
- }
- }
-
- if (rule && rule->options.inject.error) {
- return inject_error(bs, rule);
+ if (err) {
+ return err;
}
return bdrv_co_flush(bs->file->bs);
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 07/10] blkdebug: Add pass-through write_zero and discard support
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (5 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 06/10] blkdebug: Refactor error injection Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-13 22:20 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 08/10] blkdebug: Simplify override logic Eric Blake
` (3 subsequent siblings)
10 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
In order to test the effects of artificial geometry constraints
on operations like write zero or discard, we first need blkdebug
to manage these actions. It also allows us to inject errors on
those operations, just like we can for read/write/flush.
We can also test the contract promised by the block layer; namely,
if a device has specified limits on alignment or maximum size,
then those limits must be obeyed (for now, the blkdebug driver
merely inherits limits from whatever it is wrapping, but the next
patch will further enhance it to allow specific limit overrides).
This patch intentionally refuses to service requests smaller than
the requested alignments; this is because an upcoming patch adds
a qemu-iotest to prove that the block layer is correctly handling
fragmentation, but the test only works if there is a way to tell
the difference at artificial alignment boundaries when blkdebug is
using a larger-than-default alignment. If we let the blkdebug
layer always defer to the underlying layer, which potentially has
a smaller granularity, the iotest will be thwarted.
Tested by setting up an NBD server with export 'foo', then invoking:
$ ./qemu-io
qemu-io> open -o driver=blkdebug blkdebug::nbd://localhost:10809/foo
qemu-io> d 0 15M
qemu-io> w -z 0 15M
Pre-patch, the server never sees the discard (it was silently
eaten by the block layer); post-patch it is passed across the
wire. Likewise, pre-patch the write is always passed with
NBD_WRITE (with 15M of zeroes on the wire), while post-patch
it can utilize NBD_WRITE_ZEROES (for less traffic).
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v6: tighten check of unaligned requests, rebase on rule check
refactoring, drop R-b
v5: include 2017 copyright
v4: correct error injection to respect byte range, tweak formatting
v3: rebase to byte-based read/write, improve docs on why no
partial write zero passthrough
v2: new patch
---
block/blkdebug.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 74 insertions(+)
diff --git a/block/blkdebug.c b/block/blkdebug.c
index f721bcb..b3aa5c3 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -1,6 +1,7 @@
/*
* Block protocol for I/O error injection
*
+ * Copyright (C) 2016-2017 Red Hat, Inc.
* Copyright (c) 2010 Kevin Wolf <kwolf@redhat.com>
*
* Permission is hereby granted, free of charge, to any person obtaining a copy
@@ -382,6 +383,11 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
goto out;
}
+ bs->supported_write_flags = BDRV_REQ_FUA &
+ bs->file->bs->supported_write_flags;
+ bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) &
+ bs->file->bs->supported_zero_flags;
+
/* Set request alignment */
align = qemu_opt_get_size(opts, "align", 0);
if (align < INT_MAX && is_power_of_2(align)) {
@@ -496,6 +502,72 @@ static int blkdebug_co_flush(BlockDriverState *bs)
return bdrv_co_flush(bs->file->bs);
}
+static int coroutine_fn blkdebug_co_pwrite_zeroes(BlockDriverState *bs,
+ int64_t offset, int count,
+ BdrvRequestFlags flags)
+{
+ uint32_t align = MAX(bs->bl.request_alignment,
+ bs->bl.pwrite_zeroes_alignment);
+ int err;
+
+ /* Only pass through requests that are larger than requested
+ * preferred alignment (so that we test the fallback to writes on
+ * unaligned portions), and check that the block layer never hands
+ * us anything unaligned that crosses an alignment boundary. */
+ if (count < align) {
+ assert(QEMU_IS_ALIGNED(offset, align) ||
+ QEMU_IS_ALIGNED(offset + count, align) ||
+ DIV_ROUND_UP(offset, align) ==
+ DIV_ROUND_UP(offset + count, align));
+ return -ENOTSUP;
+ }
+ assert(QEMU_IS_ALIGNED(offset, align));
+ assert(QEMU_IS_ALIGNED(count, align));
+ if (bs->bl.max_pwrite_zeroes) {
+ assert(count <= bs->bl.max_pwrite_zeroes);
+ }
+
+ err = rule_check(bs, offset, count);
+ if (err) {
+ return err;
+ }
+
+ return bdrv_co_pwrite_zeroes(bs->file, offset, count, flags);
+}
+
+static int coroutine_fn blkdebug_co_pdiscard(BlockDriverState *bs,
+ int64_t offset, int count)
+{
+ uint32_t align = bs->bl.pdiscard_alignment;
+ int err;
+
+ /* Only pass through requests that are larger than requested
+ * minimum alignment, and ensure that unaligned requests do not
+ * cross optimum discard boundaries. */
+ if (count < bs->bl.request_alignment) {
+ assert(QEMU_IS_ALIGNED(offset, align) ||
+ QEMU_IS_ALIGNED(offset + count, align) ||
+ DIV_ROUND_UP(offset, align) ==
+ DIV_ROUND_UP(offset + count, align));
+ return -ENOTSUP;
+ }
+ assert(QEMU_IS_ALIGNED(offset, bs->bl.request_alignment));
+ assert(QEMU_IS_ALIGNED(count, bs->bl.request_alignment));
+ if (align && count >= align) {
+ assert(QEMU_IS_ALIGNED(offset, align));
+ assert(QEMU_IS_ALIGNED(count, align));
+ }
+ if (bs->bl.max_pdiscard) {
+ assert(count <= bs->bl.max_pdiscard);
+ }
+
+ err = rule_check(bs, offset, count);
+ if (err) {
+ return err;
+ }
+
+ return bdrv_co_pdiscard(bs->file->bs, offset, count);
+}
static void blkdebug_close(BlockDriverState *bs)
{
@@ -750,6 +822,8 @@ static BlockDriver bdrv_blkdebug = {
.bdrv_co_preadv = blkdebug_co_preadv,
.bdrv_co_pwritev = blkdebug_co_pwritev,
.bdrv_co_flush_to_disk = blkdebug_co_flush,
+ .bdrv_co_pwrite_zeroes = blkdebug_co_pwrite_zeroes,
+ .bdrv_co_pdiscard = blkdebug_co_pdiscard,
.bdrv_debug_event = blkdebug_debug_event,
.bdrv_debug_breakpoint = blkdebug_debug_breakpoint,
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 08/10] blkdebug: Simplify override logic
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (6 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 07/10] blkdebug: Add pass-through write_zero and discard support Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 09/10] blkdebug: Add ability to override unmap geometries Eric Blake
` (2 subsequent siblings)
10 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Rather than store into a local variable, then copy to the struct
if the value is valid, then reporting errors otherwise, it is
simpler to just store into the struct and report errors if the
value is invalid. This however requires that the struct store
a 64-bit number, rather than a narrower type. Move the errno
assignment into a label that will be reused from more places in
the next patch.
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Reviewed-by: Kevin Wolf <kwolf@redhat.com>
---
v6: tweak error message, but R-b kept
v5: no change
v4: fix typo in commit message, move errno assignment
v3: new patch
---
block/blkdebug.c | 14 ++++++--------
1 file changed, 6 insertions(+), 8 deletions(-)
diff --git a/block/blkdebug.c b/block/blkdebug.c
index b3aa5c3..dd5d85d 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -38,7 +38,7 @@
typedef struct BDRVBlkdebugState {
int state;
int new_state;
- int align;
+ uint64_t align;
/* For blkdebug_refresh_filename() */
char *config_file;
@@ -353,7 +353,6 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
BDRVBlkdebugState *s = bs->opaque;
QemuOpts *opts;
Error *local_err = NULL;
- uint64_t align;
int ret;
opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
@@ -389,12 +388,10 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
bs->file->bs->supported_zero_flags;
/* Set request alignment */
- align = qemu_opt_get_size(opts, "align", 0);
- if (align < INT_MAX && is_power_of_2(align)) {
- s->align = align;
- } else if (align) {
- error_setg(errp, "Invalid alignment");
- ret = -EINVAL;
+ s->align = qemu_opt_get_size(opts, "align", 0);
+ if (s->align && (s->align >= INT_MAX || !is_power_of_2(s->align))) {
+ error_setg(errp, "Cannot meet constraints with align %" PRIu64,
+ s->align);
goto fail_unref;
}
@@ -402,6 +399,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
goto out;
fail_unref:
+ ret = -EINVAL;
bdrv_unref_child(bs, bs->file);
out:
if (ret < 0) {
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 09/10] blkdebug: Add ability to override unmap geometries
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (7 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 08/10] blkdebug: Simplify override logic Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-13 22:22 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 10/10] tests: Add coverage for recent block geometry fixes Eric Blake
2017-03-08 8:40 ` [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Kevin Wolf
10 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz, Markus Armbruster
Make it easier to simulate various unusual hardware setups (for
example, recent commits 3482b9b and b8d0a98 affect the Dell
Equallogic iSCSI with its 15M preferred and maximum unmap and
write zero sizing, or b2f95fe deals with the Linux loopback
block device having a max_transfer of 64k), by allowing blkdebug
to wrap any other device with further restrictions on various
alignments.
Signed-off-by: Eric Blake <eblake@redhat.com>
---
v6: more tweaks in docs and error messages
v5: tweak docs regarding max-transfer minimum
v4: relax 512 byte minimum now that blkdebug is byte-based, fix doc typo
v3: improve legibility of bounds checking, improve docs
v2: new patch
---
qapi/block-core.json | 31 +++++++++++++++--
block/blkdebug.c | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++-
2 files changed, 124 insertions(+), 3 deletions(-)
diff --git a/qapi/block-core.json b/qapi/block-core.json
index 9bb7f4a..e8aeb8e 100644
--- a/qapi/block-core.json
+++ b/qapi/block-core.json
@@ -2476,7 +2476,32 @@
# @config: #optional filename of the configuration file
#
# @align: #optional required alignment for requests in bytes,
-# must be power of 2, or 0 for default
+# must be positive power of 2, or 0 for default
+#
+# @max-transfer: #optional maximum size for I/O transfers in bytes,
+# must be positive multiple of @align and of the
+# underlying file's request alignment (but need not be
+# a power of 2), or 0 for default (since 2.10)
+#
+# @opt-write-zero: #optional preferred alignment for write zero requests
+# in bytes, must be positive multiple of @align and of
+# the underlying file's request alignment (but need not
+# be a power of 2), or 0 for default (since 2.10)
+#
+# @max-write-zero: #optional maximum size for write zero requests in bytes,
+# must be positive multiple of @align, of @opt-write-zero,
+# and of the underlying file's request alignment (but
+# need not be a power of 2), or 0 for default (since 2.10)
+#
+# @opt-discard: #optional preferred alignment for discard requests
+# in bytes, must be positive multiple of @align and of
+# the underlying file's request alignment (but need not
+# be a power of 2), or 0 for default (since 2.10)
+#
+# @max-discard: #optional maximum size for discard requests in bytes,
+# must be positive multiple of @align, of @opt-discard,
+# and of the underlying file's request alignment (but
+# need not be a power of 2), or 0 for default (since 2.10)
#
# @inject-error: #optional array of error injection descriptions
#
@@ -2487,7 +2512,9 @@
{ 'struct': 'BlockdevOptionsBlkdebug',
'data': { 'image': 'BlockdevRef',
'*config': 'str',
- '*align': 'int',
+ '*align': 'int', '*max-transfer': 'int32',
+ '*opt-write-zero': 'int32', '*max-write-zero': 'int32',
+ '*opt-discard': 'int32', '*max-discard': 'int32',
'*inject-error': ['BlkdebugInjectErrorOptions'],
'*set-state': ['BlkdebugSetStateOptions'] } }
diff --git a/block/blkdebug.c b/block/blkdebug.c
index dd5d85d..4e2c392 100644
--- a/block/blkdebug.c
+++ b/block/blkdebug.c
@@ -39,6 +39,11 @@ typedef struct BDRVBlkdebugState {
int state;
int new_state;
uint64_t align;
+ uint64_t max_transfer;
+ uint64_t opt_write_zero;
+ uint64_t max_write_zero;
+ uint64_t opt_discard;
+ uint64_t max_discard;
/* For blkdebug_refresh_filename() */
char *config_file;
@@ -343,6 +348,31 @@ static QemuOptsList runtime_opts = {
.type = QEMU_OPT_SIZE,
.help = "Required alignment in bytes",
},
+ {
+ .name = "max-transfer",
+ .type = QEMU_OPT_SIZE,
+ .help = "Maximum transfer size in bytes",
+ },
+ {
+ .name = "opt-write-zero",
+ .type = QEMU_OPT_SIZE,
+ .help = "Optimum write zero alignment in bytes",
+ },
+ {
+ .name = "max-write-zero",
+ .type = QEMU_OPT_SIZE,
+ .help = "Maximum write zero size in bytes",
+ },
+ {
+ .name = "opt-discard",
+ .type = QEMU_OPT_SIZE,
+ .help = "Optimum discard alignment in bytes",
+ },
+ {
+ .name = "max-discard",
+ .type = QEMU_OPT_SIZE,
+ .help = "Maximum discard size in bytes",
+ },
{ /* end of list */ }
},
};
@@ -354,6 +384,7 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
QemuOpts *opts;
Error *local_err = NULL;
int ret;
+ uint64_t align;
opts = qemu_opts_create(&runtime_opts, NULL, 0, &error_abort);
qemu_opts_absorb_qdict(opts, options, &local_err);
@@ -387,13 +418,61 @@ static int blkdebug_open(BlockDriverState *bs, QDict *options, int flags,
bs->supported_zero_flags = (BDRV_REQ_FUA | BDRV_REQ_MAY_UNMAP) &
bs->file->bs->supported_zero_flags;
- /* Set request alignment */
+ /* Set alignment overrides */
s->align = qemu_opt_get_size(opts, "align", 0);
if (s->align && (s->align >= INT_MAX || !is_power_of_2(s->align))) {
error_setg(errp, "Cannot meet constraints with align %" PRIu64,
s->align);
goto fail_unref;
}
+ align = MAX(s->align, bs->file->bs->bl.request_alignment);
+
+ s->max_transfer = qemu_opt_get_size(opts, "max-transfer", 0);
+ if (s->max_transfer &&
+ (s->max_transfer >= INT_MAX ||
+ !QEMU_IS_ALIGNED(s->max_transfer, align))) {
+ error_setg(errp, "Cannot meet constraints with max-transfer %" PRIu64,
+ s->max_transfer);
+ goto fail_unref;
+ }
+
+ s->opt_write_zero = qemu_opt_get_size(opts, "opt-write-zero", 0);
+ if (s->opt_write_zero &&
+ (s->opt_write_zero >= INT_MAX ||
+ !QEMU_IS_ALIGNED(s->opt_write_zero, align))) {
+ error_setg(errp, "Cannot meet constraints with opt-write-zero %" PRIu64,
+ s->opt_write_zero);
+ goto fail_unref;
+ }
+
+ s->max_write_zero = qemu_opt_get_size(opts, "max-write-zero", 0);
+ if (s->max_write_zero &&
+ (s->max_write_zero >= INT_MAX ||
+ !QEMU_IS_ALIGNED(s->max_write_zero,
+ MAX(s->opt_write_zero, align)))) {
+ error_setg(errp, "Cannot meet constraints with max-write-zero %" PRIu64,
+ s->max_write_zero);
+ goto fail_unref;
+ }
+
+ s->opt_discard = qemu_opt_get_size(opts, "opt-discard", 0);
+ if (s->opt_discard &&
+ (s->opt_discard >= INT_MAX ||
+ !QEMU_IS_ALIGNED(s->opt_discard, align))) {
+ error_setg(errp, "Cannot meet constraints with opt-discard %" PRIu64,
+ s->opt_discard);
+ goto fail_unref;
+ }
+
+ s->max_discard = qemu_opt_get_size(opts, "max-discard", 0);
+ if (s->max_discard &&
+ (s->max_discard >= INT_MAX ||
+ !QEMU_IS_ALIGNED(s->max_discard,
+ MAX(s->opt_discard, align)))) {
+ error_setg(errp, "Cannot meet constraints with max-discard %" PRIu64,
+ s->max_discard);
+ goto fail_unref;
+ }
ret = 0;
goto out;
@@ -793,6 +872,21 @@ static void blkdebug_refresh_limits(BlockDriverState *bs, Error **errp)
if (s->align) {
bs->bl.request_alignment = s->align;
}
+ if (s->max_transfer) {
+ bs->bl.max_transfer = s->max_transfer;
+ }
+ if (s->opt_write_zero) {
+ bs->bl.pwrite_zeroes_alignment = s->opt_write_zero;
+ }
+ if (s->max_write_zero) {
+ bs->bl.max_pwrite_zeroes = s->max_write_zero;
+ }
+ if (s->opt_discard) {
+ bs->bl.pdiscard_alignment = s->opt_discard;
+ }
+ if (s->max_discard) {
+ bs->bl.max_pdiscard = s->max_discard;
+ }
}
static int blkdebug_reopen_prepare(BDRVReopenState *reopen_state,
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* [Qemu-devel] [PATCH v6 10/10] tests: Add coverage for recent block geometry fixes
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (8 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 09/10] blkdebug: Add ability to override unmap geometries Eric Blake
@ 2017-03-08 2:54 ` Eric Blake
2017-03-08 8:40 ` [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Kevin Wolf
10 siblings, 0 replies; 20+ messages in thread
From: Eric Blake @ 2017-03-08 2:54 UTC (permalink / raw)
To: qemu-devel; +Cc: kwolf, qemu-block, mreitz
Use blkdebug's new geometry constraints to emulate setups that
have caused recent regression fixes: write zeroes asserting
when running through a loopback block device with max-transfer
smaller than cluster size, and discard rounding away portions
of requests not aligned to preferred boundaries. Also, add
coverage that the block layer is honoring max transfer limits.
For now, a single iotest performs all actions, with the idea
that we can add future blkdebug constraint test cases in the
same file; but it can be split into multiple iotests if we find
reason to run one portion of the test in more setups than what
are possible in the other.
For reference, the final portion of the test (checking whether
discard passes as much as possible to the lowest layers of the
stack) works as follows:
qemu-io: discard 30M at 80000001, passed to blkdebug
blkdebug: discard 511 bytes at 80000001, -ENOTSUP (smaller than
blkdebug's 512 align)
blkdebug: discard 14371328 bytes at 80000512, passed to qcow2
qcow2: discard 739840 bytes at 80000512, -ENOTSUP (smaller than
qcow2's 1M align)
qcow2: discard 13M bytes at 77M, succeeds
blkdebug: discard 15M bytes at 90M, passed to qcow2
qcow2: discard 15M bytes at 90M, succeeds
blkdebug: discard 1356800 bytes at 105M, passed to qcow2
qcow2: discard 1M at 105M, succeeds
qcow2: discard 308224 bytes at 106M, -ENOTSUP (smaller than qcow2's
1M align)
blkdebug: discard 1 byte at 111457280, -ENOTSUP (smaller than
blkdebug's 512 align)
Signed-off-by: Eric Blake <eblake@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
---
v6: rebase to master by renumbering s/175/177/
v5: rebase to master by renumbering s/173/175/
v4: clean up some comments, nicer backing file creation, more commit message
v3: make comments tied more to test at hand, rather than the
particular hardware that led to the earlier patches being tested
v2: new patch
---
tests/qemu-iotests/177 | 114 +++++++++++++++++++++++++++++++++++++++++++++
tests/qemu-iotests/177.out | 49 +++++++++++++++++++
tests/qemu-iotests/group | 1 +
3 files changed, 164 insertions(+)
create mode 100755 tests/qemu-iotests/177
create mode 100644 tests/qemu-iotests/177.out
diff --git a/tests/qemu-iotests/177 b/tests/qemu-iotests/177
new file mode 100755
index 0000000..e4ddec7
--- /dev/null
+++ b/tests/qemu-iotests/177
@@ -0,0 +1,114 @@
+#!/bin/bash
+#
+# Test corner cases with unusual block geometries
+#
+# Copyright (C) 2016-2017 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+# creator
+owner=eblake@redhat.com
+
+seq=`basename $0`
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1 # failure is the default!
+
+_cleanup()
+{
+ _cleanup_test_img
+}
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+# get standard environment, filters and checks
+. ./common.rc
+. ./common.filter
+
+_supported_fmt qcow2
+_supported_proto file
+
+CLUSTER_SIZE=1M
+size=128M
+options=driver=blkdebug,image.driver=qcow2
+
+echo
+echo "== setting up files =="
+
+TEST_IMG="$TEST_IMG.base" _make_test_img $size
+$QEMU_IO -c "write -P 11 0 $size" "$TEST_IMG.base" | _filter_qemu_io
+_make_test_img -b "$TEST_IMG.base"
+$QEMU_IO -c "write -P 22 0 $size" "$TEST_IMG" | _filter_qemu_io
+
+# Limited to 64k max-transfer
+echo
+echo "== constrained alignment and max-transfer =="
+limits=align=4k,max-transfer=64k
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "write -P 33 1000 128k" -c "read -P 33 1000 128k" | _filter_qemu_io
+
+echo
+echo "== write zero with constrained max-transfer =="
+limits=align=512,max-transfer=64k,opt-write-zero=$CLUSTER_SIZE
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "write -z 8003584 2093056" | _filter_qemu_io
+
+# non-power-of-2 write-zero/discard alignments
+echo
+echo "== non-power-of-2 write zeroes limits =="
+
+limits=align=512,opt-write-zero=15M,max-write-zero=15M,opt-discard=15M,max-discard=15M
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "write -z 32M 32M" | _filter_qemu_io
+
+echo
+echo "== non-power-of-2 discard limits =="
+
+limits=align=512,opt-write-zero=15M,max-write-zero=15M,opt-discard=15M,max-discard=15M
+$QEMU_IO -c "open -o $options,$limits blkdebug::$TEST_IMG" \
+ -c "discard 80000001 30M" | _filter_qemu_io
+
+echo
+echo "== verify image content =="
+
+function verify_io()
+{
+ if ($QEMU_IMG info -f "$IMGFMT" "$TEST_IMG" |
+ grep "compat: 0.10" > /dev/null); then
+ # For v2 images, discarded clusters are read from the backing file
+ discarded=11
+ else
+ # Discarded clusters are zeroed for v3 or later
+ discarded=0
+ fi
+
+ echo read -P 22 0 1000
+ echo read -P 33 1000 128k
+ echo read -P 22 132072 7871512
+ echo read -P 0 8003584 2093056
+ echo read -P 22 10096640 23457792
+ echo read -P 0 32M 32M
+ echo read -P 22 64M 13M
+ echo read -P $discarded 77M 29M
+ echo read -P 22 106M 22M
+}
+
+verify_io | $QEMU_IO "$TEST_IMG" | _filter_qemu_io
+
+_check_test_img
+
+# success, all done
+echo "*** done"
+status=0
diff --git a/tests/qemu-iotests/177.out b/tests/qemu-iotests/177.out
new file mode 100644
index 0000000..e887542
--- /dev/null
+++ b/tests/qemu-iotests/177.out
@@ -0,0 +1,49 @@
+QA output created by 177
+
+== setting up files ==
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=134217728
+wrote 134217728/134217728 bytes at offset 0
+128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=134217728 backing_file=TEST_DIR/t.IMGFMT.base
+wrote 134217728/134217728 bytes at offset 0
+128 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== constrained alignment and max-transfer ==
+wrote 131072/131072 bytes at offset 1000
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 131072/131072 bytes at offset 1000
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== write zero with constrained max-transfer ==
+wrote 2093056/2093056 bytes at offset 8003584
+1.996 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== non-power-of-2 write zeroes limits ==
+wrote 33554432/33554432 bytes at offset 33554432
+32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== non-power-of-2 discard limits ==
+discard 31457280/31457280 bytes at offset 80000001
+30 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+
+== verify image content ==
+read 1000/1000 bytes at offset 0
+1000 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 131072/131072 bytes at offset 1000
+128 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 7871512/7871512 bytes at offset 132072
+7.507 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 2093056/2093056 bytes at offset 8003584
+1.996 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 23457792/23457792 bytes at offset 10096640
+22.371 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 33554432/33554432 bytes at offset 33554432
+32 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 13631488/13631488 bytes at offset 67108864
+13 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 30408704/30408704 bytes at offset 80740352
+29 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+read 23068672/23068672 bytes at offset 111149056
+22 MiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+No errors were found on the image.
+*** done
diff --git a/tests/qemu-iotests/group b/tests/qemu-iotests/group
index 43142dd..a513ec7 100644
--- a/tests/qemu-iotests/group
+++ b/tests/qemu-iotests/group
@@ -169,3 +169,4 @@
174 auto
175 auto quick
176 rw auto backing
+177 rw auto quick
--
2.9.3
^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 00/10] add blkdebug tests
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
` (9 preceding siblings ...)
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 10/10] tests: Add coverage for recent block geometry fixes Eric Blake
@ 2017-03-08 8:40 ` Kevin Wolf
10 siblings, 0 replies; 20+ messages in thread
From: Kevin Wolf @ 2017-03-08 8:40 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel, qemu-block, mreitz
Am 08.03.2017 um 03:54 hat Eric Blake geschrieben:
> Available as a tag at:
> git fetch git://repo.or.cz/qemu/ericb.git nbd-blkdebug-v6
>
> v5 was:
> https://lists.gnu.org/archive/html/qemu-devel/2017-02/msg03042.html
>
> Since then:
> - Rebase to master
> - Pull in Dan's patch that splits test 97 for qcow (patch 1)
> - Address comments from Kevin (includes creating new patch 2, 6)
> - Update patches to reflect that this is now 2.10 material, since it
> missed soft freeze and is not fixing a bug
I think if patches 1 and 2 pass review, we can take at least those into
2.9. Improving test cases is okay during the freeze (and obviously
fixing them is allowed anyway).
Kevin
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image Eric Blake
@ 2017-03-13 22:00 ` Max Reitz
2017-03-30 2:13 ` Eric Blake
0 siblings, 1 reply; 20+ messages in thread
From: Max Reitz @ 2017-03-13 22:00 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 728 bytes --]
On 08.03.2017 03:54, Eric Blake wrote:
> Tweak 097 and 176 to operate on an image that is not cluster-aligned,
> to give further coverage of clearing out an entire image.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v6: new patch
> ---
> tests/qemu-iotests/097 | 7 ++++---
> tests/qemu-iotests/097.out | 24 ++++++++++++------------
> tests/qemu-iotests/176 | 7 ++++---
> tests/qemu-iotests/176.out | 24 ++++++++++++------------
> 4 files changed, 32 insertions(+), 30 deletions(-)
I don't have anything against this patch, but maybe it would be even
more useful if it wrote some data to the final partial cluster (just an
idea).
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 03/10] qcow2: Assert that cluster operations are aligned
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 03/10] qcow2: Assert that cluster operations are aligned Eric Blake
@ 2017-03-13 22:07 ` Max Reitz
0 siblings, 0 replies; 20+ messages in thread
From: Max Reitz @ 2017-03-13 22:07 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1503 bytes --]
On 08.03.2017 03:54, Eric Blake wrote:
> qcow2_discard_clusters() is set up to silently ignore sub-cluster
> head or tail on unaligned requests. However, it is easy to audit
> the various callers: qcow2_snapshot_create() has always passed
> aligned data since the call was introduced in 1ebf561;
> qcow2_co_pdiscard() has passed aligned clusters since commit
> ecdbead taught the block layer the preferred discard alignment (the
> block layer can still pass sub-cluster values, but those are
> handled directly in qcow2_co_pdiscard()); and qcow2_make_empty()
> was fixed to pass aligned clusters in commit a3e1505. Replace
> rounding with assertions to hold us to the tighter contract,
> eliminating the now-impossible case of an early exit for a
> sub-cluster request.
>
> qcow2_zero_clusters() has always been called with cluster-aligned
> arguments from its lone caller qcow2_co_pwrite_zeroes() (like
> qcow2_co_pdiscard(), the caller takes care of sub-cluster requests
> from the block layer; and qcow2_zero_clusters() would have
> misbehaved on unaligned requests), but it deserves the same
> assertion for symmetry.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v6: avoid assertion on non-cluster-aligned image, use s->cluster_sectors
> to avoid a shift, drop R-b
> v5: no change
> v4: new patch
> ---
> block/qcow2-cluster.c | 15 ++++++++-------
> 1 file changed, 8 insertions(+), 7 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 06/10] blkdebug: Refactor error injection
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 06/10] blkdebug: Refactor error injection Eric Blake
@ 2017-03-13 22:16 ` Max Reitz
0 siblings, 0 replies; 20+ messages in thread
From: Max Reitz @ 2017-03-13 22:16 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 782 bytes --]
On 08.03.2017 03:54, Eric Blake wrote:
> Rather than repeat the logic at each caller of checking if a Rule
> exists that warrants an error injection, fold that logic into
> inject_error(); and rename it to rule_check() for legibility.
> This will help the next patch, which adds two more callers that
> need to check rules for the potential of injecting errors.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v6: new patch
> ---
> block/blkdebug.c | 74 +++++++++++++++++++++++++-------------------------------
> 1 file changed, 33 insertions(+), 41 deletions(-)
>
> diff --git a/block/blkdebug.c b/block/blkdebug.c
> index 7c804b2..f721bcb 100644
> --- a/block/blkdebug.c
> +++ b/block/blkdebug.c
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 07/10] blkdebug: Add pass-through write_zero and discard support
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 07/10] blkdebug: Add pass-through write_zero and discard support Eric Blake
@ 2017-03-13 22:20 ` Max Reitz
0 siblings, 0 replies; 20+ messages in thread
From: Max Reitz @ 2017-03-13 22:20 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 2249 bytes --]
On 08.03.2017 03:54, Eric Blake wrote:
> In order to test the effects of artificial geometry constraints
> on operations like write zero or discard, we first need blkdebug
> to manage these actions. It also allows us to inject errors on
> those operations, just like we can for read/write/flush.
>
> We can also test the contract promised by the block layer; namely,
> if a device has specified limits on alignment or maximum size,
> then those limits must be obeyed (for now, the blkdebug driver
> merely inherits limits from whatever it is wrapping, but the next
> patch will further enhance it to allow specific limit overrides).
>
> This patch intentionally refuses to service requests smaller than
> the requested alignments; this is because an upcoming patch adds
> a qemu-iotest to prove that the block layer is correctly handling
> fragmentation, but the test only works if there is a way to tell
> the difference at artificial alignment boundaries when blkdebug is
> using a larger-than-default alignment. If we let the blkdebug
> layer always defer to the underlying layer, which potentially has
> a smaller granularity, the iotest will be thwarted.
>
> Tested by setting up an NBD server with export 'foo', then invoking:
> $ ./qemu-io
> qemu-io> open -o driver=blkdebug blkdebug::nbd://localhost:10809/foo
> qemu-io> d 0 15M
> qemu-io> w -z 0 15M
>
> Pre-patch, the server never sees the discard (it was silently
> eaten by the block layer); post-patch it is passed across the
> wire. Likewise, pre-patch the write is always passed with
> NBD_WRITE (with 15M of zeroes on the wire), while post-patch
> it can utilize NBD_WRITE_ZEROES (for less traffic).
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v6: tighten check of unaligned requests, rebase on rule check
> refactoring, drop R-b
> v5: include 2017 copyright
> v4: correct error injection to respect byte range, tweak formatting
> v3: rebase to byte-based read/write, improve docs on why no
> partial write zero passthrough
> v2: new patch
> ---
> block/blkdebug.c | 74 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> 1 file changed, 74 insertions(+)
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 09/10] blkdebug: Add ability to override unmap geometries
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 09/10] blkdebug: Add ability to override unmap geometries Eric Blake
@ 2017-03-13 22:22 ` Max Reitz
0 siblings, 0 replies; 20+ messages in thread
From: Max Reitz @ 2017-03-13 22:22 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block, Markus Armbruster
[-- Attachment #1: Type: text/plain, Size: 998 bytes --]
On 08.03.2017 03:54, Eric Blake wrote:
> Make it easier to simulate various unusual hardware setups (for
> example, recent commits 3482b9b and b8d0a98 affect the Dell
> Equallogic iSCSI with its 15M preferred and maximum unmap and
> write zero sizing, or b2f95fe deals with the Linux loopback
> block device having a max_transfer of 64k), by allowing blkdebug
> to wrap any other device with further restrictions on various
> alignments.
>
> Signed-off-by: Eric Blake <eblake@redhat.com>
>
> ---
> v6: more tweaks in docs and error messages
> v5: tweak docs regarding max-transfer minimum
> v4: relax 512 byte minimum now that blkdebug is byte-based, fix doc typo
> v3: improve legibility of bounds checking, improve docs
> v2: new patch
> ---
> qapi/block-core.json | 31 +++++++++++++++--
> block/blkdebug.c | 96 +++++++++++++++++++++++++++++++++++++++++++++++++++-
> 2 files changed, 124 insertions(+), 3 deletions(-)
Reviewed-by: Max Reitz <mreitz@redhat.com>
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image
2017-03-13 22:00 ` Max Reitz
@ 2017-03-30 2:13 ` Eric Blake
2017-03-30 18:39 ` Eric Blake
0 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-30 2:13 UTC (permalink / raw)
To: Max Reitz, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1141 bytes --]
On 03/13/2017 05:00 PM, Max Reitz wrote:
> On 08.03.2017 03:54, Eric Blake wrote:
>> Tweak 097 and 176 to operate on an image that is not cluster-aligned,
>> to give further coverage of clearing out an entire image.
>>
>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>
>> ---
>> v6: new patch
>> ---
>> tests/qemu-iotests/097 | 7 ++++---
>> tests/qemu-iotests/097.out | 24 ++++++++++++------------
>> tests/qemu-iotests/176 | 7 ++++---
>> tests/qemu-iotests/176.out | 24 ++++++++++++------------
>> 4 files changed, 32 insertions(+), 30 deletions(-)
>
> I don't have anything against this patch, but maybe it would be even
> more useful if it wrote some data to the final partial cluster (just an
> idea).
And not a bad one. Kevin said patch 1 and 2 are okay for freeze (if
wanted; but not essential if they miss it), so I'll post v7 soon.
>
> Reviewed-by: Max Reitz <mreitz@redhat.com>
Of course, testing data written in the final cluster is non-trivial, so
I'll drop your R-b.
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image
2017-03-30 2:13 ` Eric Blake
@ 2017-03-30 18:39 ` Eric Blake
2017-03-30 21:25 ` Max Reitz
0 siblings, 1 reply; 20+ messages in thread
From: Eric Blake @ 2017-03-30 18:39 UTC (permalink / raw)
To: Max Reitz, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1401 bytes --]
On 03/29/2017 09:13 PM, Eric Blake wrote:
> On 03/13/2017 05:00 PM, Max Reitz wrote:
>> On 08.03.2017 03:54, Eric Blake wrote:
>>> Tweak 097 and 176 to operate on an image that is not cluster-aligned,
>>> to give further coverage of clearing out an entire image.
>>>
>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>
>>> ---
>>> v6: new patch
>>> ---
>>> tests/qemu-iotests/097 | 7 ++++---
>>> tests/qemu-iotests/097.out | 24 ++++++++++++------------
>>> tests/qemu-iotests/176 | 7 ++++---
>>> tests/qemu-iotests/176.out | 24 ++++++++++++------------
>>> 4 files changed, 32 insertions(+), 30 deletions(-)
>>
>> I don't have anything against this patch, but maybe it would be even
>> more useful if it wrote some data to the final partial cluster (just an
>> idea).
>
> And not a bad one. Kevin said patch 1 and 2 are okay for freeze (if
> wanted; but not essential if they miss it), so I'll post v7 soon.
Urrgh. Doing this uncovered the fact that we have a latent bug: we STILL
leave the unaligned tail cluster allocated, when using the slow path; or
put another way, commit a3e1505 is an incomplete fix.
Do I say "thanks" for your suggestion being the cause for me to find the
problem, or "no thanks" for it turning into more work? ;)
--
Eric Blake eblake redhat com +1-919-301-3266
Libvirt virtualization library http://libvirt.org
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 604 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image
2017-03-30 18:39 ` Eric Blake
@ 2017-03-30 21:25 ` Max Reitz
0 siblings, 0 replies; 20+ messages in thread
From: Max Reitz @ 2017-03-30 21:25 UTC (permalink / raw)
To: Eric Blake, qemu-devel; +Cc: kwolf, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1508 bytes --]
On 30.03.2017 20:39, Eric Blake wrote:
> On 03/29/2017 09:13 PM, Eric Blake wrote:
>> On 03/13/2017 05:00 PM, Max Reitz wrote:
>>> On 08.03.2017 03:54, Eric Blake wrote:
>>>> Tweak 097 and 176 to operate on an image that is not cluster-aligned,
>>>> to give further coverage of clearing out an entire image.
>>>>
>>>> Signed-off-by: Eric Blake <eblake@redhat.com>
>>>>
>>>> ---
>>>> v6: new patch
>>>> ---
>>>> tests/qemu-iotests/097 | 7 ++++---
>>>> tests/qemu-iotests/097.out | 24 ++++++++++++------------
>>>> tests/qemu-iotests/176 | 7 ++++---
>>>> tests/qemu-iotests/176.out | 24 ++++++++++++------------
>>>> 4 files changed, 32 insertions(+), 30 deletions(-)
>>>
>>> I don't have anything against this patch, but maybe it would be even
>>> more useful if it wrote some data to the final partial cluster (just an
>>> idea).
>>
>> And not a bad one. Kevin said patch 1 and 2 are okay for freeze (if
>> wanted; but not essential if they miss it), so I'll post v7 soon.
>
> Urrgh. Doing this uncovered the fact that we have a latent bug: we STILL
> leave the unaligned tail cluster allocated, when using the slow path; or
> put another way, commit a3e1505 is an incomplete fix.
>
> Do I say "thanks" for your suggestion being the cause for me to find the
> problem, or "no thanks" for it turning into more work? ;)
Just pretend you didn't see any problem. Whoops, you've already posted a
mail to the list, this will make it a bit awkward. ;-)
Max
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 512 bytes --]
^ permalink raw reply [flat|nested] 20+ messages in thread
end of thread, other threads:[~2017-03-30 21:25 UTC | newest]
Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-03-08 2:54 [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 01/10] iotests: fix 097 when run with qcow Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 02/10] iotests: Improve image-clear tests on non-aligned image Eric Blake
2017-03-13 22:00 ` Max Reitz
2017-03-30 2:13 ` Eric Blake
2017-03-30 18:39 ` Eric Blake
2017-03-30 21:25 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 03/10] qcow2: Assert that cluster operations are aligned Eric Blake
2017-03-13 22:07 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 04/10] qcow2: Discard/zero clusters by byte count Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 05/10] blkdebug: Sanity check block layer guarantees Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 06/10] blkdebug: Refactor error injection Eric Blake
2017-03-13 22:16 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 07/10] blkdebug: Add pass-through write_zero and discard support Eric Blake
2017-03-13 22:20 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 08/10] blkdebug: Simplify override logic Eric Blake
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 09/10] blkdebug: Add ability to override unmap geometries Eric Blake
2017-03-13 22:22 ` Max Reitz
2017-03-08 2:54 ` [Qemu-devel] [PATCH v6 10/10] tests: Add coverage for recent block geometry fixes Eric Blake
2017-03-08 8:40 ` [Qemu-devel] [PATCH v6 00/10] add blkdebug tests Kevin Wolf
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.