* [PULL 1/3] file-posix: Handle `EINVAL` fallocate return value
2020-07-21 15:46 [PULL 0/3] Block layer patches for 5.1.0-rc1 Kevin Wolf
@ 2020-07-21 15:46 ` Kevin Wolf
2020-07-21 15:46 ` [PULL 2/3] qcow2: Implement v2 zero writes with discard if possible Kevin Wolf
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Kevin Wolf @ 2020-07-21 15:46 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, peter.maydell, qemu-devel
From: Antoine Damhet <antoine.damhet@blade-group.com>
The `detect-zeroes=unmap` option may issue unaligned
`FALLOC_FL_PUNCH_HOLE` requests, raw block devices can (and will) return
`EINVAL`, qemu should then write the zeroes to the blockdev instead of
issuing an `IO_ERROR`.
The problem can be reprodced like this:
$ qemu-io -c 'write -P 0 42 1234' --image-opts driver=host_device,filename=/dev/loop0,detect-zeroes=unmap
write failed: Invalid argument
Signed-off-by: Antoine Damhet <antoine.damhet@blade-group.com>
Message-Id: <20200717135603.51180-1-antoine.damhet@blade-group.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/file-posix.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/block/file-posix.c b/block/file-posix.c
index 8cc39a1ef6..9a00d4190a 100644
--- a/block/file-posix.c
+++ b/block/file-posix.c
@@ -1698,7 +1698,11 @@ static int handle_aiocb_write_zeroes_unmap(void *opaque)
#ifdef CONFIG_FALLOCATE_PUNCH_HOLE
int ret = do_fallocate(s->fd, FALLOC_FL_PUNCH_HOLE | FALLOC_FL_KEEP_SIZE,
aiocb->aio_offset, aiocb->aio_nbytes);
- if (ret != -ENOTSUP) {
+ switch (ret) {
+ case -ENOTSUP:
+ case -EINVAL:
+ break;
+ default:
return ret;
}
#endif
--
2.25.4
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PULL 2/3] qcow2: Implement v2 zero writes with discard if possible
2020-07-21 15:46 [PULL 0/3] Block layer patches for 5.1.0-rc1 Kevin Wolf
2020-07-21 15:46 ` [PULL 1/3] file-posix: Handle `EINVAL` fallocate return value Kevin Wolf
@ 2020-07-21 15:46 ` Kevin Wolf
2020-07-21 15:46 ` [PULL 3/3] iotests: Test sparseness for qemu-img convert -n Kevin Wolf
2020-07-21 19:28 ` [PULL 0/3] Block layer patches for 5.1.0-rc1 Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Kevin Wolf @ 2020-07-21 15:46 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, peter.maydell, qemu-devel
qcow2 version 2 images don't support the zero flag for clusters, so for
write_zeroes requests, we return -ENOTSUP and get explicit zero buffer
writes. If the image doesn't have a backing file, we can do better: Just
discard the respective clusters.
This is relevant for 'qemu-img convert -O qcow2 -n', where qemu-img has
to assume that the existing target image may contain any data, so it has
to write zeroes. Without this patch, this results in a fully allocated
target image, even if the source image was empty.
Reported-by: Nir Soffer <nsoffer@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200721135520.72355-2-kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
block/qcow2-cluster.c | 9 ++++++++-
1 file changed, 8 insertions(+), 1 deletion(-)
diff --git a/block/qcow2-cluster.c b/block/qcow2-cluster.c
index 4b5fc8c4a7..a677ba9f5c 100644
--- a/block/qcow2-cluster.c
+++ b/block/qcow2-cluster.c
@@ -1797,8 +1797,15 @@ int qcow2_cluster_zeroize(BlockDriverState *bs, uint64_t offset,
assert(QEMU_IS_ALIGNED(end_offset, s->cluster_size) ||
end_offset >= bs->total_sectors << BDRV_SECTOR_BITS);
- /* The zero flag is only supported by version 3 and newer */
+ /*
+ * The zero flag is only supported by version 3 and newer. However, if we
+ * have no backing file, we can resort to discard in version 2.
+ */
if (s->qcow_version < 3) {
+ if (!bs->backing) {
+ return qcow2_cluster_discard(bs, offset, bytes,
+ QCOW2_DISCARD_REQUEST, false);
+ }
return -ENOTSUP;
}
--
2.25.4
^ permalink raw reply related [flat|nested] 5+ messages in thread* [PULL 3/3] iotests: Test sparseness for qemu-img convert -n
2020-07-21 15:46 [PULL 0/3] Block layer patches for 5.1.0-rc1 Kevin Wolf
2020-07-21 15:46 ` [PULL 1/3] file-posix: Handle `EINVAL` fallocate return value Kevin Wolf
2020-07-21 15:46 ` [PULL 2/3] qcow2: Implement v2 zero writes with discard if possible Kevin Wolf
@ 2020-07-21 15:46 ` Kevin Wolf
2020-07-21 19:28 ` [PULL 0/3] Block layer patches for 5.1.0-rc1 Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Kevin Wolf @ 2020-07-21 15:46 UTC (permalink / raw)
To: qemu-block; +Cc: kwolf, peter.maydell, qemu-devel
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
Message-Id: <20200721135520.72355-3-kwolf@redhat.com>
Reviewed-by: Max Reitz <mreitz@redhat.com>
Signed-off-by: Kevin Wolf <kwolf@redhat.com>
---
tests/qemu-iotests/122 | 30 ++++++++++++++++++++++++++++++
tests/qemu-iotests/122.out | 17 +++++++++++++++++
2 files changed, 47 insertions(+)
diff --git a/tests/qemu-iotests/122 b/tests/qemu-iotests/122
index dfd1cd05d6..0f3d4ca851 100755
--- a/tests/qemu-iotests/122
+++ b/tests/qemu-iotests/122
@@ -281,6 +281,36 @@ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG" "$TEST_IMG".orig
$QEMU_IMG compare "$TEST_IMG" "$TEST_IMG".orig
+echo
+echo '=== -n to an empty image ==='
+echo
+
+TEST_IMG="$TEST_IMG".orig _make_test_img 64M
+
+# Convert with -n, which should not result in a fully allocated image, not even
+# with compat=0.10 (because the target doesn't have a backing file)
+for compat in "1.1" "0.10"; do
+ IMGOPTS="compat=$compat" _make_test_img 64M
+ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG".orig "$TEST_IMG"
+ $QEMU_IMG map --output=json "$TEST_IMG"
+done
+
+echo
+echo '=== -n to an empty image with a backing file ==='
+echo
+
+TEST_IMG="$TEST_IMG".orig _make_test_img 64M
+TEST_IMG="$TEST_IMG".base _make_test_img 64M
+
+# Convert with -n, which should still not result in a fully allocated image for
+# compat=1.1 (because it can use zero clusters), but it should be fully
+# allocated with compat=0.10
+for compat in "1.1" "0.10"; do
+ IMGOPTS="compat=$compat" _make_test_img -b "$TEST_IMG".base -F $IMGFMT 64M
+ $QEMU_IMG convert -O $IMGFMT -n "$TEST_IMG".orig "$TEST_IMG"
+ $QEMU_IMG map --output=json "$TEST_IMG"
+done
+
echo
echo '=== -n -B to an image without a backing file ==='
echo
diff --git a/tests/qemu-iotests/122.out b/tests/qemu-iotests/122.out
index f1f195ed77..3a3e121d57 100644
--- a/tests/qemu-iotests/122.out
+++ b/tests/qemu-iotests/122.out
@@ -229,6 +229,23 @@ wrote 65536/65536 bytes at offset 0
64 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
Images are identical.
+=== -n to an empty image ===
+
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
+
+=== -n to an empty image with a backing file ===
+
+Formatting 'TEST_DIR/t.IMGFMT.orig', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": true, "data": false}]
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=67108864 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+[{ "start": 0, "length": 67108864, "depth": 0, "zero": false, "data": true, "offset": 327680}]
+
=== -n -B to an image without a backing file ===
Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=67108864
--
2.25.4
^ permalink raw reply related [flat|nested] 5+ messages in thread* Re: [PULL 0/3] Block layer patches for 5.1.0-rc1
2020-07-21 15:46 [PULL 0/3] Block layer patches for 5.1.0-rc1 Kevin Wolf
` (2 preceding siblings ...)
2020-07-21 15:46 ` [PULL 3/3] iotests: Test sparseness for qemu-img convert -n Kevin Wolf
@ 2020-07-21 19:28 ` Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Peter Maydell @ 2020-07-21 19:28 UTC (permalink / raw)
To: Kevin Wolf; +Cc: QEMU Developers, Qemu-block
On Tue, 21 Jul 2020 at 16:46, Kevin Wolf <kwolf@redhat.com> wrote:
>
> The following changes since commit 90218a9a393c7925f330e7dcc08658e2a01d3bd4:
>
> Merge remote-tracking branch 'remotes/huth-gitlab/tags/pull-request-2020-07-21' into staging (2020-07-21 10:24:38 +0100)
>
> are available in the Git repository at:
>
> git://repo.or.cz/qemu/kevin.git tags/for-upstream
>
> for you to fetch changes up to 4a01e27ddcb5921efd68820d05d85ba71293fea6:
>
> iotests: Test sparseness for qemu-img convert -n (2020-07-21 17:44:35 +0200)
>
> ----------------------------------------------------------------
> Block layer patches:
>
> - file-posix: Handle `EINVAL` fallocate return value
> - qemu-img convert -n: Keep qcow2 v2 target sparse
>
> ----------------------------------------------------------------
> Antoine Damhet (1):
> file-posix: Handle `EINVAL` fallocate return value
>
> Kevin Wolf (2):
> qcow2: Implement v2 zero writes with discard if possible
> iotests: Test sparseness for qemu-img convert -n
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 5+ messages in thread