[PULL 0/3] Block patches for 5.1.0-rc2?

[PULL 0/3] Block patches for 5.1.0-rc2?

[Max Reitz]

Hi,

Sorry for the very late pull request.  The iotest issue only appeared
today, and the I/O path issue was only tracked down today.  We need the
fixes for the latter in 5.1, so if they do not make it into rc2, we will
need them in rc3.


The following changes since commit 23ae28783f4674e98f7539d1c05d793166c2fc12:

  Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-27' into staging (2020-07-28 09:15:44 +0100)

are available in the Git repository at:

  https://github.com/XanClic/qemu.git tags/pull-block-2020-07-28

for you to fetch changes up to afac471b71da92d91cc56fb64c0719b8a4a2d96b:

  iotests/197: Fix for non-qcow2 formats (2020-07-28 15:28:56 +0200)

----------------------------------------------------------------
Block patches for 5.1.0:
- Fix block I/O for split transfers
- Fix iotest 197 for non-qcow2 formats

----------------------------------------------------------------
Max Reitz (3):
  block: Fix bdrv_aligned_p*v() for qiov_offset != 0
  iotests/028: Add test for cross-base-EOF reads
  iotests/197: Fix for non-qcow2 formats

 block/io.c                 | 10 ++++++----
 tests/qemu-iotests/028     | 19 +++++++++++++++++++
 tests/qemu-iotests/028.out | 11 +++++++++++
 tests/qemu-iotests/197     |  8 +++++---
 tests/qemu-iotests/197.out |  2 +-
 5 files changed, 42 insertions(+), 8 deletions(-)

-- 
2.26.2

[PULL 1/3] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Max Reitz]

Since these functions take a @qiov_offset, they must always take it into
account when working with @qiov.  There are a couple of places where
they do not, but they should.

Fixes: 65cd4424b9df03bb5195351c33e04cbbecc0705c
       ("block/io: bdrv_aligned_preadv: use and support qiov_offset")
Fixes: 28c4da28695bdbe04b336b2c9c463876cc3aaa6d
       ("block/io: bdrv_aligned_pwritev: use and support qiov_offset")
Reported-by: Claudio Fontana <cfontana@suse.de>
Reported-by: Bruce Rogers <brogers@suse.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200728120806.265916-2-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Claudio Fontana <cfontana@suse.de>
Tested-by: Bruce Rogers <brogers@suse.com>
---
 block/io.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/block/io.c b/block/io.c
index b6564e34c5..ad3a51ed53 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1524,12 +1524,13 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
             assert(num);
 
             ret = bdrv_driver_preadv(bs, offset + bytes - bytes_remaining,
-                                     num, qiov, bytes - bytes_remaining, 0);
+                                     num, qiov,
+                                     qiov_offset + bytes - bytes_remaining, 0);
             max_bytes -= num;
         } else {
             num = bytes_remaining;
-            ret = qemu_iovec_memset(qiov, bytes - bytes_remaining, 0,
-                                    bytes_remaining);
+            ret = qemu_iovec_memset(qiov, qiov_offset + bytes - bytes_remaining,
+                                    0, bytes_remaining);
         }
         if (ret < 0) {
             goto out;
@@ -2032,7 +2033,8 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
             }
 
             ret = bdrv_driver_pwritev(bs, offset + bytes - bytes_remaining,
-                                      num, qiov, bytes - bytes_remaining,
+                                      num, qiov,
+                                      qiov_offset + bytes - bytes_remaining,
                                       local_flags);
             if (ret < 0) {
                 break;
-- 
2.26.2

[PULL 2/3] iotests/028: Add test for cross-base-EOF reads

[Max Reitz]

Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200728120806.265916-3-mreitz@redhat.com>
Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Claudio Fontana <cfontana@suse.de>
---
 tests/qemu-iotests/028     | 19 +++++++++++++++++++
 tests/qemu-iotests/028.out | 11 +++++++++++
 2 files changed, 30 insertions(+)

diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028
index 5d043cef92..6dd3ae09a3 100755
--- a/tests/qemu-iotests/028
+++ b/tests/qemu-iotests/028
@@ -142,6 +142,25 @@ TEST_IMG="${TEST_IMG}.copy" io_zero readv $(( offset + 32 * 1024 )) 512 1024 32
 
 _check_test_img
 
+echo
+echo '=== Reading across backing EOF in one operation ==='
+echo
+
+# Use a cluster boundary as the base end here
+base_size=$((3 * 1024 * 1024 * 1024))
+
+TEST_IMG="$TEST_IMG.base" _make_test_img $base_size
+_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $image_size
+
+# Write 16 times 42 at the end of the base image
+$QEMU_IO -c "write -P 42 $((base_size - 16)) 16" "$TEST_IMG.base" \
+    | _filter_qemu_io
+
+# Read 32 bytes across the base EOF from the top;
+# should be 16 times 0x2a, then 16 times 0x00
+$QEMU_IO -c "read -v $((base_size - 16)) 32" "$TEST_IMG" \
+    | _filter_qemu_io
+
 # success, all done
 echo "*** done"
 rm -f $seq.full
diff --git a/tests/qemu-iotests/028.out b/tests/qemu-iotests/028.out
index 12f82c6a6c..5a68de5c46 100644
--- a/tests/qemu-iotests/028.out
+++ b/tests/qemu-iotests/028.out
@@ -730,4 +730,15 @@ read 512/512 bytes at offset 3221257728
 read 512/512 bytes at offset 3221258752
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 No errors were found on the image.
+
+=== Reading across backing EOF in one operation ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=3221225472
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4294968832 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+wrote 16/16 bytes at offset 3221225456
+16 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+bffffff0:  2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a  ................
+c0000000:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
+read 32/32 bytes at offset 3221225456
+32 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 *** done
-- 
2.26.2

[PULL 3/3] iotests/197: Fix for non-qcow2 formats

[Max Reitz]

While 197 is very much a qcow2 test, and it looks like the partial
cluster case at the end (introduced in b0ddcbbb36a66a6) is specifically
a qcow2 case, the whole test scripts actually marks itself to work with
generic formats (and generic protocols, even).

Said partial cluster case happened to work with non-qcow2 formats as
well (mostly by accident), but 1855536256 broke that, because it sets
the compat option, which does not work for non-qcow2 formats.

So go the whole way and force IMGFMT=qcow2 and IMGPROTO=file, as done in
other places in this test.

Fixes: 1855536256eb0a5708b04b85f744de69559ea323
       ("iotests/197: Fix for compat=0.10")
Reported-by: Thomas Huth <thuth@redhat.com>
Signed-off-by: Max Reitz <mreitz@redhat.com>
Message-Id: <20200728131134.902519-1-mreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
---
 tests/qemu-iotests/197     | 8 +++++---
 tests/qemu-iotests/197.out | 2 +-
 2 files changed, 6 insertions(+), 4 deletions(-)

diff --git a/tests/qemu-iotests/197 b/tests/qemu-iotests/197
index 121959a09c..a161c89816 100755
--- a/tests/qemu-iotests/197
+++ b/tests/qemu-iotests/197
@@ -114,9 +114,11 @@ echo
 
 # Force compat=1.1, because writing zeroes on a v2 image without a
 # backing file would just result in an unallocated cluster
-_make_test_img -o compat=1.1 1024
-$QEMU_IO -f $IMGFMT -C -c 'read 0 1024' "$TEST_IMG" | _filter_qemu_io
-$QEMU_IO -f $IMGFMT -c map "$TEST_IMG"
+# (Also, note that this is really a pure qcow2 test.)
+IMGPROTO=file IMGFMT=qcow2 TEST_IMG_FILE="$TEST_WRAP" \
+    _make_test_img --no-opts -o compat=1.1 1024
+$QEMU_IO -f qcow2 -C -c 'read 0 1024' "$TEST_WRAP" | _filter_qemu_io
+$QEMU_IO -f qcow2 -c map "$TEST_WRAP"
 _check_test_img
 
 # success, all done
diff --git a/tests/qemu-iotests/197.out b/tests/qemu-iotests/197.out
index 7ca46be6e4..ad414c3b0e 100644
--- a/tests/qemu-iotests/197.out
+++ b/tests/qemu-iotests/197.out
@@ -26,7 +26,7 @@ Images are identical.
 
 === Partial final cluster ===
 
-Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=1024
+Formatting 'TEST_DIR/t.wrap.IMGFMT', fmt=IMGFMT size=1024
 read 1024/1024 bytes at offset 0
 1 KiB, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 1 KiB (0x400) bytes     allocated at offset 0 bytes (0x0)
-- 
2.26.2

Re: [PULL 0/3] Block patches for 5.1.0-rc2?

[Peter Maydell]

On Tue, 28 Jul 2020 at 14:48, Max Reitz <mreitz@redhat.com> wrote:
>
> Hi,
>
> Sorry for the very late pull request.  The iotest issue only appeared
> today, and the I/O path issue was only tracked down today.  We need the
> fixes for the latter in 5.1, so if they do not make it into rc2, we will
> need them in rc3.
>
>
> The following changes since commit 23ae28783f4674e98f7539d1c05d793166c2fc12:
>
>   Merge remote-tracking branch 'remotes/maxreitz/tags/pull-block-2020-07-27' into staging (2020-07-28 09:15:44 +0100)
>
> are available in the Git repository at:
>
>   https://github.com/XanClic/qemu.git tags/pull-block-2020-07-28
>
> for you to fetch changes up to afac471b71da92d91cc56fb64c0719b8a4a2d96b:
>
>   iotests/197: Fix for non-qcow2 formats (2020-07-28 15:28:56 +0200)
>
> ----------------------------------------------------------------
> Block patches for 5.1.0:
> - Fix block I/O for split transfers
> - Fix iotest 197 for non-qcow2 formats


Applied, thanks.

Please update the changelog at https://wiki.qemu.org/ChangeLog/5.1
for any user-visible changes.

-- PMM