[PATCH for-5.1 0/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[PATCH for-5.1 0/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Max Reitz]

Hi,

Patch 1 is the fix, patch 2 adds a test.

I think we must get this into 5.1, rc2 or not.


Max Reitz (2):
  block: Fix bdrv_aligned_p*v() for qiov_offset != 0
  iotests/028: Add test for cross-base-EOF reads

 block/io.c                 | 10 ++++++----
 tests/qemu-iotests/028     | 19 +++++++++++++++++++
 tests/qemu-iotests/028.out | 11 +++++++++++
 3 files changed, 36 insertions(+), 4 deletions(-)

-- 
2.26.2

[PATCH for-5.1 1/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Max Reitz]

Since these functions take a @qiov_offset, they must always take it into
account when working with @qiov.  There are a couple of places where
they do not, but they should.

Fixes: 65cd4424b9df03bb5195351c33e04cbbecc0705c
Fixes: 28c4da28695bdbe04b336b2c9c463876cc3aaa6d
Reported-by: Claudio Fontana <cfontana@suse.de>
Reported-by: Bruce Rogers <brogers@suse.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 block/io.c | 10 ++++++----
 1 file changed, 6 insertions(+), 4 deletions(-)

diff --git a/block/io.c b/block/io.c
index b6564e34c5..ad3a51ed53 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1524,12 +1524,13 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
             assert(num);
 
             ret = bdrv_driver_preadv(bs, offset + bytes - bytes_remaining,
-                                     num, qiov, bytes - bytes_remaining, 0);
+                                     num, qiov,
+                                     qiov_offset + bytes - bytes_remaining, 0);
             max_bytes -= num;
         } else {
             num = bytes_remaining;
-            ret = qemu_iovec_memset(qiov, bytes - bytes_remaining, 0,
-                                    bytes_remaining);
+            ret = qemu_iovec_memset(qiov, qiov_offset + bytes - bytes_remaining,
+                                    0, bytes_remaining);
         }
         if (ret < 0) {
             goto out;
@@ -2032,7 +2033,8 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
             }
 
             ret = bdrv_driver_pwritev(bs, offset + bytes - bytes_remaining,
-                                      num, qiov, bytes - bytes_remaining,
+                                      num, qiov,
+                                      qiov_offset + bytes - bytes_remaining,
                                       local_flags);
             if (ret < 0) {
                 break;
-- 
2.26.2

[PATCH for-5.1 2/2] iotests/028: Add test for cross-base-EOF reads

[Max Reitz]

Signed-off-by: Max Reitz <mreitz@redhat.com>
---
 tests/qemu-iotests/028     | 19 +++++++++++++++++++
 tests/qemu-iotests/028.out | 11 +++++++++++
 2 files changed, 30 insertions(+)

diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028
index 5d043cef92..6dd3ae09a3 100755
--- a/tests/qemu-iotests/028
+++ b/tests/qemu-iotests/028
@@ -142,6 +142,25 @@ TEST_IMG="${TEST_IMG}.copy" io_zero readv $(( offset + 32 * 1024 )) 512 1024 32
 
 _check_test_img
 
+echo
+echo '=== Reading across backing EOF in one operation ==='
+echo
+
+# Use a cluster boundary as the base end here
+base_size=$((3 * 1024 * 1024 * 1024))
+
+TEST_IMG="$TEST_IMG.base" _make_test_img $base_size
+_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $image_size
+
+# Write 16 times 42 at the end of the base image
+$QEMU_IO -c "write -P 42 $((base_size - 16)) 16" "$TEST_IMG.base" \
+    | _filter_qemu_io
+
+# Read 32 bytes across the base EOF from the top;
+# should be 16 times 0x2a, then 16 times 0x00
+$QEMU_IO -c "read -v $((base_size - 16)) 32" "$TEST_IMG" \
+    | _filter_qemu_io
+
 # success, all done
 echo "*** done"
 rm -f $seq.full
diff --git a/tests/qemu-iotests/028.out b/tests/qemu-iotests/028.out
index 12f82c6a6c..5a68de5c46 100644
--- a/tests/qemu-iotests/028.out
+++ b/tests/qemu-iotests/028.out
@@ -730,4 +730,15 @@ read 512/512 bytes at offset 3221257728
 read 512/512 bytes at offset 3221258752
 512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 No errors were found on the image.
+
+=== Reading across backing EOF in one operation ===
+
+Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=3221225472
+Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4294968832 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
+wrote 16/16 bytes at offset 3221225456
+16 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
+bffffff0:  2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a  ................
+c0000000:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
+read 32/32 bytes at offset 3221225456
+32 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
 *** done
-- 
2.26.2

Re: [PATCH for-5.1 1/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Vladimir Sementsov-Ogievskiy]

28.07.2020 15:08, Max Reitz wrote:
> Since these functions take a @qiov_offset, they must always take it into
> account when working with @qiov.  There are a couple of places where
> they do not, but they should.
> 
> Fixes: 65cd4424b9df03bb5195351c33e04cbbecc0705c
> Fixes: 28c4da28695bdbe04b336b2c9c463876cc3aaa6d
> Reported-by: Claudio Fontana<cfontana@suse.de>
> Reported-by: Bruce Rogers<brogers@suse.com>
> Cc:qemu-stable@nongnu.org
> Signed-off-by: Max Reitz<mreitz@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

-- 
Best regards,
Vladimir

Re: [PATCH for-5.1 2/2] iotests/028: Add test for cross-base-EOF reads

[Vladimir Sementsov-Ogievskiy]

28.07.2020 15:08, Max Reitz wrote:
> Signed-off-by: Max Reitz<mreitz@redhat.com>

Reviewed-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Tested-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>

-- 
Best regards,
Vladimir

Re: [PATCH for-5.1 1/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Philippe Mathieu-Daudé]

On 7/28/20 2:08 PM, Max Reitz wrote:
> Since these functions take a @qiov_offset, they must always take it into
> account when working with @qiov.  There are a couple of places where
> they do not, but they should.
> 
> Fixes: 65cd4424b9df03bb5195351c33e04cbbecc0705c
> Fixes: 28c4da28695bdbe04b336b2c9c463876cc3aaa6d

Maybe:
Fixes: 65cd4424b9 ("bdrv_aligned_preadv: use and support qiov_offset")
Fixes: 28c4da2869 ("bdrv_aligned_pwritev: use and support qiov_offset")

> Reported-by: Claudio Fontana <cfontana@suse.de>
> Reported-by: Bruce Rogers <brogers@suse.com>
> Cc: qemu-stable@nongnu.org
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
>  block/io.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/block/io.c b/block/io.c
> index b6564e34c5..ad3a51ed53 100644
> --- a/block/io.c
> +++ b/block/io.c
> @@ -1524,12 +1524,13 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
>              assert(num);
>  
>              ret = bdrv_driver_preadv(bs, offset + bytes - bytes_remaining,
> -                                     num, qiov, bytes - bytes_remaining, 0);
> +                                     num, qiov,
> +                                     qiov_offset + bytes - bytes_remaining, 0);
>              max_bytes -= num;
>          } else {
>              num = bytes_remaining;
> -            ret = qemu_iovec_memset(qiov, bytes - bytes_remaining, 0,
> -                                    bytes_remaining);
> +            ret = qemu_iovec_memset(qiov, qiov_offset + bytes - bytes_remaining,
> +                                    0, bytes_remaining);
>          }
>          if (ret < 0) {
>              goto out;
> @@ -2032,7 +2033,8 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
>              }
>  
>              ret = bdrv_driver_pwritev(bs, offset + bytes - bytes_remaining,
> -                                      num, qiov, bytes - bytes_remaining,
> +                                      num, qiov,
> +                                      qiov_offset + bytes - bytes_remaining,
>                                        local_flags);
>              if (ret < 0) {
>                  break;
> 

Re: [PATCH for-5.1 1/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Claudio Fontana]

Tested-by: Claudio Fontana <cfontana@suse.de>

On 7/28/20 2:08 PM, Max Reitz wrote:
> Since these functions take a @qiov_offset, they must always take it into
> account when working with @qiov.  There are a couple of places where
> they do not, but they should.
> 
> Fixes: 65cd4424b9df03bb5195351c33e04cbbecc0705c
> Fixes: 28c4da28695bdbe04b336b2c9c463876cc3aaa6d
> Reported-by: Claudio Fontana <cfontana@suse.de>
> Reported-by: Bruce Rogers <brogers@suse.com>
> Cc: qemu-stable@nongnu.org
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
>  block/io.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/block/io.c b/block/io.c
> index b6564e34c5..ad3a51ed53 100644
> --- a/block/io.c
> +++ b/block/io.c
> @@ -1524,12 +1524,13 @@ static int coroutine_fn bdrv_aligned_preadv(BdrvChild *child,
>              assert(num);
>  
>              ret = bdrv_driver_preadv(bs, offset + bytes - bytes_remaining,
> -                                     num, qiov, bytes - bytes_remaining, 0);
> +                                     num, qiov,
> +                                     qiov_offset + bytes - bytes_remaining, 0);
>              max_bytes -= num;
>          } else {
>              num = bytes_remaining;
> -            ret = qemu_iovec_memset(qiov, bytes - bytes_remaining, 0,
> -                                    bytes_remaining);
> +            ret = qemu_iovec_memset(qiov, qiov_offset + bytes - bytes_remaining,
> +                                    0, bytes_remaining);
>          }
>          if (ret < 0) {
>              goto out;
> @@ -2032,7 +2033,8 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
>              }
>  
>              ret = bdrv_driver_pwritev(bs, offset + bytes - bytes_remaining,
> -                                      num, qiov, bytes - bytes_remaining,
> +                                      num, qiov,
> +                                      qiov_offset + bytes - bytes_remaining,
>                                        local_flags);
>              if (ret < 0) {
>                  break;
> 

Re: [PATCH for-5.1 2/2] iotests/028: Add test for cross-base-EOF reads

[Claudio Fontana]

Tested-by: Claudio Fontana <cfontana@suse.de>

On 7/28/20 2:08 PM, Max Reitz wrote:
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
>  tests/qemu-iotests/028     | 19 +++++++++++++++++++
>  tests/qemu-iotests/028.out | 11 +++++++++++
>  2 files changed, 30 insertions(+)
> 
> diff --git a/tests/qemu-iotests/028 b/tests/qemu-iotests/028
> index 5d043cef92..6dd3ae09a3 100755
> --- a/tests/qemu-iotests/028
> +++ b/tests/qemu-iotests/028
> @@ -142,6 +142,25 @@ TEST_IMG="${TEST_IMG}.copy" io_zero readv $(( offset + 32 * 1024 )) 512 1024 32
>  
>  _check_test_img
>  
> +echo
> +echo '=== Reading across backing EOF in one operation ==='
> +echo
> +
> +# Use a cluster boundary as the base end here
> +base_size=$((3 * 1024 * 1024 * 1024))
> +
> +TEST_IMG="$TEST_IMG.base" _make_test_img $base_size
> +_make_test_img -b "$TEST_IMG.base" -F $IMGFMT $image_size
> +
> +# Write 16 times 42 at the end of the base image
> +$QEMU_IO -c "write -P 42 $((base_size - 16)) 16" "$TEST_IMG.base" \
> +    | _filter_qemu_io
> +
> +# Read 32 bytes across the base EOF from the top;
> +# should be 16 times 0x2a, then 16 times 0x00
> +$QEMU_IO -c "read -v $((base_size - 16)) 32" "$TEST_IMG" \
> +    | _filter_qemu_io
> +
>  # success, all done
>  echo "*** done"
>  rm -f $seq.full
> diff --git a/tests/qemu-iotests/028.out b/tests/qemu-iotests/028.out
> index 12f82c6a6c..5a68de5c46 100644
> --- a/tests/qemu-iotests/028.out
> +++ b/tests/qemu-iotests/028.out
> @@ -730,4 +730,15 @@ read 512/512 bytes at offset 3221257728
>  read 512/512 bytes at offset 3221258752
>  512 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  No errors were found on the image.
> +
> +=== Reading across backing EOF in one operation ===
> +
> +Formatting 'TEST_DIR/t.IMGFMT.base', fmt=IMGFMT size=3221225472
> +Formatting 'TEST_DIR/t.IMGFMT', fmt=IMGFMT size=4294968832 backing_file=TEST_DIR/t.IMGFMT.base backing_fmt=IMGFMT
> +wrote 16/16 bytes at offset 3221225456
> +16 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
> +bffffff0:  2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a 2a  ................
> +c0000000:  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
> +read 32/32 bytes at offset 3221225456
> +32 bytes, X ops; XX:XX:XX.X (XXX YYY/sec and XXX ops/sec)
>  *** done
> 

Re: [PATCH for-5.1 1/2] block: Fix bdrv_aligned_p*v() for qiov_offset != 0

[Bruce Rogers]

On Tue, 2020-07-28 at 14:08 +0200, Max Reitz wrote:
> Since these functions take a @qiov_offset, they must always take it
> into
> account when working with @qiov.  There are a couple of places where
> they do not, but they should.
> 
> Fixes: 65cd4424b9df03bb5195351c33e04cbbecc0705c
> Fixes: 28c4da28695bdbe04b336b2c9c463876cc3aaa6d
> Reported-by: Claudio Fontana <cfontana@suse.de>
> Reported-by: Bruce Rogers <brogers@suse.com>
> Cc: qemu-stable@nongnu.org
> Signed-off-by: Max Reitz <mreitz@redhat.com>
> ---
>  block/io.c | 10 ++++++----
>  1 file changed, 6 insertions(+), 4 deletions(-)
> 
> diff --git a/block/io.c b/block/io.c
> index b6564e34c5..ad3a51ed53 100644
> --- a/block/io.c
> +++ b/block/io.c
> @@ -1524,12 +1524,13 @@ static int coroutine_fn
> bdrv_aligned_preadv(BdrvChild *child,
>              assert(num);
>  
>              ret = bdrv_driver_preadv(bs, offset + bytes -
> bytes_remaining,
> -                                     num, qiov, bytes -
> bytes_remaining, 0);
> +                                     num, qiov,
> +                                     qiov_offset + bytes -
> bytes_remaining, 0);
>              max_bytes -= num;
>          } else {
>              num = bytes_remaining;
> -            ret = qemu_iovec_memset(qiov, bytes - bytes_remaining,
> 0,
> -                                    bytes_remaining);
> +            ret = qemu_iovec_memset(qiov, qiov_offset + bytes -
> bytes_remaining,
> +                                    0, bytes_remaining);
>          }
>          if (ret < 0) {
>              goto out;
> @@ -2032,7 +2033,8 @@ static int coroutine_fn
> bdrv_aligned_pwritev(BdrvChild *child,
>              }
>  
>              ret = bdrv_driver_pwritev(bs, offset + bytes -
> bytes_remaining,
> -                                      num, qiov, bytes -
> bytes_remaining,
> +                                      num, qiov,
> +                                      qiov_offset + bytes -
> bytes_remaining,
>                                        local_flags);
>              if (ret < 0) {
>                  break;

This patch resolves the reported issue for me. Thanks!
Tested-by: Bruce Rogers <brogers@suse.com>

- Bruce