* [PULL 0/4] NBD patches for 2022-01-28
@ 2022-01-28 22:59 Eric Blake
2022-01-28 23:00 ` [PULL 1/4] nbd/server.c: Remove unused field Eric Blake
` (4 more replies)
0 siblings, 5 replies; 6+ messages in thread
From: Eric Blake @ 2022-01-28 22:59 UTC (permalink / raw)
To: qemu-devel
The following changes since commit 7a1043cef91739ff4b59812d30f1ed2850d3d34e:
Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2022-01-28 14:04:01 +0000)
are available in the Git repository at:
https://repo.or.cz/qemu/ericb.git tags/pull-nbd-2022-01-28
for you to fetch changes up to 6384dd534d742123d26c008d9794b20bc41359d5:
iotests/block-status-cache: New test (2022-01-28 16:55:23 -0600)
----------------------------------------------------------------
nbd patches for 2022-01-28
- Hanna Reitz: regression fix for block status caching
- Philippe Mathieu-Daude: documentation formatting
- Nir Soffer: dead code removal
----------------------------------------------------------------
Hanna Reitz (2):
block/io: Update BSC only if want_zero is true
iotests/block-status-cache: New test
Nir Soffer (1):
nbd/server.c: Remove unused field
Philippe Mathieu-Daude (1):
qapi/block: Cosmetic change in BlockExportType schema
qapi/block-export.json | 3 +-
block/io.c | 6 +-
nbd/server.c | 1 -
tests/qemu-iotests/tests/block-status-cache | 139 ++++++++++++++++++++++++
tests/qemu-iotests/tests/block-status-cache.out | 5 +
5 files changed, 151 insertions(+), 3 deletions(-)
create mode 100755 tests/qemu-iotests/tests/block-status-cache
create mode 100644 tests/qemu-iotests/tests/block-status-cache.out
--
2.34.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* [PULL 1/4] nbd/server.c: Remove unused field
2022-01-28 22:59 [PULL 0/4] NBD patches for 2022-01-28 Eric Blake
@ 2022-01-28 23:00 ` Eric Blake
2022-01-28 23:00 ` [PULL 2/4] qapi/block: Cosmetic change in BlockExportType schema Eric Blake
` (3 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Eric Blake @ 2022-01-28 23:00 UTC (permalink / raw)
To: qemu-devel
Cc: Nir Soffer, Vladimir Sementsov-Ogievskiy,
Philippe Mathieu-Daudé, open list:Network Block Dev...
From: Nir Soffer <nsoffer@redhat.com>
NBDRequestData struct has unused QSIMPLEQ_ENTRY field. It seems that
this field exists since the first git commit and was never used.
Signed-off-by: Nir Soffer <nsoffer@redhat.com>
Message-Id: <20220111194313.581486-1-nsoffer@redhat.com>
Reviewed-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Fixes: d9a73806 ("qemu-nbd: introduce NBDRequest", v1.1)
Signed-off-by: Eric Blake <eblake@redhat.com>
---
nbd/server.c | 1 -
1 file changed, 1 deletion(-)
diff --git a/nbd/server.c b/nbd/server.c
index 4630dd732250..9fb2f264023e 100644
--- a/nbd/server.c
+++ b/nbd/server.c
@@ -77,7 +77,6 @@ static int system_errno_to_nbd_errno(int err)
typedef struct NBDRequestData NBDRequestData;
struct NBDRequestData {
- QSIMPLEQ_ENTRY(NBDRequestData) entry;
NBDClient *client;
uint8_t *data;
bool complete;
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PULL 2/4] qapi/block: Cosmetic change in BlockExportType schema
2022-01-28 22:59 [PULL 0/4] NBD patches for 2022-01-28 Eric Blake
2022-01-28 23:00 ` [PULL 1/4] nbd/server.c: Remove unused field Eric Blake
@ 2022-01-28 23:00 ` Eric Blake
2022-01-28 23:00 ` [PULL 3/4] block/io: Update BSC only if want_zero is true Eric Blake
` (2 subsequent siblings)
4 siblings, 0 replies; 6+ messages in thread
From: Eric Blake @ 2022-01-28 23:00 UTC (permalink / raw)
To: qemu-devel; +Cc: Philippe Mathieu-Daude, Markus Armbruster
From: Philippe Mathieu-Daude <f4bug@amsat.org>
Fix long line introduced in commit bb01ea73110 ("qapi/block:
Restrict vhost-user-blk to CONFIG_VHOST_USER_BLK_SERVER").
Suggested-by: Markus Armbruster <armbru@redhat.com>
Acked-by: Markus Armbruster <armbru@redhat.com>
Signed-off-by: Philippe Mathieu-Daudé <f4bug@amsat.org>
Message-Id: <20220119121439.214821-1-f4bug@amsat.org>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
qapi/block-export.json | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/qapi/block-export.json b/qapi/block-export.json
index f9ce79a974b5..f183522d0d2c 100644
--- a/qapi/block-export.json
+++ b/qapi/block-export.json
@@ -278,7 +278,8 @@
##
{ 'enum': 'BlockExportType',
'data': [ 'nbd',
- { 'name': 'vhost-user-blk', 'if': 'CONFIG_VHOST_USER_BLK_SERVER' },
+ { 'name': 'vhost-user-blk',
+ 'if': 'CONFIG_VHOST_USER_BLK_SERVER' },
{ 'name': 'fuse', 'if': 'CONFIG_FUSE' } ] }
##
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PULL 3/4] block/io: Update BSC only if want_zero is true
2022-01-28 22:59 [PULL 0/4] NBD patches for 2022-01-28 Eric Blake
2022-01-28 23:00 ` [PULL 1/4] nbd/server.c: Remove unused field Eric Blake
2022-01-28 23:00 ` [PULL 2/4] qapi/block: Cosmetic change in BlockExportType schema Eric Blake
@ 2022-01-28 23:00 ` Eric Blake
2022-01-28 23:00 ` [PULL 4/4] iotests/block-status-cache: New test Eric Blake
2022-01-30 15:08 ` [PULL 0/4] NBD patches for 2022-01-28 Peter Maydell
4 siblings, 0 replies; 6+ messages in thread
From: Eric Blake @ 2022-01-28 23:00 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Kevin Wolf, open list:Block I/O path, qemu-stable,
Nir Soffer, Hanna Reitz, Stefan Hajnoczi
From: Hanna Reitz <hreitz@redhat.com>
We update the block-status cache whenever we get new information from a
bdrv_co_block_status() call to the block driver. However, if we have
passed want_zero=false to that call, it may flag areas containing zeroes
as data, and so we would update the block-status cache with wrong
information.
Therefore, we should not update the cache with want_zero=false.
Reported-by: Nir Soffer <nsoffer@redhat.com>
Fixes: 0bc329fbb00 ("block: block-status cache for data regions")
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Cc: qemu-stable@nongnu.org
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220118170000.49423-2-hreitz@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
block/io.c | 6 +++++-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/block/io.c b/block/io.c
index bb0a254def11..4e4cb556c586 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2497,8 +2497,12 @@ static int coroutine_fn bdrv_co_block_status(BlockDriverState *bs,
* non-protocol nodes, and then it is never used. However, filling
* the cache requires an RCU update, so double check here to avoid
* such an update if possible.
+ *
+ * Check want_zero, because we only want to update the cache when we
+ * have accurate information about what is zero and what is data.
*/
- if (ret == (BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID) &&
+ if (want_zero &&
+ ret == (BDRV_BLOCK_DATA | BDRV_BLOCK_OFFSET_VALID) &&
QLIST_EMPTY(&bs->children))
{
/*
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PULL 4/4] iotests/block-status-cache: New test
2022-01-28 22:59 [PULL 0/4] NBD patches for 2022-01-28 Eric Blake
` (2 preceding siblings ...)
2022-01-28 23:00 ` [PULL 3/4] block/io: Update BSC only if want_zero is true Eric Blake
@ 2022-01-28 23:00 ` Eric Blake
2022-01-30 15:08 ` [PULL 0/4] NBD patches for 2022-01-28 Peter Maydell
4 siblings, 0 replies; 6+ messages in thread
From: Eric Blake @ 2022-01-28 23:00 UTC (permalink / raw)
To: qemu-devel
Cc: Nir Soffer, Kevin Wolf, Hanna Reitz, open list:Block layer core
From: Hanna Reitz <hreitz@redhat.com>
Add a new test to verify that want_zero=false block-status calls do not
pollute the block-status cache for want_zero=true calls.
We check want_zero=true calls and their results using `qemu-img map`
(over NBD), and want_zero=false calls also using `qemu-img map` over
NBD, but using the qemu:allocation-depth context.
(This test case cannot be integrated into nbd-qemu-allocation, because
that is a qcow2 test, and this is a raw test.)
Signed-off-by: Hanna Reitz <hreitz@redhat.com>
Message-Id: <20220118170000.49423-3-hreitz@redhat.com>
Reviewed-by: Nir Soffer <nsoffer@redhat.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Tested-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
tests/qemu-iotests/tests/block-status-cache | 139 ++++++++++++++++++
.../qemu-iotests/tests/block-status-cache.out | 5 +
2 files changed, 144 insertions(+)
create mode 100755 tests/qemu-iotests/tests/block-status-cache
create mode 100644 tests/qemu-iotests/tests/block-status-cache.out
diff --git a/tests/qemu-iotests/tests/block-status-cache b/tests/qemu-iotests/tests/block-status-cache
new file mode 100755
index 000000000000..6fa10bb8f8aa
--- /dev/null
+++ b/tests/qemu-iotests/tests/block-status-cache
@@ -0,0 +1,139 @@
+#!/usr/bin/env python3
+# group: rw quick
+#
+# Test cases for the block-status cache.
+#
+# Copyright (C) 2022 Red Hat, Inc.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation; either version 2 of the License, or
+# (at your option) any later version.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program. If not, see <http://www.gnu.org/licenses/>.
+#
+
+import os
+import signal
+import iotests
+from iotests import qemu_img_create, qemu_img_pipe, qemu_nbd
+
+
+image_size = 1 * 1024 * 1024
+test_img = os.path.join(iotests.test_dir, 'test.img')
+
+nbd_pidfile = os.path.join(iotests.test_dir, 'nbd.pid')
+nbd_sock = os.path.join(iotests.sock_dir, 'nbd.sock')
+
+
+class TestBscWithNbd(iotests.QMPTestCase):
+ def setUp(self) -> None:
+ """Just create an empty image with a read-only NBD server on it"""
+ assert qemu_img_create('-f', iotests.imgfmt, test_img,
+ str(image_size)) == 0
+
+ # Pass --allocation-depth to enable the qemu:allocation-depth context,
+ # which we are going to query to provoke a block-status inquiry with
+ # want_zero=false.
+ assert qemu_nbd(f'--socket={nbd_sock}',
+ f'--format={iotests.imgfmt}',
+ '--persistent',
+ '--allocation-depth',
+ '--read-only',
+ f'--pid-file={nbd_pidfile}',
+ test_img) \
+ == 0
+
+ def tearDown(self) -> None:
+ with open(nbd_pidfile, encoding='utf-8') as f:
+ pid = int(f.read())
+ os.kill(pid, signal.SIGTERM)
+ os.remove(nbd_pidfile)
+ os.remove(test_img)
+
+ def test_with_zero_bug(self) -> None:
+ """
+ Verify that the block-status cache is not corrupted by a
+ want_zero=false call.
+ We can provoke a want_zero=false call with `qemu-img map` over NBD with
+ x-dirty-bitmap=qemu:allocation-depth, so we first run a normal `map`
+ (which results in want_zero=true), then using said
+ qemu:allocation-depth context, and finally another normal `map` to
+ verify that the cache has not been corrupted.
+ """
+
+ nbd_img_opts = f'driver=nbd,server.type=unix,server.path={nbd_sock}'
+ nbd_img_opts_alloc_depth = nbd_img_opts + \
+ ',x-dirty-bitmap=qemu:allocation-depth'
+
+ # Normal map, results in want_zero=true.
+ # This will probably detect an allocated data sector first (qemu likes
+ # to allocate the first sector to facilitate alignment probing), and
+ # then the rest to be zero. The BSC will thus contain (if anything)
+ # one range covering the first sector.
+ map_pre = qemu_img_pipe('map', '--output=json', '--image-opts',
+ nbd_img_opts)
+
+ # qemu:allocation-depth maps for want_zero=false.
+ # want_zero=false should (with the file driver, which the server is
+ # using) report everything as data. While this is sufficient for
+ # want_zero=false, this is nothing that should end up in the
+ # block-status cache.
+ # Due to a bug, this information did end up in the cache, though, and
+ # this would lead to wrong information being returned on subsequent
+ # want_zero=true calls.
+ #
+ # We need to run this map twice: On the first call, we probably still
+ # have the first sector in the cache, and so this will be served from
+ # the cache; and only the subsequent range will be queried from the
+ # block driver. This subsequent range will then be entered into the
+ # cache.
+ # If we did a want_zero=true call at this point, we would thus get
+ # correct information: The first sector is not covered by the cache, so
+ # we would get fresh block-status information from the driver, which
+ # would return a data range, and this would then go into the cache,
+ # evicting the wrong range from the want_zero=false call before.
+ #
+ # Therefore, we need a second want_zero=false map to reproduce:
+ # Since the first sector is not in the cache, the query for its status
+ # will go to the driver, which will return a result that reports the
+ # whole image to be a single data area. This result will then go into
+ # the cache, and so the cache will then report the whole image to
+ # contain data.
+ #
+ # Note that once the cache reports the whole image to contain data, any
+ # subsequent map operation will be served from the cache, and so we can
+ # never loop too many times here.
+ for _ in range(2):
+ # (Ignore the result, this is just to contaminate the cache)
+ qemu_img_pipe('map', '--output=json', '--image-opts',
+ nbd_img_opts_alloc_depth)
+
+ # Now let's see whether the cache reports everything as data, or
+ # whether we get correct information (i.e. the same as we got on our
+ # first attempt).
+ map_post = qemu_img_pipe('map', '--output=json', '--image-opts',
+ nbd_img_opts)
+
+ if map_pre != map_post:
+ print('ERROR: Map information differs before and after querying ' +
+ 'qemu:allocation-depth')
+ print('Before:')
+ print(map_pre)
+ print('After:')
+ print(map_post)
+
+ self.fail("Map information differs")
+
+
+if __name__ == '__main__':
+ # The block-status cache only works on the protocol layer, so to test it,
+ # we can only use the raw format
+ iotests.main(supported_fmts=['raw'],
+ supported_protocols=['file'])
diff --git a/tests/qemu-iotests/tests/block-status-cache.out b/tests/qemu-iotests/tests/block-status-cache.out
new file mode 100644
index 000000000000..ae1213e6f863
--- /dev/null
+++ b/tests/qemu-iotests/tests/block-status-cache.out
@@ -0,0 +1,5 @@
+.
+----------------------------------------------------------------------
+Ran 1 tests
+
+OK
--
2.34.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PULL 0/4] NBD patches for 2022-01-28
2022-01-28 22:59 [PULL 0/4] NBD patches for 2022-01-28 Eric Blake
` (3 preceding siblings ...)
2022-01-28 23:00 ` [PULL 4/4] iotests/block-status-cache: New test Eric Blake
@ 2022-01-30 15:08 ` Peter Maydell
4 siblings, 0 replies; 6+ messages in thread
From: Peter Maydell @ 2022-01-30 15:08 UTC (permalink / raw)
To: Eric Blake; +Cc: qemu-devel
On Fri, 28 Jan 2022 at 23:04, Eric Blake <eblake@redhat.com> wrote:
>
> The following changes since commit 7a1043cef91739ff4b59812d30f1ed2850d3d34e:
>
> Merge remote-tracking branch 'remotes/bonzini-gitlab/tags/for-upstream' into staging (2022-01-28 14:04:01 +0000)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/ericb.git tags/pull-nbd-2022-01-28
>
> for you to fetch changes up to 6384dd534d742123d26c008d9794b20bc41359d5:
>
> iotests/block-status-cache: New test (2022-01-28 16:55:23 -0600)
>
> ----------------------------------------------------------------
> nbd patches for 2022-01-28
>
> - Hanna Reitz: regression fix for block status caching
> - Philippe Mathieu-Daude: documentation formatting
> - Nir Soffer: dead code removal
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/7.0
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2022-01-30 15:09 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-01-28 22:59 [PULL 0/4] NBD patches for 2022-01-28 Eric Blake
2022-01-28 23:00 ` [PULL 1/4] nbd/server.c: Remove unused field Eric Blake
2022-01-28 23:00 ` [PULL 2/4] qapi/block: Cosmetic change in BlockExportType schema Eric Blake
2022-01-28 23:00 ` [PULL 3/4] block/io: Update BSC only if want_zero is true Eric Blake
2022-01-28 23:00 ` [PULL 4/4] iotests/block-status-cache: New test Eric Blake
2022-01-30 15:08 ` [PULL 0/4] NBD patches for 2022-01-28 Peter Maydell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.