* [PULL v2 00/20] NBD patches for 2021-02-02
@ 2021-02-03 14:24 Eric Blake
2021-02-03 14:24 ` [PULL v2 05/20] block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up Eric Blake
` (3 more replies)
0 siblings, 4 replies; 5+ messages in thread
From: Eric Blake @ 2021-02-03 14:24 UTC (permalink / raw)
To: qemu-devel
The following changes since commit 8360ebeb4f4a707984cafd1a22c049ec82ddcb4c:
Merge remote-tracking branch 'remotes/ehabkost-gl/tags/machine-next-pull-request' into staging (2021-02-03 09:54:21 +0000)
are available in the Git repository at:
https://repo.or.cz/qemu/ericb.git tags/pull-nbd-2021-02-02-v2
for you to fetch changes up to 5082fc82a6bc3fc06a04be47d39777c7cff61e5b:
nbd: make nbd_read* return -EIO on error (2021-02-03 08:17:12 -0600)
v2: fix accidental inclusion of .rej file from merge resolution
[only affected patches re-sent]
----------------------------------------------------------------
nbd patches for 2021-02-02
- more cleanup from iotest python conversion
- progress towards consistent use of signed 64-bit types through block layer
- fix some crashes related to NBD reconnect
----------------------------------------------------------------
Eric Blake (2):
iotests: Fix expected whitespace for 185
block: use int64_t as bytes type in tracked requests
Roman Kagan (3):
block/nbd: only detach existing iochannel from aio_context
block/nbd: only enter connection coroutine if it's present
nbd: make nbd_read* return -EIO on error
Vladimir Sementsov-Ogievskiy (15):
block: refactor bdrv_check_request: add errp
util/iov: make qemu_iovec_init_extended() honest
block: fix theoretical overflow in bdrv_init_padding()
block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up
block/io: bdrv_pad_request(): support qemu_iovec_init_extended failure
block/throttle-groups: throttle_group_co_io_limits_intercept(): 64bit bytes
block/io: improve bdrv_check_request: check qiov too
block/io: use int64_t bytes in driver wrappers
block/io: support int64_t bytes in bdrv_co_do_pwrite_zeroes()
block/io: support int64_t bytes in bdrv_aligned_pwritev()
block/io: support int64_t bytes in bdrv_co_do_copy_on_readv()
block/io: support int64_t bytes in bdrv_aligned_preadv()
block/io: support int64_t bytes in bdrv_co_p{read,write}v_part()
block/io: support int64_t bytes in read/write wrappers
block/io: use int64_t bytes in copy_range
include/block/block.h | 17 +--
include/block/block_int.h | 26 ++--
include/block/nbd.h | 7 +-
include/block/throttle-groups.h | 2 +-
include/qemu/iov.h | 2 +-
block/io.c | 274 ++++++++++++++++++++++++++++------------
block/blkverify.c | 2 +-
block/file-posix.c | 2 +-
block/nbd.c | 25 ++--
block/throttle-groups.c | 5 +-
tests/test-write-threshold.c | 5 +-
util/iov.c | 25 +++-
block/trace-events | 12 +-
tests/qemu-iotests/185.out | 2 +-
tests/qemu-iotests/206.out | 2 +-
15 files changed, 275 insertions(+), 133 deletions(-)
--
2.30.0
^ permalink raw reply [flat|nested] 5+ messages in thread
* [PULL v2 05/20] block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up
2021-02-03 14:24 [PULL v2 00/20] NBD patches for 2021-02-02 Eric Blake
@ 2021-02-03 14:24 ` Eric Blake
2021-02-03 14:24 ` [PULL v2 09/20] block: use int64_t as bytes type in tracked requests Eric Blake
` (2 subsequent siblings)
3 siblings, 0 replies; 5+ messages in thread
From: Eric Blake @ 2021-02-03 14:24 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Kevin Wolf, Vladimir Sementsov-Ogievskiy,
open list:Block I/O path, Max Reitz, Stefan Hajnoczi
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Prepare for the following patch when bdrv_pad_request() will be able to
fail. Update the comments.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-5-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
[eblake: grammar tweak]
Signed-off-by: Eric Blake <eblake@redhat.com>
---
block/io.c | 25 +++++++++++++++++++------
1 file changed, 19 insertions(+), 6 deletions(-)
diff --git a/block/io.c b/block/io.c
index c8c9dea55466..3b1aec366ede 100644
--- a/block/io.c
+++ b/block/io.c
@@ -2135,6 +2135,7 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
uint64_t align = bs->bl.request_alignment;
BdrvRequestPadding pad;
int ret;
+ bool padded = false;
trace_bdrv_co_pwritev(child->bs, offset, bytes, flags);
@@ -2166,20 +2167,32 @@ int coroutine_fn bdrv_co_pwritev_part(BdrvChild *child,
return 0;
}
+ if (!(flags & BDRV_REQ_ZERO_WRITE)) {
+ /*
+ * Pad request for following read-modify-write cycle.
+ * bdrv_co_do_zero_pwritev() does aligning by itself, so, we do
+ * alignment only if there is no ZERO flag.
+ */
+ padded = bdrv_pad_request(bs, &qiov, &qiov_offset, &offset, &bytes,
+ &pad);
+ }
+
bdrv_inc_in_flight(bs);
- /*
- * Align write if necessary by performing a read-modify-write cycle.
- * Pad qiov with the read parts and be sure to have a tracked request not
- * only for bdrv_aligned_pwritev, but also for the reads of the RMW cycle.
- */
tracked_request_begin(&req, bs, offset, bytes, BDRV_TRACKED_WRITE);
if (flags & BDRV_REQ_ZERO_WRITE) {
+ assert(!padded);
ret = bdrv_co_do_zero_pwritev(child, offset, bytes, flags, &req);
goto out;
}
- if (bdrv_pad_request(bs, &qiov, &qiov_offset, &offset, &bytes, &pad)) {
+ if (padded) {
+ /*
+ * Request was unaligned to request_alignment and therefore
+ * padded. We are going to do read-modify-write, and must
+ * serialize the request to prevent interactions of the
+ * widened region with other transactions.
+ */
bdrv_make_request_serialising(&req, align);
bdrv_padding_rmw_read(child, &req, &pad, false);
}
--
2.30.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PULL v2 09/20] block: use int64_t as bytes type in tracked requests
2021-02-03 14:24 [PULL v2 00/20] NBD patches for 2021-02-02 Eric Blake
2021-02-03 14:24 ` [PULL v2 05/20] block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up Eric Blake
@ 2021-02-03 14:24 ` Eric Blake
2021-02-03 14:24 ` [PULL v2 12/20] block/io: support int64_t bytes in bdrv_aligned_pwritev() Eric Blake
2021-02-03 19:35 ` [PULL v2 00/20] NBD patches for 2021-02-02 Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Eric Blake @ 2021-02-03 14:24 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Kevin Wolf, Vladimir Sementsov-Ogievskiy,
open list:Block I/O path, Max Reitz, Stefan Hajnoczi
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.
Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.
We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).
All requests in block/io must not overflow BDRV_MAX_LENGTH, all
external users of BdrvTrackedRequest already have corresponding
assertions, so we are safe. Add some assertions still.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-9-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
include/block/block_int.h | 4 ++--
block/io.c | 14 +++++++++-----
2 files changed, 11 insertions(+), 7 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index 5bbbf9ee0af9..7f41f0990cc0 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -79,12 +79,12 @@ enum BdrvTrackedRequestType {
typedef struct BdrvTrackedRequest {
BlockDriverState *bs;
int64_t offset;
- uint64_t bytes;
+ int64_t bytes;
enum BdrvTrackedRequestType type;
bool serialising;
int64_t overlap_offset;
- uint64_t overlap_bytes;
+ int64_t overlap_bytes;
QLIST_ENTRY(BdrvTrackedRequest) list;
Coroutine *co; /* owner, used for deadlock detection */
diff --git a/block/io.c b/block/io.c
index b56db913da30..1c23587d18c6 100644
--- a/block/io.c
+++ b/block/io.c
@@ -717,10 +717,10 @@ static void tracked_request_end(BdrvTrackedRequest *req)
static void tracked_request_begin(BdrvTrackedRequest *req,
BlockDriverState *bs,
int64_t offset,
- uint64_t bytes,
+ int64_t bytes,
enum BdrvTrackedRequestType type)
{
- assert(bytes <= INT64_MAX && offset <= INT64_MAX - bytes);
+ bdrv_check_request(offset, bytes, &error_abort);
*req = (BdrvTrackedRequest){
.bs = bs,
@@ -741,8 +741,10 @@ static void tracked_request_begin(BdrvTrackedRequest *req,
}
static bool tracked_request_overlaps(BdrvTrackedRequest *req,
- int64_t offset, uint64_t bytes)
+ int64_t offset, int64_t bytes)
{
+ bdrv_check_request(offset, bytes, &error_abort);
+
/* aaaa bbbb */
if (offset >= req->overlap_offset + req->overlap_bytes) {
return false;
@@ -810,8 +812,10 @@ static void tracked_request_set_serialising(BdrvTrackedRequest *req,
uint64_t align)
{
int64_t overlap_offset = req->offset & ~(align - 1);
- uint64_t overlap_bytes = ROUND_UP(req->offset + req->bytes, align)
- - overlap_offset;
+ int64_t overlap_bytes =
+ ROUND_UP(req->offset + req->bytes, align) - overlap_offset;
+
+ bdrv_check_request(req->offset, req->bytes, &error_abort);
if (!req->serialising) {
qatomic_inc(&req->bs->serialising_in_flight);
--
2.30.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* [PULL v2 12/20] block/io: support int64_t bytes in bdrv_aligned_pwritev()
2021-02-03 14:24 [PULL v2 00/20] NBD patches for 2021-02-02 Eric Blake
2021-02-03 14:24 ` [PULL v2 05/20] block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up Eric Blake
2021-02-03 14:24 ` [PULL v2 09/20] block: use int64_t as bytes type in tracked requests Eric Blake
@ 2021-02-03 14:24 ` Eric Blake
2021-02-03 19:35 ` [PULL v2 00/20] NBD patches for 2021-02-02 Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Eric Blake @ 2021-02-03 14:24 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Kevin Wolf, Vladimir Sementsov-Ogievskiy,
open list:Block I/O path, Max Reitz, Stefan Hajnoczi
From: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
We are generally moving to int64_t for both offset and bytes parameters
on all io paths.
Main motivation is realization of 64-bit write_zeroes operation for
fast zeroing large disk chunks, up to the whole disk.
We chose signed type, to be consistent with off_t (which is signed) and
with possibility for signed return type (where negative value means
error).
So, prepare bdrv_aligned_pwritev() now and convert the dependencies:
bdrv_co_write_req_prepare() and bdrv_co_write_req_finish() to signed
type bytes.
Conversion of bdrv_co_write_req_prepare() and
bdrv_co_write_req_finish() is definitely safe, as all requests in
block/io must not overflow BDRV_MAX_LENGTH. Still add assertions.
For bdrv_aligned_pwritev() 'bytes' type is widened, so callers are
safe. Let's check usage of the parameter inside the function.
Passing to bdrv_co_write_req_prepare() and bdrv_co_write_req_finish()
is OK.
Passing to qemu_iovec_* is OK after new assertion. All other callees
are already updated to int64_t.
Checking alignment is not changed, offset + bytes and qiov_offset +
bytes calculations are safe (thanks to new assertions).
max_transfer is kept to be int for now. It has a default of INT_MAX
here, and some drivers may rely on it. It's to be refactored later.
Signed-off-by: Vladimir Sementsov-Ogievskiy <vsementsov@virtuozzo.com>
Message-Id: <20201211183934.169161-12-vsementsov@virtuozzo.com>
Reviewed-by: Eric Blake <eblake@redhat.com>
Signed-off-by: Eric Blake <eblake@redhat.com>
---
block/io.c | 21 +++++++++++++--------
1 file changed, 13 insertions(+), 8 deletions(-)
diff --git a/block/io.c b/block/io.c
index 98d9f5bdf48a..59ae0a110da1 100644
--- a/block/io.c
+++ b/block/io.c
@@ -1932,11 +1932,12 @@ fail:
}
static inline int coroutine_fn
-bdrv_co_write_req_prepare(BdrvChild *child, int64_t offset, uint64_t bytes,
+bdrv_co_write_req_prepare(BdrvChild *child, int64_t offset, int64_t bytes,
BdrvTrackedRequest *req, int flags)
{
BlockDriverState *bs = child->bs;
- int64_t end_sector = DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE);
+
+ bdrv_check_request(offset, bytes, &error_abort);
if (bs->read_only) {
return -EPERM;
@@ -1963,7 +1964,8 @@ bdrv_co_write_req_prepare(BdrvChild *child, int64_t offset, uint64_t bytes,
assert(req->overlap_offset <= offset);
assert(offset + bytes <= req->overlap_offset + req->overlap_bytes);
- assert(end_sector <= bs->total_sectors || child->perm & BLK_PERM_RESIZE);
+ assert(offset + bytes <= bs->total_sectors * BDRV_SECTOR_SIZE ||
+ child->perm & BLK_PERM_RESIZE);
switch (req->type) {
case BDRV_TRACKED_WRITE:
@@ -1984,12 +1986,14 @@ bdrv_co_write_req_prepare(BdrvChild *child, int64_t offset, uint64_t bytes,
}
static inline void coroutine_fn
-bdrv_co_write_req_finish(BdrvChild *child, int64_t offset, uint64_t bytes,
+bdrv_co_write_req_finish(BdrvChild *child, int64_t offset, int64_t bytes,
BdrvTrackedRequest *req, int ret)
{
int64_t end_sector = DIV_ROUND_UP(offset + bytes, BDRV_SECTOR_SIZE);
BlockDriverState *bs = child->bs;
+ bdrv_check_request(offset, bytes, &error_abort);
+
qatomic_inc(&bs->write_gen);
/*
@@ -2026,16 +2030,18 @@ bdrv_co_write_req_finish(BdrvChild *child, int64_t offset, uint64_t bytes,
* after possibly fragmenting it.
*/
static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
- BdrvTrackedRequest *req, int64_t offset, unsigned int bytes,
+ BdrvTrackedRequest *req, int64_t offset, int64_t bytes,
int64_t align, QEMUIOVector *qiov, size_t qiov_offset, int flags)
{
BlockDriverState *bs = child->bs;
BlockDriver *drv = bs->drv;
int ret;
- uint64_t bytes_remaining = bytes;
+ int64_t bytes_remaining = bytes;
int max_transfer;
+ bdrv_check_qiov_request(offset, bytes, qiov, qiov_offset, &error_abort);
+
if (!drv) {
return -ENOMEDIUM;
}
@@ -2047,7 +2053,6 @@ static int coroutine_fn bdrv_aligned_pwritev(BdrvChild *child,
assert(is_power_of_2(align));
assert((offset & (align - 1)) == 0);
assert((bytes & (align - 1)) == 0);
- assert(!qiov || qiov_offset + bytes <= qiov->size);
max_transfer = QEMU_ALIGN_DOWN(MIN_NON_ZERO(bs->bl.max_transfer, INT_MAX),
align);
@@ -2146,7 +2151,7 @@ static int coroutine_fn bdrv_co_do_zero_pwritev(BdrvChild *child,
assert(!bytes || (offset & (align - 1)) == 0);
if (bytes >= align) {
/* Write the aligned part in the middle. */
- uint64_t aligned_bytes = bytes & ~(align - 1);
+ int64_t aligned_bytes = bytes & ~(align - 1);
ret = bdrv_aligned_pwritev(child, req, offset, aligned_bytes, align,
NULL, 0, flags);
if (ret < 0) {
--
2.30.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [PULL v2 00/20] NBD patches for 2021-02-02
2021-02-03 14:24 [PULL v2 00/20] NBD patches for 2021-02-02 Eric Blake
` (2 preceding siblings ...)
2021-02-03 14:24 ` [PULL v2 12/20] block/io: support int64_t bytes in bdrv_aligned_pwritev() Eric Blake
@ 2021-02-03 19:35 ` Peter Maydell
3 siblings, 0 replies; 5+ messages in thread
From: Peter Maydell @ 2021-02-03 19:35 UTC (permalink / raw)
To: Eric Blake; +Cc: QEMU Developers
On Wed, 3 Feb 2021 at 14:28, Eric Blake <eblake@redhat.com> wrote:
>
> The following changes since commit 8360ebeb4f4a707984cafd1a22c049ec82ddcb4c:
>
> Merge remote-tracking branch 'remotes/ehabkost-gl/tags/machine-next-pull-request' into staging (2021-02-03 09:54:21 +0000)
>
> are available in the Git repository at:
>
> https://repo.or.cz/qemu/ericb.git tags/pull-nbd-2021-02-02-v2
>
> for you to fetch changes up to 5082fc82a6bc3fc06a04be47d39777c7cff61e5b:
>
> nbd: make nbd_read* return -EIO on error (2021-02-03 08:17:12 -0600)
>
> v2: fix accidental inclusion of .rej file from merge resolution
> [only affected patches re-sent]
>
> ----------------------------------------------------------------
> nbd patches for 2021-02-02
>
> - more cleanup from iotest python conversion
> - progress towards consistent use of signed 64-bit types through block layer
> - fix some crashes related to NBD reconnect
>
Applied, thanks.
Please update the changelog at https://wiki.qemu.org/ChangeLog/6.0
for any user-visible changes.
-- PMM
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2021-02-03 19:59 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-02-03 14:24 [PULL v2 00/20] NBD patches for 2021-02-02 Eric Blake
2021-02-03 14:24 ` [PULL v2 05/20] block/io: refactor bdrv_pad_request(): move bdrv_pad_request() up Eric Blake
2021-02-03 14:24 ` [PULL v2 09/20] block: use int64_t as bytes type in tracked requests Eric Blake
2021-02-03 14:24 ` [PULL v2 12/20] block/io: support int64_t bytes in bdrv_aligned_pwritev() Eric Blake
2021-02-03 19:35 ` [PULL v2 00/20] NBD patches for 2021-02-02 Peter Maydell
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.