* [Qemu-devel] [PATCH 0/3] add bdrv_co_drain_begin/end BlockDriver callbacks
@ 2017-09-20 10:23 Manos Pitsidianakis
2017-09-20 10:23 ` [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback Manos Pitsidianakis
` (2 more replies)
0 siblings, 3 replies; 9+ messages in thread
From: Manos Pitsidianakis @ 2017-09-20 10:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-block, Kevin Wolf, Stefan Hajnoczi, Max Reitz, Fam Zheng
This patch series renames bdrv_co_drain to bdrv_co_drain_begin and adds a new
bdrv_co_drain_end callback to match bdrv_drained_begin/end and
drained_begin/end of BdrvChild. This is needed because the throttle driver
(block/throttle.c) needs a way to mark the end of the drain in order to toggle
io_limits_disabled correctly.
Based-on: <20170918202529.28379-1-el13635@mail.ntua.gr>
"block/throttle-groups.c: allocate RestartData on the heap"
Which fixes a coroutine crash in block/throttle-groups.c
Manos Pitsidianakis (3):
block: add bdrv_co_drain_end callback
block: rename bdrv_co_drain to bdrv_co_drain_begin
block/throttle.c: add bdrv_co_drain_begin/end callbacks
include/block/block_int.h | 4 +++-
block/io.c | 42 ++++++++++++++++++++++++++++++------------
block/qed.c | 6 +++---
block/throttle.c | 18 ++++++++++++++++++
4 files changed, 54 insertions(+), 16 deletions(-)
--
2.11.0
^ permalink raw reply [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback
2017-09-20 10:23 [Qemu-devel] [PATCH 0/3] add bdrv_co_drain_begin/end BlockDriver callbacks Manos Pitsidianakis
@ 2017-09-20 10:23 ` Manos Pitsidianakis
2017-09-20 14:26 ` Stefan Hajnoczi
2017-09-20 14:28 ` Stefan Hajnoczi
2017-09-20 10:23 ` [Qemu-devel] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin Manos Pitsidianakis
2017-09-20 10:23 ` [Qemu-devel] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks Manos Pitsidianakis
2 siblings, 2 replies; 9+ messages in thread
From: Manos Pitsidianakis @ 2017-09-20 10:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-block, Kevin Wolf, Stefan Hajnoczi, Max Reitz, Fam Zheng
BlockDriverState has a bdrv_do_drain() callback but no equivalent for the end
of the drain. The throttle driver (block/throttle.c) needs a way to mark the
end of the drain in order to toggle io_limits_disabled correctly, thus
bdrv_co_drain_end is needed.
Signed-off-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
---
include/block/block_int.h | 2 ++
block/io.c | 43 +++++++++++++++++++++++++++++++------------
2 files changed, 33 insertions(+), 12 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index ba4c383393..ea1326e3c7 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -359,6 +359,8 @@ struct BlockDriver {
*/
void coroutine_fn (*bdrv_co_drain)(BlockDriverState *bs);
+ void coroutine_fn (*bdrv_co_drain_end)(BlockDriverState *bs);
+
void (*bdrv_add_child)(BlockDriverState *parent, BlockDriverState *child,
Error **errp);
void (*bdrv_del_child)(BlockDriverState *parent, BdrvChild *child,
diff --git a/block/io.c b/block/io.c
index 4378ae4c7d..465345289d 100644
--- a/block/io.c
+++ b/block/io.c
@@ -153,6 +153,7 @@ typedef struct {
Coroutine *co;
BlockDriverState *bs;
bool done;
+ bool begin;
} BdrvCoDrainData;
static void coroutine_fn bdrv_drain_invoke_entry(void *opaque)
@@ -160,18 +161,23 @@ static void coroutine_fn bdrv_drain_invoke_entry(void *opaque)
BdrvCoDrainData *data = opaque;
BlockDriverState *bs = data->bs;
- bs->drv->bdrv_co_drain(bs);
+ if (data->begin) {
+ bs->drv->bdrv_co_drain(bs);
+ } else {
+ bs->drv->bdrv_co_drain_end(bs);
+ }
/* Set data->done before reading bs->wakeup. */
atomic_mb_set(&data->done, true);
bdrv_wakeup(bs);
}
-static void bdrv_drain_invoke(BlockDriverState *bs)
+static void bdrv_drain_invoke(BlockDriverState *bs, bool begin)
{
- BdrvCoDrainData data = { .bs = bs, .done = false };
+ BdrvCoDrainData data = { .bs = bs, .done = false, .begin = begin};
- if (!bs->drv || !bs->drv->bdrv_co_drain) {
+ if (!bs->drv || (begin && !bs->drv->bdrv_co_drain) ||
+ (!begin && !bs->drv->bdrv_co_drain_end)) {
return;
}
@@ -180,7 +186,7 @@ static void bdrv_drain_invoke(BlockDriverState *bs)
BDRV_POLL_WHILE(bs, !data.done);
}
-static bool bdrv_drain_recurse(BlockDriverState *bs)
+static bool bdrv_drain_recurse(BlockDriverState *bs, bool begin)
{
BdrvChild *child, *tmp;
bool waited;
@@ -188,7 +194,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs)
waited = BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0);
/* Ensure any pending metadata writes are submitted to bs->file. */
- bdrv_drain_invoke(bs);
+ bdrv_drain_invoke(bs, begin);
QLIST_FOREACH_SAFE(child, &bs->children, next, tmp) {
BlockDriverState *bs = child->bs;
@@ -205,7 +211,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs)
*/
bdrv_ref(bs);
}
- waited |= bdrv_drain_recurse(bs);
+ waited |= bdrv_drain_recurse(bs, begin);
if (in_main_loop) {
bdrv_unref(bs);
}
@@ -221,12 +227,18 @@ static void bdrv_co_drain_bh_cb(void *opaque)
BlockDriverState *bs = data->bs;
bdrv_dec_in_flight(bs);
- bdrv_drained_begin(bs);
+ if (data->begin) {
+ bdrv_drained_begin(bs);
+ } else {
+ bdrv_drained_end(bs);
+ }
+
data->done = true;
aio_co_wake(co);
}
-static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs)
+static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs,
+ bool begin)
{
BdrvCoDrainData data;
@@ -239,6 +251,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs)
.co = qemu_coroutine_self(),
.bs = bs,
.done = false,
+ .begin = begin,
};
bdrv_inc_in_flight(bs);
aio_bh_schedule_oneshot(bdrv_get_aio_context(bs),
@@ -253,7 +266,7 @@ static void coroutine_fn bdrv_co_yield_to_drain(BlockDriverState *bs)
void bdrv_drained_begin(BlockDriverState *bs)
{
if (qemu_in_coroutine()) {
- bdrv_co_yield_to_drain(bs);
+ bdrv_co_yield_to_drain(bs, true);
return;
}
@@ -262,17 +275,22 @@ void bdrv_drained_begin(BlockDriverState *bs)
bdrv_parent_drained_begin(bs);
}
- bdrv_drain_recurse(bs);
+ bdrv_drain_recurse(bs, true);
}
void bdrv_drained_end(BlockDriverState *bs)
{
+ if (qemu_in_coroutine()) {
+ bdrv_co_yield_to_drain(bs, false);
+ return;
+ }
assert(bs->quiesce_counter > 0);
if (atomic_fetch_dec(&bs->quiesce_counter) > 1) {
return;
}
bdrv_parent_drained_end(bs);
+ bdrv_drain_recurse(bs, false);
aio_enable_external(bdrv_get_aio_context(bs));
}
@@ -350,7 +368,7 @@ void bdrv_drain_all_begin(void)
aio_context_acquire(aio_context);
for (bs = bdrv_first(&it); bs; bs = bdrv_next(&it)) {
if (aio_context == bdrv_get_aio_context(bs)) {
- waited |= bdrv_drain_recurse(bs);
+ waited |= bdrv_drain_recurse(bs, true);
}
}
aio_context_release(aio_context);
@@ -371,6 +389,7 @@ void bdrv_drain_all_end(void)
aio_context_acquire(aio_context);
aio_enable_external(aio_context);
bdrv_parent_drained_end(bs);
+ bdrv_drain_recurse(bs, false);
aio_context_release(aio_context);
}
--
2.11.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin
2017-09-20 10:23 [Qemu-devel] [PATCH 0/3] add bdrv_co_drain_begin/end BlockDriver callbacks Manos Pitsidianakis
2017-09-20 10:23 ` [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback Manos Pitsidianakis
@ 2017-09-20 10:23 ` Manos Pitsidianakis
2017-09-20 14:28 ` Stefan Hajnoczi
2017-09-20 10:23 ` [Qemu-devel] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks Manos Pitsidianakis
2 siblings, 1 reply; 9+ messages in thread
From: Manos Pitsidianakis @ 2017-09-20 10:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-block, Kevin Wolf, Stefan Hajnoczi, Max Reitz, Fam Zheng
Signed-off-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
---
include/block/block_int.h | 2 +-
block/io.c | 4 ++--
block/qed.c | 6 +++---
3 files changed, 6 insertions(+), 6 deletions(-)
diff --git a/include/block/block_int.h b/include/block/block_int.h
index ea1326e3c7..fd10f53b43 100644
--- a/include/block/block_int.h
+++ b/include/block/block_int.h
@@ -357,7 +357,7 @@ struct BlockDriver {
* Drain and stop any internal sources of requests in the driver, and
* remain so until next I/O callback (e.g. bdrv_co_writev) is called.
*/
- void coroutine_fn (*bdrv_co_drain)(BlockDriverState *bs);
+ void coroutine_fn (*bdrv_co_drain_begin)(BlockDriverState *bs);
void coroutine_fn (*bdrv_co_drain_end)(BlockDriverState *bs);
diff --git a/block/io.c b/block/io.c
index 465345289d..6b4b13b777 100644
--- a/block/io.c
+++ b/block/io.c
@@ -162,7 +162,7 @@ static void coroutine_fn bdrv_drain_invoke_entry(void *opaque)
BlockDriverState *bs = data->bs;
if (data->begin) {
- bs->drv->bdrv_co_drain(bs);
+ bs->drv->bdrv_co_drain_begin(bs);
} else {
bs->drv->bdrv_co_drain_end(bs);
}
@@ -176,7 +176,7 @@ static void bdrv_drain_invoke(BlockDriverState *bs, bool begin)
{
BdrvCoDrainData data = { .bs = bs, .done = false, .begin = begin};
- if (!bs->drv || (begin && !bs->drv->bdrv_co_drain) ||
+ if (!bs->drv || (begin && !bs->drv->bdrv_co_drain_begin) ||
(!begin && !bs->drv->bdrv_co_drain_end)) {
return;
}
diff --git a/block/qed.c b/block/qed.c
index 28e2ec89e8..821dcaa055 100644
--- a/block/qed.c
+++ b/block/qed.c
@@ -265,7 +265,7 @@ static bool qed_plug_allocating_write_reqs(BDRVQEDState *s)
assert(!s->allocating_write_reqs_plugged);
if (s->allocating_acb != NULL) {
/* Another allocating write came concurrently. This cannot happen
- * from bdrv_qed_co_drain, but it can happen when the timer runs.
+ * from bdrv_qed_co_drain_begin, but it can happen when the timer runs.
*/
qemu_co_mutex_unlock(&s->table_lock);
return false;
@@ -358,7 +358,7 @@ static void bdrv_qed_attach_aio_context(BlockDriverState *bs,
}
}
-static void coroutine_fn bdrv_qed_co_drain(BlockDriverState *bs)
+static void coroutine_fn bdrv_qed_co_drain_begin(BlockDriverState *bs)
{
BDRVQEDState *s = bs->opaque;
@@ -1608,7 +1608,7 @@ static BlockDriver bdrv_qed = {
.bdrv_check = bdrv_qed_check,
.bdrv_detach_aio_context = bdrv_qed_detach_aio_context,
.bdrv_attach_aio_context = bdrv_qed_attach_aio_context,
- .bdrv_co_drain = bdrv_qed_co_drain,
+ .bdrv_co_drain_begin = bdrv_qed_co_drain_begin,
};
static void bdrv_qed_init(void)
--
2.11.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* [Qemu-devel] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks
2017-09-20 10:23 [Qemu-devel] [PATCH 0/3] add bdrv_co_drain_begin/end BlockDriver callbacks Manos Pitsidianakis
2017-09-20 10:23 ` [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback Manos Pitsidianakis
2017-09-20 10:23 ` [Qemu-devel] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin Manos Pitsidianakis
@ 2017-09-20 10:23 ` Manos Pitsidianakis
2017-09-20 14:26 ` Stefan Hajnoczi
2 siblings, 1 reply; 9+ messages in thread
From: Manos Pitsidianakis @ 2017-09-20 10:23 UTC (permalink / raw)
To: qemu-devel; +Cc: qemu-block, Kevin Wolf, Stefan Hajnoczi, Max Reitz, Fam Zheng
Signed-off-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
---
block/throttle.c | 18 ++++++++++++++++++
1 file changed, 18 insertions(+)
diff --git a/block/throttle.c b/block/throttle.c
index 5bca76300f..833175ac77 100644
--- a/block/throttle.c
+++ b/block/throttle.c
@@ -197,6 +197,21 @@ static bool throttle_recurse_is_first_non_filter(BlockDriverState *bs,
return bdrv_recurse_is_first_non_filter(bs->file->bs, candidate);
}
+static void coroutine_fn throttle_co_drain_begin(BlockDriverState *bs)
+{
+ ThrottleGroupMember *tgm = bs->opaque;
+ if (atomic_fetch_inc(&tgm->io_limits_disabled) == 0) {
+ throttle_group_restart_tgm(tgm);
+ }
+}
+
+static void coroutine_fn throttle_co_drain_end(BlockDriverState *bs)
+{
+ ThrottleGroupMember *tgm = bs->opaque;
+ assert(tgm->io_limits_disabled);
+ atomic_dec(&tgm->io_limits_disabled);
+}
+
static BlockDriver bdrv_throttle = {
.format_name = "throttle",
.protocol_name = "throttle",
@@ -226,6 +241,9 @@ static BlockDriver bdrv_throttle = {
.bdrv_reopen_abort = throttle_reopen_abort,
.bdrv_co_get_block_status = bdrv_co_get_block_status_from_file,
+ .bdrv_co_drain_begin = throttle_co_drain_begin,
+ .bdrv_co_drain_end = throttle_co_drain_end,
+
.is_filter = true,
};
--
2.11.0
^ permalink raw reply related [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback
2017-09-20 10:23 ` [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback Manos Pitsidianakis
@ 2017-09-20 14:26 ` Stefan Hajnoczi
2017-09-20 17:32 ` Manos Pitsidianakis
2017-09-20 14:28 ` Stefan Hajnoczi
1 sibling, 1 reply; 9+ messages in thread
From: Stefan Hajnoczi @ 2017-09-20 14:26 UTC (permalink / raw)
To: Manos Pitsidianakis
Cc: qemu-devel, qemu-block, Kevin Wolf, Max Reitz, Fam Zheng
On Wed, Sep 20, 2017 at 01:23:09PM +0300, Manos Pitsidianakis wrote:
> @@ -188,7 +194,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs)
> waited = BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0);
>
> /* Ensure any pending metadata writes are submitted to bs->file. */
> - bdrv_drain_invoke(bs);
> + bdrv_drain_invoke(bs, begin);
Do you need to move bdrv_drain_invoke(bs, begin) before
BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0)?
This will ensure that throttling is disabled and the TGM restarted
before we wait for requests to complete.
Stefan
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks
2017-09-20 10:23 ` [Qemu-devel] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks Manos Pitsidianakis
@ 2017-09-20 14:26 ` Stefan Hajnoczi
0 siblings, 0 replies; 9+ messages in thread
From: Stefan Hajnoczi @ 2017-09-20 14:26 UTC (permalink / raw)
To: Manos Pitsidianakis
Cc: qemu-devel, qemu-block, Kevin Wolf, Max Reitz, Fam Zheng
On Wed, Sep 20, 2017 at 01:23:11PM +0300, Manos Pitsidianakis wrote:
> Signed-off-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
> ---
> block/throttle.c | 18 ++++++++++++++++++
> 1 file changed, 18 insertions(+)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback
2017-09-20 10:23 ` [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback Manos Pitsidianakis
2017-09-20 14:26 ` Stefan Hajnoczi
@ 2017-09-20 14:28 ` Stefan Hajnoczi
1 sibling, 0 replies; 9+ messages in thread
From: Stefan Hajnoczi @ 2017-09-20 14:28 UTC (permalink / raw)
To: Manos Pitsidianakis
Cc: qemu-devel, qemu-block, Kevin Wolf, Max Reitz, Fam Zheng
On Wed, Sep 20, 2017 at 01:23:09PM +0300, Manos Pitsidianakis wrote:
> diff --git a/include/block/block_int.h b/include/block/block_int.h
> index ba4c383393..ea1326e3c7 100644
> --- a/include/block/block_int.h
> +++ b/include/block/block_int.h
> @@ -359,6 +359,8 @@ struct BlockDriver {
> */
> void coroutine_fn (*bdrv_co_drain)(BlockDriverState *bs);
>
> + void coroutine_fn (*bdrv_co_drain_end)(BlockDriverState *bs);
Please update the doc comment to describe the environment under which
begin() and end() are invoked. It should be clear when to implement
begin() and/or end().
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin
2017-09-20 10:23 ` [Qemu-devel] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin Manos Pitsidianakis
@ 2017-09-20 14:28 ` Stefan Hajnoczi
0 siblings, 0 replies; 9+ messages in thread
From: Stefan Hajnoczi @ 2017-09-20 14:28 UTC (permalink / raw)
To: Manos Pitsidianakis
Cc: qemu-devel, qemu-block, Kevin Wolf, Max Reitz, Fam Zheng
On Wed, Sep 20, 2017 at 01:23:10PM +0300, Manos Pitsidianakis wrote:
> Signed-off-by: Manos Pitsidianakis <el13635@mail.ntua.gr>
> ---
> include/block/block_int.h | 2 +-
> block/io.c | 4 ++--
> block/qed.c | 6 +++---
> 3 files changed, 6 insertions(+), 6 deletions(-)
Reviewed-by: Stefan Hajnoczi <stefanha@redhat.com>
^ permalink raw reply [flat|nested] 9+ messages in thread
* Re: [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback
2017-09-20 14:26 ` Stefan Hajnoczi
@ 2017-09-20 17:32 ` Manos Pitsidianakis
0 siblings, 0 replies; 9+ messages in thread
From: Manos Pitsidianakis @ 2017-09-20 17:32 UTC (permalink / raw)
To: Stefan Hajnoczi; +Cc: qemu-devel, qemu-block, Kevin Wolf, Max Reitz, Fam Zheng
[-- Attachment #1: Type: text/plain, Size: 813 bytes --]
On Wed, Sep 20, 2017 at 03:26:32PM +0100, Stefan Hajnoczi wrote:
>On Wed, Sep 20, 2017 at 01:23:09PM +0300, Manos Pitsidianakis wrote:
>> @@ -188,7 +194,7 @@ static bool bdrv_drain_recurse(BlockDriverState *bs)
>> waited = BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0);
>>
>> /* Ensure any pending metadata writes are submitted to bs->file. */
>> - bdrv_drain_invoke(bs);
>> + bdrv_drain_invoke(bs, begin);
>
>Do you need to move bdrv_drain_invoke(bs, begin) before
>BDRV_POLL_WHILE(bs, atomic_read(&bs->in_flight) > 0)?
>
>This will ensure that throttling is disabled and the TGM restarted
>before we wait for requests to complete.
>
Hm yes. Before, the order was irrelevant because BlockBackend issued the
drain first by restarting the tgm in blk_root_drained_begin.
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 9+ messages in thread
end of thread, other threads:[~2017-09-20 17:32 UTC | newest]
Thread overview: 9+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-09-20 10:23 [Qemu-devel] [PATCH 0/3] add bdrv_co_drain_begin/end BlockDriver callbacks Manos Pitsidianakis
2017-09-20 10:23 ` [Qemu-devel] [PATCH 1/3] block: add bdrv_co_drain_end callback Manos Pitsidianakis
2017-09-20 14:26 ` Stefan Hajnoczi
2017-09-20 17:32 ` Manos Pitsidianakis
2017-09-20 14:28 ` Stefan Hajnoczi
2017-09-20 10:23 ` [Qemu-devel] [PATCH 2/3] block: rename bdrv_co_drain to bdrv_co_drain_begin Manos Pitsidianakis
2017-09-20 14:28 ` Stefan Hajnoczi
2017-09-20 10:23 ` [Qemu-devel] [PATCH 3/3] block/throttle.c: add bdrv_co_drain_begin/end callbacks Manos Pitsidianakis
2017-09-20 14:26 ` Stefan Hajnoczi
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.