* [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free
@ 2019-07-23 19:06 Stefan Hajnoczi
2019-07-23 19:09 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
` (2 more replies)
0 siblings, 3 replies; 5+ messages in thread
From: Stefan Hajnoczi @ 2019-07-23 19:06 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Paolo Bonzini, Philippe Mathieu-Daudé,
qemu-block, Stefan Hajnoczi
The tests/test-bdrv-drain /bdrv-drain/iothread/drain test case does the
following:
1. The preadv coroutine calls aio_bh_schedule_oneshot() and then yields.
2. The one-shot BH executes in another AioContext. All it does is call
aio_co_wakeup(preadv_co).
3. The preadv coroutine is re-entered and returns.
There is a race condition in aio_co_wake() where the preadv coroutine
returns and the test case destroys the preadv IOThread. aio_co_wake()
can still be running in the other AioContext and it performs an access
to the freed IOThread AioContext.
Here is the race in aio_co_schedule():
QSLIST_INSERT_HEAD_ATOMIC(&ctx->scheduled_coroutines,
co, co_scheduled_next);
<-- race: co may execute before we invoke qemu_bh_schedule()!
qemu_bh_schedule(ctx->co_schedule_bh);
So if co causes ctx to be freed then we're in trouble. Fix this problem
by holding a reference to ctx.
Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
---
util/async.c | 8 ++++++++
1 file changed, 8 insertions(+)
diff --git a/util/async.c b/util/async.c
index 8d2105729c..4e4c7af51e 100644
--- a/util/async.c
+++ b/util/async.c
@@ -459,9 +459,17 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
abort();
}
+ /* The coroutine might run and release the last ctx reference before we
+ * invoke qemu_bh_schedule(). Take a reference to keep ctx alive until
+ * we're done.
+ */
+ aio_context_ref(ctx);
+
QSLIST_INSERT_HEAD_ATOMIC(&ctx->scheduled_coroutines,
co, co_scheduled_next);
qemu_bh_schedule(ctx->co_schedule_bh);
+
+ aio_context_unref(ctx);
}
void aio_co_wake(struct Coroutine *co)
--
2.21.0
^ permalink raw reply related [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH] util/async: hold AioContext ref to prevent use-after-free
2019-07-23 19:06 [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free Stefan Hajnoczi
@ 2019-07-23 19:09 ` Stefan Hajnoczi
2019-07-24 10:54 ` Philippe Mathieu-Daudé
2019-07-24 8:47 ` [Qemu-devel] " Paolo Bonzini
2019-07-24 11:42 ` Stefan Hajnoczi
2 siblings, 1 reply; 5+ messages in thread
From: Stefan Hajnoczi @ 2019-07-23 19:09 UTC (permalink / raw)
To: Stefan Hajnoczi
Cc: Fam Zheng, Paolo Bonzini, Philippe Mathieu-Daudé,
qemu-devel, qemu block
On Tue, Jul 23, 2019 at 8:06 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
> So if co causes ctx to be freed then we're in trouble. Fix this problem
> by holding a reference to ctx.
For QEMU 4.2. I'm not aware of a way to trigger this bug in QEMU
proper. This fix just makes tests/test-bdrv-drain more reliable.
Stefan
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free
2019-07-23 19:06 [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free Stefan Hajnoczi
2019-07-23 19:09 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
@ 2019-07-24 8:47 ` Paolo Bonzini
2019-07-24 11:42 ` Stefan Hajnoczi
2 siblings, 0 replies; 5+ messages in thread
From: Paolo Bonzini @ 2019-07-24 8:47 UTC (permalink / raw)
To: Stefan Hajnoczi, qemu-devel
Cc: Fam Zheng, Philippe Mathieu-Daudé, qemu-block
On 23/07/19 21:06, Stefan Hajnoczi wrote:
> The tests/test-bdrv-drain /bdrv-drain/iothread/drain test case does the
> following:
>
> 1. The preadv coroutine calls aio_bh_schedule_oneshot() and then yields.
> 2. The one-shot BH executes in another AioContext. All it does is call
> aio_co_wakeup(preadv_co).
> 3. The preadv coroutine is re-entered and returns.
>
> There is a race condition in aio_co_wake() where the preadv coroutine
> returns and the test case destroys the preadv IOThread. aio_co_wake()
> can still be running in the other AioContext and it performs an access
> to the freed IOThread AioContext.
>
> Here is the race in aio_co_schedule():
>
> QSLIST_INSERT_HEAD_ATOMIC(&ctx->scheduled_coroutines,
> co, co_scheduled_next);
> <-- race: co may execute before we invoke qemu_bh_schedule()!
> qemu_bh_schedule(ctx->co_schedule_bh);
>
> So if co causes ctx to be freed then we're in trouble. Fix this problem
> by holding a reference to ctx.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> util/async.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
>
> diff --git a/util/async.c b/util/async.c
> index 8d2105729c..4e4c7af51e 100644
> --- a/util/async.c
> +++ b/util/async.c
> @@ -459,9 +459,17 @@ void aio_co_schedule(AioContext *ctx, Coroutine *co)
> abort();
> }
>
> + /* The coroutine might run and release the last ctx reference before we
> + * invoke qemu_bh_schedule(). Take a reference to keep ctx alive until
> + * we're done.
> + */
> + aio_context_ref(ctx);
> +
> QSLIST_INSERT_HEAD_ATOMIC(&ctx->scheduled_coroutines,
> co, co_scheduled_next);
> qemu_bh_schedule(ctx->co_schedule_bh);
> +
> + aio_context_unref(ctx);
> }
>
> void aio_co_wake(struct Coroutine *co)
>
This must have been painful to debug.
Reviewed-by: Paolo Bonzini <pbonzini@redhat.com>
Paolo
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [Qemu-block] [PATCH] util/async: hold AioContext ref to prevent use-after-free
2019-07-23 19:09 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
@ 2019-07-24 10:54 ` Philippe Mathieu-Daudé
0 siblings, 0 replies; 5+ messages in thread
From: Philippe Mathieu-Daudé @ 2019-07-24 10:54 UTC (permalink / raw)
To: Stefan Hajnoczi, Stefan Hajnoczi
Cc: Fam Zheng, Paolo Bonzini, Alex Bennée, qemu-devel, qemu block
On 7/23/19 9:09 PM, Stefan Hajnoczi wrote:
> On Tue, Jul 23, 2019 at 8:06 PM Stefan Hajnoczi <stefanha@redhat.com> wrote:
>> So if co causes ctx to be freed then we're in trouble. Fix this problem
>> by holding a reference to ctx.
>
> For QEMU 4.2. I'm not aware of a way to trigger this bug in QEMU
> proper. This fix just makes tests/test-bdrv-drain more reliable.
This looks harmless for 4.1-rc3.
^ permalink raw reply [flat|nested] 5+ messages in thread
* Re: [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free
2019-07-23 19:06 [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free Stefan Hajnoczi
2019-07-23 19:09 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2019-07-24 8:47 ` [Qemu-devel] " Paolo Bonzini
@ 2019-07-24 11:42 ` Stefan Hajnoczi
2 siblings, 0 replies; 5+ messages in thread
From: Stefan Hajnoczi @ 2019-07-24 11:42 UTC (permalink / raw)
To: qemu-devel
Cc: Fam Zheng, Paolo Bonzini, Philippe Mathieu-Daudé, qemu-block
[-- Attachment #1: Type: text/plain, Size: 1275 bytes --]
On Tue, Jul 23, 2019 at 08:06:23PM +0100, Stefan Hajnoczi wrote:
> The tests/test-bdrv-drain /bdrv-drain/iothread/drain test case does the
> following:
>
> 1. The preadv coroutine calls aio_bh_schedule_oneshot() and then yields.
> 2. The one-shot BH executes in another AioContext. All it does is call
> aio_co_wakeup(preadv_co).
> 3. The preadv coroutine is re-entered and returns.
>
> There is a race condition in aio_co_wake() where the preadv coroutine
> returns and the test case destroys the preadv IOThread. aio_co_wake()
> can still be running in the other AioContext and it performs an access
> to the freed IOThread AioContext.
>
> Here is the race in aio_co_schedule():
>
> QSLIST_INSERT_HEAD_ATOMIC(&ctx->scheduled_coroutines,
> co, co_scheduled_next);
> <-- race: co may execute before we invoke qemu_bh_schedule()!
> qemu_bh_schedule(ctx->co_schedule_bh);
>
> So if co causes ctx to be freed then we're in trouble. Fix this problem
> by holding a reference to ctx.
>
> Signed-off-by: Stefan Hajnoczi <stefanha@redhat.com>
> ---
> util/async.c | 8 ++++++++
> 1 file changed, 8 insertions(+)
Thanks, applied to my block tree:
https://github.com/stefanha/qemu/commits/block
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
^ permalink raw reply [flat|nested] 5+ messages in thread
end of thread, other threads:[~2019-07-24 11:42 UTC | newest]
Thread overview: 5+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-07-23 19:06 [Qemu-devel] [PATCH] util/async: hold AioContext ref to prevent use-after-free Stefan Hajnoczi
2019-07-23 19:09 ` [Qemu-devel] [Qemu-block] " Stefan Hajnoczi
2019-07-24 10:54 ` Philippe Mathieu-Daudé
2019-07-24 8:47 ` [Qemu-devel] " Paolo Bonzini
2019-07-24 11:42 ` Stefan Hajnoczi
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).