[Qemu-devel] AioContext of block jobs

[Qemu-devel] AioContext of block jobs

[Fam Zheng]

I was looking at block jobs' AioContext and realized that the block job
coroutines are actually started in main loop.

I'm confused because 5a7e7a0bad17c96e03f55ed7019e2d7545e21a96 and friends in
the series [1] seem to move the coroutines to the BDS's iothreads, but it
didn't do that.

(Although after the first block_job_yield or sleep, the coroutines ARE resumed
in the right AioContext.)

Why is it safe to start the jobs from the main thread where QMP command is
handled? I see no guarantee that the jobs won't access BDS before first yield
but after releasing the AioContext.

Is this a bug?

[1]: http://lists.gnu.org/archive/html/qemu-devel/2014-10/msg02191.html

Thanks,
Fam

Re: [Qemu-devel] AioContext of block jobs

[Paolo Bonzini]

On 25/03/2015 09:31, Fam Zheng wrote:
> I was looking at block jobs' AioContext and realized that the block job
> coroutines are actually started in main loop.
> 
> I'm confused because 5a7e7a0bad17c96e03f55ed7019e2d7545e21a96 and friends in
> the series [1] seem to move the coroutines to the BDS's iothreads, but it
> didn't do that.
> 
> (Although after the first block_job_yield or sleep, the coroutines ARE resumed
> in the right AioContext.)
> 
> Why is it safe to start the jobs from the main thread where QMP command is
> handled? I see no guarantee that the jobs won't access BDS before first yield
> but after releasing the AioContext.
> 
> Is this a bug?

It's okay because the coroutine is started while the main thread is
holding the AioContext.  So the first "stint" of the coroutine, until
the first yield, is done in the main thread but still with the
AioContext held.

After the first yield, the main thread releases the AioContext, the
dataplane thread gets it back and the coroutine is moved to the
dataplane thread.

Paolo

Re: [Qemu-devel] AioContext of block jobs

[Fam Zheng]

On Wed, 03/25 12:25, Paolo Bonzini wrote:
> 
> 
> On 25/03/2015 09:31, Fam Zheng wrote:
> > I was looking at block jobs' AioContext and realized that the block job
> > coroutines are actually started in main loop.
> > 
> > I'm confused because 5a7e7a0bad17c96e03f55ed7019e2d7545e21a96 and friends in
> > the series [1] seem to move the coroutines to the BDS's iothreads, but it
> > didn't do that.
> > 
> > (Although after the first block_job_yield or sleep, the coroutines ARE resumed
> > in the right AioContext.)
> > 
> > Why is it safe to start the jobs from the main thread where QMP command is
> > handled? I see no guarantee that the jobs won't access BDS before first yield
> > but after releasing the AioContext.
> > 
> > Is this a bug?
> 
> It's okay because the coroutine is started while the main thread is
> holding the AioContext.  So the first "stint" of the coroutine, until
> the first yield, is done in the main thread but still with the
> AioContext held.

I see! That's what I missed. Thanks!

Fam

> 
> After the first yield, the main thread releases the AioContext, the
> dataplane thread gets it back and the coroutine is moved to the
> dataplane thread.
> 

Re: [Qemu-devel] AioContext of block jobs

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1418 bytes --]

On Wed, Mar 25, 2015 at 04:31:39PM +0800, Fam Zheng wrote:
> I was looking at block jobs' AioContext and realized that the block job
> coroutines are actually started in main loop.
> 
> I'm confused because 5a7e7a0bad17c96e03f55ed7019e2d7545e21a96 and friends in
> the series [1] seem to move the coroutines to the BDS's iothreads, but it
> didn't do that.
> 
> (Although after the first block_job_yield or sleep, the coroutines ARE resumed
> in the right AioContext.)
> 
> Why is it safe to start the jobs from the main thread where QMP command is
> handled? I see no guarantee that the jobs won't access BDS before first yield
> but after releasing the AioContext.

Is there a concrete case you are worried about?

For example:
void qmp_block_stream(const char *device,
...
    stream_start(bs, base_bs, base_name, has_speed ? speed : 0,
                 on_error, block_job_cb, bs, &local_err);
    if (local_err) {
        error_propagate(errp, local_err);
        goto out;
    }

    trace_qmp_block_stream(bs, bs->job);

out:
    aio_context_release(aio_context);   <----
}

Since the BDS AioContext is held during stream_start, there is no race
condition.

The resources used by the coroutine (i.e. timers or the BDS) bind their event
handler functions to the AioContext.  This means the coroutine will only be
entered again under the AioContext in the future.

Stefan

[-- Attachment #2: Type: application/pgp-signature, Size: 473 bytes --]

Re: [Qemu-devel] AioContext of block jobs

[Fam Zheng]

On Wed, 03/25 13:31, Stefan Hajnoczi wrote:
> On Wed, Mar 25, 2015 at 04:31:39PM +0800, Fam Zheng wrote:
> > I was looking at block jobs' AioContext and realized that the block job
> > coroutines are actually started in main loop.
> > 
> > I'm confused because 5a7e7a0bad17c96e03f55ed7019e2d7545e21a96 and friends in
> > the series [1] seem to move the coroutines to the BDS's iothreads, but it
> > didn't do that.
> > 
> > (Although after the first block_job_yield or sleep, the coroutines ARE resumed
> > in the right AioContext.)
> > 
> > Why is it safe to start the jobs from the main thread where QMP command is
> > handled? I see no guarantee that the jobs won't access BDS before first yield
> > but after releasing the AioContext.
> 
> Is there a concrete case you are worried about?

No, I just missed that aio_context_release is only called after the first block
job coroutine yield.

Thanks for explaining.

Fam

> 
> For example:
> void qmp_block_stream(const char *device,
> ...
>     stream_start(bs, base_bs, base_name, has_speed ? speed : 0,
>                  on_error, block_job_cb, bs, &local_err);
>     if (local_err) {
>         error_propagate(errp, local_err);
>         goto out;
>     }
> 
>     trace_qmp_block_stream(bs, bs->job);
> 
> out:
>     aio_context_release(aio_context);   <----
> }
> 
> Since the BDS AioContext is held during stream_start, there is no race
> condition.
> 
> The resources used by the coroutine (i.e. timers or the BDS) bind their event
> handler functions to the AioContext.  This means the coroutine will only be
> entered again under the AioContext in the future.
> 
> Stefan