Re: [Qemu-devel] [Qemu-block] [RFC PATCH] coroutines: generate wrapper code

[Stefan Hajnoczi <stefanha@gmail.com>]

[-- Attachment #1: Type: text/plain, Size: 3210 bytes --]

On Mon, Feb 11, 2019 at 09:38:37AM +0000, Vladimir Sementsov-Ogievskiy wrote:
> 11.02.2019 6:42, Stefan Hajnoczi wrote:
> > On Fri, Feb 08, 2019 at 05:11:22PM +0300, Vladimir Sementsov-Ogievskiy wrote:
> >> Hi all!
> >>
> >> We have a very frequent pattern of wrapping a coroutine_fn function
> >> to be called from non-coroutine context:
> >>
> >>    - create structure to pack parameters
> >>    - create function to call original function taking parameters from
> >>      struct
> >>    - create wrapper, which in case of non-coroutine context will
> >>      create a coroutine, enter it and start poll-loop.
> >>
> >> Here is a draft of template code + example how it can be used to drop a
> >> lot of similar code.
> >>
> >> Hope someone like it except me)
> > 
> > My 2 cents.  Cons:
> > 
> >   * Synchronous poll loops are an anti-pattern.  They block all of QEMU
> >     with the big mutex held.  Making them easier to write is
> >     questionable because we should aim to have as few of these as
> >     possible.
> 
> Understand. Do we have a concept or a kind of target for a future to get rid of
> these a lot of poll-loops? What is the right way? At least for block-layer?

It's non-trivial.  The nested event loop could be flattened if there was
a mechanism to stop further activity on a specific object only (e.g.
BlockDriverState).  That way the event loop can continue processing
events for other objects and device emulation could continue for other
objects.

Unfortunately there are interactions between objects like in block jobs
that act on multiple BDSes, so it becomes even tricky.

A simple way of imagining this is to make each object an "actor"
coroutine.  The coroutine processes a single message (request) at a time
and yields when it needs to wait.  Callers send messages and expect
asynchronous responses.  This model is bad for efficiency (parallelism
is necessary) but at least it offers a sane way of thinking about
multiple asynchronous components coordinating together.  (It's another
way of saying, let's put everything into coroutines.)

The advantage of a flat event loop is that a hang in one object (e.g.
I/O getting stuck in one file) doesn't freeze the entire event loop.

> > 
> >   * Code generation makes the code easier to write but harder to read.
> >     Code is read more than written.  In this case I think open coding
> >     isn't too bad and I prefer it to reading a code generation script to
> >     understand how it works.
> 
> But you can read generated code in same way. You only need to read generator
> script if it generates something wrong, but should be rare.

Generated code isn't visible unless the code has been built and indexed
(if you're using ctags).  This makes it harder for people to navigate
the code.

> > 
> > If we were planning to add lots more of these then I agree code
> > generation would help.  But in this case I'd rather not.
> > 
> 
> What do you think at least of generating code to create a coroutine from a function
> with multiple arguments?

If it's easy to read without requiring one to figure out how the magic
works, then I like it.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]