Re: [Qemu-devel] [RFC] docs: vhost-user: add in-band kick/call messages

[Johannes Berg <johannes@sipsolutions.net>]

On Tue, 2019-09-10 at 17:03 +0200, Stefan Hajnoczi wrote:
> 
> > Now, this means that the CPU (that's part of the simulation) has to
> > *wait* for the device to add an entry to the simulation calendar in
> > response to the kick... That means that it really has to look like
> > 
> > CPU               device                   calendar
> >      ---[kick]-->
> >                          ---[add entry]-->
> >                          <---[return]-----
> 
> What are the semantics of returning from the calendar?  Does it mean
> "it's now your turn to run?", "your entry has been added and you'll be
> notified later when it's time to run?", or something else?

The latter - the entry was added, and you'll be notified when it's time
to run; but we need to have that state on the calendar so the CPU won't
converse with the calendar before that state is committed.

> >      <-[return]--
> 
> What are the semantics of returning to the CPU?  Does it mean the
> submitted I/O request is now complete?

No, it just means that the interrupt was triggered; it will be handled
once the calendar decide that it's the device's turn to run its
interrupt processing event.

The I/O request completion will be done later in a very similar fashion
(just inverting the participants, and replacing "kick" by "call").

> Sorry for these questions.  If this becomes tedious for you I will look
> into the paper you linked.

No no, don't worry, it's fine. Also, the paper doesn't really concern
itself with issues such as this, they just assume everything is
synchronous and make their code that way - they have a simulated device
in qemu that they wrote themselves, and it just happens to be
synchronous by way of their implementation...

What I'm trying to do here now is not really replicate that paper or
anything, but - because I'm working on similar things and simulation -
get some pieces into the upstreams (qemu for vhost-user, Linux for user-
mode-Linux running in a simulation) so (a) I don't have to maintain it
out-of-tree, and (b) it's available for others for use without a monster
patch to an ancient version of the software stack :-)

Happy to answer any questions.

Aside from the interrupt processing semantics in UML which I'll work on
next I do have the whole simulation calendar working etc., i.e. I have
full UML "VM" controlled entirely by the simulation calendar, also over
virtio.



Btw, that reminds me ... there's yet another process to add a virtio
device ID. I'm currently defining a virtio simulation calendar protocol
like this:

+/*
+ * A message passed on either of the VQs (input and output).
+ */
+struct virtio_simtime_msg {
+       __u64 op;               /* see below */
+       __u64 time;             /* time in nanoseconds */
+};
+
+/**
+ * enum virtio_simtime_ops - Operation codes
+ */
+enum virtio_simtime_ops {
+       /**
+        * @VIRTIO_SIMTIME_REQUEST: request from device to run at the given time
+        */
+       VIRTIO_SIMTIME_REQUEST          = 1,
+
+       /**
+        * @VIRTIO_SIMTIME_WAIT: wait for the requested runtime, new requests
+        *      may be made while waiting (due to interrupts); the time field
+        *      is ignored
+        */
+       VIRTIO_SIMTIME_WAIT             = 2,
+
+       /**
+        * @VIRTIO_SIMTIME_GET: ask device to fill the buffer with the current
+        *      simulation time
+        */
+       VIRTIO_SIMTIME_GET              = 3,
+
+       /**
+        * @VIRTIO_SIMTIME_UPDATE: time update to/from the control device
+        */
+       VIRTIO_SIMTIME_UPDATE           = 4,
+
+       /**
+        * @VIRTIO_SIMTIME_RUN: run time request granted, current time is in
+        *      the time field
+        */
+       VIRTIO_SIMTIME_RUN              = 5,
+
+       /**
+        * @VIRTIO_SIMTIME_FREE_UNTIL: enable free-running until the given time,
+        *      this is a messag from the device telling the user that it can do
+        *      its own scheduling for anything before the given time
+        */
+       VIRTIO_SIMTIME_FREE_UNTIL       = 6,
+};


Above, I've basically only described _REQUEST, _WAIT and _RUN, I'm
pretty sure I need _GET for interrupt handling (or maybe more
efficiently a _REQUEST_NOW, rather than using _GET && _REQUEST); I think
I need _UPDATE before a "kick" to the device if I have _FREE_UNTIL which
is an optimization to not have to talk to the calendar all the time.


Is any of you familiar with the process of getting a virtio device ID
assigned, and if so, do you think it'd be feasible? Without that, it'd
probably be difficult to upstream the patch to support this protocol to
user-mode Linux.

Thanks,
johannes