From: Stefan Hajnoczi <stefanha@redhat.com>
To: Johannes Berg <johannes@sipsolutions.net>
Cc: "Marc-André Lureau" <marcandre.lureau@redhat.com>,
qemu-devel@nongnu.org, mst@redhat.com
Subject: Re: [Qemu-devel] [RFC] docs: vhost-user: add in-band kick/call messages
Date: Wed, 11 Sep 2019 09:35:03 +0200 [thread overview]
Message-ID: <20190911073503.GB4181@stefanha-x1.localdomain> (raw)
In-Reply-To: <e114b68dffecd9b4c4666327b15a28098c83f7ec.camel@sipsolutions.net>
[-- Attachment #1: Type: text/plain, Size: 3070 bytes --]
On Tue, Sep 10, 2019 at 05:14:36PM +0200, Johannes Berg wrote:
> On Tue, 2019-09-10 at 17:03 +0200, Stefan Hajnoczi wrote:
> >
> > > Now, this means that the CPU (that's part of the simulation) has to
> > > *wait* for the device to add an entry to the simulation calendar in
> > > response to the kick... That means that it really has to look like
> > >
> > > CPU device calendar
> > > ---[kick]-->
> > > ---[add entry]-->
> > > <---[return]-----
> >
> > What are the semantics of returning from the calendar? Does it mean
> > "it's now your turn to run?", "your entry has been added and you'll be
> > notified later when it's time to run?", or something else?
>
> The latter - the entry was added, and you'll be notified when it's time
> to run; but we need to have that state on the calendar so the CPU won't
> converse with the calendar before that state is committed.
Is the device only adding a calendar entry and not doing any actual
device emulation at this stage?
If yes, then this suggests the system could be structured more cleanly.
The vhost-user device process should focus on device emulation. It
should not be aware of the calendar. The vhost-user protocol also
shouldn't require modifications.
Instead, Linux arch/um code would add the entry to the calendar when the
CPU wants to kick a vhost-user device. I assume the CPU is suspended
until arch/um code completes adding the entry to the calendar.
When the calendar decides to run the device entry it signals the
vhost-user kick eventfd. The vhost-user device processes the virtqueue
as if it had been directly signalled by the CPU, totally unaware that
it's running within a simulation system.
The irq path is similar: the device signals the callfd and the calendar
adds an entry to notify UML that the request has completed.
Some pseudo-code:
arch/um/drivers/.../vhost-user.c:
void um_vu_kick(struct um_vu_vq *vq)
{
if (simulation_mode) {
calendar_add_entry({
.type = CAL_ENTRY_TYPE_VHOST_USER_KICK,
.device = vq->dev,
.vq_idx = vq->idx,
});
return;
}
/* The normal code path: signal the kickfd */
uint64_t val = 1;
write(vq->kickfd, &val, sizeof(val));
}
I'm suggesting this because it seems like a cleaner approach than
exposing the calendar concept to the vhost-user devices. I'm not 100%
sure it offers the semantics you need to make everything deterministic
though.
A different topic: vhost-user does not have a 1:1 vq buffer:kick
relationship. It's possible to add multiple buffers and kick only once.
It is also possible for the device to complete multiple buffers and only
call once.
This could pose a problem for simulation because it allows a degree of
non-determinism. But as long as the both the CPU and the I/O completion
of the device happen on a strict schedule this isn't a problem.
Stefan
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]
next prev parent reply other threads:[~2019-09-11 7:36 UTC|newest]
Thread overview: 38+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-09-02 12:12 [Qemu-devel] [RFC] vhost-user simulation extension Johannes Berg
2019-09-02 12:12 ` [Qemu-devel] [RFC] docs: vhost-user: add in-band kick/call messages Johannes Berg
2019-09-05 20:28 ` Johannes Berg
2019-09-09 16:00 ` Stefan Hajnoczi
2019-09-09 17:34 ` Johannes Berg
2019-09-10 15:03 ` Stefan Hajnoczi
2019-09-10 15:14 ` Johannes Berg
2019-09-10 15:33 ` Michael S. Tsirkin
2019-09-10 15:34 ` Johannes Berg
2019-09-11 6:56 ` Stefan Hajnoczi
2019-09-11 7:35 ` Stefan Hajnoczi [this message]
2019-09-11 8:26 ` Johannes Berg
2019-09-11 15:17 ` Stefan Hajnoczi
2019-09-11 15:31 ` Stefan Hajnoczi
2019-09-11 15:36 ` Johannes Berg
2019-09-11 15:38 ` Johannes Berg
2019-09-12 12:22 ` Stefan Hajnoczi
2019-09-12 20:37 ` Johannes Berg
2019-09-06 12:13 ` [Qemu-devel] [RFC] libvhost-user: implement VHOST_USER_PROTOCOL_F_KICK_CALL_MSGS Johannes Berg
2019-09-06 14:22 ` Michael S. Tsirkin
2019-09-06 14:48 ` Johannes Berg
2019-09-06 15:12 ` Michael S. Tsirkin
2019-09-06 15:32 ` Johannes Berg
2019-09-08 13:13 ` Michael S. Tsirkin
2019-09-09 11:35 ` Johannes Berg
2019-09-09 12:41 ` Michael S. Tsirkin
2019-09-09 13:05 ` Johannes Berg
2019-09-09 13:48 ` Michael S. Tsirkin
2019-09-09 13:50 ` Johannes Berg
2019-09-09 14:59 ` Michael S. Tsirkin
2019-09-09 15:26 ` Johannes Berg
2019-09-09 15:34 ` Johannes Berg
2019-09-09 15:45 ` Michael S. Tsirkin
2019-09-09 15:47 ` Johannes Berg
2019-09-10 15:52 ` Johannes Berg
2019-09-11 9:16 ` Michael S. Tsirkin
2019-09-11 9:20 ` Johannes Berg
2019-09-11 9:54 ` Michael S. Tsirkin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190911073503.GB4181@stefanha-x1.localdomain \
--to=stefanha@redhat.com \
--cc=johannes@sipsolutions.net \
--cc=marcandre.lureau@redhat.com \
--cc=mst@redhat.com \
--cc=qemu-devel@nongnu.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).