Re: [virtio-dev] Memory sharing device

From: Frank Yang <lfy@google.com>
To: "Michael S. Tsirkin" <mst@redhat.com>
Cc: "Dr. David Alan Gilbert" <dgilbert@redhat.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Roman Kiryanov <rkir@google.com>,
	Gerd Hoffmann <kraxel@redhat.com>,
	Stefan Hajnoczi <stefanha@redhat.com>,
	virtio-dev@lists.oasis-open.org,
	Greg Hartman <ghartman@google.com>
Subject: Re: [virtio-dev] Memory sharing device
Date: Tue, 12 Feb 2019 18:50:29 -0800	[thread overview]
Message-ID: <CAEkmjvVzwfNW4QL8E52=AEXvN_RoeEoOHCODaefsa4wSLOMmZg@mail.gmail.com> (raw)
In-Reply-To: <20190212135924-mutt-send-email-mst@kernel.org>

[-- Attachment #1: Type: text/plain, Size: 2798 bytes --]

On Tue, Feb 12, 2019 at 11:06 AM Michael S. Tsirkin <mst@redhat.com> wrote:

> On Tue, Feb 12, 2019 at 09:26:10AM -0800, Frank Yang wrote:
> > BTW, the other unique aspect is that the ping messages allow a _host_
> pointer
> > to serve as the lump of shared memory;
> > then there is no need to track buffers in the guest kernel and the device
> > implementation can perform specialize buffer space management.
> > Because it is also host pointer shared memory, it is also physically
> contiguous
> > and there is no scatterlist needed to process the traffic.
>
> Yes at the moment virtio descriptors all pass addresses guest to host.
>
> Ability to reverse that was part of the vhost-pci proposal a while ago.
> BTW that also at least originally had ability to tunnel
> multiple devices over a single connection.
>
>
Can there be a similar proposal for virtio-pci without vhsot?

> There was nothing wrong with the proposals I think, they
> just had to be polished a bit before making it into the spec.
> And that runneling was dropped but I think it can be brought back
> if desired, we just didn't see a use for it.
>
> Thinking about it more, I think vhost-pci might be too much for us due to
the vhost requirement (sockets and IPC while we desire a highly process
local solution)

But there's nothing preventing us from having the same reversals for
virtio-pci devices without vhost, right?
That's kind of what's being proposed with the shared memory stuff at the
moment, though it is not a device type by itself yet (Arguably, it should
be).

How about that? That sounds close to what you were looking for,
> does it not? That would be something to look into -
> if your ideas can be used to implement a virtio device
> backend by code running within a VM, that would be very interesting.
>

What about a device type, say, virtio-metapci, that relies on virtio-pci
for device enumeration and shared memory handling
(assuming it's going to be compatible with the host pointer shared memory
implementation),
so there's no duplication of the concept of device enumeration nor shared
memory operations.
But, it works in terms of the ping / event virtqueues, and relies on the
host hypervisor to dispatch to device implementation callbacks.
A potential issue is that such metapci device share the same device id
namespace as other virtio-pci devices...but maybe that's OK?
If this can build on virtio-pci, I might be able to come up with a spec
that assumes virtio-pci as the transport,
and assumes (via the WIP host memory sharing work) that host memory can be
used as buffer storage.
The difference is that it will not contain most of the config virtqueue
stuff (except maybe for create/destroy instance),
and it should also work with the existing ecosystem around virtio-pci.

>
> --
> MST
>

[-- Attachment #2: Type: text/html, Size: 3834 bytes --]