Re: [virtio-dev] Memory sharing device

[Frank Yang <lfy@google.com>]

[-- Attachment #1: Type: text/plain, Size: 4726 bytes --]

On Tue, Feb 19, 2019 at 10:25 PM Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
>
> > > That might simlify pass-through of v4l2 host devices, but isn't
> > > necessarily the best choice long-term.
>
> > Right, but if we're not emulating at the v4l2 api level, then it starts
> > looking a lot
> > like the proposed virtio-hostmem; there's a common pattern of
> > direct access to host memory
>
> Typically camera hardware can DMA the image data to any place in memory,
> including guest ram.  There is no need to expose host memory to the
> guest for that.  The normal virtio workflow with guest-allocated buffers
> will work just fine.
>
>
True, thanks for pointing it out.
Although, I'd still think it is still a bit nicer to do it via a host
memory buffer,
because it'd be outside of guest ram;
we wouldn't have to interface as tightly with the kernel in terms of not
interfering with
other guest RAM usages,
such as equivalents of dma_alloc_coherent with the limitation on CMA and
all.

> In addition, we'd like to do things in a way
> > that allows virtual drivers/devices to be defined in a manner that
> doesn't
> > require the guest kernel to be updated.
>
> Hmm.  I'm wondering whenever you just need a different virtio transport.
> Very simliar to virtio-pci, but instead of using all guest ram (modulo
> iommu quirks) as address space use a pci memory bar as address space.
> virtio rings would live there, all buffers would live there, addresses
> passed around would be offsets into that pci bar.  Then your userspace
> driver can just mmap() that pci bar and handle (almost?) everything on
> its own.  Maybe a little stub driver in the kernel is needed for ring
> notifications.
>
>
Yeah, that's (almost?) exactly what we want to do / we're doing already
pretty much
with the memory sharing scheme we're using currently.

Defining it as a transport would also require definition of another virtio
device type
that the userspace drivers talk to, right?
As a transport/device pair, would it break down as, the transport ends up
restricting what kinds of addresses can be referred to in the ring messages
(in the pci memory bar),
then the device provides the userspace drivers with
implementation of mmap() + memory allocation/sharing ops and notification?


> > For us, maintaining and upgrading guest kernels in response to tweaks to
> > virtual devices
> > is much more work than modifying a userspace shared library driver that
> > communicates to some virtio driver.
>
> Well, one of the design goals of virtio is that such an upgrade is not
> required.  Capabilities of device and driver are negotiated using
> feature flags, which removes the need for lockstep updates of guest and
> host.
>
>
We also have a similar feature flag system,
but yeah it would be great to use the capabilities/flags
to their fullest extent here.


> > Thus far, it's suggested that socket or
> > network devices
> > be used for this, because they are general guest/host communication,
> > but they don't have the portability or performance characteristics we
> want.
>
> For high-bandwidth fully agree.
>
> For low-bandwidth (like the sensors discussed elsewhere in the thread) I
> think running some protocol (mqtt, something json-based, ...) over
> virtio-serial or virtio-vsock isn't too bad.
>
> But creating virtio-sensors makes sense too.
>
>
Yeah for performance considerations, sockets seem to be fine.
It's just that I've also got concerns on how well sockets work on non-Linux.

> > Alternatively emulate something existing, USB Video Class device for
> > > example.
> >
> > Using usb or faking some other transport generally also involves being
> > coupled to that set of kernel code in the guest, which introduces
> > complexity and overhead.  Exposing RAM can be a more flexible
> > abstraction.
>
> Well, you get guest drivers for free and all existing software will run
> just fine out-of-the-box.
>
> Which is the reason why there is no virtio-usb for example.  xhci has a
> hardware design which can be emulated without much overhead, so the
> performance benefit of virtio-usb over xhci would be pretty close to
> zero.
>
>
That's pretty interesting; from a brief glance at xhci, it looks like what
you're proposing
is that since there usb passthrough already with xhci,
and xhci has something that looks like a virtqueue,
perhaps we can create our own USB driver on the host,
then pass it through.

It does make things more inconvenient for users though since they would
have to install
usb drivers out of nowhere. Wonder if a usb forwarder virtual device exists
that can talk to some shared library on the host for the xhci queue, though
that might be considered an unsafe plugin.


> cheers,
>   Gerd
>
>

[-- Attachment #2: Type: text/html, Size: 6495 bytes --]