On Tue, Feb 12, 2019 at 9:20 AM Frank Yang wrote: > > > On Tue, Feb 12, 2019 at 8:46 AM Dr. David Alan Gilbert < > dgilbert@redhat.com> wrote: > >> * Frank Yang (lfy@google.com) wrote: >> > Thanks Roman for the reply. Yes, we need sensors, sound, codecs, etc. as >> > well. >> > >> > For general string passing, yes, perhaps virtio-vsock can be used. >> However, >> > I have some concerns about virtio-serial and virtio-vsock (mentioned >> > elsewhere in the thread in rely to Stefan's similar comments) around >> socket >> > API specialization. >> > >> > Stepping back to standardization and portability concerns, it is also >> not >> > necessarily desirable to use general pipes to do what we want, because >> even >> > though that device exists and is part of the spec already, that results >> in >> > _de-facto_ non-portability. If we had some kind of spec to enumerate >> such >> > 'user-defined' devices, at least we can have _de-jure_ non-portability; >> an >> > enumerated device doesn't work as advertised. >> > >> > virtio-gpu: we have concerns around its specialization to virgl and >> > de-facto gallium-based protocol, while we tend to favor API forwarding >> due >> > to its debuggability and flexibility. We may use virtio-gpu in the >> future >> > if/when it provides that general "send api data" capability.] >> > >> > In any case, I now have a very rough version of the spec in mind >> (attached >> > as a patch and as a pdf). >> >> Some thoughts (and remember I'm fairly new to virtio): >> > > Thanks for the feedback! These are good points. > > >> >> a) Please don't call it virito-user - we have vhost-user as one of the >> implementations of virtio and that would just get confusing (especially >> when we have a vhost-user-user implementation) >> >> > Ok, the name isn't really important to me. virto-meta perhaps? > > >> b) Your ping and event queues confuse me - they seem to be >> reimplementing exactly what virtio-queues already are; aren't virtio >> queues already lumps of shared memory with a 'kick' mechanism to wake up >> the other end when something interesting happens? >> >> Right, it does expose that aspect of virtqueues, but there is a unique > part; it adds on a device-specific component > (the instance handle and metadata) to dispatch to and trigger the > device-specific callback when it comes on the host. > I wonder if there's a way to define just the instance > handle/metadata/callback parts, with other virtqueue stuff being standard. > > BTW, the other unique aspect is that the ping messages allow a _host_ pointer to serve as the lump of shared memory; then there is no need to track buffers in the guest kernel and the device implementation can perform specialize buffer space management. Because it is also host pointer shared memory, it is also physically contiguous and there is no scatterlist needed to process the traffic. > > >> c) I think you actually have two separate types of devices that >> should be treated differently; >> 1) your high bandwidth gpu/codec >> 2) Low bandwidth batteries/sensors >> > > Yeah, perhaps there's some way to have general virtio devices for > different bandwidth levels. > Note that latency can also be an important consideration; while sensors > don't send much data per second, > the time of their arrival can matter a lot. Sensors themselves can "ping" > at 10^2 Hz or even higher. > > I can imagine you having a total of two device definitions and drivers >> for (1) and (2). >> > (2) feels like it's pretty similar to a socket/pipe/serial - but it >> needs a way to enumerate the sensors you have, their ranges etc and a >> defined format for transmitting the data. I'm not sure if it's possible >> to take one of the existing socket/pipe/serial things and layer on top >> of it. (is there any HID like standard for sensors like that?) >> >> I'm not sure about there being an existing standard > (and it's possible to send different kinds of sensor / battery signals; > i.e., a fake source for testing purposes, raw sensor output from the > actual hardware, > and an integrated virtual sensor that passes through derived pose > information. > There's a similar breakdown when one is considering other devices such as > battery and camera.) > Though I have reservations about using actual sockets/pipes to communicate, > for compatibility and latency reasons. > > As another example, consider camera / video playback. I've heard that the > way to approach this > is to build v4l2 and build a virtio-v4l2 layer underneath. However, this > would mean either assuming v4l2 on host > which is not portable to non-Linux, or requiring additional translation > layers between v4l2 in the guest and > $library on the host. With the proposed spec, a more general 'codec' > device can be implemented, > tested on actual programs on multiple guests/hosts easily, and then, if it > makes sense, > "promoted" to a new "virtio-codec" device type. > > I think there's a central tension here that is process-related and is > similar to the reasons userspace drivers are a thing in Linux; > There's a set of existing guests/hosts that use virtio on some sort of > development schedule that > doesn't match the pace of development in kernels supporting virtio / the > virtio spec itself. > In Virtio, we have a centralized definition of drivers and devices that is > set in the middle layers > between chrome on top of VMMs and guest userspace: the guest kernel and > core VMM code. > When developing for programs such as the Android Emulator, > we aim most of our changes at the guest userspace and the chrome on top of > the VMM, > but it can often involve new forms of guest/host communication. > > It's possible that what's being proposed also serves as a kind of L2 cache > (in the form of such a virtio meta device) to deal with changes > in third-party programs like the Android Emulator with servicing the cache > miss being standardization out of the meta device and into an existing > concrete virtio device type. > Seen from this perspective, it can be beneficial process-wise as a way to > pull in more new devices / standardize at a faster rate, that would > otherwise not have been visible. > > >> Perhaps for (1) for your GPU stuff, maybe a single virtio device >> would work, with a small number of shared memory arenas but multiple >> virtio queues; each (set of) queues would represent a subdevice (say >> a bunch of queues for the GPU another bunch for the CODEC etc). > > > Well, if one device definition works well with both bandwidth regimes and > has better latency properties, > I like having the one definition better. > > Along these lines, I would also be open to not putting up a new standard, > and revising virtio-vsock to incorporate all the unique parts of the > features proposed: > dropping socket API requirement from guest/host, > device enumeration, operation via ping+shared memory, > and callback-based operation with device definitions in the shared > libraries. > > The way we're approaching GPU, with the workloads we tend to run > (just UI drawing, or in 3d apps, not too many draw calls per second / > limited by host GPU doing work), > still having one virtqueue seems fine as it's only used for kick > (API call parameters themselves would also live in shared memory), > though there are probably workloads that benefit from having one virtqueue > per "instance". > > >> > Dave >> >> > The part of the intro in there that is relevant to the current thread: >> > >> > """ >> > Note that virtio-serial/virtio-vsock is not considered because they do >> not >> > standardize the set of devices that operate on top of them, but in >> practice, >> > are often used for fully general devices. Spec-wise, this is not a >> great >> > situation because we would still have potentially non portable device >> > implementations where there is no standard mechanism to determine >> whether or >> > not things are portable. virtio-user provides a device enumeration >> > mechanism >> > to better control this. >> > >> > In addition, for performance considerations in applications such as >> graphics >> > and media, virtio-serial/virtio-vsock have the overhead of sending >> actual >> > traffic through the virtqueue, while an approach based on shared memory >> can >> > result in having fewer copies and virtqueue messages. virtio-serial is >> also >> > limited in being specialized for console forwarding and having a cap on >> the >> > number of clients. virtio-vsock is also not optimal in its choice of >> > sockets >> > API for transport; shared memory cannot be used, arbitrary strings can >> be >> > passed without an designation of the device/driver being run de-facto, >> and >> > the >> > guest must have additional machinery to handle socket APIs. In >> addition, on >> > the host, sockets are only dependable on Linux, with less predictable >> > behavior >> > from Windows/macOS regarding Unix sockets. Waiting for socket traffic >> on >> > the >> > host also requires a poll() loop, which is suboptimal for latency. With >> > virtio-user, only the bare set of standard driver calls >> > (open/close/ioctl/mmap/read) is needed, and RAM is a more universal >> > transport >> > abstraction. We also explicitly spec out callbacks on host that are >> > triggered >> > by virtqueue messages, which results in lower latency and makes it easy >> to >> > dispatch to a particular device implementation without polling. >> > >> > """ >> > >> > On Tue, Feb 12, 2019 at 6:03 AM Michael S. Tsirkin >> wrote: >> > >> > > On Tue, Feb 12, 2019 at 02:47:41PM +0100, Cornelia Huck wrote: >> > > > On Tue, 12 Feb 2019 11:25:47 +0000 >> > > > "Dr. David Alan Gilbert" wrote: >> > > > >> > > > > * Roman Kiryanov (rkir@google.com) wrote: >> > > > > > > > Our long term goal is to have as few kernel drivers as >> possible >> > > and to move >> > > > > > > > "drivers" into userspace. If we go with the virtqueues, is >> there >> > > > > > > > general a purpose >> > > > > > > > device/driver to talk between our host and guest to support >> > > custom hardware >> > > > > > > > (with own blobs)? >> > > > > > > >> > > > > > > The challenge is to answer the following question: >> > > > > > > how to do this without losing the benefits of standartization? >> > > > > > >> > > > > > We looked into UIO and it still requires some kernel driver to >> tell >> > > > > > where the device is, it also has limitations on sharing a device >> > > > > > between processes. The benefit of standardization could be in >> > > avoiding >> > > > > > everybody writing their own UIO drivers for virtual devices. >> > > > > > >> > > > > > Our emulator uses a battery, sound, accelerometer and more. We >> need >> > > to >> > > > > > support all of this. I looked into the spec, "5 Device types", >> and >> > > > > > seems "battery" is not there. We can invent our own drivers but >> we >> > > see >> > > > > > having one flexible driver is a better idea. >> > > > > >> > > > > Can you group these devices together at all in their requirements? >> > > > > For example, battery and accelerometers (to me) sound like >> > > low-bandwidth >> > > > > 'sensors' with a set of key,value pairs that update occasionally >> > > > > and a limited (no?) amount of control from the VM->host. >> > > > > A 'virtio-values' device that carried a string list of keys that >> it >> > > > > supported might make sense and be enough for at least two of your >> > > > > device types. >> > > > >> > > > Maybe not a 'virtio-values' device -- but a 'virtio-sensors' device >> > > > looks focused enough without being too inflexible. It can easily >> > > > advertise its type (battery, etc.) and therefore avoid the mismatch >> > > > problem that a too loosely defined device would be susceptible to. >> > > >> > > Isn't virtio-vsock/vhost-vsock a good fit for this kind of general >> > > string passing? People seem to use it exactly for this. >> > > >> > > > > > Yes, I realize that a guest could think it is using the same >> device >> > > as >> > > > > > the host advertised (because strings matched) while it is not. >> We >> > > > > > control both the host and the guest and we can live with this. >> > > > >> > > > The problem is that this is not true for the general case if you >> have a >> > > > standardized device type. It must be possible in theory to switch >> to an >> > > > alternative implementation of the device or the driver, as long as >> they >> > > > conform to the spec. I think a more concretely specified device type >> > > > (like the suggested virtio-values or virtio-sensors) is needed for >> that. >> > > > >> > > > >> --------------------------------------------------------------------- >> > > > To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org >> > > > For additional commands, e-mail: >> virtio-dev-help@lists.oasis-open.org >> > > >> >> >> >> -- >> Dr. David Alan Gilbert / dgilbert@redhat.com / Manchester, UK >> >