On Tue, Feb 12, 2019 at 11:15 AM Michael S. Tsirkin wrote: > On Tue, Feb 12, 2019 at 11:01:21AM -0800, Frank Yang wrote: > > > > > > > > On Tue, Feb 12, 2019 at 10:22 AM Michael S. Tsirkin > wrote: > > > > On Tue, Feb 12, 2019 at 07:56:58AM -0800, Frank Yang wrote: > > > Stepping back to standardization and portability concerns, it is > also not > > > necessarily desirable to use general pipes to do what we want, > because > > even > > > though that device exists and is part of the spec already, that > results > > in > > > _de-facto_ non-portability. > > > > That's not different from e.g. TCP. > > > > > If we had some kind of spec to enumerate such > > > 'user-defined' devices, at least we can have _de-jure_ > non-portability; > > an > > > enumerated device doesn't work as advertised. > > > > I am not sure distinguishing between different types of non > portability > > will be in scope for virtio. Actually having devices that are > portable > > would be. > > > > > > The device itself is portable; the user-defined drivers that run on them > will > > work or not depending on > > negotiating device IDs. > > > > ... > > > > > Note that virtio-serial/virtio-vsock is not considered because > they do > > not > > > standardize the set of devices that operate on top of them, but in > > practice, > > > are often used for fully general devices. Spec-wise, this is not > a great > > > situation because we would still have potentially non portable > device > > > implementations where there is no standard mechanism to determine > whether > > or > > > not things are portable. > > > > Well it's easy to add an enumeration on top of sockets, and several > well > > known solutions exist. There's an advantage to just reusing these. > > > > > > Sure, but there are many unique features/desirable properties of having > the > > virtio meta device > > because (as explained in the spec) there are limitations to > network/socket > > based communication. > > > > > > > virtio-user provides a device enumeration mechanism > > > to better control this. > > > > We'll have to see what it all looks like. For virtio pci transport > it's > > important that you can reason about the device at a basic level > based on > > it's PCI ID, and that is quite fundamental. > > > > > > The spec contains more details; basically the device itself is always > portable, > > and there is a configuration protocol > > to negotiate whether a particular use of the device is available. This is > > similar to PCI, > > but with more defined ways to operate the device in terms of callbacks in > > shared libraries on the host. > > > > > > Maybe what you are looking for is a new virtio transport then? > > > > > > > > Perhaps, something like virtio host memory transport? But > > at the same time, it needs to interact with shared memory which is best > set as > > a PCI device. > > Can we mix transport types? In any case, the analog of "PCI ID"'s here > (the > > vendor/device/version numbers) > > are meaningful, with the contract being that the user of the device > needs to > > match on vendor/device id and > > negotiate on version number. > > Virtio is fundamentally using feature bits not versions. > It's been pretty successful in maintaining compatiblity > across a wide range of hypervisor/guest revisions. > The proposed device should be compatible with all hypervisor/guest revisions as is, without feature bits. > > Wha are the advantages of defining a new virtio transport type? > > it would be something that has the IDs, and be able to handle resolving > offsets > > to > > physical addresses to host memory addresses, > > in addition to dispatching to callbacks on the host. > > But it would be effectively equivalent to having a new virtio device > type with > > device ID enumeration, right? > > Under virtio PCI Device IDs are all defined in virtio spec. > If you want your own ID scheme you want an alternative transport. > But now what you describe looks kind of like vhost pci to me. > > Yet, vhost is still not a good fit, due to reliance on sockets/network functionality. It looks like we want to build something that has aspects in common with other virtio devices, but there is no combination of existing virtio devices that works. - the locality of virtio-fs, but not specialized to fuse - the device id enumeration from virtio pci, but not in the pci id space - the general communication of virtio-vsock, but not specialized to sockets and allowing host memory-based transport So, I still think it should be a new device (or transport if that works better). > > > > > > > In addition, for performance considerations in applications such as > > graphics > > > and media, virtio-serial/virtio-vsock have the overhead of sending > actual > > > traffic through the virtqueue, while an approach based on shared > memory > > can > > > result in having fewer copies and virtqueue messages. > virtio-serial is > > also > > > limited in being specialized for console forwarding and having a > cap on > > the > > > number of clients. virtio-vsock is also not optimal in its choice > of > > sockets > > > API for transport; shared memory cannot be used, arbitrary strings > can be > > > passed without an designation of the device/driver being run > de-facto, > > and the > > > guest must have additional machinery to handle socket APIs. In > addition, > > on > > > the host, sockets are only dependable on Linux, with less > predictable > > behavior > > > from Windows/macOS regarding Unix sockets. Waiting for socket > traffic on > > the > > > host also requires a poll() loop, which is suboptimal for > latency. With > > > virtio-user, only the bare set of standard driver calls > > > (open/close/ioctl/mmap/read) is needed, and RAM is a more universal > > transport > > > abstraction. We also explicitly spec out callbacks on host that > are > > triggered > > > by virtqueue messages, which results in lower latency and makes it > easy > > to > > > dispatch to a particular device implementation without polling. > > > > open/close/mmap/read seem to make sense. ioctl gives one pause. > > > > > > ioctl would be to send ping messages, but I'm not fixated on that > choice. write > > () is also a possibility to send ping messages; I preferred ioctl() > because it > > should be clear that it's a control message not a data message. > > Yes if ioctls supported are white-listed and not blindly passed through > (e.g. send a ping message), then it does not matter. > > There would be one whitelisted ioctl: IOCTL_PING, with struct close to what is specified in the proposed spec, with the instance handle field populated by the kernel. > > > > > Given open/close this begins to look a bit like virtio-fs. > > Have you looked at that? > > > > > > > > That's an interesting possibility since virtio-fs maps host pointers as > well, > > which fits our use cases. > > Another alternative is to add the features unique about virtio-user to > > virtio-fs: > > device enumeration, memory sharing operations, operation in terms of > callbacks > > on the host. > > However, it doesn't seem like a good fit due to being specialized to > filesystem > > operations. > > Well everything is a file :) > > Not according to virtio-fs; it specializes in fuse support in the guest kernel. However, it would work for me if the unique features about virtio-user were added to it, and dropping the requirement to use fuse. > > > > > > -- > > MST > > >