[virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

Hello everyone,

I recently took over Tomeu's task of upstreaming virtio-wayland. After
spending quite a bit of time collecting information from his different
attempts [1][2] I wanted to sync with all the people that were involved
in the previous discussions (if I missed some of them, feel free to add
them back).

The goal here is to get a rough idea of the general direction this
should take so I can start implementing a PoC and see if it fits
everyone's needs.

virtio-wayland [3] started as a solution to pass wayland messages
between host and guests so the guest can execute wayland apps whose
surface buffers are passed to the wayland compositor running on the
host. While this was its primary use case, I've heard it's been used to
transport other protocols. And that's not surprising, when looking at
the code I noticed it was providing a protocol-agnostic message passing
interface between host and guests, similar to what VSOCK provides but
with FD passing as an extra feature.

Based on all previous discussions, I could identify 3 different
approaches:

    1/ Use VSOCK and extend it to support passing (some) FDs
    2/ Use a user space VSOCK-based proxy that's in charge of
       a/ passing regular messages
       b/ passing specific handles to describe objects shared
          between host and guest (most focus has been on dmabufs as
          this is what we really care about for the gfx use case,
          but other kind of FDs can be emulated through a
          VSOCK <-> UNIX_SOCK bridging)
    3/ Have a dedicated kernel space solution that provides features
       exposed by #1 but through a virtio device interface (basically
       what virtio-wayland does today)

Each of them has its pros and cons, which I'll try to sum-up (please
correct me if I'm wrong, and add new things if you think they are
missing).

#1 might require extra care if we want to make it safe, as pointed
out by Stefan here [4] (but I wonder if the problem is not the same
for a virtio-wayland based solution). Of course you also need a bit of
infrastructure to register FD <-> VFD mappings (VFD being a virtual
file descriptor that's only used as unique IDs identifying the resource
backed by the local FD). FD <-> VFD mappings would have to be created
by the subsystem in charge of the object backing the FD (virtio-gpu for
exported GEM buffers, virtio-vdec for video buffers, vsock for unix
sockets if we decide to bridge unix and vsock sockets to make it
transparent, ...). The FD <-> VFD mapping would also have to be created
on the host side, probably by the virtio device implementation
(virglrenderer for GEM bufs for instance), which means host and guest
need a way to inform the other end that a new FD <-> VFD mapping has
been created so the other end can create a similar mapping (I guess this
requires extra device-specific commands to work). Note that this
solution doesn't look so different from the virtio-dmabuf [5] approach
proposed by Gerd a few months back, it's just extended to be a global
VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One
great thing about this approach is that we can re-use it for any kind
of FD sharing, not just dmabufs.

#2 is a bit challenging, since it requires the proxy to know about all
possible kind of FDs and do a FD <-> unique handle conversion with some
help from the subsystem backing the FD. For dmabufs, that means we
need to know who created the dmabuf, or assume that only one device is
used for all allocations (virtio-gpu?). AFAIU, there's also a security
issue as one could pass random (but potentially valid) handles to the
host proxy (pointed out by Tomasz [6]).

#3 is pretty similar to #1 in its design except that, instead of using
the VSOCK infrastructure it's using a new type of virtio device. I
guess it has the same pros and cons #1 has, and the name should probably
be changed to reflect the fact that it can transmit any kind of data not
just wayland.

This is just a high level view of the problem and the solutions proposed
by various people over the years. I'm sure I'm missing tons of details
and don't realize yet all the complexity behind solution #1, but looking
at this summary, I wonder if I should investigate this solution in
priority. An alternative could be to rebrand virtio-wayland, but as I
said, it's close enough to VSOCK to try to merge the missing features
in VSOCK instead. This being said, I'm not yet set on any of those
solutions, and the point of this email is to see with all of you which
option I should investigate first.

Note that option #3 is already implemented (would have to be polished
for upstream), IIRC option #2 has been partially implemented by Tomeu
but I'm not sure it was finished, and option #1 has just been discussed
so far [2].

Any feedback/comment is welcome.

Thanks,

Boris

[1]https://lore.kernel.org/patchwork/patch/878268/
[2]https://www.spinics.net/lists/kvm/msg159206.html
[3]https://chromium.googlesource.com/chromiumos/third_party/kernel/+/chromeos-4.19/drivers/virtio/virtio_wl.c
[4]https://www.spinics.net/lists/kvm/msg159206.html
[5]https://markmail.org/message/jeh5xjjxvylyrbur
[6]https://www.spinics.net/lists/kvm/msg185688.html

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[David Stevens]

> FD <-> VFD mappings would have to be created
> by the subsystem in charge of the object backing the FD (virtio-gpu for
> exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> sockets if we decide to bridge unix and vsock sockets to make it
> transparent, ...). The FD <-> VFD mapping would also have to be created
> on the host side, probably by the virtio device implementation
> (virglrenderer for GEM bufs for instance), which means host and guest
> need a way to inform the other end that a new FD <-> VFD mapping has
> been created so the other end can create a similar mapping (I guess this
> requires extra device-specific commands to work).

My recent proposal for cross device resource sharing seems like it
could be relevant here: https://markmail.org/thread/jsaoqy7phrqdcpqu.


-David

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> #1 might require extra care if we want to make it safe, as pointed
> out by Stefan here [4] (but I wonder if the problem is not the same
> for a virtio-wayland based solution). Of course you also need a bit of
> infrastructure to register FD <-> VFD mappings (VFD being a virtual
> file descriptor that's only used as unique IDs identifying the resource
> backed by the local FD). FD <-> VFD mappings would have to be created
> by the subsystem in charge of the object backing the FD (virtio-gpu for
> exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> sockets if we decide to bridge unix and vsock sockets to make it
> transparent, ...). The FD <-> VFD mapping would also have to be created
> on the host side, probably by the virtio device implementation
> (virglrenderer for GEM bufs for instance), which means host and guest
> need a way to inform the other end that a new FD <-> VFD mapping has
> been created so the other end can create a similar mapping (I guess this
> requires extra device-specific commands to work). Note that this
> solution doesn't look so different from the virtio-dmabuf [5] approach
> proposed by Gerd a few months back, it's just extended to be a global
> VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One
> great thing about this approach is that we can re-use it for any kind
> of FD sharing, not just dmabufs.

A dedicated is most likely not going to happen.  Probably virtio-gpu and
possibly other devices too will support attaching a unique handle (a
uuid probably) to a dma-buf, then share buffers like this:

  (1) guest userspace export virtio-gpu resource as dma-buf
  (2) guest kernel generates uuid, sends uuid to the host (or maybe we
      let the host generate the uuid, not clear yet), attaches it to
      the (guest) dma-buf.
  (3) guest userspace imports the dma-buf into virtio-vdev
  (4) virtio-vdec driver finds the uuid attached to the buffer and passes
      it on to the host,
  (5) virtio-vdev device can use the uuid to lookup the buffer on the
      host, then have the host decoder send the data directly to the
      host gpu ...

It certainly makes sense to use this for wayland too, so you just pass
around the uuid if you want pass around a dma-buf (no matter which of
the three solutions).

> #2 is a bit challenging, since it requires the proxy to know about all
> possible kind of FDs and do a FD <-> unique handle conversion with some
> help from the subsystem backing the FD. For dmabufs, that means we
> need to know who created the dmabuf, or assume that only one device is
> used for all allocations (virtio-gpu?).

See above, the uuid idea should simplify this.

> #3 is pretty similar to #1 in its design except that, instead of using
> the VSOCK infrastructure it's using a new type of virtio device. I
> guess it has the same pros and cons #1 has, and the name should probably
> be changed to reflect the fact that it can transmit any kind of data not
> just wayland.

Even though vsock looks simple at first it isn't when you look at the
details.  You'll want support more streams than virtqueues.  So you'll
go multiplex.  You want good performance, but you also don't want allow
streams to DoS the device by filling up the queue.

Thats why I don't like the new virtio device idea much and would prefer
vhost being reused, either directly (#1) or via proxy (#2).

Note that vhost aims to be hypervisor-agnostic and we have (unless I
missed something) three transports for it: virtio, vmware and hyperv.
So extending that with virtio-only features might not be the best idea.

Also it is a fact that approach #1 didn't went anywhere so far but we
have a working implementation of approach #3.  So I guess I wouldn't
veto approach #3 if you pick it after evaluating the options on the
table.

Final note: We have a new kid on the block: virtio-fs.  I think
virtio-fs with dax enabled should allow for shared file mappings between
host and guest.  That is something a proxy (#2) might be able to make
use of.

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 1592 bytes --]

On Mon, Feb 10, 2020 at 02:38:56PM +0100, Gerd Hoffmann wrote:
> > #3 is pretty similar to #1 in its design except that, instead of using
> > the VSOCK infrastructure it's using a new type of virtio device. I
> > guess it has the same pros and cons #1 has, and the name should probably
> > be changed to reflect the fact that it can transmit any kind of data not
> > just wayland.
> 
> Even though vsock looks simple at first it isn't when you look at the
> details.  You'll want support more streams than virtqueues.  So you'll
> go multiplex.  You want good performance, but you also don't want allow
> streams to DoS the device by filling up the queue.
> 
> Thats why I don't like the new virtio device idea much and would prefer
> vhost being reused, either directly (#1) or via proxy (#2).
> 
> Note that vhost aims to be hypervisor-agnostic and we have (unless I
> missed something) three transports for it: virtio, vmware and hyperv.
> So extending that with virtio-only features might not be the best idea.

s/vsock/vhost/

> Also it is a fact that approach #1 didn't went anywhere so far but we
> have a working implementation of approach #3.  So I guess I wouldn't
> veto approach #3 if you pick it after evaluating the options on the
> table.
> 
> Final note: We have a new kid on the block: virtio-fs.  I think
> virtio-fs with dax enabled should allow for shared file mappings between
> host and guest.  That is something a proxy (#2) might be able to make
> use of.

Yes, virtio-fs allows the guest to directly map files from the host.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 488 bytes --]

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

Hi David,

On Mon, 10 Feb 2020 14:06:21 +0900
David Stevens <stevensd@chromium.org> wrote:

> > FD <-> VFD mappings would have to be created
> > by the subsystem in charge of the object backing the FD (virtio-gpu for
> > exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> > sockets if we decide to bridge unix and vsock sockets to make it
> > transparent, ...). The FD <-> VFD mapping would also have to be created
> > on the host side, probably by the virtio device implementation
> > (virglrenderer for GEM bufs for instance), which means host and guest
> > need a way to inform the other end that a new FD <-> VFD mapping has
> > been created so the other end can create a similar mapping (I guess this
> > requires extra device-specific commands to work).  
> 
> My recent proposal for cross device resource sharing seems like it
> could be relevant here: https://markmail.org/thread/jsaoqy7phrqdcpqu.

Thanks for sharing this link. I had a quick look at this proposal, and, 
maybe I'm wrong, but I'm not sure it actually addresses Tomasz' concern
[1] if we keep letting a userspace proxy do the FD <-> UUID conversion
and sending the UUID through the VSOCK. To me, a UUID only guarantees
that 2 buffers will get different UUIDs (assuming they use the same
algorithm to generate this UUID), but nothing prevents a malicious app
from opening a connection to the host proxy and sending valid wayland
messages with forged UUIDs, in the hope that one of them will match an
already exported resource.

Regards,

Boris

[1]https://www.spinics.net/lists/kvm/msg185688.html

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[David Stevens]

> > > FD <-> VFD mappings would have to be created
> > > by the subsystem in charge of the object backing the FD (virtio-gpu for
> > > exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> > > sockets if we decide to bridge unix and vsock sockets to make it
> > > transparent, ...). The FD <-> VFD mapping would also have to be created
> > > on the host side, probably by the virtio device implementation
> > > (virglrenderer for GEM bufs for instance), which means host and guest
> > > need a way to inform the other end that a new FD <-> VFD mapping has
> > > been created so the other end can create a similar mapping (I guess this
> > > requires extra device-specific commands to work).
> >
> > My recent proposal for cross device resource sharing seems like it
> > could be relevant here: https://markmail.org/thread/jsaoqy7phrqdcpqu.
>
> Thanks for sharing this link. I had a quick look at this proposal, and,
> maybe I'm wrong, but I'm not sure it actually addresses Tomasz' concern
> [1] if we keep letting a userspace proxy do the FD <-> UUID conversion
> and sending the UUID through the VSOCK. To me, a UUID only guarantees
> that 2 buffers will get different UUIDs (assuming they use the same
> algorithm to generate this UUID), but nothing prevents a malicious app
> from opening a connection to the host proxy and sending valid wayland
> messages with forged UUIDs, in the hope that one of them will match an
> already exported resource.

You're correct that it wouldn't really protect against malicious apps.
My email was more about pointing out a mechanism that could
potentially help address the FD <-> VFD mapping.

-David

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

Hi Gerd,

Sorry for the delay, I was OOO last week.

On Mon, 10 Feb 2020 14:38:56 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
> 
> > #1 might require extra care if we want to make it safe, as pointed
> > out by Stefan here [4] (but I wonder if the problem is not the same
> > for a virtio-wayland based solution). Of course you also need a bit of
> > infrastructure to register FD <-> VFD mappings (VFD being a virtual
> > file descriptor that's only used as unique IDs identifying the resource
> > backed by the local FD). FD <-> VFD mappings would have to be created
> > by the subsystem in charge of the object backing the FD (virtio-gpu for
> > exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> > sockets if we decide to bridge unix and vsock sockets to make it
> > transparent, ...). The FD <-> VFD mapping would also have to be created
> > on the host side, probably by the virtio device implementation
> > (virglrenderer for GEM bufs for instance), which means host and guest
> > need a way to inform the other end that a new FD <-> VFD mapping has
> > been created so the other end can create a similar mapping (I guess this
> > requires extra device-specific commands to work). Note that this
> > solution doesn't look so different from the virtio-dmabuf [5] approach
> > proposed by Gerd a few months back, it's just extended to be a global
> > VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One
> > great thing about this approach is that we can re-use it for any kind
> > of FD sharing, not just dmabufs.  
> 
> A dedicated is most likely not going to happen.  Probably virtio-gpu and
> possibly other devices too will support attaching a unique handle (a
> uuid probably) to a dma-buf, then share buffers like this:
> 
>   (1) guest userspace export virtio-gpu resource as dma-buf
>   (2) guest kernel generates uuid, sends uuid to the host (or maybe we
>       let the host generate the uuid, not clear yet), attaches it to
>       the (guest) dma-buf.
>   (3) guest userspace imports the dma-buf into virtio-vdev
>   (4) virtio-vdec driver finds the uuid attached to the buffer and passes
>       it on to the host,
>   (5) virtio-vdev device can use the uuid to lookup the buffer on the
>       host, then have the host decoder send the data directly to the
>       host gpu ...
> 
> It certainly makes sense to use this for wayland too, so you just pass
> around the uuid if you want pass around a dma-buf (no matter which of
> the three solutions).

As pointed in my reply to David's email, I'm a bit worried by the
security implications of this approach. As long as the dmabuf -> UUID
conversion stays in kernel space we should be safe, but if we start
allowing a guest proxy (running in userland) to send raw UUIDs on a
VSOCK connection, we lose the ability to check if the process is
actually allowed to use this resource. Those checks happen kernel side
when a FD is passed on a UNIX socket, which guarantees that the process
passing this FD actually has access to this resource (either because it
created it or because someone else passed the resource through another
UNIX socket). Are we expecting that the wayland proxy runs with high
privilege (root?), and do we have a way to enforce that at the VSOCK
level?

> 
> > #2 is a bit challenging, since it requires the proxy to know about all
> > possible kind of FDs and do a FD <-> unique handle conversion with some
> > help from the subsystem backing the FD. For dmabufs, that means we
> > need to know who created the dmabuf, or assume that only one device is
> > used for all allocations (virtio-gpu?).  
> 
> See above, the uuid idea should simplify this.

Well, it gets more complicated if you start adding allocators
(we currently only have virtio-gpu, but what if virtio-vdec starts
providing its own buffer allocator) and don't add a generic interface
to attach UUIDs to resources. This could be addressed with the
virtio-dmabuf infrastructure you proposed, but it looks like David's
proposal does not generalize this interface (it's a virtio-gpu-only
thing right now).

> 
> > #3 is pretty similar to #1 in its design except that, instead of using
> > the VSOCK infrastructure it's using a new type of virtio device. I
> > guess it has the same pros and cons #1 has, and the name should probably
> > be changed to reflect the fact that it can transmit any kind of data not
> > just wayland.  
> 
> Even though vsock looks simple at first it isn't when you look at the
> details.  You'll want support more streams than virtqueues.  So you'll
> go multiplex.  You want good performance, but you also don't want allow
> streams to DoS the device by filling up the queue.
> 
> Thats why I don't like the new virtio device idea much and would prefer
> vhost being reused, either directly (#1) or via proxy (#2).

Okay.

> 
> Note that vhost aims to be hypervisor-agnostic and we have (unless I
> missed something) three transports for it: virtio, vmware and hyperv.
> So extending that with virtio-only features might not be the best idea.

Hm, should we add a dedicated virtio-fd-passing side-channel then. I
mean, I could pass extra information on the wayland-VSOCK to let the
other end know that some VFDs (or UUIDs, or resource handles, depending
on how you want to call them) need to be retrieved through this
FD-passing pipe.

> 
> Also it is a fact that approach #1 didn't went anywhere so far but we
> have a working implementation of approach #3.  So I guess I wouldn't
> veto approach #3 if you pick it after evaluating the options on the
> table.

I'd really like to investigate option #1, unless you have strong
reasons to think this is a dead end.

> 
> Final note: We have a new kid on the block: virtio-fs.  I think
> virtio-fs with dax enabled should allow for shared file mappings between
> host and guest.  That is something a proxy (#2) might be able to make
> use of.

We'll have a userland proxy anyway, no matter the solution we choose,
so yes, for zero-copy file mapping sharing, virtio-fs is likely to be
used.

Regards,

Boris

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

Hi Stéphane,

On Mon, 10 Feb 2020 12:01:02 -0800
Stéphane Marchesin <marcheu@chromium.org> wrote:

> On Fri, Feb 7, 2020 at 9:28 AM Boris Brezillon <
> boris.brezillon@collabora.com> wrote:  
> 
> > Hello everyone,
> >
> > I recently took over Tomeu's task of upstreaming virtio-wayland. After
> > spending quite a bit of time collecting information from his different
> > attempts [1][2] I wanted to sync with all the people that were involved
> > in the previous discussions (if I missed some of them, feel free to add
> > them back).
> >
> > The goal here is to get a rough idea of the general direction this
> > should take so I can start implementing a PoC and see if it fits
> > everyone's needs.
> >
> > virtio-wayland [3] started as a solution to pass wayland messages
> > between host and guests so the guest can execute wayland apps whose
> > surface buffers are passed to the wayland compositor running on the
> > host. While this was its primary use case, I've heard it's been used to
> > transport other protocols. And that's not surprising, when looking at
> > the code I noticed it was providing a protocol-agnostic message passing
> > interface between host and guests, similar to what VSOCK provides but
> > with FD passing as an extra feature.
> >
> > Based on all previous discussions, I could identify 3 different
> > approaches:
> >
> >     1/ Use VSOCK and extend it to support passing (some) FDs
> >     2/ Use a user space VSOCK-based proxy that's in charge of
> >        a/ passing regular messages
> >        b/ passing specific handles to describe objects shared
> >           between host and guest (most focus has been on dmabufs as
> >           this is what we really care about for the gfx use case,
> >           but other kind of FDs can be emulated through a
> >           VSOCK <-> UNIX_SOCK bridging)
> >     3/ Have a dedicated kernel space solution that provides features
> >        exposed by #1 but through a virtio device interface (basically
> >        what virtio-wayland does today)
> >
> > Each of them has its pros and cons, which I'll try to sum-up (please
> > correct me if I'm wrong, and add new things if you think they are
> > missing).
> >
> > #1 might require extra care if we want to make it safe, as pointed
> > out by Stefan here [4] (but I wonder if the problem is not the same
> > for a virtio-wayland based solution). Of course you also need a bit of
> > infrastructure to register FD <-> VFD mappings (VFD being a virtual
> > file descriptor that's only used as unique IDs identifying the resource
> > backed by the local FD). FD <-> VFD mappings would have to be created
> > by the subsystem in charge of the object backing the FD (virtio-gpu for
> > exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> > sockets if we decide to bridge unix and vsock sockets to make it
> > transparent, ...). The FD <-> VFD mapping would also have to be created
> > on the host side, probably by the virtio device implementation
> > (virglrenderer for GEM bufs for instance), which means host and guest
> > need a way to inform the other end that a new FD <-> VFD mapping has
> > been created so the other end can create a similar mapping (I guess this
> > requires extra device-specific commands to work). Note that this
> > solution doesn't look so different from the virtio-dmabuf [5] approach
> > proposed by Gerd a few months back, it's just extended to be a global
> > VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One
> > great thing about this approach is that we can re-use it for any kind
> > of FD sharing, not just dmabufs.
> >
> > #2 is a bit challenging, since it requires the proxy to know about all
> > possible kind of FDs and do a FD <-> unique handle conversion with some
> > help from the subsystem backing the FD. For dmabufs, that means we
> > need to know who created the dmabuf, or assume that only one device is
> > used for all allocations (virtio-gpu?). AFAIU, there's also a security
> > issue as one could pass random (but potentially valid) handles to the
> > host proxy (pointed out by Tomasz [6]).
> >
> > #3 is pretty similar to #1 in its design except that, instead of using
> > the VSOCK infrastructure it's using a new type of virtio device. I
> > guess it has the same pros and cons #1 has, and the name should probably
> > be changed to reflect the fact that it can transmit any kind of data not
> > just wayland.
> >
> > This is just a high level view of the problem and the solutions proposed
> > by various people over the years. I'm sure I'm missing tons of details
> > and don't realize yet all the complexity behind solution #1, but looking
> > at this summary, I wonder if I should investigate this solution in
> > priority. An alternative could be to rebrand virtio-wayland, but as I
> > said, it's close enough to VSOCK to try to merge the missing features
> > in VSOCK instead. This being said, I'm not yet set on any of those
> > solutions, and the point of this email is to see with all of you which
> > option I should investigate first.
> >
> > Note that option #3 is already implemented (would have to be polished
> > for upstream), IIRC option #2 has been partially implemented by Tomeu
> > but I'm not sure it was finished, and option #1 has just been discussed
> > so far [2].
> >
> > Any feedback/comment is welcome.
> >  
> 
> One of the key things I feel we need is for the host-side to be aware of
> what's going on with the life cycle of the buffers in one single place.
> 
> Today we are making suboptimal buffer allocations because we have to
> pessimistically assume that all buffers with the display flag are going to
> be displayed (even though they might not be used that way). Furthermore, we
> can't reallocate buffers on the fly to fit changing usage models, for
> example to go to different overlays. If virglrenderer knew about the usage
> in real time, it could make more optimal allocation decisions. Today this
> is one of the main aspects limiting the performance we can get out of our
> VMs.

Could you elaborate on that specific aspect? Where/when the
re-allocation/copy would happen (I suspect it's done at
wl_surface.commit() time but I'm not sure). I was also wondering if
this wp_linux_dmabuf_hints extension [1] couldn't help getting the most
optimal format/modifier without requiring this re-allocation/copy on
the host side.

Regards,

Boris

[1]https://gitlab.freedesktop.org/wayland/wayland-protocols/merge_requests/8

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> As pointed in my reply to David's email, I'm a bit worried by the
> security implications of this approach. As long as the dmabuf -> UUID
> conversion stays in kernel space we should be safe, but if we start
> allowing a guest proxy (running in userland) to send raw UUIDs on a
> VSOCK connection, we lose the ability to check if the process is
> actually allowed to use this resource.

Correct.

The UUID namespace is huge (128bit), so playing guessing games to access
resources you don't own isn't exactly trivial.  I would not be concerned
too much.  If you want avoid uuids nevertheless the only way is option
#1 (extent vsock) I think.

> to attach UUIDs to resources. This could be addressed with the
> virtio-dmabuf infrastructure you proposed, but it looks like David's
> proposal does not generalize this interface (it's a virtio-gpu-only
> thing right now).

The idea is that virtio-gpu guest driver adds a uuid to any dma-buf it
exports.  Then other drivers importing the dma-buf can figure what the
uuid is.  We could also add a ioctl (on the dma-buf fd) to query the
uuid.

> > Also it is a fact that approach #1 didn't went anywhere so far but we
> > have a working implementation of approach #3.  So I guess I wouldn't
> > veto approach #3 if you pick it after evaluating the options on the
> > table.
> 
> I'd really like to investigate option #1, unless you have strong
> reasons to think this is a dead end.

Last time I discussed this with Stefan Hajnoczi he didn't like the idea
too much.  IIRC the main concern was that it would work on specific
kinds of file handles only and that vsock would need special support
code for every file handle type.

It quite a while ago though, maybe time for a re-evaluation.  The
limitation to specific file handles wouldn't go away, and likewise the
need for special support code.  We have a more clear plan for dma-buf
support though.  Also we got virtio-fs meanwhile and being able to pass
virtio-fs file handles over vsock looks like a pretty cool feature to
me.

Stefan?

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 17 Feb 2020 13:32:16 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
> 
> > As pointed in my reply to David's email, I'm a bit worried by the
> > security implications of this approach. As long as the dmabuf -> UUID
> > conversion stays in kernel space we should be safe, but if we start
> > allowing a guest proxy (running in userland) to send raw UUIDs on a
> > VSOCK connection, we lose the ability to check if the process is
> > actually allowed to use this resource.  
> 
> Correct.
> 
> The UUID namespace is huge (128bit), so playing guessing games to access
> resources you don't own isn't exactly trivial.

It depends on the UUID algorithm [1] I guess. Versions 1, 2, 3 and 5
seem easier to guess than version 4 (not sure which one you're planning
to use here).

> I would not be concerned
> too much. 

Any potential security hole should be considered IMHO, even if it's not
easily exploitable.

> If you want avoid uuids nevertheless the only way is option
> #1 (extent vsock) I think.
> 
> > to attach UUIDs to resources. This could be addressed with the
> > virtio-dmabuf infrastructure you proposed, but it looks like David's
> > proposal does not generalize this interface (it's a virtio-gpu-only
> > thing right now).  
> 
> The idea is that virtio-gpu guest driver adds a uuid to any dma-buf it
> exports.  Then other drivers importing the dma-buf can figure what the
> uuid is.  We could also add a ioctl (on the dma-buf fd) to query the
> uuid.

Okay.

> 
> > > Also it is a fact that approach #1 didn't went anywhere so far but we
> > > have a working implementation of approach #3.  So I guess I wouldn't
> > > veto approach #3 if you pick it after evaluating the options on the
> > > table.  
> > 
> > I'd really like to investigate option #1, unless you have strong
> > reasons to think this is a dead end.  
> 
> Last time I discussed this with Stefan Hajnoczi he didn't like the idea
> too much.  IIRC the main concern was that it would work on specific
> kinds of file handles only and that vsock would need special support
> code for every file handle type.

According to [2], he was more worried by the risk of DoS than the fact
that only some FDs might be able to transit on a VSOCK (which I find
totally reasonable BTW, since it requires some coordination between
host and guest to get that working).

> 
> It quite a while ago though, maybe time for a re-evaluation.  The
> limitation to specific file handles wouldn't go away, and likewise the
> need for special support code.  We have a more clear plan for dma-buf
> support though.  Also we got virtio-fs meanwhile and being able to pass
> virtio-fs file handles over vsock looks like a pretty cool feature to
> me.
> 
> Stefan?
> 

[1]https://en.wikipedia.org/wiki/Universally_unique_identifier
[2]https://www.spinics.net/lists/kvm/msg159206.html

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Fri, 7 Feb 2020 18:28:42 +0100
Boris Brezillon <boris.brezillon@collabora.com> wrote:

> Based on all previous discussions, I could identify 3 different
> approaches:
> 
>     1/ Use VSOCK and extend it to support passing (some) FDs
>     2/ Use a user space VSOCK-based proxy that's in charge of
>        a/ passing regular messages
>        b/ passing specific handles to describe objects shared
>           between host and guest (most focus has been on dmabufs as
>           this is what we really care about for the gfx use case,
>           but other kind of FDs can be emulated through a
>           VSOCK <-> UNIX_SOCK bridging)
>     3/ Have a dedicated kernel space solution that provides features
>        exposed by #1 but through a virtio device interface (basically
>        what virtio-wayland does today)
> 
> Each of them has its pros and cons, which I'll try to sum-up (please
> correct me if I'm wrong, and add new things if you think they are
> missing).
> 
> #1 might require extra care if we want to make it safe, as pointed
> out by Stefan here [4] (but I wonder if the problem is not the same
> for a virtio-wayland based solution). Of course you also need a bit of
> infrastructure to register FD <-> VFD mappings (VFD being a virtual
> file descriptor that's only used as unique IDs identifying the resource
> backed by the local FD). FD <-> VFD mappings would have to be created
> by the subsystem in charge of the object backing the FD (virtio-gpu for
> exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> sockets if we decide to bridge unix and vsock sockets to make it
> transparent, ...). The FD <-> VFD mapping would also have to be created
> on the host side, probably by the virtio device implementation
> (virglrenderer for GEM bufs for instance), which means host and guest
> need a way to inform the other end that a new FD <-> VFD mapping has
> been created so the other end can create a similar mapping (I guess this
> requires extra device-specific commands to work).

We might be able to do that lazily if we add
->virtio_fd_to_vfd()/->virtio_vfd_to_fd() hooks to the file_operations
struct, instead of expecting the subsystem to pro-actively create those
mappings (when those hooks are not implemented, that means the resource
backed by this FD can't be passed on a VSOCK). Anyway, that's just an
implementation detail, the first question being, should we pursue in
this direction or not?

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 17 Feb 2020 09:47:46 -0800
Zach Reizner <zachr@google.com> wrote:

> > Thats why I don't like the new virtio device idea much and would prefer
> > vhost being reused, either directly (#1) or via proxy (#2).
> 
> For crosvm's purposes, we are looking at ways to reduce vhost usage in
> order to reduce host kernel exposure to untrusted guest input,
> including from the guest kernel. That is why a non-vhost based
> solution would be prefered.

Okay, I didn't know you were avoiding vhost-based solutions to
reduce the attack surface.

> 
> > Any potential security hole should be considered IMHO, even if it's not
> > easily exploitable.
> 
> I second Boris's statement. In the age of Spectre-like exploits,
> keeping secrets from other processes should be considered riskier than
> using FDs to prove that a process can access a resource.
> 
> > We might be able to do that lazily if we add
> > ->virtio_fd_to_vfd()/->virtio_vfd_to_fd() hooks to the file_operations
> > struct, instead of expecting the subsystem to pro-actively create those
> > mappings (when those hooks are not implemented, that means the resource
> > backed by this FD can't be passed on a VSOCK).
> 
> virtio-wayland actually will handle passing arbitrary FDs back and
> forth, but it does have special handling for virtio-gpu dmabufs. For
> everything else, virtio-wayland will determine if the FD is
> readable/writable/mappable and will recreate a pipe/memfd on the other
> side to emulate the FD. For mappable FDs, this is limited host->guest
> passing because in practice this is used only by the compositor to
> send the keyboard mapping to clients. The pipes are needed for both
> host->guest and guest->host because they are used to request clipboard
> content. The only FDs that require special handling are ones where
> there already is a host-side FD corresponding to the guest side FD,
> such as the case of wayland socket connections or dmabufs.
> 
> As for the question of which solution to go for, I would suggest
> either 3 (dedicated virtio device) or combining features from
> virtio-wayland with virtio-gpu. I don't think extending vsock is a
> good idea because there's enough going on with virtio-wayland with
> respect to FD passing and emulation that a generic interface will be
> too opinionated.

Can you be more specific? What would be the limitations of such a
generic interface that would make it unusable for the virtio-wayland
use case?

> I like the combination with virtio-gpu because it
> already has two related but separate graphics concepts combined into
> one device (display and rendering), so wayland would feel right at
> home.

Except that, looking at the virtio-wayland code nothing in there looks
fundamentally wayland specific, or even display-server specific. Sure,
FD passing support is focused on what wayland needs (pipes, dmabuf FDs,
...), but I see nothing that would prevent making it generic enough to
pass other kind of resources if we ever need to.

> Because wayland benefits from allocation and sharing of
> virtio-gpu buffers, a virtio-gpu combo device simplifies access to
> those buffers, whereas the separate virtio devices as implemented in
> crosvm requires bridging of resource handles (in guest kernel) and FDs
> (in crosvm).

I must admit I'm not a huge fan of integrating a protocol-agnostic
message passing interface (with FD-passing support) to virtio-gpu.
Sounds like something that could be re-used without virtio-gpu being
involved to me. As for the resource bridging thing, that's more or less
what I proposed with the generic interface to translate resource handles
(what I called VFDs) to local FDs. It would probably be easier with a
vhost-based solution, but your resource bridge interface in crosvm
seems to provide the same kind of abstraction. Any reason you want to
avoid this split?


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Tue, 18 Feb 2020 13:15:04 -0800
Zach Reizner <zachr@google.com> wrote:

> >> <snip> I don't think extending vsock is a
> >> good idea because there's enough going on with virtio-wayland with
> >> respect to FD passing and emulation that a generic interface will be
> >> too opinionated.  
> >
> > Can you be more specific? What would be the limitations of such a
> > generic interface that would make it unusable for the virtio-wayland
> > use case?  
> 
> What I meant by that statement was that virtio-wayland implements
> juuuust enough FD semantics to make wayland work. It's true that some
> other protocols also work over virtio-wayland, but that was a
> combination of filling in the holes for that protocol and coincidence.
> I'm afraid that a generic interface would have to contend with a long
> tail of FD semantics that don't line up with most other protocols. To
> give you a specific example, when virtio-wayland encounters an FD in
> the guest->host direction that it doesn't recognize, but that can be
> written or read to, it will create a unix pipe on the host side and
> send it along the wayland connection. This choice works fine for
> wayland, but that's because that was the entire point of the design. A
> more generic system would have to realize that interaction more
> authentically (e.g. for sending a bi-directional socket),

Indeed, but is that really a bad thing? I mean, do we really want
any kind of FD to transit between guest and host as soon as they
implement ->read/write(). For instance, should we forward device FDs
which have ->read/write() implemented but on which guest users might
want to call ioctl().

> and I'm not
> sure it's going to be possible.
> 
> >  
> > > I like the combination with virtio-gpu because it
> > > already has two related but separate graphics concepts combined into
> > > one device (display and rendering), so wayland would feel right at
> > > home.  
> >
> > Except that, looking at the virtio-wayland code nothing in there looks
> > fundamentally wayland specific, or even display-server specific. Sure,
> > FD passing support is focused on what wayland needs (pipes, dmabuf FDs,
> > ...), but I see nothing that would prevent making it generic enough to
> > pass other kind of resources if we ever need to.  
> 
> That was intentional, and you have a good point. The reason I was
> suggesting integration with virtio-gpu was because I saw that as the
> most viable path towards upstreaming. It seems that creating a
> dedicated virtio device is highly contentious.
> 
> >
> > I must admit I'm not a huge fan of integrating a protocol-agnostic
> > message passing interface (with FD-passing support) to virtio-gpu.
> > Sounds like something that could be re-used without virtio-gpu being
> > involved to me. As for the resource bridging thing, that's more or less
> > what I proposed with the generic interface to translate resource handles
> > (what I called VFDs) to local FDs. It would probably be easier with a
> > vhost-based solution, but your resource bridge interface in crosvm
> > seems to provide the same kind of abstraction. Any reason you want to
> > avoid this split?  
> 
> If the upstreamed version of this actually becomes agnostic, than
> keeping it separate from virtio-gpu would make the most sense, but
> that seems to be unsettled.

Gerd, Stefan, any comment?

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 17 Feb 2020 19:21:50 +0100
Boris Brezillon <boris.brezillon@collabora.com> wrote:

> > > Thats why I don't like the new virtio device idea much and would prefer
> > > vhost being reused, either directly (#1) or via proxy (#2).  
> > 
> > For crosvm's purposes, we are looking at ways to reduce vhost usage in
> > order to reduce host kernel exposure to untrusted guest input,
> > including from the guest kernel. That is why a non-vhost based
> > solution would be prefered.  
> 
> Okay, I didn't know you were avoiding vhost-based solutions to
> reduce the attack surface.

Looks like they implemented vhost-less vsock in Firecracker[1]. Not
sure how much work that would be to port this implementation to crosvm,
but maybe that's an option.

[1]https://github.com/firecracker-microvm/firecracker/pull/1176

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

On Mon, Feb 24, 2020 at 11:33:48AM +0100, Boris Brezillon wrote:
> On Mon, 17 Feb 2020 19:21:50 +0100
> Boris Brezillon <boris.brezillon@collabora.com> wrote:
> 
> > > > Thats why I don't like the new virtio device idea much and would prefer
> > > > vhost being reused, either directly (#1) or via proxy (#2).  
> > > 
> > > For crosvm's purposes, we are looking at ways to reduce vhost usage in
> > > order to reduce host kernel exposure to untrusted guest input,
> > > including from the guest kernel. That is why a non-vhost based
> > > solution would be prefered.  
> > 
> > Okay, I didn't know you were avoiding vhost-based solutions to
> > reduce the attack surface.
> 
> Looks like they implemented vhost-less vsock in Firecracker[1]. Not
> sure how much work that would be to port this implementation to crosvm,
> but maybe that's an option.
> 
> [1]https://github.com/firecracker-microvm/firecracker/pull/1176

Well, the nice thing about vsock is that (a) you have the same interface
on guest and host, and (b) it is hypervisor-agnostic.  Applications can
simply use the usual system calls (socket, bind, listen, connect) with
AF_VSOCK socket type and be done with it.

When not using vhost this goes away (on the host side).  firecracker
seems to have a vsock <-> unix socket bridge in vmm userspace instead
(pull req msg sounds like that, didn't check the code).

I think there can be only one vsock device per guest.  So you can't have
multiple instances, one with a vsock backend on the host and one with a
userspace backend on the host, then pick one of the two depending on the
use case.

So, one advantage of a separate device would be that we'll gain some
flexibility in terms of the host side implementation.  vsock vhost can
be used together with a wayland userspace backend.

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> > Because wayland benefits from allocation and sharing of
> > virtio-gpu buffers, a virtio-gpu combo device simplifies access to
> > those buffers, whereas the separate virtio devices as implemented in
> > crosvm requires bridging of resource handles (in guest kernel) and FDs
> > (in crosvm).
> 
> I must admit I'm not a huge fan of integrating a protocol-agnostic
> message passing interface (with FD-passing support) to virtio-gpu.
> Sounds like something that could be re-used without virtio-gpu being
> involved to me.

Agree.

> As for the resource bridging thing, that's more or less
> what I proposed with the generic interface to translate resource handles
> (what I called VFDs) to local FDs.

Yes, I think it makes sense to figure a sensible way to translate/manage
resource handles and have multiple virtio device work together.  What if
you want pass a handle to a virtio-fs file instead of a virtio-gpu
dma-buf between host and guest?

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 24 Feb 2020 13:12:55 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

> On Mon, Feb 24, 2020 at 11:33:48AM +0100, Boris Brezillon wrote:
> > On Mon, 17 Feb 2020 19:21:50 +0100
> > Boris Brezillon <boris.brezillon@collabora.com> wrote:
> >   
> > > > > Thats why I don't like the new virtio device idea much and would prefer
> > > > > vhost being reused, either directly (#1) or via proxy (#2).    
> > > > 
> > > > For crosvm's purposes, we are looking at ways to reduce vhost usage in
> > > > order to reduce host kernel exposure to untrusted guest input,
> > > > including from the guest kernel. That is why a non-vhost based
> > > > solution would be prefered.    
> > > 
> > > Okay, I didn't know you were avoiding vhost-based solutions to
> > > reduce the attack surface.  
> > 
> > Looks like they implemented vhost-less vsock in Firecracker[1]. Not
> > sure how much work that would be to port this implementation to crosvm,
> > but maybe that's an option.
> > 
> > [1]https://github.com/firecracker-microvm/firecracker/pull/1176  
> 
> Well, the nice thing about vsock is that (a) you have the same interface
> on guest and host, and (b) it is hypervisor-agnostic.  Applications can
> simply use the usual system calls (socket, bind, listen, connect) with
> AF_VSOCK socket type and be done with it.
> 
> When not using vhost this goes away (on the host side).  firecracker
> seems to have a vsock <-> unix socket bridge in vmm userspace instead
> (pull req msg sounds like that, didn't check the code).

That's also my understanding. Note that you still keep the genericity
on the guest side, which is a good thing I guess.

> 
> I think there can be only one vsock device per guest.  So you can't have
> multiple instances, one with a vsock backend on the host and one with a
> userspace backend on the host, then pick one of the two depending on the
> use case.
> 
> So, one advantage of a separate device would be that we'll gain some
> flexibility in terms of the host side implementation.  vsock vhost can
> be used together with a wayland userspace backend.

Yes, though if you really want to avoid vhost to reduce the attack
surface, you'll probably want to never use vhost instead of having this
restriction on a per-use-case basis.

But let's say we go for a dedicated virtio-device to preserve this
granularity. Should we aim at providing a generic virtio-msg device or
should we keep this so-called wayland-specific virtio device (I'd like
to remind you that it's actually protocol-agnostic)?

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 24 Feb 2020 13:45:13 +0100
Boris Brezillon <boris.brezillon@collabora.com> wrote:

> On Mon, 24 Feb 2020 13:12:55 +0100
> Gerd Hoffmann <kraxel@redhat.com> wrote:
> 
> > On Mon, Feb 24, 2020 at 11:33:48AM +0100, Boris Brezillon wrote:  
> > > On Mon, 17 Feb 2020 19:21:50 +0100
> > > Boris Brezillon <boris.brezillon@collabora.com> wrote:
> > >     
> > > > > > Thats why I don't like the new virtio device idea much and would prefer
> > > > > > vhost being reused, either directly (#1) or via proxy (#2).      
> > > > > 
> > > > > For crosvm's purposes, we are looking at ways to reduce vhost usage in
> > > > > order to reduce host kernel exposure to untrusted guest input,
> > > > > including from the guest kernel. That is why a non-vhost based
> > > > > solution would be prefered.      
> > > > 
> > > > Okay, I didn't know you were avoiding vhost-based solutions to
> > > > reduce the attack surface.    
> > > 
> > > Looks like they implemented vhost-less vsock in Firecracker[1]. Not
> > > sure how much work that would be to port this implementation to crosvm,
> > > but maybe that's an option.
> > > 
> > > [1]https://github.com/firecracker-microvm/firecracker/pull/1176    
> > 
> > Well, the nice thing about vsock is that (a) you have the same interface
> > on guest and host, and (b) it is hypervisor-agnostic.  Applications can
> > simply use the usual system calls (socket, bind, listen, connect) with
> > AF_VSOCK socket type and be done with it.
> > 
> > When not using vhost this goes away (on the host side).  firecracker
> > seems to have a vsock <-> unix socket bridge in vmm userspace instead
> > (pull req msg sounds like that, didn't check the code).  
> 
> That's also my understanding. Note that you still keep the genericity
> on the guest side, which is a good thing I guess.
> 
> > 
> > I think there can be only one vsock device per guest.  So you can't have
> > multiple instances, one with a vsock backend on the host and one with a
> > userspace backend on the host, then pick one of the two depending on the
> > use case.
> > 
> > So, one advantage of a separate device would be that we'll gain some
> > flexibility in terms of the host side implementation.  vsock vhost can
> > be used together with a wayland userspace backend.  
> 
> Yes, though if you really want to avoid vhost to reduce the attack
> surface, you'll probably want to never use vhost instead of having this
> restriction on a per-use-case basis.
> 
> But let's say we go for a dedicated virtio-device to preserve this
> granularity. Should we aim at providing a generic virtio-msg device or
> should we keep this so-called wayland-specific virtio device (I'd like
> to remind you that it's actually protocol-agnostic)?

Maybe there's another option (not sure I mentioned it previously):
de-correlate the message passing and fd-passing interfaces. If we add a
virtio-fd device that takes care of FD passing between guest and host,
we can still use a regular vsock to pass messages (can be vhost or
!vhost depending on your level of paranoïa). The wayland proxies would
then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> But let's say we go for a dedicated virtio-device to preserve this
> granularity. Should we aim at providing a generic virtio-msg device or
> should we keep this so-called wayland-specific virtio device (I'd like
> to remind you that it's actually protocol-agnostic)?

I think it totally makes sense to have something protocol-agnostic,
especially given that people figured it isn't really wayland-specific
and already using the existing implementation for non-wayland things.

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> > But let's say we go for a dedicated virtio-device to preserve this
> > granularity. Should we aim at providing a generic virtio-msg device or
> > should we keep this so-called wayland-specific virtio device (I'd like
> > to remind you that it's actually protocol-agnostic)?
> 
> Maybe there's another option (not sure I mentioned it previously):
> de-correlate the message passing and fd-passing interfaces. If we add a
> virtio-fd device that takes care of FD passing between guest and host,
> we can still use a regular vsock to pass messages (can be vhost or
> !vhost depending on your level of paranoïa). The wayland proxies would
> then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.

Hmm, I suspect supporting unix socket passing (by creating a tunnel for
them) is going to be tricky in with that approach.

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

[virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 24 Feb 2020 14:42:31 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
> 
> > > But let's say we go for a dedicated virtio-device to preserve this
> > > granularity. Should we aim at providing a generic virtio-msg device or
> > > should we keep this so-called wayland-specific virtio device (I'd like
> > > to remind you that it's actually protocol-agnostic)?  
> > 
> > Maybe there's another option (not sure I mentioned it previously):
> > de-correlate the message passing and fd-passing interfaces. If we add a
> > virtio-fd device that takes care of FD passing between guest and host,
> > we can still use a regular vsock to pass messages (can be vhost or
> > !vhost depending on your level of paranoïa). The wayland proxies would
> > then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.  
> 
> Hmm, I suspect supporting unix socket passing (by creating a tunnel for
> them) is going to be tricky in with that approach.

Yes, If you want a generic vsock <-> unix proxy, you'll have to
encapsulate the 'message+number of FDs passed' information so the other
end knows how many FDs should be dequeued on the virtio-fd dev. The
other option being to implement proxies that can parse the messages and
extract the 'number of FDs' from there.


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] Re: [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 24 Feb 2020 14:57:43 +0100
Boris Brezillon <boris.brezillon@collabora.com> wrote:

> On Mon, 24 Feb 2020 14:42:31 +0100
> Gerd Hoffmann <kraxel@redhat.com> wrote:
> 
> >   Hi,
> >   
> > > > But let's say we go for a dedicated virtio-device to preserve this
> > > > granularity. Should we aim at providing a generic virtio-msg device or
> > > > should we keep this so-called wayland-specific virtio device (I'd like
> > > > to remind you that it's actually protocol-agnostic)?    
> > > 
> > > Maybe there's another option (not sure I mentioned it previously):
> > > de-correlate the message passing and fd-passing interfaces. If we add a
> > > virtio-fd device that takes care of FD passing between guest and host,
> > > we can still use a regular vsock to pass messages (can be vhost or
> > > !vhost depending on your level of paranoïa). The wayland proxies would
> > > then queue/dequeue messages to/from the vsock and FDs to/from virtio-fd.    
> > 
> > Hmm, I suspect supporting unix socket passing (by creating a tunnel for
> > them) is going to be tricky in with that approach.  
> 
> Yes, If you want a generic vsock <-> unix proxy, you'll have to
> encapsulate the 'message+number of FDs passed' information so the other
> end knows how many FDs should be dequeued on the virtio-fd dev. The
> other option being to implement proxies that can parse the messages and
> extract the 'number of FDs' from there.

Nevermind, I misunderstood your concern. Indeed, supporting unix FD
passing would be trickier (you'd need to attach both a vosck and a
virtio-fd instance).

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

Hi all,

On Fri, 7 Feb 2020 18:28:42 +0100
Boris Brezillon <boris.brezillon@collabora.com> wrote:

> Hello everyone,
> 
> I recently took over Tomeu's task of upstreaming virtio-wayland. After
> spending quite a bit of time collecting information from his different
> attempts [1][2] I wanted to sync with all the people that were involved
> in the previous discussions (if I missed some of them, feel free to add
> them back).
> 
> The goal here is to get a rough idea of the general direction this
> should take so I can start implementing a PoC and see if it fits
> everyone's needs.
> 
> virtio-wayland [3] started as a solution to pass wayland messages
> between host and guests so the guest can execute wayland apps whose
> surface buffers are passed to the wayland compositor running on the
> host. While this was its primary use case, I've heard it's been used to
> transport other protocols. And that's not surprising, when looking at
> the code I noticed it was providing a protocol-agnostic message passing
> interface between host and guests, similar to what VSOCK provides but
> with FD passing as an extra feature.
> 
> Based on all previous discussions, I could identify 3 different
> approaches:
> 
>     1/ Use VSOCK and extend it to support passing (some) FDs
>     2/ Use a user space VSOCK-based proxy that's in charge of
>        a/ passing regular messages
>        b/ passing specific handles to describe objects shared
>           between host and guest (most focus has been on dmabufs as
>           this is what we really care about for the gfx use case,
>           but other kind of FDs can be emulated through a
>           VSOCK <-> UNIX_SOCK bridging)
>     3/ Have a dedicated kernel space solution that provides features
>        exposed by #1 but through a virtio device interface (basically
>        what virtio-wayland does today)
> 
> Each of them has its pros and cons, which I'll try to sum-up (please
> correct me if I'm wrong, and add new things if you think they are
> missing).
> 
> #1 might require extra care if we want to make it safe, as pointed
> out by Stefan here [4] (but I wonder if the problem is not the same
> for a virtio-wayland based solution). Of course you also need a bit of
> infrastructure to register FD <-> VFD mappings (VFD being a virtual
> file descriptor that's only used as unique IDs identifying the resource
> backed by the local FD). FD <-> VFD mappings would have to be created
> by the subsystem in charge of the object backing the FD (virtio-gpu for
> exported GEM buffers, virtio-vdec for video buffers, vsock for unix
> sockets if we decide to bridge unix and vsock sockets to make it
> transparent, ...). The FD <-> VFD mapping would also have to be created
> on the host side, probably by the virtio device implementation
> (virglrenderer for GEM bufs for instance), which means host and guest
> need a way to inform the other end that a new FD <-> VFD mapping has
> been created so the other end can create a similar mapping (I guess this
> requires extra device-specific commands to work). Note that this
> solution doesn't look so different from the virtio-dmabuf [5] approach
> proposed by Gerd a few months back, it's just extended to be a global
> VFD <-> FD registry instead of a dmabuf <-> unique-handle one. One
> great thing about this approach is that we can re-use it for any kind
> of FD sharing, not just dmabufs.
> 
> #2 is a bit challenging, since it requires the proxy to know about all
> possible kind of FDs and do a FD <-> unique handle conversion with some
> help from the subsystem backing the FD. For dmabufs, that means we
> need to know who created the dmabuf, or assume that only one device is
> used for all allocations (virtio-gpu?). AFAIU, there's also a security
> issue as one could pass random (but potentially valid) handles to the
> host proxy (pointed out by Tomasz [6]).
> 
> #3 is pretty similar to #1 in its design except that, instead of using
> the VSOCK infrastructure it's using a new type of virtio device. I
> guess it has the same pros and cons #1 has, and the name should probably
> be changed to reflect the fact that it can transmit any kind of data not
> just wayland.
> 
> This is just a high level view of the problem and the solutions proposed
> by various people over the years. I'm sure I'm missing tons of details
> and don't realize yet all the complexity behind solution #1, but looking
> at this summary, I wonder if I should investigate this solution in
> priority. An alternative could be to rebrand virtio-wayland, but as I
> said, it's close enough to VSOCK to try to merge the missing features
> in VSOCK instead. This being said, I'm not yet set on any of those
> solutions, and the point of this email is to see with all of you which
> option I should investigate first.
> 
> Note that option #3 is already implemented (would have to be polished
> for upstream), IIRC option #2 has been partially implemented by Tomeu
> but I'm not sure it was finished, and option #1 has just been discussed
> so far [2].

I'd like to sum-up the discussion that has been happening in this
thread and share my plans so you can acknowledge/reject the solution I'm
about to start working on. Looks like we're heading to a slightly
adjusted option #3. Option #2 hasn't received much interest, and Zach
and I think letting user space do the FD -> UUID translation is not that
great from a security perspective. Option #1 sounds like a good idea on
the paper, but the fact that Google wants a solution that does not
involve vhost and given Gerd's concern about being able to use vsock
in some cases and do !vsock message-passing in others, it might not be
possible to use vsock (there can only be one vsock implementation
active). That leaves us with option #3: have a dedicated virtio device
providing a message+FD passing interface.

So, I'm about to start working on virtio-pipe (I realize the name is
not that great since pipes are normally unidirectional, but I'm sure
we'll have plenty of time to bikeshed on that particular aspect once
the other bits are sorted out :)). This device would be a singleton
(there can only be one per VM), and would provide the following
features:

* allow you to connect to named/public pipes (similar to named unix
  sockets). That's an easy way to expose host services to guests, and
  AFAICT, that's not so far from the virtio-wayland way of doing things
  (VIRTWL_IOCTL_NEW_CTX_NAMED).
* manage a central UUID <-> 'struct file' map that allows virtio-pipe
  to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
  UUIDs back to FDs on the other end
  - we need to expose an API to let each subsystem register/unregister
    their UUID <-> FD mapping (subsystems are responsible for the UUID
    creation/negotiation)
* allow you to create private/unnamed pipes whose FDs can be passed to
  the other end (that implies creating a UUID <-> FD mapping for each
  unnamed pipe). This feature is here to allow unamed unix-socket <->
  virtio-pipe bridging. AFAICT, that's what VIRTWL_IOCTL_NEW_CTX is
  providing. Looks like virtio-wayland also provides support for actual
  pipes (uni-directional msg-passing) with its VIRTWL_IOCTL_NEW_PIPE_*
  ioctls, which we can always support at some point if we see the need.

Once we have those basic building blocks in place, we can start
patching virtio-gpu to create a UUID <-> FD mapping when a buffer is
exported as a dmabuf (using the virtio-pipe public API). And this can
easily be applied to any kind of FDs we want to share.

Note that this solution could be extended to support FD passing on top
of VSOCK if we ever need/want to. We'd only have to move the UUID <->
FD map/API out of virtio-pipe (virtio-fd?) and let virtio-vsock use it.

Gerd, Stefan, Zach, what do you think? Should I start looking into
that, maybe work on a PoC and post it as an RFC, or do you see any
problem making this proposal not possible/irrelevant.

Any feedback is welcome!

Thanks,

Boris

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Alex Bennée]

Boris Brezillon <boris.brezillon@collabora.com> writes:

> Hi all,
>
> On Fri, 7 Feb 2020 18:28:42 +0100
> Boris Brezillon <boris.brezillon@collabora.com> wrote:
>
>> Hello everyone,
>> 
>> I recently took over Tomeu's task of upstreaming virtio-wayland. After
>> spending quite a bit of time collecting information from his different
>> attempts [1][2] I wanted to sync with all the people that were involved
>> in the previous discussions (if I missed some of them, feel free to add
>> them back).
>> 
>> The goal here is to get a rough idea of the general direction this
>> should take so I can start implementing a PoC and see if it fits
>> everyone's needs.
>> 
>> virtio-wayland [3] started as a solution to pass wayland messages
>> between host and guests so the guest can execute wayland apps whose
>> surface buffers are passed to the wayland compositor running on the
>> host. While this was its primary use case, I've heard it's been used to
>> transport other protocols. And that's not surprising, when looking at
>> the code I noticed it was providing a protocol-agnostic message passing
>> interface between host and guests, similar to what VSOCK provides but
>> with FD passing as an extra feature.
>> 
>> Based on all previous discussions, I could identify 3 different
>> approaches:
>> 
>>     1/ Use VSOCK and extend it to support passing (some) FDs
>>     2/ Use a user space VSOCK-based proxy that's in charge of
>>        a/ passing regular messages
>>        b/ passing specific handles to describe objects shared
>>           between host and guest (most focus has been on dmabufs as
>>           this is what we really care about for the gfx use case,
>>           but other kind of FDs can be emulated through a
>>           VSOCK <-> UNIX_SOCK bridging)
>>     3/ Have a dedicated kernel space solution that provides features
>>        exposed by #1 but through a virtio device interface (basically
>>        what virtio-wayland does today)
>> 
<snip>

> Option #1 sounds like a good idea on
> the paper, but the fact that Google wants a solution that does not
> involve vhost and given Gerd's concern about being able to use vsock
> in some cases and do !vsock message-passing in others, it might not be
> possible to use vsock (there can only be one vsock implementation
> active).

Forgive me for jumping in but what is the concern about vhost here? Is
it purely the increased attack surface to the kernel or some more
fundamental issue? Is this something that can be addressed with a
vhost-user backend where all the complications of the host end are dealt
with in user-space?

> That leaves us with option #3: have a dedicated virtio device
> providing a message+FD passing interface.
>
> So, I'm about to start working on virtio-pipe (I realize the name is
> not that great since pipes are normally unidirectional, but I'm sure
> we'll have plenty of time to bikeshed on that particular aspect once
> the other bits are sorted out :)). This device would be a singleton
> (there can only be one per VM), and would provide the following
> features:
>
> * allow you to connect to named/public pipes (similar to named unix
>   sockets). That's an easy way to expose host services to guests, and
>   AFAICT, that's not so far from the virtio-wayland way of doing things
>   (VIRTWL_IOCTL_NEW_CTX_NAMED).
> * manage a central UUID <-> 'struct file' map that allows virtio-pipe
>   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
>   UUIDs back to FDs on the other end
>   - we need to expose an API to let each subsystem register/unregister
>     their UUID <-> FD mapping (subsystems are responsible for the UUID
>     creation/negotiation)
> * allow you to create private/unnamed pipes whose FDs can be passed to
>   the other end (that implies creating a UUID <-> FD mapping for each
>   unnamed pipe). This feature is here to allow unamed unix-socket <->
>   virtio-pipe bridging. AFAICT, that's what VIRTWL_IOCTL_NEW_CTX is
>   providing. Looks like virtio-wayland also provides support for actual
>   pipes (uni-directional msg-passing) with its VIRTWL_IOCTL_NEW_PIPE_*
>   ioctls, which we can always support at some point if we see the need.
>
> Once we have those basic building blocks in place, we can start
> patching virtio-gpu to create a UUID <-> FD mapping when a buffer is
> exported as a dmabuf (using the virtio-pipe public API). And this can
> easily be applied to any kind of FDs we want to share.
>
> Note that this solution could be extended to support FD passing on top
> of VSOCK if we ever need/want to. We'd only have to move the UUID <->
> FD map/API out of virtio-pipe (virtio-fd?) and let virtio-vsock use it.
>
> Gerd, Stefan, Zach, what do you think? Should I start looking into
> that, maybe work on a PoC and post it as an RFC, or do you see any
> problem making this proposal not possible/irrelevant.
>
> Any feedback is welcome!
>
> Thanks,
>
> Boris
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
> For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org


-- 
Alex Bennée

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Tue, 25 Feb 2020 15:55:12 +0000
Alex Bennée <alex.bennee@linaro.org> wrote:

> Boris Brezillon <boris.brezillon@collabora.com> writes:
> 
> > Hi all,
> >
> > On Fri, 7 Feb 2020 18:28:42 +0100
> > Boris Brezillon <boris.brezillon@collabora.com> wrote:
> >  
> >> Hello everyone,
> >> 
> >> I recently took over Tomeu's task of upstreaming virtio-wayland. After
> >> spending quite a bit of time collecting information from his different
> >> attempts [1][2] I wanted to sync with all the people that were involved
> >> in the previous discussions (if I missed some of them, feel free to add
> >> them back).
> >> 
> >> The goal here is to get a rough idea of the general direction this
> >> should take so I can start implementing a PoC and see if it fits
> >> everyone's needs.
> >> 
> >> virtio-wayland [3] started as a solution to pass wayland messages
> >> between host and guests so the guest can execute wayland apps whose
> >> surface buffers are passed to the wayland compositor running on the
> >> host. While this was its primary use case, I've heard it's been used to
> >> transport other protocols. And that's not surprising, when looking at
> >> the code I noticed it was providing a protocol-agnostic message passing
> >> interface between host and guests, similar to what VSOCK provides but
> >> with FD passing as an extra feature.
> >> 
> >> Based on all previous discussions, I could identify 3 different
> >> approaches:
> >> 
> >>     1/ Use VSOCK and extend it to support passing (some) FDs
> >>     2/ Use a user space VSOCK-based proxy that's in charge of
> >>        a/ passing regular messages
> >>        b/ passing specific handles to describe objects shared
> >>           between host and guest (most focus has been on dmabufs as
> >>           this is what we really care about for the gfx use case,
> >>           but other kind of FDs can be emulated through a
> >>           VSOCK <-> UNIX_SOCK bridging)
> >>     3/ Have a dedicated kernel space solution that provides features
> >>        exposed by #1 but through a virtio device interface (basically
> >>        what virtio-wayland does today)
> >>   
> <snip>
> 
> > Option #1 sounds like a good idea on
> > the paper, but the fact that Google wants a solution that does not
> > involve vhost and given Gerd's concern about being able to use vsock
> > in some cases and do !vsock message-passing in others, it might not be
> > possible to use vsock (there can only be one vsock implementation
> > active).  
> 
> Forgive me for jumping in but what is the concern about vhost here? Is
> it purely the increased attack surface to the kernel or some more
> fundamental issue? Is this something that can be addressed with a
> vhost-user backend where all the complications of the host end are dealt
> with in user-space?

That's a question for Zach or Stéphane, but AFAIU it has to do with the
increased attack surface to the host kernel. Firecraker (which, IIUC, is
a fork of crosvm) had the same request [1] and implemented a !vhost
vsock backend [2] to address that.

[1]https://github.com/firecracker-microvm/firecracker/issues/650
[2]https://github.com/firecracker-microvm/firecracker/pull/1176

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,
 
> So, I'm about to start working on virtio-pipe (I realize the name is
> not that great since pipes are normally unidirectional, but I'm sure
> we'll have plenty of time to bikeshed on that particular aspect once
> the other bits are sorted out :)).

virtio-ipc?

> This device would be a singleton
> (there can only be one per VM),

Why?

Alex already mentioned vhost-user.  This is basically a way to emulate
virtio devices outside qemu (not sure whenever other vmms support that
too).  With vhost emulation is done in the kernel, with vhost-user
emulation is done in another process instead.  Communication with qemu
runs over a socked with messages roughly equivalent to vhost ioctls.

There is a vhost-user implementation of virtio-gpu (see
contrib/vhost-user-gpu/ in qemu git repo).  Main motivation is that a
separate process can have a much stricter sandbox (no need to have
access to network or guest disk images), which is especially useful for
a complex thing like 3D rendering.

So one possible approach for the host side implementation would be to
write the virtio protocol parser as support library, then implement the
host applications (i.e. the host wayland proxy) as vhost-user process
using that library.

That would not work well with the singleton device approach though.

A vhost-user approach would allow for multiple vmms sharing the
implementation.  It would allow to easily pick a language other
than C (mostly relevant for qemu, crosvm isn't C anyway ...).

> * manage a central UUID <-> 'struct file' map that allows virtio-pipe
>   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
>   UUIDs back to FDs on the other end
>   - we need to expose an API to let each subsystem register/unregister
>     their UUID <-> FD mapping (subsystems are responsible for the UUID
>     creation/negotiation)

Do you have a rough plan how to do that?
On the guest side?
On the host side?

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Wed, 26 Feb 2020 16:12:24 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
>  
> > So, I'm about to start working on virtio-pipe (I realize the name is
> > not that great since pipes are normally unidirectional, but I'm sure
> > we'll have plenty of time to bikeshed on that particular aspect once
> > the other bits are sorted out :)).  
> 
> virtio-ipc?
> 
> > This device would be a singleton
> > (there can only be one per VM),  
> 
> Why?
> 
> Alex already mentioned vhost-user.  This is basically a way to emulate
> virtio devices outside qemu (not sure whenever other vmms support that
> too).  With vhost emulation is done in the kernel, with vhost-user
> emulation is done in another process instead.  Communication with qemu
> runs over a socked with messages roughly equivalent to vhost ioctls.
> 
> There is a vhost-user implementation of virtio-gpu (see
> contrib/vhost-user-gpu/ in qemu git repo).  Main motivation is that a
> separate process can have a much stricter sandbox (no need to have
> access to network or guest disk images), which is especially useful for
> a complex thing like 3D rendering.

Okay, thanks for this explanation.

> 
> So one possible approach for the host side implementation would be to
> write the virtio protocol parser as support library, then implement the
> host applications (i.e. the host wayland proxy) as vhost-user process
> using that library.
> 
> That would not work well with the singleton device approach though.

Hm, so you'd have several virtio-ipc devices exposed to the guest and
the guest user would have to select one, or would the guest only see
one virtio-ipc device? When I said singleton it was from the guest
perspective (though I thought this would also apply to the host).

> 
> A vhost-user approach would allow for multiple vmms sharing the
> implementation.  It would allow to easily pick a language other
> than C (mostly relevant for qemu, crosvm isn't C anyway ...).
> 
> > * manage a central UUID <-> 'struct file' map that allows virtio-pipe
> >   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
> >   UUIDs back to FDs on the other end
> >   - we need to expose an API to let each subsystem register/unregister
> >     their UUID <-> FD mapping (subsystems are responsible for the UUID
> >     creation/negotiation)  
> 
> Do you have a rough plan how to do that?
> On the guest side?
> On the host side?

On the guest side yes, but I need to familiarize a bit more with the
host side of things. This part is not entirely clear yet, but it would
probably involve the same kind of central UUID <-> FD tracking with an
API to add/remove a mapping.


                    |
          host      |         guest
 ____________       |       ____________
|            |      |      |            |
| virtio-ipc |<---link2--->| virtio-ipc |
|____________|      |      |____________|
      A             |             A
      |             |             |
     link1          |            link4
      |             |             |
 _____|______       |       ______|_____
|            |      |      |            |
| virtio-gpu |<---link3--->| virtio-gpu |
|____________|      |      |____________|
                    |


guest side prime export + FD transfer on an IPC connection:
-----------------------------------------------------------

* guest_virtio_gpu->prime_handle_to_fd()
  + virtio_gpu_resource_to_uuid() request sent on link3 to get a UUID
    - virglrenderer_get_resource_uuid() (the UUID is generated if it
      doesn't exist yet)
    - host_virtio_gpu exports the resource to a pseudo-'struct file'
      representation ('struct file' only exists kernel side, but maybe
      we can add the concept of resource on the host user space side
      and define an interface for this resource object?)
      We'll also need resource objs to be exported as plain FDs so we
      can pass them through a unix socket.
    - host_virtio_gpu calls host_virtio_ipc_add_mapping(UUID, host_side_res_obj)
      on link4
  + guest_virtio_gpu calls guest_virtio_ipc_add_mapping(UUID, file)

* guest_virtio_ipc->ioctl(create_connection, service_name_or_path)
  + virtio_ipc_create_connection() request on link2
    - create an ipc connection obj
    - open unix socket at service_name_or_path
    - bridge the ipc connection send/recvmsg requests to the  local
      unix socket
[optional, only needed if we want to pass the connection FD to the host
  + virtio_ipc_connection_to_uuid() request sent on link2
    - create/get the UUID for the ipc connection object
    - attach the UUID to the ipc connection resource obj using
      host_virtio_ipc_add_mapping(UUID, host_side_connection_obj)
      (both virtio_gpu_res and virtio_ipc_conn should implement a
      common virtio_ipc_resource interface which needs to be defined)
  + guest_virtio_ipc calls
    guest_virtio_ipc_add_mapping(UUID, file)
]

* guest_virtio_ipc_connection->ioctl(sendmsg, in-band-data, fds)
  + translate FDs/'struct file's to UUIDs (error out if any of the
    mappings does not exists)
  + virtio_ipc_connection_send_msg(connection, in-band-data, UUIDs)
    request sent on link2
    - translate UUIDs back to resource objects + get a plain FDs out
      of the resource objects (->get_fd() method ?)
    - call sendmsg() on the unix sock


I'm still unsure how FDs coming from a host application can be
converted to resource objects (and then UUIDs so they can be passed
to the ipc_connection) if they've not been previously created/passed
by the guest though. If we want to support that, we probably need to
expose new per-subsystem interfaces to import resources in a VM (those
methods would create a new FD <-> resource mapping for each imported
resource). Is that a use case we care about?

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[David Stevens]

> * manage a central UUID <-> 'struct file' map that allows virtio-pipe
>   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
>   UUIDs back to FDs on the other end
>   - we need to expose an API to let each subsystem register/unregister
>     their UUID <-> FD mapping (subsystems are responsible for the UUID
>     creation/negotiation)

Can you provide more detail about the envisioned scope of this
framework? What sort of FDs and subsystems will use the framework? How
do operations on the guest FD affect the host FD, and vice versa?

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Thu, 27 Feb 2020 13:20:51 +0900
David Stevens <stevensd@chromium.org> wrote:

> > * manage a central UUID <-> 'struct file' map that allows virtio-pipe
> >   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
> >   UUIDs back to FDs on the other end
> >   - we need to expose an API to let each subsystem register/unregister
> >     their UUID <-> FD mapping (subsystems are responsible for the UUID
> >     creation/negotiation)  
> 
> Can you provide more detail about the envisioned scope of this
> framework?

The scope is "generic message+FD passing" interface, which is pretty
much what virtio-wl provides.

> What sort of FDs and subsystems will use the framework?

Basically any subsystem/driver that wants to share resources with the
other end (host/guest depending on where you place yourself):
virtio-gpu, virtio-vdev, virtio-fs, ...

As for the sort of FDs, it can be any kind of resources that needs to
be shared: dmabuf, virtio-ipc-connection, opened-file, ...

> How
> do operations on the guest FD affect the host FD, and vice versa?

Depends what you mean by operations. If we're talking about regular
read/write/ioctl/mmap operations on the guest side, it's up to the
subsystem/driver to implement the expected behavior. Note that on the
host side, we won't manipulate plain FDs since we are in userspace and
the 'struct file' abstraction does not exists there. There will be
local-FD <-> resource mappings though, but how this is done is
resource-type dependent (for IPC FDs we create a unix socket and bridge
it to the virtio-ipc connection, for video buffers we'll ask the
gpu/videodev to export the resource as a dmabuf, ...).

What's not entirely clear is whether we need to support cases where the
host create its own resources (without being asked by the guest) and
decides to pass them to a virtio-ipc-connection (through the unix <->
virtio-ipc bridge). If that's needed, we need interfaces to let host
apps import their resources in the VM (again, those interfaces are
likely to be resource-type dependent).

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Thu, 27 Feb 2020 01:22:11 -0800
Zach Reizner <zachr@google.com> wrote:

> On Thu, Feb 27, 2020 at 1:09 AM Boris Brezillon
> <boris.brezillon@collabora.com> wrote:
> >
> > What's not entirely clear is whether we need to support cases where the
> > host create its own resources (without being asked by the guest) and
> > decides to pass them to a virtio-ipc-connection (through the unix <->
> > virtio-ipc bridge). If that's needed, we need interfaces to let host
> > apps import their resources in the VM (again, those interfaces are
> > likely to be resource-type dependent).  
> 
> When the wayland compositor sends the keyboard map or requests
> clipboard content, the compositor (or another client on the host) will
> create a resource (shm or pipe) and send it over the unix domain
> socket. Would this be a case of the above situation you mentioned
> above? It seems like we need to support the host creating resources
> passed over the virtio-ipc-connection.

Okay. There's also another problem I noticed just now: every time an FD
is passed on a unix socket a new FD is created on the receiver's side,
even if the receiver already has an opened-FD pointing to the exact
same file, which means we can't easily create an FD <-> virtio-resource
mapping on the host (there's this kcmp() [1] syscall, but it doesn't
seem to have a wrapper in glibc which is not a good sign, plus you need
to enable CONFIG_CHECKPOINT_RESTORE).

[1]http://man7.org/linux/man-pages/man2/kcmp.2.html

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> > So one possible approach for the host side implementation would be to
> > write the virtio protocol parser as support library, then implement the
> > host applications (i.e. the host wayland proxy) as vhost-user process
> > using that library.
> > 
> > That would not work well with the singleton device approach though.
> 
> Hm, so you'd have several virtio-ipc devices exposed to the guest and
> the guest user would have to select one,

Yes.  We would have to attach names to the devices so the guest can
pick the one it wants, something like protocol=org.freedesktop.wayland

> or would the guest only see one virtio-ipc device? When I said
> singleton it was from the guest perspective (though I thought this
> would also apply to the host).

Yes, you'll have the same model on both guest and host.

A singleton device would work too, but would require an extra middle man
on the host side.  The vmm directly (qemu/crosvm), or a vhost-user
process, or a vhost kernel module would have to dispatch messages to the
actual applications (wayland proxy, ...).

A protocol-specific vhost-user process could drive the virtio rings
directly instead.  Of course this approach has its drawbacks too.
Porting applications is probably easier if the familiar socket interface
is available.

> guest side prime export + FD transfer on an IPC connection:
> -----------------------------------------------------------
> 
> * guest_virtio_gpu->prime_handle_to_fd()
>   + virtio_gpu_resource_to_uuid() request sent on link3 to get a UUID
>     - virglrenderer_get_resource_uuid() (the UUID is generated if it
>       doesn't exist yet)
>     - host_virtio_gpu exports the resource to a pseudo-'struct file'
>       representation ('struct file' only exists kernel side, but maybe
>       we can add the concept of resource on the host user space side
>       and define an interface for this resource object?)

Yep, that needs be sorted.  If everything happens inside the same
process this should be easy, it'll be just an vmm implementation detail.

Supporting multi-process vmms (see vhost-user) will be more tricky.
Let the vmm manage in this case too?  Or should the kernel help here?
Or should we have a dbus service for that?  Other ideas?

>       We'll also need resource objs to be exported as plain FDs so we
>       can pass them through a unix socket.

virtio-gpu would use dma-bufs for that.

> I'm still unsure how FDs coming from a host application can be
> converted to resource objects (and then UUIDs so they can be passed
> to the ipc_connection) if they've not been previously created/passed
> by the guest though.

Depends on the kind of fd I think.  For unix sockets or virtio-fs files
it should be easy.  dma-bufs or sysv shmem would be more tricky because
we have to map them into the guest address space (if we want support
that).  Not impossible though, we could use the new shared memory
support for that.

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[David Stevens]

On Thu, Feb 27, 2020 at 6:09 PM Boris Brezillon
<boris.brezillon@collabora.com> wrote:
>
> On Thu, 27 Feb 2020 13:20:51 +0900
> David Stevens <stevensd@chromium.org> wrote:
>
> > > * manage a central UUID <-> 'struct file' map that allows virtio-pipe
> > >   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
> > >   UUIDs back to FDs on the other end
> > >   - we need to expose an API to let each subsystem register/unregister
> > >     their UUID <-> FD mapping (subsystems are responsible for the UUID
> > >     creation/negotiation)
> >
> > Can you provide more detail about the envisioned scope of this
> > framework?
>
> The scope is "generic message+FD passing" interface, which is pretty
> much what virtio-wl provides.

I think that scope is too broad. A socket is a 'generic message+FD'
interface. Unless there's the expectation that the interface should
eventually be as flexible as a regular domain socket, I think it would
be a good idea to frame the scope of the interface more precisely.

Part of this ambiguity comes from the informal usage of the term 'FD'.
An FD is a concept in Linux and other operating systems (and not even
all operating systems - e.g. Fuchsia). At present, FDs are not a
concept in virtio. Talking about sending FDs over virtio handwaves a
lot of details about what that's actually going on.

> > How
> > do operations on the guest FD affect the host FD, and vice versa?
>
> Depends what you mean by operations. If we're talking about regular
> read/write/ioctl/mmap operations on the guest side, it's up to the
> subsystem/driver to implement the expected behavior.

I think part of my confusion comes from the fact that virtio-wayland
seems to provide both the IPC mechanism described for virtio-ipc as
well as some additional guest/host file sharing support. If that is
actually the case, then I guess a striped down version of
virtio-wayland would still be necessary.

-David

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Thu, 27 Feb 2020 20:14:22 +0900
David Stevens <stevensd@chromium.org> wrote:

> On Thu, Feb 27, 2020 at 6:09 PM Boris Brezillon
> <boris.brezillon@collabora.com> wrote:
> >
> > On Thu, 27 Feb 2020 13:20:51 +0900
> > David Stevens <stevensd@chromium.org> wrote:
> >  
> > > > * manage a central UUID <-> 'struct file' map that allows virtio-pipe
> > > >   to convert FDs to UUIDs, pass UUIDs through a pipe and convert those
> > > >   UUIDs back to FDs on the other end
> > > >   - we need to expose an API to let each subsystem register/unregister
> > > >     their UUID <-> FD mapping (subsystems are responsible for the UUID
> > > >     creation/negotiation)  
> > >
> > > Can you provide more detail about the envisioned scope of this
> > > framework?  
> >
> > The scope is "generic message+FD passing" interface, which is pretty
> > much what virtio-wl provides.  
> 
> I think that scope is too broad. A socket is a 'generic message+FD'
> interface. Unless there's the expectation that the interface should
> eventually be as flexible as a regular domain socket, I think it would
> be a good idea to frame the scope of the interface more precisely.

Generic message passing still stands I think (generic as in
protocol-agnostic). For the FD part, of course we won't support all
kind of FDs on day 1, but the idea is to abstract things so we can
extend the solution easily.

> 
> Part of this ambiguity comes from the informal usage of the term 'FD'.
> An FD is a concept in Linux and other operating systems (and not even
> all operating systems - e.g. Fuchsia). At present, FDs are not a
> concept in virtio. Talking about sending FDs over virtio handwaves a
> lot of details about what that's actually going on.

Correct, we're actually not passing Linux FDs at the virtio level,
we're passing resource handles, but I note that those handles are
called VFD (Virtual File Descriptor) in the virtio-wayland too :).

> 
> > > How
> > > do operations on the guest FD affect the host FD, and vice versa?  
> >
> > Depends what you mean by operations. If we're talking about regular
> > read/write/ioctl/mmap operations on the guest side, it's up to the
> > subsystem/driver to implement the expected behavior.  
> 
> I think part of my confusion comes from the fact that virtio-wayland
> seems to provide both the IPC mechanism described for virtio-ipc as
> well as some additional guest/host file sharing support.

I guess that'd be another kind of resource, and would require a
specific implementation, yes. Is it related to what Gerd was mentioning
with virtio-fs?

> If that is
> actually the case, then I guess a striped down version of
> virtio-wayland would still be necessary.

Maybe, until we add support for this file sharing feature upstream.
I'll have a closer look at how file sharing is exposed by virtio-wl.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Thu, 27 Feb 2020 11:29:17 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

> > I'm still unsure how FDs coming from a host application can be
> > converted to resource objects (and then UUIDs so they can be passed
> > to the ipc_connection) if they've not been previously created/passed
> > by the guest though.  
> 
> Depends on the kind of fd I think.  For unix sockets or virtio-fs files
> it should be easy.  dma-bufs or sysv shmem would be more tricky because
> we have to map them into the guest address space (if we want support
> that).  Not impossible though, we could use the new shared memory
> support for that.

I was more worried about the complexity an import_resource() API would
put on the host proxy: it will have to know which subsystem each of the
resources/FDs it tries to pass to the virtio-ipc socket should be
attached to, and none of those resources have actually been created by
the proxy itself. That implies having a way to get that information from
a FD (is it a dmabuf FD, a unix socket fd, ... and depending on the kind
of FD, which device/subsystem is in charge of this FD, for instance, for
dmabufs, you might need to know who allocated the buffer).



---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> > > Can you provide more detail about the envisioned scope of this
> > > framework?
> >
> > The scope is "generic message+FD passing" interface, which is pretty
> > much what virtio-wl provides.
> 
> I think that scope is too broad. A socket is a 'generic message+FD'
> interface. Unless there's the expectation that the interface should
> eventually be as flexible as a regular domain socket, I think it would
> be a good idea to frame the scope of the interface more precisely.

Yes, sure, we need to exactly specify the different kinds of file
handles / resources.  I think it makes sense to have a virtio feature
flag for each of them, so guest+host can easily negotiate what they are
able to handle and what not.

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

Hi Gerd,

On Thu, 27 Feb 2020 15:43:22 +0100
Gerd Hoffmann <kraxel@redhat.com> wrote:

>   Hi,
> 
> > > > Can you provide more detail about the envisioned scope of this
> > > > framework?  
> > >
> > > The scope is "generic message+FD passing" interface, which is pretty
> > > much what virtio-wl provides.  
> > 
> > I think that scope is too broad. A socket is a 'generic message+FD'
> > interface. Unless there's the expectation that the interface should
> > eventually be as flexible as a regular domain socket, I think it would
> > be a good idea to frame the scope of the interface more precisely.  
> 
> Yes, sure, we need to exactly specify the different kinds of file
> handles / resources. I think it makes sense to have a virtio feature
> flag for each of them, so guest+host can easily negotiate what they are
> able to handle and what not.

I was expecting that to be a feature of the resource producers
(virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc
itself. If we go for a model where UUID <-> resource/'struct file'
mappings are created by the subsystems that are in charge of those
resources, it's hard for virtio-ipc to know what kind of resources can
be passed in advance.

Regards,

Boris

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

  Hi,

> > Yes, sure, we need to exactly specify the different kinds of file
> > handles / resources. I think it makes sense to have a virtio feature
> > flag for each of them, so guest+host can easily negotiate what they are
> > able to handle and what not.
> 
> I was expecting that to be a feature of the resource producers
> (virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc
> itself.

"resources from other virtio devices" would be one virtio-ipc feature
flag.  And, yes, that would for the most part have the other device
handle the problem.

But there also is "unix socket", or maybe a somewhat broader "stream",
which would be another feature flag I guess because virtio-ipc would
just tunnel the stream without the help from other devices.

Possibly there will be more ...

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[David Stevens]

> > > Yes, sure, we need to exactly specify the different kinds of file
> > > handles / resources. I think it makes sense to have a virtio feature
> > > flag for each of them, so guest+host can easily negotiate what they are
> > > able to handle and what not.
> >
> > I was expecting that to be a feature of the resource producers
> > (virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc
> > itself.
>
> "resources from other virtio devices" would be one virtio-ipc feature
> flag.  And, yes, that would for the most part have the other device
> handle the problem.
>
> But there also is "unix socket", or maybe a somewhat broader "stream",
> which would be another feature flag I guess because virtio-ipc would
> just tunnel the stream without the help from other devices.

Can you elaborate on what you mean by this? I can envision how
virtio-ipc would be a generic mechanism for passing data+virtio
resources, including any new types of resources it itself defines.
However, if "unix sockets" or a generic "stream" expands beyond
virtio, that seems too broad, with too many edge cases to be feasible
to implement.

-David

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Gerd Hoffmann]

On Fri, Feb 28, 2020 at 07:11:40PM +0900, David Stevens wrote:
> > But there also is "unix socket", or maybe a somewhat broader "stream",
> > which would be another feature flag I guess because virtio-ipc would
> > just tunnel the stream without the help from other devices.
> 
> Can you elaborate on what you mean by this? I can envision how
> virtio-ipc would be a generic mechanism for passing data+virtio
> resources, including any new types of resources it itself defines.
> However, if "unix sockets" or a generic "stream" expands beyond
> virtio, that seems too broad, with too many edge cases to be feasible
> to implement.

As far I know this is exactly what virtio-wayland does today if you try
to pass a unix socket file descriptor to the other side, so I assume
this functionality is needed ...

cheers,
  Gerd


---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Fri, 28 Feb 2020 19:11:40 +0900
David Stevens <stevensd@chromium.org> wrote:

> > > > Yes, sure, we need to exactly specify the different kinds of file
> > > > handles / resources. I think it makes sense to have a virtio feature
> > > > flag for each of them, so guest+host can easily negotiate what they are
> > > > able to handle and what not.  
> > >
> > > I was expecting that to be a feature of the resource producers
> > > (virtio-gpu, virtio-fs, ...) rather than a feature of virtio-ipc
> > > itself.  
> >
> > "resources from other virtio devices" would be one virtio-ipc feature
> > flag.  And, yes, that would for the most part have the other device
> > handle the problem.
> >
> > But there also is "unix socket", or maybe a somewhat broader "stream",
> > which would be another feature flag I guess because virtio-ipc would
> > just tunnel the stream without the help from other devices.  
> 
> Can you elaborate on what you mean by this? I can envision how
> virtio-ipc would be a generic mechanism for passing data+virtio
> resources, including any new types of resources it itself defines.
> However, if "unix sockets" or a generic "stream" expands beyond
> virtio, that seems too broad, with too many edge cases to be feasible
> to implement.

I don't think we need to bridge unix sockets or any kind of stream
interface, like pipes, regular sockets, ... in kernel space. If
virtio-ipc provides a way to create anonymous virtio-ipc connections
whose FDs can be passed to a opened virtio-ipc connection, we can
implement those bridges in user space. fstat() allows us to know what
kind of FD we're receiving from the unix socket (socket, regular
file, fifo), and for sockets, we even have
getsockopt({SO_DOMAIN,SO_TYPE}) to get a more precise information.

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[David Stevens]

On Fri, Feb 28, 2020 at 7:30 PM Gerd Hoffmann <kraxel@redhat.com> wrote:
>
> On Fri, Feb 28, 2020 at 07:11:40PM +0900, David Stevens wrote:
> > > But there also is "unix socket", or maybe a somewhat broader "stream",
> > > which would be another feature flag I guess because virtio-ipc would
> > > just tunnel the stream without the help from other devices.
> >
> > Can you elaborate on what you mean by this? I can envision how
> > virtio-ipc would be a generic mechanism for passing data+virtio
> > resources, including any new types of resources it itself defines.
> > However, if "unix sockets" or a generic "stream" expands beyond
> > virtio, that seems too broad, with too many edge cases to be feasible
> > to implement.
>
> As far I know this is exactly what virtio-wayland does today if you try
> to pass a unix socket file descriptor to the other side, so I assume
> this functionality is needed ...

As Zach said earlier, virtio-wayland implements just enough FD sharing
support to get wayland working. However, there are many other
situations where virtio-wayland's unix socket support isn't
sufficient. I think adding support for sharing unix sockets to a
generic virtio-ipc would imply that the support is more comprehensive
than would be feasible to implement. So while virtio-ipc should
support wayland, either directly or with the support of
userspace/another virtio device, it also shouldn't overpromise what it
is capable of doing.

-David

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org

Re: [virtio-dev] [RFC] Upstreaming virtio-wayland (or an alternative)

[Boris Brezillon]

On Mon, 2 Mar 2020 11:24:50 +0900
David Stevens <stevensd@chromium.org> wrote:

> On Fri, Feb 28, 2020 at 7:30 PM Gerd Hoffmann <kraxel@redhat.com> wrote:
> >
> > On Fri, Feb 28, 2020 at 07:11:40PM +0900, David Stevens wrote:  
> > > > But there also is "unix socket", or maybe a somewhat broader "stream",
> > > > which would be another feature flag I guess because virtio-ipc would
> > > > just tunnel the stream without the help from other devices.  
> > >
> > > Can you elaborate on what you mean by this? I can envision how
> > > virtio-ipc would be a generic mechanism for passing data+virtio
> > > resources, including any new types of resources it itself defines.
> > > However, if "unix sockets" or a generic "stream" expands beyond
> > > virtio, that seems too broad, with too many edge cases to be feasible
> > > to implement.  
> >
> > As far I know this is exactly what virtio-wayland does today if you try
> > to pass a unix socket file descriptor to the other side, so I assume
> > this functionality is needed ...  
> 
> As Zach said earlier, virtio-wayland implements just enough FD sharing
> support to get wayland working. However, there are many other
> situations where virtio-wayland's unix socket support isn't
> sufficient. I think adding support for sharing unix sockets to a
> generic virtio-ipc would imply that the support is more comprehensive
> than would be feasible to implement. So while virtio-ipc should
> support wayland, either directly or with the support of
> userspace/another virtio device, it also shouldn't overpromise what it
> is capable of doing.

I feel like some of the terms I used were confusing, but I think we're
on the same page when it comes to the actual implementation and
features we want to see supported by this virtio-ipc device. So, how
about rephrasing it this way:

"
virtio-ipc aims at providing a protocol-agnostic message passing
interface with extensible resource-passing capabilities. The initial
implementation should cover the wayland use case, and as such,
should support passing the following type of resources:

* opened virtio-ipc connections (similar to passing an FD to one end of
  a unix pipe, except communication would be bi-directional). With this
  feature we can let user space implement all sort of proxies
  (sockets <-> virtio-ipc, pipes <-> virtio-ipc, ...)
* virtio-gpu dmabufs
* ... [need to look in more details what's needed for keymap and file
  sharing]
"

Note that it's not that far from what virtio-wayland provides today with
2 main differences:

1/ the name of the device no longer implies that the interface is
   wayland-specific
2/ virtio-ipc is resource-type agnostic, and FD <-> resource ID mapping
   creation is left to the subsystem handling the resource. That means
   we shouldn't see this sort of 'convert FD to res-type X' logic [1]
   in the virtio-ipc driver, which should make it easier to extend while
   keeping the code base small enough/sane, no matter how far we
   decide to go with resource passing.

[1]https://chromium.googlesource.com/chromiumos/third_party/kernel/+/chromeos-4.19/drivers/virtio/virtio_wl.c#786

---------------------------------------------------------------------
To unsubscribe, e-mail: virtio-dev-unsubscribe@lists.oasis-open.org
For additional commands, e-mail: virtio-dev-help@lists.oasis-open.org