Re: KVM Forum VFIO BoF summary

Re: KVM Forum VFIO BoF summary

[Felipe Franciosi]

Hi Alex,

I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).

> On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@redhat.com> wrote:
> 
> There were mostly two main threads of discussion, the first was Filipe's
> discussion of a socket interface making use of the VFIO ABI to
> implement a userspace device model.  For a VM use case, QEMU would be
> modified to use a socket for the VFIO ABI, including shared memory for
> DMA.  Ideally this would be transparent for much of the QEMU vfio code
> outside of setup.  Effectively this becomes a userspace implementation
> of mdev as this could already be done using mdev, but it requires a
> vendor driver to expose the userspace interface and likely a longer
> round trip through the kernel and back to userspace. Either path
> allows device models in userspace that can be open or proprietary, but
> this new proposal eliminates the vendor kernel driver component.  Of
> course non-upstream kernel mdev vendor drivers do taint the host
> kernel, so they at least leave a breadcrumb.

Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.

This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.

As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.

> I believe Filipe
> mentioned a conversation after the BoF with Alex Graf who had a related
> concern about using the socket approach to use this as a non-GPL
> backdoor for device models in QEMU and a suggested approach was a GPL
> handshake via the socket interface.

Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.

Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.

The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.

I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).

This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.

Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).

Thanks,
Felipe

Re: [Qemu-devel] KVM Forum VFIO BoF summary

[Felipe Franciosi]

Hi Alex,

I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).

> On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@redhat.com> wrote:
> 
> There were mostly two main threads of discussion, the first was Filipe's
> discussion of a socket interface making use of the VFIO ABI to
> implement a userspace device model.  For a VM use case, QEMU would be
> modified to use a socket for the VFIO ABI, including shared memory for
> DMA.  Ideally this would be transparent for much of the QEMU vfio code
> outside of setup.  Effectively this becomes a userspace implementation
> of mdev as this could already be done using mdev, but it requires a
> vendor driver to expose the userspace interface and likely a longer
> round trip through the kernel and back to userspace. Either path
> allows device models in userspace that can be open or proprietary, but
> this new proposal eliminates the vendor kernel driver component.  Of
> course non-upstream kernel mdev vendor drivers do taint the host
> kernel, so they at least leave a breadcrumb.

Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.

This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.

As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.

> I believe Filipe
> mentioned a conversation after the BoF with Alex Graf who had a related
> concern about using the socket approach to use this as a non-GPL
> backdoor for device models in QEMU and a suggested approach was a GPL
> handshake via the socket interface.

Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.

Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.

The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.

I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).

This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.

Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).

Thanks,
Felipe

Re: KVM Forum VFIO BoF summary

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 5586 bytes --]

On Fri, Nov 23, 2018 at 10:09:53PM +0000, Felipe Franciosi wrote:
> Hi Alex,
> 
> I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).
> 
> > On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@redhat.com> wrote:
> > 
> > There were mostly two main threads of discussion, the first was Filipe's
> > discussion of a socket interface making use of the VFIO ABI to
> > implement a userspace device model.  For a VM use case, QEMU would be
> > modified to use a socket for the VFIO ABI, including shared memory for
> > DMA.  Ideally this would be transparent for much of the QEMU vfio code
> > outside of setup.  Effectively this becomes a userspace implementation
> > of mdev as this could already be done using mdev, but it requires a
> > vendor driver to expose the userspace interface and likely a longer
> > round trip through the kernel and back to userspace. Either path
> > allows device models in userspace that can be open or proprietary, but
> > this new proposal eliminates the vendor kernel driver component.  Of
> > course non-upstream kernel mdev vendor drivers do taint the host
> > kernel, so they at least leave a breadcrumb.
> 
> Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.
> 
> This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.
> 
> As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.
> 
> > I believe Filipe
> > mentioned a conversation after the BoF with Alex Graf who had a related
> > concern about using the socket approach to use this as a non-GPL
> > backdoor for device models in QEMU and a suggested approach was a GPL
> > handshake via the socket interface.
> 
> Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.
> 
> Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.
> 
> The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.
> 
> I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).
> 
> This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.
> 
> Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).

Device emulation is the core of what GPL QEMU does.  Mechanisms for
out-of-tree device emulation get rejected because they are likely to be
misused by parties who do not wish to contribute to QEMU.

That would result in the QEMU community maintaining infrastructure for
third parties who ship code of questionable quality and profit from it.
That's not healthy for the QEMU community and there's no incentive to
allow it.

A bit more about specifics you mentioned:

vhost-user is an exception for vhost devices.  They are based on the
open VIRTIO specification.  QEMU has in-tree implementations of these
devices so users can fall back to them if they need to request support
(otherwise it's impossible to support users with third-party devices).

Generic out-of-tree PCI device emulation is way too big a loophole.  The
QEMU community cannot support users who run arbitrary third-party PCI
devices.

The fact that there are ways to get around this by writing software mdev
devices doesn't mean it makes sense for QEMU to open a loophole itself.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: [Qemu-devel] KVM Forum VFIO BoF summary

[Stefan Hajnoczi]

[-- Attachment #1: Type: text/plain, Size: 5586 bytes --]

On Fri, Nov 23, 2018 at 10:09:53PM +0000, Felipe Franciosi wrote:
> Hi Alex,
> 
> I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).
> 
> > On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@redhat.com> wrote:
> > 
> > There were mostly two main threads of discussion, the first was Filipe's
> > discussion of a socket interface making use of the VFIO ABI to
> > implement a userspace device model.  For a VM use case, QEMU would be
> > modified to use a socket for the VFIO ABI, including shared memory for
> > DMA.  Ideally this would be transparent for much of the QEMU vfio code
> > outside of setup.  Effectively this becomes a userspace implementation
> > of mdev as this could already be done using mdev, but it requires a
> > vendor driver to expose the userspace interface and likely a longer
> > round trip through the kernel and back to userspace. Either path
> > allows device models in userspace that can be open or proprietary, but
> > this new proposal eliminates the vendor kernel driver component.  Of
> > course non-upstream kernel mdev vendor drivers do taint the host
> > kernel, so they at least leave a breadcrumb.
> 
> Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.
> 
> This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.
> 
> As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.
> 
> > I believe Filipe
> > mentioned a conversation after the BoF with Alex Graf who had a related
> > concern about using the socket approach to use this as a non-GPL
> > backdoor for device models in QEMU and a suggested approach was a GPL
> > handshake via the socket interface.
> 
> Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.
> 
> Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.
> 
> The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.
> 
> I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).
> 
> This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.
> 
> Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).

Device emulation is the core of what GPL QEMU does.  Mechanisms for
out-of-tree device emulation get rejected because they are likely to be
misused by parties who do not wish to contribute to QEMU.

That would result in the QEMU community maintaining infrastructure for
third parties who ship code of questionable quality and profit from it.
That's not healthy for the QEMU community and there's no incentive to
allow it.

A bit more about specifics you mentioned:

vhost-user is an exception for vhost devices.  They are based on the
open VIRTIO specification.  QEMU has in-tree implementations of these
devices so users can fall back to them if they need to request support
(otherwise it's impossible to support users with third-party devices).

Generic out-of-tree PCI device emulation is way too big a loophole.  The
QEMU community cannot support users who run arbitrary third-party PCI
devices.

The fact that there are ways to get around this by writing software mdev
devices doesn't mean it makes sense for QEMU to open a loophole itself.

Stefan

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 455 bytes --]

Re: KVM Forum VFIO BoF summary

[Alexander Graf]

On 11/26/2018 01:59 PM, Stefan Hajnoczi wrote:
> On Fri, Nov 23, 2018 at 10:09:53PM +0000, Felipe Franciosi wrote:
>> Hi Alex,
>>
>> I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).
>>
>>> On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@redhat.com> wrote:
>>>
>>> There were mostly two main threads of discussion, the first was Filipe's
>>> discussion of a socket interface making use of the VFIO ABI to
>>> implement a userspace device model.  For a VM use case, QEMU would be
>>> modified to use a socket for the VFIO ABI, including shared memory for
>>> DMA.  Ideally this would be transparent for much of the QEMU vfio code
>>> outside of setup.  Effectively this becomes a userspace implementation
>>> of mdev as this could already be done using mdev, but it requires a
>>> vendor driver to expose the userspace interface and likely a longer
>>> round trip through the kernel and back to userspace. Either path
>>> allows device models in userspace that can be open or proprietary, but
>>> this new proposal eliminates the vendor kernel driver component.  Of
>>> course non-upstream kernel mdev vendor drivers do taint the host
>>> kernel, so they at least leave a breadcrumb.
>> Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.
>>
>> This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.
>>
>> As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.
>>
>>> I believe Filipe
>>> mentioned a conversation after the BoF with Alex Graf who had a related
>>> concern about using the socket approach to use this as a non-GPL
>>> backdoor for device models in QEMU and a suggested approach was a GPL
>>> handshake via the socket interface.
>> Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.
>>
>> Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.
>>
>> The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.
>>
>> I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).
>>
>> This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.
>>
>> Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).
> Device emulation is the core of what GPL QEMU does.  Mechanisms for
> out-of-tree device emulation get rejected because they are likely to be
> misused by parties who do not wish to contribute to QEMU.

Are you sure? Maybe people just want to use out of tree device emulation 
because it allows for sophisticated use cases:

   * RTL simulation (see the Oracle talk on this from KVM Forum a few 
years back)
   * Security isolation (eg an SPDK based target, you don't want to give 
all of QEMU access to the SPDK device)
   * Reuse other OSS projects (share device emulation between different 
emulators)
   * Share emulation with a big broker entity (which could then poll 
every VM on the entire system on 1 core, giving possibly lower latencies)
   * Different programming language (say I want to implement a device in 
rust, because it's more secure, or easier to prototype with)

I can see the FUD around closed source and terrible quality of out of 
tree devices, but that doesn't mean they are the only reason people want 
this. And I also don't think it's a good reason to hold back 
compartmented device emulation.

> That would result in the QEMU community maintaining infrastructure for
> third parties who ship code of questionable quality and profit from it.

You mean like we do today for the Ceph block module? In-tree vs 
out-of-tree doesn't say too much about code quality. And I surely hope 
people understand that a separate host process from a separate entity 
means that the overall product can only be as good as its components.

> That's not healthy for the QEMU community and there's no incentive to
> allow it.

I disagree - I think it's quite healthy, because it allows us to grow 
into use cases we currently neglect or have random downstream projects 
for at this point. There are numerous forks of QEMU to allow RTL 
simulation of external devices for example.

> A bit more about specifics you mentioned:
>
> vhost-user is an exception for vhost devices.  They are based on the
> open VIRTIO specification.  QEMU has in-tree implementations of these
> devices so users can fall back to them if they need to request support
> (otherwise it's impossible to support users with third-party devices).
>
> Generic out-of-tree PCI device emulation is way too big a loophole.  The
> QEMU community cannot support users who run arbitrary third-party PCI
> devices.
>
> The fact that there are ways to get around this by writing software mdev
> devices doesn't mean it makes sense for QEMU to open a loophole itself.

IMHO it would be much more sensible to give people a path we can support 
rather than encourage hacks, where hacks are either:

   * downstream patches to do out of tree emulation
   * mdev based implementations of the same

So how about we give people a reasonable interface that allows them to 
support their sophisticated things without us getting penalized for it? 
What really are the problems you're seeing? From all discussions I know 
so far, it's mostly been:

   1) Code Quality

We can always have a tainted flag. IIRC Gerd already worked on bits for 
unmaintained code inside of QEMU itself. There's little reason to not 
extend this to external modules. We should treat "someone uses random 
emulated device nobody maintains" and "someone uses external code" the 
same way.

   2) GPL

One idea I had here was to enforce licensing via legal methods. Linux 
for example has an explicit tag that allows kernel modules to expose 
their license information to Linux. Based on that, it can expose 
interfaces or refuse to do so. We could do the same - and for starters 
only expose PCIe device emulation to GPL peers.


Alex

Re: [Qemu-devel] KVM Forum VFIO BoF summary

[Alexander Graf]

On 11/26/2018 01:59 PM, Stefan Hajnoczi wrote:
> On Fri, Nov 23, 2018 at 10:09:53PM +0000, Felipe Franciosi wrote:
>> Hi Alex,
>>
>> I'm also terribly sorry for the delay in responding to this. I'm only now having the time resources to come back to this topic and figure out a way forward with my proposal. Please see my notes below (quoting only the relevant sections).
>>
>>> On Nov 6, 2018, at 9:32 PM, Alex Williamson <alex.williamson@redhat.com> wrote:
>>>
>>> There were mostly two main threads of discussion, the first was Filipe's
>>> discussion of a socket interface making use of the VFIO ABI to
>>> implement a userspace device model.  For a VM use case, QEMU would be
>>> modified to use a socket for the VFIO ABI, including shared memory for
>>> DMA.  Ideally this would be transparent for much of the QEMU vfio code
>>> outside of setup.  Effectively this becomes a userspace implementation
>>> of mdev as this could already be done using mdev, but it requires a
>>> vendor driver to expose the userspace interface and likely a longer
>>> round trip through the kernel and back to userspace. Either path
>>> allows device models in userspace that can be open or proprietary, but
>>> this new proposal eliminates the vendor kernel driver component.  Of
>>> course non-upstream kernel mdev vendor drivers do taint the host
>>> kernel, so they at least leave a breadcrumb.
>> Spot on. This can be done today via mdev, but it would unnecessarily involve the kernel in the setup of devices. Also, since vfio is the only existing bus driver for mdev, it would make little sense to implement an mdev "physical" driver (which wouldn't have a physical backing device) just to talk a vfio-like abi back to userspace.
>>
>> This "vfio-user" proposal is a perfect fit for Qemu. I think the correct design is to slightly rearchitect the vfio implementation in Qemu to allow for userspace backends through unix sockets, very much like vhost-user does it for virtio devices today.
>>
>> As a matter of fact, with something like this in place, I think the Qemu code could potentially be simplified by moving some of the existing vhost-user offloading code underneath it.
>>
>>> I believe Filipe
>>> mentioned a conversation after the BoF with Alex Graf who had a related
>>> concern about using the socket approach to use this as a non-GPL
>>> backdoor for device models in QEMU and a suggested approach was a GPL
>>> handshake via the socket interface.
>> Not exactly. The GPL-violating concerns came from Paolo and Stefan (cc'd). Alex Graf (cc'd) came up with a solution for the concern which involves adding a copyrighted "poem" to the protocol handshake. Qemu would then grant the copyright for GPL applications, hence limiting who can use the protocol.
>>
>> Vendors can obviously bypass this by using a GPL proxy application that talks to Qemu, but then uses a separate mechanism (possibly another unix socket) to talk to other non-GPL applications. Vendors which distribute Qemu can possibly also bypass this by modifying the protocol not to include the copyrighted messages. Using either legal loophole approach, however, can lead to a lot of public point and shame.
>>
>> The point I would like to discuss more widely (before going forwards with code) is whether vendors can actually make use of a GPL-only protocol. I understand the community desire of gathering efforts into GPL software, Qemu code and Virtio. A non-GPL implementation means that maybe some vendors cannot use this.
>>
>> I envision this protocol being used by userspace applications which _need_ to emulate a device outside Qemu for performance reasons, not license reasons (eg. a separate process can efficiently poll VQs from multiple VMs from a single core). Applications that already do that today, albeit open-source, are not GPL (eg. OVS, DPDK, SPDK).
>>
>> This is why I'd like to hear more from Paolo and Stefan on why making this non-GPL is so bad. It is already possible today (via mdev), and similar approaches already exist for a large number of device implementations via vhost-user. Additionally, I would also like to hear from willing vendors that feel they could _only_ benefit from this if it does not enforce GPL.
>>
>> Finally, I want to clarify I am not opposing making this GPL-only. I just want to make sure the effort is justifiable (ie. there is reason for this to enforce GPL peer applications given the existing vhost-user and mdev alternatives) and not in vain (ie. there are GPL use cases for this out there).
> Device emulation is the core of what GPL QEMU does.  Mechanisms for
> out-of-tree device emulation get rejected because they are likely to be
> misused by parties who do not wish to contribute to QEMU.

Are you sure? Maybe people just want to use out of tree device emulation 
because it allows for sophisticated use cases:

   * RTL simulation (see the Oracle talk on this from KVM Forum a few 
years back)
   * Security isolation (eg an SPDK based target, you don't want to give 
all of QEMU access to the SPDK device)
   * Reuse other OSS projects (share device emulation between different 
emulators)
   * Share emulation with a big broker entity (which could then poll 
every VM on the entire system on 1 core, giving possibly lower latencies)
   * Different programming language (say I want to implement a device in 
rust, because it's more secure, or easier to prototype with)

I can see the FUD around closed source and terrible quality of out of 
tree devices, but that doesn't mean they are the only reason people want 
this. And I also don't think it's a good reason to hold back 
compartmented device emulation.

> That would result in the QEMU community maintaining infrastructure for
> third parties who ship code of questionable quality and profit from it.

You mean like we do today for the Ceph block module? In-tree vs 
out-of-tree doesn't say too much about code quality. And I surely hope 
people understand that a separate host process from a separate entity 
means that the overall product can only be as good as its components.

> That's not healthy for the QEMU community and there's no incentive to
> allow it.

I disagree - I think it's quite healthy, because it allows us to grow 
into use cases we currently neglect or have random downstream projects 
for at this point. There are numerous forks of QEMU to allow RTL 
simulation of external devices for example.

> A bit more about specifics you mentioned:
>
> vhost-user is an exception for vhost devices.  They are based on the
> open VIRTIO specification.  QEMU has in-tree implementations of these
> devices so users can fall back to them if they need to request support
> (otherwise it's impossible to support users with third-party devices).
>
> Generic out-of-tree PCI device emulation is way too big a loophole.  The
> QEMU community cannot support users who run arbitrary third-party PCI
> devices.
>
> The fact that there are ways to get around this by writing software mdev
> devices doesn't mean it makes sense for QEMU to open a loophole itself.

IMHO it would be much more sensible to give people a path we can support 
rather than encourage hacks, where hacks are either:

   * downstream patches to do out of tree emulation
   * mdev based implementations of the same

So how about we give people a reasonable interface that allows them to 
support their sophisticated things without us getting penalized for it? 
What really are the problems you're seeing? From all discussions I know 
so far, it's mostly been:

   1) Code Quality

We can always have a tainted flag. IIRC Gerd already worked on bits for 
unmaintained code inside of QEMU itself. There's little reason to not 
extend this to external modules. We should treat "someone uses random 
emulated device nobody maintains" and "someone uses external code" the 
same way.

   2) GPL

One idea I had here was to enforce licensing via legal methods. Linux 
for example has an explicit tag that allows kernel modules to expose 
their license information to Linux. Based on that, it can expose 
interfaces or refuse to do so. We could do the same - and for starters 
only expose PCIe device emulation to GPL peers.


Alex