RE: [PATCH RFC 00/15] Add VFIO mediated device support and IMS support for the idxd driver.

["Tian, Kevin" <kevin.tian@intel.com>]

> From: Jason Gunthorpe <jgg@mellanox.com>
> Sent: Saturday, April 25, 2020 2:12 AM
> 
> > > > idxd is just the first device that supports Scalable IOV. We have a
> > > > lot more coming later, in different types. Then putting such
> > > > emulation in user space means that Qemu needs to support all those
> > > > vendor specific interfaces for every new device which supports
> > >
> > > It would be very sad to see an endless amount of device emulation code
> > > crammed into the kernel. Userspace is where device emulation is
> > > supposed to live. For security
> >
> > I think providing an unified abstraction to userspace is also important,
> > which is what VFIO provides today. The merit of using one set of VFIO
> > API to manage all kinds of mediated devices and VF devices is a major
> > gain. Instead, inventing a new vDPA-like interface for every Scalable-IOV
> > or equivalent device is just overkill and doesn't scale. Also the actual
> > emulation code in idxd driver is actually small, if putting aside the PCI
> > config space part for which I already explained most logic could be shared
> > between mdev device drivers.
> 
> If it was just config space you might have an argument, VFIO already
> does some config space mangling, but emulating BAR space is out of
> scope of VFIO, IMHO.

out of scope of vfio-pci, but in scope of vfio-mdev. btw I feel that most
of your objections are actually related to the general idea of vfio-mdev.
Scalable IOV just uses PASID to harden DMA isolation in mediated
pass-through usage which vfio-mdev enables. Then are you just opposing
the whole vfio-mdev? If not, I'm curious about the criteria in your mind 
about when using vfio-mdev is good...

> 
> I also think it is disingenuous to pretend this is similar to
> SR-IOV. SR-IOV is self contained and the BAR does not require
> emulation. What you have here sounds like it is just an ordinary

technically Scalable IOV is definitely different from SR-IOV. It's 
simpler in hardware. And we're not emulating SR-IOV. The point
is just in usage-wise we want to present a consistent user 
experience just like passing through a PCI endpoint (PF or VF) device
through vfio eco-system, including various userspace VMMs (Qemu,
firecracker, rust-vmm, etc.), middleware (Libvirt), and higher level 
management stacks. 

> multi-queue device with the ability to PASID tag queues for IOMMU
> handling. This is absolutely not SRIOV - it is much closer to VDPA,
> which isn't using mdev.
> 
> Further, I disagree with your assessment that this doesn't scale. You
> already said you plan a normal user interface for idxd, so instead of
> having a single sane user interface (ala VDPA) idxd now needs *two*. If
> this is the general pattern of things to come, it is a bad path.
> 
> The only thing we get out of this is someone doesn't have to write a
> idxd emulation driver in qemu, instead they have to write it in the
> kernel. I don't see how that is a win for the ecosystem.
> 

No. The clear win is on leveraging classic VFIO iommu and its eco-system
as explained above.

Thanks
Kevin