From: Alex Williamson <firstname.lastname@example.org> To: Jason Gunthorpe <email@example.com> Cc: "Tian, Kevin" <firstname.lastname@example.org>, "Raj, Ashok" <email@example.com>, "Jiang, Dave" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "email@example.com" <firstname.lastname@example.org>, "Pan, Jacob jun" <email@example.com>, "Liu, Yi L" <firstname.lastname@example.org>, "Lu, Baolu" <email@example.com>, "Kumar, Sanjay K" <firstname.lastname@example.org>, "Luck, Tony" <email@example.com>, "Lin, Jing" <firstname.lastname@example.org>, "Williams, Dan J" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com>, "firstname.lastname@example.org" <email@example.com> Subject: Re: [PATCH RFC 00/15] Add VFIO mediated device support and IMS support for the idxd driver. Date: Sun, 26 Apr 2020 21:43:55 -0600 [thread overview] Message-ID: <firstname.lastname@example.org> (raw) In-Reply-To: <20200426191357.GB13640@mellanox.com> On Sun, 26 Apr 2020 16:13:57 -0300 Jason Gunthorpe <email@example.com> wrote: > On Sun, Apr 26, 2020 at 05:18:59AM +0000, Tian, Kevin wrote: > > > > > I think providing an unified abstraction to userspace is also important, > > > > which is what VFIO provides today. The merit of using one set of VFIO > > > > API to manage all kinds of mediated devices and VF devices is a major > > > > gain. Instead, inventing a new vDPA-like interface for every Scalable-IOV > > > > or equivalent device is just overkill and doesn't scale. Also the actual > > > > emulation code in idxd driver is actually small, if putting aside the PCI > > > > config space part for which I already explained most logic could be shared > > > > between mdev device drivers. > > > > > > If it was just config space you might have an argument, VFIO already > > > does some config space mangling, but emulating BAR space is out of > > > scope of VFIO, IMHO. > > > > out of scope of vfio-pci, but in scope of vfio-mdev. btw I feel that most > > of your objections are actually related to the general idea of > > vfio-mdev. > > There have been several abusive proposals of vfio-mdev, everything > from a way to create device drivers to this kind of generic emulation > framework. > > > Scalable IOV just uses PASID to harden DMA isolation in mediated > > pass-through usage which vfio-mdev enables. Then are you just opposing > > the whole vfio-mdev? If not, I'm curious about the criteria in your mind > > about when using vfio-mdev is good... > > It is appropriate when non-PCI standard techniques are needed to do > raw device assignment, just like VFIO. > > Basically if vfio-pci is already doing it then it seems reasonable > that vfio-mdev should do the same. This mission creep where vfio-mdev > gains functionality far beyond VFIO is the problem. Ehm, vfio-pci emulates BARs too. We also emulate FLR, power management, DisINTx, and VPD. FLR, PM, and VPD all have device specific quirks in the host kernel, and I've generally taken the stance that would should take advantage of those quirks, not duplicate them in userspace and not invent new access mechanisms/ioctls for each of them. Emulating DisINTx is convenient since we must have a mechanism to mask INTx, whether it's at the device or the APIC, so we can pretend the hardware supports it. BAR emulation is really too trivial to argue about, the BARs mean nothing to the physical device mapping, they're simply scratch registers that we mask out the alignment bits on read. vfio-pci is a mix of things that we decide are too complicated or irrelevant to emulate in the kernel and things that take advantage of shared quirks or are just too darn easy to worry about. BARs fall into that latter category, any sort of mapping into VM address spaces is necessarily done in userspace, but scratch registers that are masked on read, *shrug*, vfio-pci does that. Thanks, Alex > > technically Scalable IOV is definitely different from SR-IOV. It's > > simpler in hardware. And we're not emulating SR-IOV. The point > > is just in usage-wise we want to present a consistent user > > experience just like passing through a PCI endpoint (PF or VF) device > > through vfio eco-system, including various userspace VMMs (Qemu, > > firecracker, rust-vmm, etc.), middleware (Libvirt), and higher level > > management stacks. > > Yes, I understand your desire, but at the same time we have not been > doing device emulation in the kernel. You should at least be > forthwright about that major change in the cover letters/etc. > > > > The only thing we get out of this is someone doesn't have to write a > > > idxd emulation driver in qemu, instead they have to write it in the > > > kernel. I don't see how that is a win for the ecosystem. > > > > No. The clear win is on leveraging classic VFIO iommu and its eco-system > > as explained above. > > vdpa had no problem implementing iommu support without VFIO. This was > their original argument too, it turned out to be erroneous. > > Jason >
next prev parent reply other threads:[~2020-04-27 3:44 UTC|newest] Thread overview: 89+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-04-21 23:33 Dave Jiang 2020-04-21 23:33 ` [PATCH RFC 01/15] drivers/base: Introduce platform_msi_ops Dave Jiang 2020-04-26 7:01 ` Greg KH 2020-04-27 21:38 ` Dave Jiang 2020-04-28 7:34 ` Greg KH 2020-04-21 23:33 ` [PATCH RFC 02/15] drivers/base: Introduce a new platform-msi list Dave Jiang 2020-04-25 21:13 ` Thomas Gleixner 2020-05-04 0:08 ` Dey, Megha 2020-04-21 23:34 ` [PATCH RFC 03/15] drivers/base: Allocate/free platform-msi interrupts by group Dave Jiang 2020-04-25 21:23 ` Thomas Gleixner 2020-05-04 0:08 ` Dey, Megha 2020-04-21 23:34 ` [PATCH RFC 04/15] drivers/base: Add support for a new IMS irq domain Dave Jiang 2020-04-23 20:11 ` Jason Gunthorpe 2020-05-01 22:30 ` Dey, Megha 2020-05-03 22:25 ` Jason Gunthorpe 2020-05-03 22:40 ` Dey, Megha 2020-05-03 22:46 ` Jason Gunthorpe 2020-05-04 0:25 ` Dey, Megha 2020-05-04 12:14 ` Jason Gunthorpe 2020-05-06 10:27 ` Tian, Kevin 2020-04-25 21:38 ` Thomas Gleixner 2020-05-04 0:11 ` Dey, Megha 2020-04-21 23:34 ` [PATCH RFC 05/15] ims-msi: Add mask/unmask routines Dave Jiang 2020-04-25 21:49 ` Thomas Gleixner 2020-05-04 0:16 ` Dey, Megha 2020-04-21 23:34 ` [PATCH RFC 06/15] ims-msi: Enable IMS interrupts Dave Jiang 2020-04-25 22:13 ` Thomas Gleixner 2020-05-04 0:17 ` Dey, Megha 2020-04-21 23:34 ` [PATCH RFC 07/15] Documentation: Interrupt Message store Dave Jiang 2020-04-23 20:04 ` Jason Gunthorpe 2020-05-01 22:32 ` Dey, Megha 2020-05-03 22:28 ` Jason Gunthorpe 2020-05-03 22:41 ` Dey, Megha 2020-04-21 23:34 ` [PATCH RFC 08/15] vfio/mdev: Add a member for iommu domain in mdev_device Dave Jiang 2020-04-21 23:34 ` [PATCH RFC 09/15] vfio/type1: Save domain when attach domain to mdev Dave Jiang 2020-04-21 23:34 ` [PATCH RFC 10/15] dmaengine: idxd: add config support for readonly devices Dave Jiang 2020-04-21 23:34 ` [PATCH RFC 11/15] dmaengine: idxd: add IMS support in base driver Dave Jiang 2020-04-21 23:35 ` [PATCH RFC 12/15] dmaengine: idxd: add device support functions in prep for mdev Dave Jiang 2020-04-21 23:35 ` [PATCH RFC 13/15] dmaengine: idxd: add support for VFIO mediated device Dave Jiang 2020-04-21 23:35 ` [PATCH RFC 14/15] dmaengine: idxd: add error notification from host driver to " Dave Jiang 2020-04-21 23:35 ` [PATCH RFC 15/15] dmaengine: idxd: add ABI documentation for mediated device support Dave Jiang 2020-04-21 23:54 ` [PATCH RFC 00/15] Add VFIO mediated device support and IMS support for the idxd driver Jason Gunthorpe 2020-04-22 0:53 ` Tian, Kevin 2020-04-22 11:50 ` Jason Gunthorpe 2020-04-22 21:14 ` Raj, Ashok 2020-04-23 19:12 ` Jason Gunthorpe 2020-04-24 3:27 ` Tian, Kevin 2020-04-24 12:44 ` Jason Gunthorpe 2020-04-24 16:25 ` Tian, Kevin 2020-04-24 18:12 ` Jason Gunthorpe 2020-04-26 5:18 ` Tian, Kevin 2020-04-26 19:13 ` Jason Gunthorpe 2020-04-27 3:43 ` Alex Williamson [this message] 2020-04-27 11:58 ` Jason Gunthorpe 2020-04-27 13:19 ` Alex Williamson 2020-04-27 13:22 ` Jason Gunthorpe 2020-04-27 14:18 ` Alex Williamson 2020-04-27 14:25 ` Jason Gunthorpe 2020-04-27 15:41 ` Alex Williamson 2020-04-27 16:16 ` Jason Gunthorpe 2020-04-27 16:25 ` Dave Jiang 2020-04-27 21:56 ` Jason Gunthorpe 2020-04-29 9:42 ` Tian, Kevin 2020-05-08 20:47 ` Raj, Ashok 2020-05-08 23:16 ` Jason Gunthorpe 2020-05-08 23:52 ` Dave Jiang 2020-05-09 0:09 ` Raj, Ashok 2020-05-09 12:21 ` Jason Gunthorpe 2020-05-13 2:29 ` Jason Wang 2020-05-13 8:30 ` Tian, Kevin 2020-05-13 12:40 ` Jason Gunthorpe 2020-04-27 12:13 ` Tian, Kevin 2020-04-27 12:55 ` Jason Gunthorpe 2020-04-22 21:24 ` Dan Williams 2020-04-23 19:17 ` Dan Williams 2020-04-23 19:49 ` Jason Gunthorpe 2020-05-01 22:31 ` Dey, Megha 2020-05-03 22:21 ` Jason Gunthorpe 2020-05-03 22:32 ` Dey, Megha 2020-04-23 19:18 ` Jason Gunthorpe 2020-05-01 22:31 ` Dey, Megha 2020-05-03 22:22 ` Jason Gunthorpe 2020-05-03 22:31 ` Dey, Megha 2020-05-03 22:36 ` Jason Gunthorpe 2020-05-04 0:20 ` Dey, Megha 2020-04-22 23:04 ` Dey, Megha 2020-04-23 19:44 ` Jason Gunthorpe 2020-05-01 22:32 ` Dey, Megha 2020-04-24 6:31 ` Jason Wang
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --email@example.com \ --firstname.lastname@example.org \ --subject='Re: [PATCH RFC 00/15] Add VFIO mediated device support and IMS support for the idxd driver.' \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: link
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).