From: Alex Williamson <alex.williamson@redhat.com> To: "Tian, Kevin" <kevin.tian@intel.com> Cc: "Daniel P. Berrangé" <berrange@redhat.com>, "Liu, Yi L" <yi.l.liu@intel.com>, "akrowiak@linux.ibm.com" <akrowiak@linux.ibm.com>, "jjherne@linux.ibm.com" <jjherne@linux.ibm.com>, "Peng, Chao P" <chao.p.peng@intel.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, "Laine Stump" <laine@redhat.com>, "libvir-list@redhat.com" <libvir-list@redhat.com>, "jasowang@redhat.com" <jasowang@redhat.com>, "cohuck@redhat.com" <cohuck@redhat.com>, "thuth@redhat.com" <thuth@redhat.com>, "peterx@redhat.com" <peterx@redhat.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "pasic@linux.ibm.com" <pasic@linux.ibm.com>, "eric.auger@redhat.com" <eric.auger@redhat.com>, "Sun, Yi Y" <yi.y.sun@intel.com>, "nicolinc@nvidia.com" <nicolinc@nvidia.com>, "jgg@nvidia.com" <jgg@nvidia.com>, "eric.auger.pro@gmail.com" <eric.auger.pro@gmail.com>, "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au> Subject: Re: [RFC 00/18] vfio: Adopt iommufd Date: Thu, 28 Apr 2022 08:24:48 -0600 [thread overview] Message-ID: <20220428082448.318385ed.alex.williamson@redhat.com> (raw) In-Reply-To: <BN9PR11MB5276189A2A8EACFBF75B22238CFD9@BN9PR11MB5276.namprd11.prod.outlook.com> On Thu, 28 Apr 2022 03:21:45 +0000 "Tian, Kevin" <kevin.tian@intel.com> wrote: > > From: Alex Williamson <alex.williamson@redhat.com> > > Sent: Wednesday, April 27, 2022 12:22 AM > > > > > > > > My expectation would be that libvirt uses: > > > > > > > > -object iommufd,id=iommufd0,fd=NNN > > > > -device vfio-pci,fd=MMM,iommufd=iommufd0 > > > > > > > > Whereas simple QEMU command line would be: > > > > > > > > -object iommufd,id=iommufd0 > > > > -device vfio-pci,iommufd=iommufd0,host=0000:02:00.0 > > > > > > > > The iommufd object would open /dev/iommufd itself. Creating an > > > > implicit iommufd object is someone problematic because one of the > > > > things I forgot to highlight in my previous description is that the > > > > iommufd object is meant to be shared across not only various vfio > > > > devices (platform, ccw, ap, nvme, etc), but also across subsystems, ex. > > > > vdpa. > > > > > > Out of curiosity - in concept one iommufd is sufficient to support all > > > ioas requirements across subsystems while having multiple iommufd's > > > instead lose the benefit of centralized accounting. The latter will also > > > cause some trouble when we start virtualizing ENQCMD which requires > > > VM-wide PASID virtualization thus further needs to share that > > > information across iommufd's. Not unsolvable but really no gain by > > > adding such complexity. So I'm curious whether Qemu provide > > > a way to restrict that certain object type can only have one instance > > > to discourage such multi-iommufd attempt? > > > > I don't see any reason for QEMU to restrict iommufd objects. The QEMU > > philosophy seems to be to let users create whatever configuration they > > want. For libvirt though, the assumption would be that a single > > iommufd object can be used across subsystems, so libvirt would never > > automatically create multiple objects. > > I like the flexibility what the objection approach gives in your proposal. > But with the said complexity in mind (with no foreseen benefit), I wonder What's the actual complexity? Front-end/backend splits are very common in QEMU. We're making the object connection via name, why is it significantly more complicated to allow multiple iommufd objects? On the contrary, it seems to me that we'd need to go out of our way to add code to block multiple iommufd objects. > whether an alternative approach which treats iommufd as a global > property instead of an object is acceptable in Qemu, i.e.: > > -iommufd on/off > -device vfio-pci,iommufd,[fd=MMM/host=0000:02:00.0] > > All devices with iommufd specified then implicitly share a single iommufd > object within Qemu. QEMU requires key-value pairs AFAIK, so the above doesn't work, then we're just back to the iommufd=on/off. > This still allows vfio devices to be specified via fd but just requires Libvirt > to grant file permission on /dev/iommu. Is it a worthwhile tradeoff to be > considered or just not a typical way in Qemu philosophy e.g. any object > associated with a device must be explicitly specified? Avoiding QEMU opening files was a significant focus of my alternate proposal. Also note that we must be able to support hotplug, so we need to be able to dynamically add and remove the iommufd object, I don't see that a global property allows for that. Implicit associations of devices to shared resources doesn't seem particularly desirable to me. Thanks, Alex
WARNING: multiple messages have this Message-ID (diff)
From: Alex Williamson <alex.williamson@redhat.com> To: "Tian, Kevin" <kevin.tian@intel.com> Cc: "akrowiak@linux.ibm.com" <akrowiak@linux.ibm.com>, "jjherne@linux.ibm.com" <jjherne@linux.ibm.com>, "thuth@redhat.com" <thuth@redhat.com>, "Peng, Chao P" <chao.p.peng@intel.com>, "Daniel P. Berrangé" <berrange@redhat.com>, "jgg@nvidia.com" <jgg@nvidia.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, "libvir-list@redhat.com" <libvir-list@redhat.com>, "jasowang@redhat.com" <jasowang@redhat.com>, "cohuck@redhat.com" <cohuck@redhat.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "peterx@redhat.com" <peterx@redhat.com>, "pasic@linux.ibm.com" <pasic@linux.ibm.com>, "eric.auger@redhat.com" <eric.auger@redhat.com>, "Sun, Yi Y" <yi.y.sun@intel.com>, "Liu, Yi L" <yi.l.liu@intel.com>, "nicolinc@nvidia.com" <nicolinc@nvidia.com>, "Laine Stump" <laine@redhat.com>, "david@gibson.dropbear.id.au" <david@gibson.dropbear.id.au>, "eric.auger.pro@gmail.com" <eric.auger.pro@gmail.com> Subject: Re: [RFC 00/18] vfio: Adopt iommufd Date: Thu, 28 Apr 2022 08:24:48 -0600 [thread overview] Message-ID: <20220428082448.318385ed.alex.williamson@redhat.com> (raw) In-Reply-To: <BN9PR11MB5276189A2A8EACFBF75B22238CFD9@BN9PR11MB5276.namprd11.prod.outlook.com> On Thu, 28 Apr 2022 03:21:45 +0000 "Tian, Kevin" <kevin.tian@intel.com> wrote: > > From: Alex Williamson <alex.williamson@redhat.com> > > Sent: Wednesday, April 27, 2022 12:22 AM > > > > > > > > My expectation would be that libvirt uses: > > > > > > > > -object iommufd,id=iommufd0,fd=NNN > > > > -device vfio-pci,fd=MMM,iommufd=iommufd0 > > > > > > > > Whereas simple QEMU command line would be: > > > > > > > > -object iommufd,id=iommufd0 > > > > -device vfio-pci,iommufd=iommufd0,host=0000:02:00.0 > > > > > > > > The iommufd object would open /dev/iommufd itself. Creating an > > > > implicit iommufd object is someone problematic because one of the > > > > things I forgot to highlight in my previous description is that the > > > > iommufd object is meant to be shared across not only various vfio > > > > devices (platform, ccw, ap, nvme, etc), but also across subsystems, ex. > > > > vdpa. > > > > > > Out of curiosity - in concept one iommufd is sufficient to support all > > > ioas requirements across subsystems while having multiple iommufd's > > > instead lose the benefit of centralized accounting. The latter will also > > > cause some trouble when we start virtualizing ENQCMD which requires > > > VM-wide PASID virtualization thus further needs to share that > > > information across iommufd's. Not unsolvable but really no gain by > > > adding such complexity. So I'm curious whether Qemu provide > > > a way to restrict that certain object type can only have one instance > > > to discourage such multi-iommufd attempt? > > > > I don't see any reason for QEMU to restrict iommufd objects. The QEMU > > philosophy seems to be to let users create whatever configuration they > > want. For libvirt though, the assumption would be that a single > > iommufd object can be used across subsystems, so libvirt would never > > automatically create multiple objects. > > I like the flexibility what the objection approach gives in your proposal. > But with the said complexity in mind (with no foreseen benefit), I wonder What's the actual complexity? Front-end/backend splits are very common in QEMU. We're making the object connection via name, why is it significantly more complicated to allow multiple iommufd objects? On the contrary, it seems to me that we'd need to go out of our way to add code to block multiple iommufd objects. > whether an alternative approach which treats iommufd as a global > property instead of an object is acceptable in Qemu, i.e.: > > -iommufd on/off > -device vfio-pci,iommufd,[fd=MMM/host=0000:02:00.0] > > All devices with iommufd specified then implicitly share a single iommufd > object within Qemu. QEMU requires key-value pairs AFAIK, so the above doesn't work, then we're just back to the iommufd=on/off. > This still allows vfio devices to be specified via fd but just requires Libvirt > to grant file permission on /dev/iommu. Is it a worthwhile tradeoff to be > considered or just not a typical way in Qemu philosophy e.g. any object > associated with a device must be explicitly specified? Avoiding QEMU opening files was a significant focus of my alternate proposal. Also note that we must be able to support hotplug, so we need to be able to dynamically add and remove the iommufd object, I don't see that a global property allows for that. Implicit associations of devices to shared resources doesn't seem particularly desirable to me. Thanks, Alex
next prev parent reply other threads:[~2022-04-28 14:24 UTC|newest] Thread overview: 125+ messages / expand[flat|nested] mbox.gz Atom feed top 2022-04-14 10:46 [RFC 00/18] vfio: Adopt iommufd Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-14 10:46 ` [RFC 01/18] scripts/update-linux-headers: Add iommufd.h Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-14 10:46 ` [RFC 02/18] linux-headers: Import latest vfio.h and iommufd.h Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-14 10:46 ` [RFC 03/18] hw/vfio/pci: fix vfio_pci_hot_reset_result trace point Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-14 10:46 ` [RFC 04/18] vfio/pci: Use vbasedev local variable in vfio_realize() Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-14 10:46 ` [RFC 05/18] vfio/common: Rename VFIOGuestIOMMU::iommu into ::iommu_mr Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-14 10:46 ` [RFC 06/18] vfio/common: Split common.c into common.c, container.c and as.c Yi Liu 2022-04-14 10:46 ` [RFC 07/18] vfio: Add base object for VFIOContainer Yi Liu 2022-04-14 10:46 ` Yi Liu 2022-04-29 6:29 ` David Gibson 2022-04-29 6:29 ` David Gibson 2022-05-03 13:05 ` Yi Liu 2022-04-14 10:47 ` [RFC 08/18] vfio/container: Introduce vfio_[attach/detach]_device Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 09/18] vfio/platform: Use vfio_[attach/detach]_device Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 10/18] vfio/ap: " Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 11/18] vfio/ccw: " Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 12/18] vfio/container-obj: Introduce [attach/detach]_device container callbacks Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 13/18] vfio/container-obj: Introduce VFIOContainer reset callback Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 14/18] hw/iommufd: Creation Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 15/18] vfio/iommufd: Implement iommufd backend Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-22 14:58 ` Jason Gunthorpe 2022-04-22 21:33 ` Alex Williamson 2022-04-22 21:33 ` Alex Williamson 2022-04-26 9:55 ` Yi Liu 2022-04-26 9:55 ` Yi Liu 2022-04-26 10:41 ` Tian, Kevin 2022-04-26 10:41 ` Tian, Kevin 2022-04-26 13:41 ` Jason Gunthorpe 2022-04-26 14:08 ` Yi Liu 2022-04-26 14:08 ` Yi Liu 2022-04-26 14:11 ` Jason Gunthorpe 2022-04-26 18:45 ` Alex Williamson 2022-04-26 18:45 ` Alex Williamson 2022-04-26 19:27 ` Jason Gunthorpe 2022-04-26 20:59 ` Alex Williamson 2022-04-26 20:59 ` Alex Williamson 2022-04-26 23:08 ` Jason Gunthorpe 2022-04-26 13:53 ` Jason Gunthorpe 2022-04-14 10:47 ` [RFC 16/18] vfio/iommufd: Add IOAS_COPY_DMA support Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 17/18] vfio/as: Allow the selection of a given iommu backend Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-14 10:47 ` [RFC 18/18] vfio/pci: Add an iommufd option Yi Liu 2022-04-14 10:47 ` Yi Liu 2022-04-15 8:37 ` [RFC 00/18] vfio: Adopt iommufd Nicolin Chen 2022-04-17 10:30 ` Eric Auger 2022-04-17 10:30 ` Eric Auger 2022-04-19 3:26 ` Nicolin Chen 2022-04-25 19:40 ` Eric Auger 2022-04-25 19:40 ` Eric Auger 2022-04-18 8:49 ` Tian, Kevin 2022-04-18 8:49 ` Tian, Kevin 2022-04-18 12:09 ` Yi Liu 2022-04-18 12:09 ` Yi Liu 2022-04-25 19:51 ` Eric Auger 2022-04-25 19:51 ` Eric Auger 2022-04-25 19:55 ` Eric Auger 2022-04-25 19:55 ` Eric Auger 2022-04-26 8:39 ` Tian, Kevin 2022-04-26 8:39 ` Tian, Kevin 2022-04-22 22:09 ` Alex Williamson 2022-04-22 22:09 ` Alex Williamson 2022-04-25 10:10 ` Daniel P. Berrangé 2022-04-25 10:10 ` Daniel P. Berrangé 2022-04-25 13:36 ` Jason Gunthorpe 2022-04-25 14:37 ` Alex Williamson 2022-04-25 14:37 ` Alex Williamson 2022-04-26 8:37 ` Tian, Kevin 2022-04-26 8:37 ` Tian, Kevin 2022-04-26 12:33 ` Jason Gunthorpe 2022-04-26 16:21 ` Alex Williamson 2022-04-26 16:21 ` Alex Williamson 2022-04-26 16:42 ` Jason Gunthorpe 2022-04-26 19:24 ` Alex Williamson 2022-04-26 19:24 ` Alex Williamson 2022-04-26 19:36 ` Jason Gunthorpe 2022-04-28 3:21 ` Tian, Kevin 2022-04-28 3:21 ` Tian, Kevin 2022-04-28 14:24 ` Alex Williamson [this message] 2022-04-28 14:24 ` Alex Williamson 2022-04-28 16:20 ` Daniel P. Berrangé 2022-04-28 16:20 ` Daniel P. Berrangé 2022-04-29 0:45 ` Tian, Kevin 2022-04-29 0:45 ` Tian, Kevin 2022-04-25 20:23 ` Eric Auger 2022-04-25 20:23 ` Eric Auger 2022-04-25 22:53 ` Alex Williamson 2022-04-25 22:53 ` Alex Williamson 2022-04-26 9:47 ` Shameerali Kolothum Thodi via 2022-04-26 9:47 ` Shameerali Kolothum Thodi 2022-04-26 11:44 ` Eric Auger 2022-04-26 11:44 ` Eric Auger 2022-04-26 12:43 ` Shameerali Kolothum Thodi 2022-04-26 12:43 ` Shameerali Kolothum Thodi via 2022-04-26 16:35 ` Alex Williamson 2022-04-26 16:35 ` Alex Williamson 2022-05-09 14:24 ` Zhangfei Gao 2022-05-10 3:17 ` Yi Liu 2022-05-10 6:51 ` Eric Auger 2022-05-10 12:35 ` Zhangfei Gao 2022-05-10 12:45 ` Jason Gunthorpe 2022-05-10 14:08 ` Yi Liu 2022-05-11 14:17 ` zhangfei.gao 2022-05-12 9:01 ` zhangfei.gao 2022-05-17 8:55 ` Yi Liu 2022-05-18 7:22 ` zhangfei.gao 2022-05-18 14:00 ` Yi Liu 2022-06-28 8:14 ` Shameerali Kolothum Thodi 2022-06-28 8:14 ` Shameerali Kolothum Thodi via 2022-06-28 8:58 ` Eric Auger 2022-05-17 8:52 ` Yi Liu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20220428082448.318385ed.alex.williamson@redhat.com \ --to=alex.williamson@redhat.com \ --cc=akrowiak@linux.ibm.com \ --cc=berrange@redhat.com \ --cc=chao.p.peng@intel.com \ --cc=cohuck@redhat.com \ --cc=david@gibson.dropbear.id.au \ --cc=eric.auger.pro@gmail.com \ --cc=eric.auger@redhat.com \ --cc=jasowang@redhat.com \ --cc=jgg@nvidia.com \ --cc=jjherne@linux.ibm.com \ --cc=kevin.tian@intel.com \ --cc=kvm@vger.kernel.org \ --cc=laine@redhat.com \ --cc=libvir-list@redhat.com \ --cc=nicolinc@nvidia.com \ --cc=pasic@linux.ibm.com \ --cc=peterx@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=thuth@redhat.com \ --cc=yi.l.liu@intel.com \ --cc=yi.y.sun@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.