From: "Liu, Yi L" <yi.l.liu@intel.com> To: Peter Xu <peterx@redhat.com> Cc: David Gibson <david@gibson.dropbear.id.au>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "pbonzini@redhat.com" <pbonzini@redhat.com>, "alex.williamson@redhat.com" <alex.williamson@redhat.com>, "mst@redhat.com" <mst@redhat.com>, "eric.auger@redhat.com" <eric.auger@redhat.com>, "Tian, Kevin" <kevin.tian@intel.com>, "Tian, Jun J" <jun.j.tian@intel.com>, "Sun, Yi Y" <yi.y.sun@intel.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, "Wu, Hao" <hao.wu@intel.com>, Jacob Pan <jacob.jun.pan@linux.intel.com>, "Yi Sun" <yi.y.sun@linux.intel.com> Subject: RE: [RFC v3 03/25] hw/iommu: introduce IOMMUContext Date: Wed, 12 Feb 2020 07:15:13 +0000 [thread overview] Message-ID: <A2975661238FB949B60364EF0F2C25743A1BA4D8@SHSMSX104.ccr.corp.intel.com> (raw) In-Reply-To: <20200211165843.GG984290@xz-x1> Hi Peter, > From: Peter Xu <peterx@redhat.com> > Sent: Wednesday, February 12, 2020 12:59 AM > To: Liu, Yi L <yi.l.liu@intel.com> > Subject: Re: [RFC v3 03/25] hw/iommu: introduce IOMMUContext > > On Fri, Jan 31, 2020 at 11:42:13AM +0000, Liu, Yi L wrote: > > > I'm not very clear on the relationship betwen an IOMMUContext and a > > > DualStageIOMMUObject. Can there be many IOMMUContexts to a > > > DualStageIOMMUOBject? The other way around? Or is it just > > > zero-or-one DualStageIOMMUObjects to an IOMMUContext? > > > > It is possible. As the below patch shows, DualStageIOMMUObject is per vfio > > container. IOMMUContext can be either per-device or shared across devices, > > it depends on vendor specific vIOMMU emulators. > > Is there an example when an IOMMUContext can be not per-device? No, I don’t have such example so far. But as IOMMUContext is got from pci_device_iommu_context(), in concept it possible to be not per-device. It is kind of leave to vIOMMU to decide if different devices could share a single IOMMUContext. > It makes sense to me to have an object that is per-container (in your > case, the DualStageIOMMUObject, IIUC), then we can connect that object > to a device. However I'm a bit confused on why we've got two abstract > layers (the other one is IOMMUContext)? That was previously for the > whole SVA new APIs, now it's all moved over to the other new object, > then IOMMUContext only register/unregister... Your understanding is correct. Actually, I also struggled on adding two abstract layer. But, you know, there are two function calling requirements around vSVA enabling. First one is explicit method for vIOMMU calls into VFIO (pasid allocation, bind guest page table, cache invalidate). Second one is explicit method for VFIO calls into vIOMMU (DMA fault/PRQ injection which is not included in this series yet, but will be upstreamed later). So I added the DualStageIOMMUObject to cover vIOMMU to VFIO callings, and IOMMUContext to cover VFIO to vIOMMU callings. As IOMMUContext covers VFIO to vIOMMU callings, so I made it include register/unregister. > Can we put the reg/unreg > procedures into DualStageIOMMUObject as well? Then we drop the > IOMMUContext (or say, keep IOMMUContext and drop DualStageIOMMUObject > but let IOMMUContext to be per-vfio-container, the major difference is > the naming here, say, PASID allocation does not seem to be related to > dual-stage at all). > > Besides that, not sure I read it right... but even with your current > series, the container->iommu_ctx will always only be bound to the > first device created within that container, since you've got: > > group = vfio_get_group(groupid, pci_device_iommu_address_space(pdev), > pci_device_iommu_context(pdev), errp); > > And: > > if (vfio_connect_container(group, as, iommu_ctx, errp)) { > error_prepend(errp, "failed to setup container for group %d: ", > groupid); > goto close_fd_exit; > } > > The iommu_ctx will be set to container->iommu_ctx if there's no > existing container. yes, it's true. May need to add a iommu_ctx list in VFIO container or add check on the input iommu_ctx of vfio_get_group() if sticking on this direction. While considering your suggestion on dropping one of the two abstract layers. I came up a new proposal as below: We may drop the IOMMUContext in this series, and rename DualStageIOMMUObject to HostIOMMUContext, which is per-vfio-container. Add an interface in PCI layer(e.g. an callback in PCIDevice) to let vIOMMU get HostIOMMUContext. I think this could cover the requirement of providing explicit method for vIOMMU to call into VFIO and then program host IOMMU. While for the requirement of VFIO to vIOMMU callings (e.g. PRQ), I think it could be done via PCI layer by adding an operation in PCIIOMMUOps. Thoughts? Thanks, Yi Liu
WARNING: multiple messages have this Message-ID (diff)
From: "Liu, Yi L" <yi.l.liu@intel.com> To: Peter Xu <peterx@redhat.com> Cc: "Tian, Kevin" <kevin.tian@intel.com>, Jacob Pan <jacob.jun.pan@linux.intel.com>, Yi Sun <yi.y.sun@linux.intel.com>, "kvm@vger.kernel.org" <kvm@vger.kernel.org>, "mst@redhat.com" <mst@redhat.com>, "Tian, Jun J" <jun.j.tian@intel.com>, "qemu-devel@nongnu.org" <qemu-devel@nongnu.org>, "eric.auger@redhat.com" <eric.auger@redhat.com>, "alex.williamson@redhat.com" <alex.williamson@redhat.com>, "pbonzini@redhat.com" <pbonzini@redhat.com>, "Wu, Hao" <hao.wu@intel.com>, "Sun, Yi Y" <yi.y.sun@intel.com>, David Gibson <david@gibson.dropbear.id.au> Subject: RE: [RFC v3 03/25] hw/iommu: introduce IOMMUContext Date: Wed, 12 Feb 2020 07:15:13 +0000 [thread overview] Message-ID: <A2975661238FB949B60364EF0F2C25743A1BA4D8@SHSMSX104.ccr.corp.intel.com> (raw) In-Reply-To: <20200211165843.GG984290@xz-x1> Hi Peter, > From: Peter Xu <peterx@redhat.com> > Sent: Wednesday, February 12, 2020 12:59 AM > To: Liu, Yi L <yi.l.liu@intel.com> > Subject: Re: [RFC v3 03/25] hw/iommu: introduce IOMMUContext > > On Fri, Jan 31, 2020 at 11:42:13AM +0000, Liu, Yi L wrote: > > > I'm not very clear on the relationship betwen an IOMMUContext and a > > > DualStageIOMMUObject. Can there be many IOMMUContexts to a > > > DualStageIOMMUOBject? The other way around? Or is it just > > > zero-or-one DualStageIOMMUObjects to an IOMMUContext? > > > > It is possible. As the below patch shows, DualStageIOMMUObject is per vfio > > container. IOMMUContext can be either per-device or shared across devices, > > it depends on vendor specific vIOMMU emulators. > > Is there an example when an IOMMUContext can be not per-device? No, I don’t have such example so far. But as IOMMUContext is got from pci_device_iommu_context(), in concept it possible to be not per-device. It is kind of leave to vIOMMU to decide if different devices could share a single IOMMUContext. > It makes sense to me to have an object that is per-container (in your > case, the DualStageIOMMUObject, IIUC), then we can connect that object > to a device. However I'm a bit confused on why we've got two abstract > layers (the other one is IOMMUContext)? That was previously for the > whole SVA new APIs, now it's all moved over to the other new object, > then IOMMUContext only register/unregister... Your understanding is correct. Actually, I also struggled on adding two abstract layer. But, you know, there are two function calling requirements around vSVA enabling. First one is explicit method for vIOMMU calls into VFIO (pasid allocation, bind guest page table, cache invalidate). Second one is explicit method for VFIO calls into vIOMMU (DMA fault/PRQ injection which is not included in this series yet, but will be upstreamed later). So I added the DualStageIOMMUObject to cover vIOMMU to VFIO callings, and IOMMUContext to cover VFIO to vIOMMU callings. As IOMMUContext covers VFIO to vIOMMU callings, so I made it include register/unregister. > Can we put the reg/unreg > procedures into DualStageIOMMUObject as well? Then we drop the > IOMMUContext (or say, keep IOMMUContext and drop DualStageIOMMUObject > but let IOMMUContext to be per-vfio-container, the major difference is > the naming here, say, PASID allocation does not seem to be related to > dual-stage at all). > > Besides that, not sure I read it right... but even with your current > series, the container->iommu_ctx will always only be bound to the > first device created within that container, since you've got: > > group = vfio_get_group(groupid, pci_device_iommu_address_space(pdev), > pci_device_iommu_context(pdev), errp); > > And: > > if (vfio_connect_container(group, as, iommu_ctx, errp)) { > error_prepend(errp, "failed to setup container for group %d: ", > groupid); > goto close_fd_exit; > } > > The iommu_ctx will be set to container->iommu_ctx if there's no > existing container. yes, it's true. May need to add a iommu_ctx list in VFIO container or add check on the input iommu_ctx of vfio_get_group() if sticking on this direction. While considering your suggestion on dropping one of the two abstract layers. I came up a new proposal as below: We may drop the IOMMUContext in this series, and rename DualStageIOMMUObject to HostIOMMUContext, which is per-vfio-container. Add an interface in PCI layer(e.g. an callback in PCIDevice) to let vIOMMU get HostIOMMUContext. I think this could cover the requirement of providing explicit method for vIOMMU to call into VFIO and then program host IOMMU. While for the requirement of VFIO to vIOMMU callings (e.g. PRQ), I think it could be done via PCI layer by adding an operation in PCIIOMMUOps. Thoughts? Thanks, Yi Liu
next prev parent reply other threads:[~2020-02-12 7:15 UTC|newest] Thread overview: 136+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-01-29 12:16 [RFC v3 00/25] intel_iommu: expose Shared Virtual Addressing to VMs Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 01/25] hw/pci: modify pci_setup_iommu() to set PCIIOMMUOps Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 02/25] hw/iommu: introduce DualStageIOMMUObject Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-31 3:59 ` David Gibson 2020-01-31 3:59 ` David Gibson 2020-01-31 11:42 ` Liu, Yi L 2020-01-31 11:42 ` Liu, Yi L 2020-02-12 6:32 ` David Gibson 2020-02-12 6:32 ` David Gibson 2020-01-29 12:16 ` [RFC v3 03/25] hw/iommu: introduce IOMMUContext Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-31 4:06 ` David Gibson 2020-01-31 4:06 ` David Gibson 2020-01-31 11:42 ` Liu, Yi L 2020-01-31 11:42 ` Liu, Yi L 2020-02-11 16:58 ` Peter Xu 2020-02-11 16:58 ` Peter Xu 2020-02-12 7:15 ` Liu, Yi L [this message] 2020-02-12 7:15 ` Liu, Yi L 2020-02-12 15:59 ` Peter Xu 2020-02-12 15:59 ` Peter Xu 2020-02-13 2:46 ` Liu, Yi L 2020-02-13 2:46 ` Liu, Yi L 2020-02-14 5:36 ` David Gibson 2020-02-14 5:36 ` David Gibson 2020-02-15 6:25 ` Liu, Yi L 2020-02-15 6:25 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 04/25] hw/pci: introduce pci_device_iommu_context() Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 05/25] intel_iommu: provide get_iommu_context() callback Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 06/25] scripts/update-linux-headers: Import iommu.h Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:25 ` Cornelia Huck 2020-01-29 12:25 ` Cornelia Huck 2020-01-31 11:40 ` Liu, Yi L 2020-01-31 11:40 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 07/25] header file update VFIO/IOMMU vSVA APIs Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:28 ` Cornelia Huck 2020-01-29 12:28 ` Cornelia Huck 2020-01-31 11:41 ` Liu, Yi L 2020-01-31 11:41 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 08/25] vfio: pass IOMMUContext into vfio_get_group() Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 09/25] vfio: check VFIO_TYPE1_NESTING_IOMMU support Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 19:08 ` Peter Xu 2020-02-11 19:08 ` Peter Xu 2020-02-12 7:16 ` Liu, Yi L 2020-02-12 7:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 10/25] vfio: register DualStageIOMMUObject to vIOMMU Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 11/25] vfio: get stage-1 pasid formats from Kernel Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 19:30 ` Peter Xu 2020-02-11 19:30 ` Peter Xu 2020-02-12 7:19 ` Liu, Yi L 2020-02-12 7:19 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 12/25] vfio/common: add pasid_alloc/free support Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 19:31 ` Peter Xu 2020-02-11 19:31 ` Peter Xu 2020-02-12 7:20 ` Liu, Yi L 2020-02-12 7:20 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 13/25] intel_iommu: modify x-scalable-mode to be string option Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 19:43 ` Peter Xu 2020-02-11 19:43 ` Peter Xu 2020-02-12 7:28 ` Liu, Yi L 2020-02-12 7:28 ` Liu, Yi L 2020-02-12 16:05 ` Peter Xu 2020-02-12 16:05 ` Peter Xu 2020-02-13 2:44 ` Liu, Yi L 2020-02-13 2:44 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 14/25] intel_iommu: add virtual command capability support Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 20:16 ` Peter Xu 2020-02-11 20:16 ` Peter Xu 2020-02-12 7:32 ` Liu, Yi L 2020-02-12 7:32 ` Liu, Yi L 2020-02-11 21:56 ` Peter Xu 2020-02-11 21:56 ` Peter Xu 2020-02-13 2:40 ` Liu, Yi L 2020-02-13 2:40 ` Liu, Yi L 2020-02-13 14:31 ` Peter Xu 2020-02-13 14:31 ` Peter Xu 2020-02-13 15:08 ` Peter Xu 2020-02-13 15:08 ` Peter Xu 2020-02-15 8:49 ` Liu, Yi L 2020-02-15 8:49 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 15/25] intel_iommu: process pasid cache invalidation Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 20:17 ` Peter Xu 2020-02-11 20:17 ` Peter Xu 2020-02-12 7:33 ` Liu, Yi L 2020-02-12 7:33 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 16/25] intel_iommu: add PASID cache management infrastructure Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-02-11 23:35 ` Peter Xu 2020-02-11 23:35 ` Peter Xu 2020-02-12 8:37 ` Liu, Yi L 2020-02-12 8:37 ` Liu, Yi L 2020-02-12 15:26 ` Peter Xu 2020-02-12 15:26 ` Peter Xu 2020-02-13 2:59 ` Liu, Yi L 2020-02-13 2:59 ` Liu, Yi L 2020-02-13 15:14 ` Peter Xu 2020-02-13 15:14 ` Peter Xu 2020-02-15 8:50 ` Liu, Yi L 2020-02-15 8:50 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 17/25] vfio: add bind stage-1 page table support Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 18/25] intel_iommu: bind/unbind guest page table to host Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 19/25] intel_iommu: replay guest pasid bindings " Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 20/25] intel_iommu: replay pasid binds after context cache invalidation Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 21/25] intel_iommu: do not pass down pasid bind for PASID #0 Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 22/25] vfio: add support for flush iommu stage-1 cache Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 23/25] intel_iommu: process PASID-based iotlb invalidation Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 24/25] intel_iommu: propagate PASID-based iotlb invalidation to host Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 12:16 ` [RFC v3 25/25] intel_iommu: process PASID-based Device-TLB invalidation Liu, Yi L 2020-01-29 12:16 ` Liu, Yi L 2020-01-29 13:44 ` [RFC v3 00/25] intel_iommu: expose Shared Virtual Addressing to VMs no-reply 2020-01-29 13:44 ` no-reply 2020-01-29 13:48 ` no-reply 2020-01-29 13:48 ` no-reply
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=A2975661238FB949B60364EF0F2C25743A1BA4D8@SHSMSX104.ccr.corp.intel.com \ --to=yi.l.liu@intel.com \ --cc=alex.williamson@redhat.com \ --cc=david@gibson.dropbear.id.au \ --cc=eric.auger@redhat.com \ --cc=hao.wu@intel.com \ --cc=jacob.jun.pan@linux.intel.com \ --cc=jun.j.tian@intel.com \ --cc=kevin.tian@intel.com \ --cc=kvm@vger.kernel.org \ --cc=mst@redhat.com \ --cc=pbonzini@redhat.com \ --cc=peterx@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=yi.y.sun@intel.com \ --cc=yi.y.sun@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.