From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> To: "Liu, Yi L" <yi.l.liu@intel.com>, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, peterx@redhat.com Cc: jasowang@redhat.com, qemu-devel@nongnu.org, kevin.tian@intel.com, ashok.raj@intel.com, jacob.jun.pan@intel.com, tianyu.lan@intel.com, "Liu, Yi L" <yi.l.liu@linux.intel.com> Subject: Re: [RFC PATCH 7/8] VFIO: Add new IOCTL for IOMMU TLB invalidate propagation Date: Fri, 12 May 2017 13:11:02 +0100 [thread overview] Message-ID: <cc330a8f-e087-9b6f-2a40-38b58688d300@arm.com> (raw) In-Reply-To: <1493201525-14418-8-git-send-email-yi.l.liu@intel.com> Hi Yi, On 26/04/17 11:12, Liu, Yi L wrote: > From: "Liu, Yi L" <yi.l.liu@linux.intel.com> > > This patch adds VFIO_IOMMU_TLB_INVALIDATE to propagate IOMMU TLB > invalidate request from guest to host. > > In the case of SVM virtualization on VT-d, host IOMMU driver has > no knowledge of caching structure updates unless the guest > invalidation activities are passed down to the host. So a new > IOCTL is needed to propagate the guest cache invalidation through > VFIO. > > Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com> > --- > include/uapi/linux/vfio.h | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h > index 6b97987..50c51f8 100644 > --- a/include/uapi/linux/vfio.h > +++ b/include/uapi/linux/vfio.h > @@ -564,6 +564,15 @@ struct vfio_device_svm { > > #define VFIO_IOMMU_SVM_BIND_TASK _IO(VFIO_TYPE, VFIO_BASE + 22) > > +/* For IOMMU TLB Invalidation Propagation */ > +struct vfio_iommu_tlb_invalidate { > + __u32 argsz; > + __u32 length; > + __u8 data[]; > +}; We initially discussed something a little more generic than this, with most info explicitly described and only pIOMMU-specific quirks and hints in an opaque structure. Out of curiosity, why the change? I'm not against a fully opaque structure, but there seem to be a large overlap between TLB invalidations across architectures. For what it's worth, when prototyping the paravirtualized IOMMU I came up with the following. (From the paravirtualized POV, the SMMU also has to swizzle endianess after unpacking an opaque structure, since userspace doesn't know what's in it and guest might use a different endianess. So we need to force all opaque data to be e.g. little-endian.) struct vfio_iommu_tlb_invalidate { __u32 argsz; __u32 scope; __u32 flags; __u32 pasid; __u64 vaddr; __u64 size; __u8 data[]; }; Scope is a bitfields restricting the invalidation scope. By default invalidate the whole container (all PASIDs and all VAs). @pasid, @vaddr and @size are unused. Adding VFIO_IOMMU_INVALIDATE_PASID (1 << 0) restricts the invalidation scope to the pasid described by @pasid. Adding VFIO_IOMMU_INVALIDATE_VADDR (1 << 1) restricts the invalidation scope to the address range described by (@vaddr, @size). So setting scope = VFIO_IOMMU_INVALIDATE_VADDR would invalidate the VA range for *all* pasids (as well as no_pasid). Setting scope = (VFIO_IOMMU_INVALIDATE_VADDR|VFIO_IOMMU_INVALIDATE_PASID) would invalidate the VA range only for @pasid. Flags depend on the selected scope: VFIO_IOMMU_INVALIDATE_NO_PASID, indicating that invalidation (either without scope or with INVALIDATE_VADDR) targets non-pasid mappings exclusively (some architectures, e.g. SMMU, allow this) VFIO_IOMMU_INVALIDATE_VADDR_LEAF, indicating that the pIOMMU doesn't need to invalidate all intermediate tables cached as part of the PTW for vaddr, only the last-level entry (pte). This is a hint. I guess what's missing for Intel IOMMU and would go in @data is the "global" hint (which we don't have in SMMU invalidations). Do you see anything else, that the pIOMMU cannot deduce from this structure? Thanks, Jean > +#define VFIO_IOMMU_TLB_INVALIDATE _IO(VFIO_TYPE, VFIO_BASE + 23) > + > /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ > > /* >
WARNING: multiple messages have this Message-ID (diff)
From: Jean-Philippe Brucker <jean-philippe.brucker@arm.com> To: "Liu, Yi L" <yi.l.liu@intel.com>, kvm@vger.kernel.org, iommu@lists.linux-foundation.org, alex.williamson@redhat.com, peterx@redhat.com Cc: jasowang@redhat.com, qemu-devel@nongnu.org, kevin.tian@intel.com, ashok.raj@intel.com, jacob.jun.pan@intel.com, tianyu.lan@intel.com, "Liu, Yi L" <yi.l.liu@linux.intel.com> Subject: Re: [Qemu-devel] [RFC PATCH 7/8] VFIO: Add new IOCTL for IOMMU TLB invalidate propagation Date: Fri, 12 May 2017 13:11:02 +0100 [thread overview] Message-ID: <cc330a8f-e087-9b6f-2a40-38b58688d300@arm.com> (raw) In-Reply-To: <1493201525-14418-8-git-send-email-yi.l.liu@intel.com> Hi Yi, On 26/04/17 11:12, Liu, Yi L wrote: > From: "Liu, Yi L" <yi.l.liu@linux.intel.com> > > This patch adds VFIO_IOMMU_TLB_INVALIDATE to propagate IOMMU TLB > invalidate request from guest to host. > > In the case of SVM virtualization on VT-d, host IOMMU driver has > no knowledge of caching structure updates unless the guest > invalidation activities are passed down to the host. So a new > IOCTL is needed to propagate the guest cache invalidation through > VFIO. > > Signed-off-by: Liu, Yi L <yi.l.liu@linux.intel.com> > --- > include/uapi/linux/vfio.h | 9 +++++++++ > 1 file changed, 9 insertions(+) > > diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h > index 6b97987..50c51f8 100644 > --- a/include/uapi/linux/vfio.h > +++ b/include/uapi/linux/vfio.h > @@ -564,6 +564,15 @@ struct vfio_device_svm { > > #define VFIO_IOMMU_SVM_BIND_TASK _IO(VFIO_TYPE, VFIO_BASE + 22) > > +/* For IOMMU TLB Invalidation Propagation */ > +struct vfio_iommu_tlb_invalidate { > + __u32 argsz; > + __u32 length; > + __u8 data[]; > +}; We initially discussed something a little more generic than this, with most info explicitly described and only pIOMMU-specific quirks and hints in an opaque structure. Out of curiosity, why the change? I'm not against a fully opaque structure, but there seem to be a large overlap between TLB invalidations across architectures. For what it's worth, when prototyping the paravirtualized IOMMU I came up with the following. (From the paravirtualized POV, the SMMU also has to swizzle endianess after unpacking an opaque structure, since userspace doesn't know what's in it and guest might use a different endianess. So we need to force all opaque data to be e.g. little-endian.) struct vfio_iommu_tlb_invalidate { __u32 argsz; __u32 scope; __u32 flags; __u32 pasid; __u64 vaddr; __u64 size; __u8 data[]; }; Scope is a bitfields restricting the invalidation scope. By default invalidate the whole container (all PASIDs and all VAs). @pasid, @vaddr and @size are unused. Adding VFIO_IOMMU_INVALIDATE_PASID (1 << 0) restricts the invalidation scope to the pasid described by @pasid. Adding VFIO_IOMMU_INVALIDATE_VADDR (1 << 1) restricts the invalidation scope to the address range described by (@vaddr, @size). So setting scope = VFIO_IOMMU_INVALIDATE_VADDR would invalidate the VA range for *all* pasids (as well as no_pasid). Setting scope = (VFIO_IOMMU_INVALIDATE_VADDR|VFIO_IOMMU_INVALIDATE_PASID) would invalidate the VA range only for @pasid. Flags depend on the selected scope: VFIO_IOMMU_INVALIDATE_NO_PASID, indicating that invalidation (either without scope or with INVALIDATE_VADDR) targets non-pasid mappings exclusively (some architectures, e.g. SMMU, allow this) VFIO_IOMMU_INVALIDATE_VADDR_LEAF, indicating that the pIOMMU doesn't need to invalidate all intermediate tables cached as part of the PTW for vaddr, only the last-level entry (pte). This is a hint. I guess what's missing for Intel IOMMU and would go in @data is the "global" hint (which we don't have in SMMU invalidations). Do you see anything else, that the pIOMMU cannot deduce from this structure? Thanks, Jean > +#define VFIO_IOMMU_TLB_INVALIDATE _IO(VFIO_TYPE, VFIO_BASE + 23) > + > /* -------- Additional API for SPAPR TCE (Server POWERPC) IOMMU -------- */ > > /* >
next prev parent reply other threads:[~2017-05-12 12:07 UTC|newest] Thread overview: 116+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-04-26 10:11 [RFC PATCH 0/8] Shared Virtual Memory virtualization for VT-d Liu, Yi L 2017-04-26 10:11 ` [Qemu-devel] " Liu, Yi L 2017-04-26 10:11 ` [RFC PATCH 1/8] iommu: Introduce bind_pasid_table API function Liu, Yi L 2017-04-26 10:11 ` [Qemu-devel] " Liu, Yi L 2017-04-26 16:56 ` Jean-Philippe Brucker 2017-04-26 16:56 ` [Qemu-devel] " Jean-Philippe Brucker 2017-04-27 6:36 ` Liu, Yi L 2017-04-27 6:36 ` [Qemu-devel] " Liu, Yi L 2017-04-27 10:12 ` Jean-Philippe Brucker 2017-04-27 10:12 ` [Qemu-devel] " Jean-Philippe Brucker [not found] ` <772ca9de-50ba-a379-002d-5ff1f6a2e297-5wv7dgnIgG8@public.gmane.org> 2017-04-28 7:59 ` Liu, Yi L 2017-04-28 7:59 ` Liu, Yi L [not found] ` <c042bf90-d48b-4ebf-c01a-fca7c4875277-5wv7dgnIgG8@public.gmane.org> 2017-04-26 18:29 ` jacob pan 2017-04-26 18:29 ` [Qemu-devel] " jacob pan [not found] ` <20170426112948.00004520-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> 2017-04-26 18:59 ` Jean-Philippe Brucker 2017-04-26 18:59 ` [Qemu-devel] " Jean-Philippe Brucker 2017-04-28 9:04 ` Liu, Yi L 2017-04-28 9:04 ` Liu, Yi L 2017-04-28 12:51 ` Jean-Philippe Brucker 2017-04-28 12:51 ` Jean-Philippe Brucker [not found] ` <3adb4e33-db96-4133-0510-412c3bfb24fe-5wv7dgnIgG8@public.gmane.org> 2017-05-23 7:50 ` Liu, Yi L 2017-05-23 7:50 ` Liu, Yi L 2017-05-25 12:33 ` Jean-Philippe Brucker 2017-05-25 12:33 ` Jean-Philippe Brucker [not found] ` <1493201525-14418-2-git-send-email-yi.l.liu-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> 2017-05-12 21:59 ` Alex Williamson 2017-05-12 21:59 ` [Qemu-devel] " Alex Williamson [not found] ` <20170512155914.73bad777-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> 2017-05-14 10:56 ` Liu, Yi L 2017-05-14 10:56 ` [Qemu-devel] " Liu, Yi L 2017-04-26 10:11 ` [RFC PATCH 2/8] iommu/vt-d: add bind_pasid_table function Liu, Yi L 2017-04-26 10:11 ` [Qemu-devel] " Liu, Yi L [not found] ` <1493201525-14418-3-git-send-email-yi.l.liu-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> 2017-05-12 21:59 ` Alex Williamson 2017-05-12 21:59 ` [Qemu-devel] " Alex Williamson [not found] ` <20170512155929.66809113-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> 2017-05-15 13:14 ` jacob pan 2017-05-15 13:14 ` [Qemu-devel] " jacob pan 2017-04-26 10:12 ` [RFC PATCH 3/8] iommu: Introduce iommu do invalidate API function Liu, Yi L 2017-04-26 10:12 ` [Qemu-devel] " Liu, Yi L [not found] ` <1493201525-14418-4-git-send-email-yi.l.liu-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> 2017-05-12 21:59 ` Alex Williamson 2017-05-12 21:59 ` [Qemu-devel] " Alex Williamson [not found] ` <20170512155924.755ee17f-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> 2017-05-17 10:23 ` Liu, Yi L 2017-05-17 10:23 ` [Qemu-devel] " Liu, Yi L 2017-04-26 10:12 ` [RFC PATCH 4/8] iommu/vt-d: Add iommu do invalidate function Liu, Yi L 2017-04-26 10:12 ` [Qemu-devel] " Liu, Yi L [not found] ` <1493201525-14418-5-git-send-email-yi.l.liu-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> 2017-05-12 21:59 ` Alex Williamson 2017-05-12 21:59 ` [Qemu-devel] " Alex Williamson [not found] ` <20170512155918.5251fb94-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> 2017-05-17 10:24 ` Liu, Yi L 2017-05-17 10:24 ` [Qemu-devel] " Liu, Yi L 2017-04-26 10:12 ` [RFC PATCH 5/8] VFIO: Add new IOTCL for PASID Table bind propagation Liu, Yi L 2017-04-26 10:12 ` [Qemu-devel] " Liu, Yi L 2017-04-26 16:56 ` Jean-Philippe Brucker 2017-04-26 16:56 ` [Qemu-devel] " Jean-Philippe Brucker 2017-04-27 5:43 ` Liu, Yi L [not found] ` <1493201525-14418-6-git-send-email-yi.l.liu-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org> 2017-05-11 10:29 ` Liu, Yi L 2017-05-11 10:29 ` Liu, Yi L 2017-05-12 21:58 ` Alex Williamson 2017-05-12 21:58 ` [Qemu-devel] " Alex Williamson [not found] ` <20170512155851.627409ed-1yVPhWWZRC1BDLzU/O5InQ@public.gmane.org> 2017-05-17 10:27 ` Liu, Yi L 2017-05-17 10:27 ` Liu, Yi L [not found] ` <20170517102759.GF22110-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-05-18 11:29 ` Jean-Philippe Brucker 2017-05-18 11:29 ` Jean-Philippe Brucker 2017-04-26 10:12 ` [RFC PATCH 6/8] VFIO: do pasid table binding Liu, Yi L 2017-04-26 10:12 ` [Qemu-devel] " Liu, Yi L 2017-05-09 7:55 ` Xiao Guangrong 2017-05-09 7:55 ` [Qemu-devel] " Xiao Guangrong 2017-05-11 10:29 ` Liu, Yi L 2017-05-12 21:59 ` Alex Williamson 2017-05-12 21:59 ` [Qemu-devel] " Alex Williamson 2017-04-26 10:12 ` [RFC PATCH 7/8] VFIO: Add new IOCTL for IOMMU TLB invalidate propagation Liu, Yi L 2017-04-26 10:12 ` [Qemu-devel] " Liu, Yi L 2017-05-12 12:11 ` Jean-Philippe Brucker [this message] 2017-05-12 12:11 ` Jean-Philippe Brucker [not found] ` <cc330a8f-e087-9b6f-2a40-38b58688d300-5wv7dgnIgG8@public.gmane.org> 2017-05-14 10:12 ` Liu, Yi L 2017-05-14 10:12 ` [Qemu-devel] " Liu, Yi L 2017-05-15 12:14 ` Jean-Philippe Brucker 2017-05-15 12:14 ` [Qemu-devel] " Jean-Philippe Brucker 2017-07-02 10:06 ` Liu, Yi L 2017-07-02 10:06 ` Liu, Yi L 2017-07-03 11:52 ` Jean-Philippe Brucker [not found] ` <0e4f2dd4-d553-b1b7-7bec-fe0ff5242c54-5wv7dgnIgG8@public.gmane.org> 2017-07-03 10:31 ` Liu, Yi L 2017-07-03 10:31 ` Liu, Yi L [not found] ` <20170703103115.GB22053-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-07-05 6:45 ` Tian, Kevin 2017-07-05 6:45 ` Tian, Kevin [not found] ` <AADFC41AFE54684AB9EE6CBC0274A5D190D25919-0J0gbvR4kThpB2pF5aRoyrfspsVTdybXVpNB7YpNyf8@public.gmane.org> 2017-07-05 12:42 ` Jean-Philippe Brucker 2017-07-05 12:42 ` Jean-Philippe Brucker [not found] ` <1d63c1ae-ca10-0f9d-91de-0d9c9823c104-5wv7dgnIgG8@public.gmane.org> 2017-07-05 17:28 ` Alex Williamson 2017-07-05 17:28 ` Alex Williamson [not found] ` <20170705112816.56554f65-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org> 2017-07-05 22:26 ` Tian, Kevin 2017-07-05 22:26 ` Tian, Kevin 2017-07-14 8:58 ` Liu, Yi L 2017-07-14 8:58 ` Liu, Yi L [not found] ` <A2975661238FB949B60364EF0F2C2574390A7C4F-E2R4CRU6q/6iAffOGbnezLfspsVTdybXVpNB7YpNyf8@public.gmane.org> 2017-07-14 18:15 ` Alex Williamson 2017-07-14 18:15 ` Alex Williamson [not found] ` <20170714121555.7e64d849-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org> 2017-07-17 10:58 ` Liu, Yi L 2017-07-17 10:58 ` Liu, Yi L 2017-07-17 22:45 ` Alex Williamson [not found] ` <20170717164515.2491b3bf-DGNDKt5SQtizQB+pC5nmwQ@public.gmane.org> 2017-07-18 9:38 ` Jean-Philippe Brucker 2017-07-18 9:38 ` Jean-Philippe Brucker [not found] ` <d0abeefc-adcf-85c3-f5d9-8c90a18f8011-5wv7dgnIgG8@public.gmane.org> 2017-07-18 14:29 ` Alex Williamson 2017-07-18 14:29 ` Alex Williamson 2017-07-18 15:03 ` Jean-Philippe Brucker 2017-07-19 10:45 ` Liu, Yi L 2017-07-19 10:45 ` Liu, Yi L 2017-07-19 21:50 ` Jacob Pan 2017-07-19 21:50 ` Jacob Pan 2017-07-05 22:31 ` Tian, Kevin 2017-07-05 22:31 ` Tian, Kevin 2017-05-12 21:58 ` Alex Williamson 2017-05-12 21:58 ` [Qemu-devel] " Alex Williamson 2017-05-14 10:55 ` Liu, Yi L 2017-05-14 10:55 ` [Qemu-devel] " Liu, Yi L [not found] ` <20170514105507.GB22110-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org> 2017-07-05 5:32 ` Tian, Kevin 2017-07-05 5:32 ` [Qemu-devel] " Tian, Kevin 2017-04-26 10:12 ` [RFC PATCH 8/8] VFIO: do IOMMU TLB invalidation from guest Liu, Yi L 2017-04-26 10:12 ` [Qemu-devel] " Liu, Yi L 2017-05-08 4:09 ` [RFC PATCH 0/8] Shared Virtual Memory virtualization for VT-d Xiao Guangrong 2017-05-08 4:09 ` [Qemu-devel] " Xiao Guangrong 2017-05-07 7:33 ` Liu, Yi L
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=cc330a8f-e087-9b6f-2a40-38b58688d300@arm.com \ --to=jean-philippe.brucker@arm.com \ --cc=alex.williamson@redhat.com \ --cc=ashok.raj@intel.com \ --cc=iommu@lists.linux-foundation.org \ --cc=jacob.jun.pan@intel.com \ --cc=jasowang@redhat.com \ --cc=kevin.tian@intel.com \ --cc=kvm@vger.kernel.org \ --cc=peterx@redhat.com \ --cc=qemu-devel@nongnu.org \ --cc=tianyu.lan@intel.com \ --cc=yi.l.liu@intel.com \ --cc=yi.l.liu@linux.intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.