From: Jacob Pan <jacob.jun.pan@linux.intel.com> To: iommu@lists.linux-foundation.org, LKML <linux-kernel@vger.kernel.org>, Joerg Roedel <joro@8bytes.org>, Jason Gunthorpe <jgg@nvidia.com>, "Christoph Hellwig" <hch@infradead.org> Cc: "Lu Baolu" <baolu.lu@linux.intel.com>, Raj Ashok <ashok.raj@intel.com>, "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>, Dave Jiang <dave.jiang@intel.com>, Tony Luck <tony.luck@intel.com>, mike.campin@intel.com, Yi Liu <yi.l.liu@intel.com>, "Tian, Kevin" <kevin.tian@intel.com> Subject: [RFC 0/7] Support in-kernel DMA with PASID and SVA Date: Tue, 21 Sep 2021 13:29:34 -0700 [thread overview] Message-ID: <1632256181-36071-1-git-send-email-jacob.jun.pan@linux.intel.com> (raw) Hi Joerg/Jason/Christoph et all, The current in-kernel supervisor PASID support is based on the SVM/SVA machinery in sva-lib. Kernel SVA is achieved by extending a special flag to indicate the binding of the device and a page table should be performed on init_mm instead of the mm of the current process.Page requests and other differences between user and kernel SVA are handled as special cases. This unrestricted binding with the kernel page table is being challenged for security and the convention that in-kernel DMA must be compatible with DMA APIs. (https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/) There is also the lack of IOTLB synchronization upon kernel page table updates. This patchset is trying to address these concerns by having an explicit DMA API compatible model while continue to support in-kernel use of DMA requests with PASID. Specifically, the following DMA-IOMMU APIs are introduced: int iommu_dma_pasid_enable/disable(struct device *dev, struct iommu_domain **domain, enum iommu_dma_pasid_mode mode); int iommu_map/unmap_kva(struct iommu_domain *domain, void *cpu_addr,size_t size, int prot); The following three addressing modes are supported with example API usages by device drivers. 1. Physical address (bypass) mode. Similar to DMA direct where trusted devices can DMA pass through IOMMU on a per PASID basis. Example: pasid = iommu_dma_pasid_enable(dev, NULL, IOMMU_DMA_PASID_BYPASS); /* Use the returning PASID and PA for work submission */ 2. IOVA mode. DMA API compatible. Map a supervisor PASID the same way as the PCI requester ID (RID) Example: pasid = iommu_dma_pasid_enable(dev, NULL, IOMMU_DMA_PASID_IOVA); /* Use the PASID and DMA API allocated IOVA for work submission */ 3. KVA mode. New kva map/unmap APIs. Support fast and strict sub-modes transparently based on device trustfulness. Example: pasid = iommu_dma_pasid_enable(dev, &domain, IOMMU_DMA_PASID_KVA); iommu_map_kva(domain, &buf, size, prot); /* Use the returned PASID and KVA to submit work */ Where: Fast mode: Shared CPU page tables for trusted devices only Strict mode: IOMMU domain returned for the untrusted device to replicate KVA-PA mapping in IOMMU page tables. On a per device basis, DMA address and performance modes are enabled by the device drivers. Platform information such as trustability, user command line input (not included in this set) could also be taken into consideration (not implemented in this RFC). This RFC is intended to communicate the API directions. Little testing is done outside IDXD and DMA engine tests. For PA and IOVA modes, the implementation is straightforward and tested with Intel IDXD driver. But several opens remain in KVA fast mode thus not tested: 1. Lack of IOTLB synchronization, kernel direct map alias can be updated as a result of module loading/eBPF load. Adding kernel mmu notifier? 2. The use of the auxiliary domain for KVA map, will aux domain stay in the long term? Is there another way to represent sub-device granu isolation? 3. Is limiting the KVA sharing to the direct map range reasonable and practical for all architectures? Many thanks to Ashok Raj, Kevin Tian, and Baolu who provided feedback and many ideas in this set. Thanks, Jacob Jacob Pan (7): ioasid: reserve special PASID for in-kernel DMA dma-iommu: Add API for DMA request with PASID iommu/vt-d: Add DMA w/ PASID support for PA and IOVA dma-iommu: Add support for DMA w/ PASID in KVA iommu/vt-d: Add support for KVA PASID mode iommu: Add KVA map API dma/idxd: Use dma-iommu PASID API instead of SVA lib drivers/dma/idxd/idxd.h | 4 +- drivers/dma/idxd/init.c | 36 ++-- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/dma-iommu.c | 123 +++++++++++++- drivers/iommu/intel/iommu.c | 154 +++++++++++++++++- drivers/iommu/ioasid.c | 2 + drivers/iommu/iommu-sva-lib.c | 1 + drivers/iommu/iommu.c | 63 +++++++ include/linux/dma-iommu.h | 14 ++ include/linux/intel-iommu.h | 7 +- include/linux/ioasid.h | 4 + include/linux/iommu.h | 13 ++ 12 files changed, 390 insertions(+), 33 deletions(-) -- 2.25.1
WARNING: multiple messages have this Message-ID (diff)
From: Jacob Pan <jacob.jun.pan@linux.intel.com> To: iommu@lists.linux-foundation.org, LKML <linux-kernel@vger.kernel.org>, Joerg Roedel <joro@8bytes.org>, Jason Gunthorpe <jgg@nvidia.com>, "Christoph Hellwig" <hch@infradead.org> Cc: "Tian, Kevin" <kevin.tian@intel.com>, Tony Luck <tony.luck@intel.com>, Dave Jiang <dave.jiang@intel.com>, Raj Ashok <ashok.raj@intel.com>, "Kumar, Sanjay K" <sanjay.k.kumar@intel.com>, mike.campin@intel.com Subject: [RFC 0/7] Support in-kernel DMA with PASID and SVA Date: Tue, 21 Sep 2021 13:29:34 -0700 [thread overview] Message-ID: <1632256181-36071-1-git-send-email-jacob.jun.pan@linux.intel.com> (raw) Hi Joerg/Jason/Christoph et all, The current in-kernel supervisor PASID support is based on the SVM/SVA machinery in sva-lib. Kernel SVA is achieved by extending a special flag to indicate the binding of the device and a page table should be performed on init_mm instead of the mm of the current process.Page requests and other differences between user and kernel SVA are handled as special cases. This unrestricted binding with the kernel page table is being challenged for security and the convention that in-kernel DMA must be compatible with DMA APIs. (https://lore.kernel.org/linux-iommu/20210511194726.GP1002214@nvidia.com/) There is also the lack of IOTLB synchronization upon kernel page table updates. This patchset is trying to address these concerns by having an explicit DMA API compatible model while continue to support in-kernel use of DMA requests with PASID. Specifically, the following DMA-IOMMU APIs are introduced: int iommu_dma_pasid_enable/disable(struct device *dev, struct iommu_domain **domain, enum iommu_dma_pasid_mode mode); int iommu_map/unmap_kva(struct iommu_domain *domain, void *cpu_addr,size_t size, int prot); The following three addressing modes are supported with example API usages by device drivers. 1. Physical address (bypass) mode. Similar to DMA direct where trusted devices can DMA pass through IOMMU on a per PASID basis. Example: pasid = iommu_dma_pasid_enable(dev, NULL, IOMMU_DMA_PASID_BYPASS); /* Use the returning PASID and PA for work submission */ 2. IOVA mode. DMA API compatible. Map a supervisor PASID the same way as the PCI requester ID (RID) Example: pasid = iommu_dma_pasid_enable(dev, NULL, IOMMU_DMA_PASID_IOVA); /* Use the PASID and DMA API allocated IOVA for work submission */ 3. KVA mode. New kva map/unmap APIs. Support fast and strict sub-modes transparently based on device trustfulness. Example: pasid = iommu_dma_pasid_enable(dev, &domain, IOMMU_DMA_PASID_KVA); iommu_map_kva(domain, &buf, size, prot); /* Use the returned PASID and KVA to submit work */ Where: Fast mode: Shared CPU page tables for trusted devices only Strict mode: IOMMU domain returned for the untrusted device to replicate KVA-PA mapping in IOMMU page tables. On a per device basis, DMA address and performance modes are enabled by the device drivers. Platform information such as trustability, user command line input (not included in this set) could also be taken into consideration (not implemented in this RFC). This RFC is intended to communicate the API directions. Little testing is done outside IDXD and DMA engine tests. For PA and IOVA modes, the implementation is straightforward and tested with Intel IDXD driver. But several opens remain in KVA fast mode thus not tested: 1. Lack of IOTLB synchronization, kernel direct map alias can be updated as a result of module loading/eBPF load. Adding kernel mmu notifier? 2. The use of the auxiliary domain for KVA map, will aux domain stay in the long term? Is there another way to represent sub-device granu isolation? 3. Is limiting the KVA sharing to the direct map range reasonable and practical for all architectures? Many thanks to Ashok Raj, Kevin Tian, and Baolu who provided feedback and many ideas in this set. Thanks, Jacob Jacob Pan (7): ioasid: reserve special PASID for in-kernel DMA dma-iommu: Add API for DMA request with PASID iommu/vt-d: Add DMA w/ PASID support for PA and IOVA dma-iommu: Add support for DMA w/ PASID in KVA iommu/vt-d: Add support for KVA PASID mode iommu: Add KVA map API dma/idxd: Use dma-iommu PASID API instead of SVA lib drivers/dma/idxd/idxd.h | 4 +- drivers/dma/idxd/init.c | 36 ++-- .../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +- drivers/iommu/dma-iommu.c | 123 +++++++++++++- drivers/iommu/intel/iommu.c | 154 +++++++++++++++++- drivers/iommu/ioasid.c | 2 + drivers/iommu/iommu-sva-lib.c | 1 + drivers/iommu/iommu.c | 63 +++++++ include/linux/dma-iommu.h | 14 ++ include/linux/intel-iommu.h | 7 +- include/linux/ioasid.h | 4 + include/linux/iommu.h | 13 ++ 12 files changed, 390 insertions(+), 33 deletions(-) -- 2.25.1 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next reply other threads:[~2021-09-22 5:13 UTC|newest] Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top 2021-09-21 20:29 Jacob Pan [this message] 2021-09-21 20:29 ` [RFC 0/7] Support in-kernel DMA with PASID and SVA Jacob Pan 2021-09-21 20:29 ` [RFC 1/7] ioasid: reserve special PASID for in-kernel DMA Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-21 20:29 ` [RFC 2/7] dma-iommu: Add API for DMA request with PASID Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-21 20:29 ` [RFC 3/7] iommu/vt-d: Add DMA w/ PASID support for PA and IOVA Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-21 20:29 ` [RFC 4/7] dma-iommu: Add support for DMA w/ PASID in KVA Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-21 20:29 ` [RFC 5/7] iommu/vt-d: Add support for KVA PASID mode Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-21 20:29 ` [RFC 6/7] iommu: Add KVA map API Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-21 20:29 ` [RFC 7/7] dma/idxd: Use dma-iommu PASID API instead of SVA lib Jacob Pan 2021-09-21 20:29 ` Jacob Pan 2021-09-22 17:04 ` [RFC 0/7] Support in-kernel DMA with PASID and SVA Jason Gunthorpe 2021-09-22 17:04 ` Jason Gunthorpe via iommu 2021-09-29 19:37 ` Jacob Pan 2021-09-29 19:37 ` Jacob Pan 2021-09-29 19:39 ` Jason Gunthorpe 2021-09-29 19:39 ` Jason Gunthorpe via iommu 2021-09-29 22:57 ` Jacob Pan 2021-09-29 22:57 ` Jacob Pan 2021-09-29 23:43 ` Jason Gunthorpe 2021-09-29 23:43 ` Jason Gunthorpe via iommu 2021-09-30 14:22 ` Campin, Mike 2021-09-30 14:22 ` Campin, Mike 2021-09-30 15:21 ` Jacob Pan 2021-09-30 15:21 ` Jacob Pan 2021-10-01 12:24 ` Barry Song 2021-10-01 12:24 ` Barry Song 2021-10-01 12:36 ` Jason Gunthorpe 2021-10-01 12:36 ` Jason Gunthorpe via iommu 2021-10-01 12:45 ` Barry Song 2021-10-01 12:45 ` Barry Song 2021-10-04 16:40 ` Jacob Pan 2021-10-04 16:40 ` Jacob Pan 2021-10-04 18:21 ` Jason Gunthorpe 2021-10-04 18:21 ` Jason Gunthorpe via iommu 2021-10-07 5:43 ` Barry Song 2021-10-07 5:43 ` Barry Song 2021-10-07 11:32 ` Jason Gunthorpe 2021-10-07 11:32 ` Jason Gunthorpe via iommu 2021-10-07 11:54 ` Barry Song 2021-10-07 11:54 ` Barry Song 2021-10-07 11:59 ` Jason Gunthorpe 2021-10-07 11:59 ` Jason Gunthorpe via iommu 2021-10-07 17:50 ` Jacob Pan 2021-10-07 17:50 ` Jacob Pan 2021-10-07 17:48 ` Jason Gunthorpe 2021-10-07 17:48 ` Jason Gunthorpe via iommu 2021-10-07 18:08 ` Jacob Pan 2021-10-07 18:08 ` Jacob Pan 2021-10-07 19:11 ` Jacob Pan 2021-10-07 19:11 ` Jacob Pan 2021-10-07 19:10 ` Jason Gunthorpe 2021-10-07 19:10 ` Jason Gunthorpe via iommu
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1632256181-36071-1-git-send-email-jacob.jun.pan@linux.intel.com \ --to=jacob.jun.pan@linux.intel.com \ --cc=ashok.raj@intel.com \ --cc=baolu.lu@linux.intel.com \ --cc=dave.jiang@intel.com \ --cc=hch@infradead.org \ --cc=iommu@lists.linux-foundation.org \ --cc=jgg@nvidia.com \ --cc=joro@8bytes.org \ --cc=kevin.tian@intel.com \ --cc=linux-kernel@vger.kernel.org \ --cc=mike.campin@intel.com \ --cc=sanjay.k.kumar@intel.com \ --cc=tony.luck@intel.com \ --cc=yi.l.liu@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.