From: Liu Yi L <yi.l.liu@intel.com> To: alex.williamson@redhat.com, eric.auger@redhat.com, baolu.lu@linux.intel.com, joro@8bytes.org Cc: kevin.tian@intel.com, jacob.jun.pan@linux.intel.com, ashok.raj@intel.com, yi.l.liu@intel.com, jun.j.tian@intel.com, yi.y.sun@intel.com, jean-philippe@linaro.org, peterx@redhat.com, hao.wu@intel.com, iommu@lists.linux-foundation.org, kvm@vger.kernel.org, linux-kernel@vger.kernel.org Subject: [PATCH v2 03/15] vfio/type1: Report iommu nesting info to userspace Date: Thu, 11 Jun 2020 05:15:22 -0700 [thread overview] Message-ID: <1591877734-66527-4-git-send-email-yi.l.liu@intel.com> (raw) In-Reply-To: <1591877734-66527-1-git-send-email-yi.l.liu@intel.com> This patch exports iommu nesting capability info to user space through VFIO. User space is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to. The nesting info is available only after the nesting iommu type is set for a container. Current implementation imposes one limitation - one nesting container should include at most one group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd-level address space and multiple 1st-level address spaces. While the 2nd-leve address space is reasonably sharable by multiple groups , blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/ super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this decision. https://lkml.org/lkml/2020/5/15/1028 Cc: Kevin Tian <kevin.tian@intel.com> CC: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Alex Williamson <alex.williamson@redhat.com> Cc: Eric Auger <eric.auger@redhat.com> Cc: Jean-Philippe Brucker <jean-philippe@linaro.org> Cc: Joerg Roedel <joro@8bytes.org> Cc: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> --- drivers/vfio/vfio_iommu_type1.c | 73 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 9 +++++ 2 files changed, 82 insertions(+) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 402aad3..22432cf 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -71,6 +71,7 @@ struct vfio_iommu { unsigned int dma_avail; bool v2; bool nesting; + struct iommu_nesting_info *nesting_info; }; struct vfio_domain { @@ -125,6 +126,9 @@ struct vfio_regions { #define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu) \ (!list_empty(&iommu->domain_list)) +#define IS_DOMAIN_IN_CONTAINER(iommu) ((iommu->external_domain) || \ + (!list_empty(&iommu->domain_list))) + static int put_pfn(unsigned long pfn, int prot); /* @@ -1641,6 +1645,12 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, } } + /* Nesting type container can include only one group */ + if (iommu->nesting && IS_DOMAIN_IN_CONTAINER(iommu)) { + mutex_unlock(&iommu->lock); + return -EINVAL; + } + group = kzalloc(sizeof(*group), GFP_KERNEL); domain = kzalloc(sizeof(*domain), GFP_KERNEL); if (!group || !domain) { @@ -1700,6 +1710,36 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, if (ret) goto out_domain; + /* Nesting cap info is available only after attaching */ + if (iommu->nesting) { + struct iommu_nesting_info tmp; + struct iommu_nesting_info *info; + + /* First get the size of vendor specific nesting info */ + ret = iommu_domain_get_attr(domain->domain, + DOMAIN_ATTR_NESTING_INFO, + &tmp); + if (ret) + goto out_detach; + + info = kzalloc(tmp.size, GFP_KERNEL); + if (!info) { + ret = -ENOMEM; + goto out_detach; + } + + /* Now get the nesting info */ + info->size = tmp.size; + ret = iommu_domain_get_attr(domain->domain, + DOMAIN_ATTR_NESTING_INFO, + info); + if (ret) { + kfree(info); + goto out_detach; + } + iommu->nesting_info = info; + } + /* Get aperture info */ iommu_domain_get_attr(domain->domain, DOMAIN_ATTR_GEOMETRY, &geo); @@ -1801,6 +1841,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, return 0; out_detach: + kfree(iommu->nesting_info); vfio_iommu_detach_group(domain, group); out_domain: iommu_domain_free(domain->domain); @@ -1998,6 +2039,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, vfio_iommu_unmap_unpin_all(iommu); else vfio_iommu_unmap_unpin_reaccount(iommu); + + kfree(iommu->nesting_info); } iommu_domain_free(domain->domain); list_del(&domain->next); @@ -2190,6 +2233,30 @@ static int vfio_iommu_iova_build_caps(struct vfio_iommu *iommu, return ret; } +static int vfio_iommu_info_add_nesting_cap(struct vfio_iommu *iommu, + struct vfio_info_cap *caps) +{ + struct vfio_info_cap_header *header; + struct vfio_iommu_type1_info_cap_nesting *nesting_cap; + size_t size; + + size = sizeof(*nesting_cap) + iommu->nesting_info->size; + + header = vfio_info_cap_add(caps, size, + VFIO_IOMMU_TYPE1_INFO_CAP_NESTING, 1); + if (IS_ERR(header)) + return PTR_ERR(header); + + nesting_cap = container_of(header, + struct vfio_iommu_type1_info_cap_nesting, + header); + + memcpy(&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->size); + + return 0; +} + static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu, unsigned long arg) { @@ -2223,6 +2290,12 @@ static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu, if (ret) return ret; + if (iommu->nesting_info) { + ret = vfio_iommu_info_add_nesting_cap(iommu, &caps); + if (ret) + return ret; + } + if (caps.size) { info.flags |= VFIO_IOMMU_INFO_CAPS; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 015516b..26e3dce 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -14,6 +14,7 @@ #include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0 @@ -785,6 +786,14 @@ struct vfio_iommu_type1_info_cap_iova_range { struct vfio_iova_range iova_ranges[]; }; +#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 2 + +struct vfio_iommu_type1_info_cap_nesting { + struct vfio_info_cap_header header; + __u32 flags; + __u8 info[]; +}; + #define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12) /** -- 2.7.4
WARNING: multiple messages have this Message-ID (diff)
From: Liu Yi L <yi.l.liu@intel.com> To: alex.williamson@redhat.com, eric.auger@redhat.com, baolu.lu@linux.intel.com, joro@8bytes.org Cc: jean-philippe@linaro.org, kevin.tian@intel.com, ashok.raj@intel.com, kvm@vger.kernel.org, jun.j.tian@intel.com, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, yi.y.sun@intel.com, hao.wu@intel.com Subject: [PATCH v2 03/15] vfio/type1: Report iommu nesting info to userspace Date: Thu, 11 Jun 2020 05:15:22 -0700 [thread overview] Message-ID: <1591877734-66527-4-git-send-email-yi.l.liu@intel.com> (raw) In-Reply-To: <1591877734-66527-1-git-send-email-yi.l.liu@intel.com> This patch exports iommu nesting capability info to user space through VFIO. User space is expected to check this info for supported uAPIs (e.g. PASID alloc/free, bind page table, and cache invalidation) and the vendor specific format information for first level/stage page table that will be bound to. The nesting info is available only after the nesting iommu type is set for a container. Current implementation imposes one limitation - one nesting container should include at most one group. The philosophy of vfio container is having all groups/devices within the container share the same IOMMU context. When vSVA is enabled, one IOMMU context could include one 2nd-level address space and multiple 1st-level address spaces. While the 2nd-leve address space is reasonably sharable by multiple groups , blindly sharing 1st-level address spaces across all groups within the container might instead break the guest expectation. In the future sub/ super container concept might be introduced to allow partial address space sharing within an IOMMU context. But for now let's go with this restriction by requiring singleton container for using nesting iommu features. Below link has the related discussion about this decision. https://lkml.org/lkml/2020/5/15/1028 Cc: Kevin Tian <kevin.tian@intel.com> CC: Jacob Pan <jacob.jun.pan@linux.intel.com> Cc: Alex Williamson <alex.williamson@redhat.com> Cc: Eric Auger <eric.auger@redhat.com> Cc: Jean-Philippe Brucker <jean-philippe@linaro.org> Cc: Joerg Roedel <joro@8bytes.org> Cc: Lu Baolu <baolu.lu@linux.intel.com> Signed-off-by: Liu Yi L <yi.l.liu@intel.com> --- drivers/vfio/vfio_iommu_type1.c | 73 +++++++++++++++++++++++++++++++++++++++++ include/uapi/linux/vfio.h | 9 +++++ 2 files changed, 82 insertions(+) diff --git a/drivers/vfio/vfio_iommu_type1.c b/drivers/vfio/vfio_iommu_type1.c index 402aad3..22432cf 100644 --- a/drivers/vfio/vfio_iommu_type1.c +++ b/drivers/vfio/vfio_iommu_type1.c @@ -71,6 +71,7 @@ struct vfio_iommu { unsigned int dma_avail; bool v2; bool nesting; + struct iommu_nesting_info *nesting_info; }; struct vfio_domain { @@ -125,6 +126,9 @@ struct vfio_regions { #define IS_IOMMU_CAP_DOMAIN_IN_CONTAINER(iommu) \ (!list_empty(&iommu->domain_list)) +#define IS_DOMAIN_IN_CONTAINER(iommu) ((iommu->external_domain) || \ + (!list_empty(&iommu->domain_list))) + static int put_pfn(unsigned long pfn, int prot); /* @@ -1641,6 +1645,12 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, } } + /* Nesting type container can include only one group */ + if (iommu->nesting && IS_DOMAIN_IN_CONTAINER(iommu)) { + mutex_unlock(&iommu->lock); + return -EINVAL; + } + group = kzalloc(sizeof(*group), GFP_KERNEL); domain = kzalloc(sizeof(*domain), GFP_KERNEL); if (!group || !domain) { @@ -1700,6 +1710,36 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, if (ret) goto out_domain; + /* Nesting cap info is available only after attaching */ + if (iommu->nesting) { + struct iommu_nesting_info tmp; + struct iommu_nesting_info *info; + + /* First get the size of vendor specific nesting info */ + ret = iommu_domain_get_attr(domain->domain, + DOMAIN_ATTR_NESTING_INFO, + &tmp); + if (ret) + goto out_detach; + + info = kzalloc(tmp.size, GFP_KERNEL); + if (!info) { + ret = -ENOMEM; + goto out_detach; + } + + /* Now get the nesting info */ + info->size = tmp.size; + ret = iommu_domain_get_attr(domain->domain, + DOMAIN_ATTR_NESTING_INFO, + info); + if (ret) { + kfree(info); + goto out_detach; + } + iommu->nesting_info = info; + } + /* Get aperture info */ iommu_domain_get_attr(domain->domain, DOMAIN_ATTR_GEOMETRY, &geo); @@ -1801,6 +1841,7 @@ static int vfio_iommu_type1_attach_group(void *iommu_data, return 0; out_detach: + kfree(iommu->nesting_info); vfio_iommu_detach_group(domain, group); out_domain: iommu_domain_free(domain->domain); @@ -1998,6 +2039,8 @@ static void vfio_iommu_type1_detach_group(void *iommu_data, vfio_iommu_unmap_unpin_all(iommu); else vfio_iommu_unmap_unpin_reaccount(iommu); + + kfree(iommu->nesting_info); } iommu_domain_free(domain->domain); list_del(&domain->next); @@ -2190,6 +2233,30 @@ static int vfio_iommu_iova_build_caps(struct vfio_iommu *iommu, return ret; } +static int vfio_iommu_info_add_nesting_cap(struct vfio_iommu *iommu, + struct vfio_info_cap *caps) +{ + struct vfio_info_cap_header *header; + struct vfio_iommu_type1_info_cap_nesting *nesting_cap; + size_t size; + + size = sizeof(*nesting_cap) + iommu->nesting_info->size; + + header = vfio_info_cap_add(caps, size, + VFIO_IOMMU_TYPE1_INFO_CAP_NESTING, 1); + if (IS_ERR(header)) + return PTR_ERR(header); + + nesting_cap = container_of(header, + struct vfio_iommu_type1_info_cap_nesting, + header); + + memcpy(&nesting_cap->info, iommu->nesting_info, + iommu->nesting_info->size); + + return 0; +} + static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu, unsigned long arg) { @@ -2223,6 +2290,12 @@ static int vfio_iommu_type1_get_info(struct vfio_iommu *iommu, if (ret) return ret; + if (iommu->nesting_info) { + ret = vfio_iommu_info_add_nesting_cap(iommu, &caps); + if (ret) + return ret; + } + if (caps.size) { info.flags |= VFIO_IOMMU_INFO_CAPS; diff --git a/include/uapi/linux/vfio.h b/include/uapi/linux/vfio.h index 015516b..26e3dce 100644 --- a/include/uapi/linux/vfio.h +++ b/include/uapi/linux/vfio.h @@ -14,6 +14,7 @@ #include <linux/types.h> #include <linux/ioctl.h> +#include <linux/iommu.h> #define VFIO_API_VERSION 0 @@ -785,6 +786,14 @@ struct vfio_iommu_type1_info_cap_iova_range { struct vfio_iova_range iova_ranges[]; }; +#define VFIO_IOMMU_TYPE1_INFO_CAP_NESTING 2 + +struct vfio_iommu_type1_info_cap_nesting { + struct vfio_info_cap_header header; + __u32 flags; + __u8 info[]; +}; + #define VFIO_IOMMU_GET_INFO _IO(VFIO_TYPE, VFIO_BASE + 12) /** -- 2.7.4 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2020-06-11 12:10 UTC|newest] Thread overview: 74+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-06-11 12:15 [PATCH v2 00/15] vfio: expose virtual Shared Virtual Addressing to VMs Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 01/15] vfio/type1: Refactor vfio_iommu_type1_ioctl() Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 02/15] iommu: Report domain nesting info Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 19:30 ` Alex Williamson 2020-06-11 19:30 ` Alex Williamson 2020-06-12 9:05 ` Liu, Yi L 2020-06-12 9:05 ` Liu, Yi L 2020-06-15 1:22 ` Tian, Kevin 2020-06-15 1:22 ` Tian, Kevin 2020-06-15 6:04 ` Liu, Yi L 2020-06-15 6:04 ` Liu, Yi L 2020-06-16 1:56 ` Tian, Kevin 2020-06-16 1:56 ` Tian, Kevin 2020-06-16 2:24 ` Liu, Yi L 2020-06-16 2:24 ` Liu, Yi L 2020-06-17 14:39 ` Jean-Philippe Brucker 2020-06-17 14:39 ` Jean-Philippe Brucker 2020-06-18 11:46 ` Liu, Yi L 2020-06-18 11:46 ` Liu, Yi L 2020-06-11 12:15 ` Liu Yi L [this message] 2020-06-11 12:15 ` [PATCH v2 03/15] vfio/type1: Report iommu nesting info to userspace Liu Yi L 2020-06-11 12:15 ` [PATCH v2 04/15] vfio: Add PASID allocation/free support Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 05/15] iommu/vt-d: Support setting ioasid set to domain Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 06/15] vfio/type1: Add VFIO_IOMMU_PASID_REQUEST (alloc/free) Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 07/15] iommu/uapi: Add iommu_gpasid_unbind_data Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 08/15] iommu: Pass domain and unbind_data to sva_unbind_gpasid() Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 09/15] iommu/vt-d: Check ownership for PASIDs from user-space Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 10/15] vfio/type1: Support binding guest page tables to PASID Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 11/15] vfio/type1: Allow invalidating first-level/stage IOMMU cache Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 12/15] vfio/type1: Add vSVA support for IOMMU-backed mdevs Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 13/15] vfio/pci: Expose PCIe PASID capability to guest Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-11 12:15 ` [PATCH v2 14/15] vfio: Document dual stage control Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-15 9:41 ` Stefan Hajnoczi 2020-06-15 9:41 ` Stefan Hajnoczi 2020-06-17 6:27 ` Liu, Yi L 2020-06-17 6:27 ` Liu, Yi L 2020-06-22 12:51 ` Stefan Hajnoczi 2020-06-22 12:51 ` Stefan Hajnoczi 2020-06-23 6:43 ` Liu, Yi L 2020-06-23 6:43 ` Liu, Yi L 2020-06-11 12:15 ` [PATCH v2 15/15] iommu/vt-d: Support reporting nesting capability info Liu Yi L 2020-06-11 12:15 ` Liu Yi L 2020-06-15 10:02 ` [PATCH v2 00/15] vfio: expose virtual Shared Virtual Addressing to VMs Stefan Hajnoczi 2020-06-15 10:02 ` Stefan Hajnoczi 2020-06-15 12:39 ` Liu, Yi L 2020-06-15 12:39 ` Liu, Yi L 2020-06-16 15:34 ` Stefan Hajnoczi 2020-06-16 15:34 ` Stefan Hajnoczi 2020-06-16 2:26 ` Tian, Kevin 2020-06-16 2:26 ` Tian, Kevin 2020-06-16 15:49 ` Stefan Hajnoczi 2020-06-16 15:49 ` Stefan Hajnoczi 2020-06-16 16:09 ` Peter Xu 2020-06-16 16:09 ` Peter Xu 2020-06-22 12:49 ` Stefan Hajnoczi 2020-06-22 12:49 ` Stefan Hajnoczi 2020-06-16 17:00 ` Raj, Ashok 2020-06-16 17:00 ` Raj, Ashok 2020-06-22 12:49 ` Stefan Hajnoczi 2020-06-22 12:49 ` Stefan Hajnoczi
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=1591877734-66527-4-git-send-email-yi.l.liu@intel.com \ --to=yi.l.liu@intel.com \ --cc=alex.williamson@redhat.com \ --cc=ashok.raj@intel.com \ --cc=baolu.lu@linux.intel.com \ --cc=eric.auger@redhat.com \ --cc=hao.wu@intel.com \ --cc=iommu@lists.linux-foundation.org \ --cc=jacob.jun.pan@linux.intel.com \ --cc=jean-philippe@linaro.org \ --cc=joro@8bytes.org \ --cc=jun.j.tian@intel.com \ --cc=kevin.tian@intel.com \ --cc=kvm@vger.kernel.org \ --cc=linux-kernel@vger.kernel.org \ --cc=peterx@redhat.com \ --cc=yi.y.sun@intel.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.