From: Eric Auger <eric.auger@redhat.com> To: eric.auger.pro@gmail.com, eric.auger@redhat.com, joro@8bytes.org, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, will.deacon@arm.com, robin.murphy@arm.com, dwmw2@infradead.org, alex.williamson@redhat.com Cc: baolu.lu@linux.intel.com, shameerali.kolothum.thodi@huawei.com, jean-philippe.brucker@arm.com Subject: [RFC 2/3] iommu: Account for dma_mask and iommu aperture in IOVA reserved regions Date: Mon, 28 Sep 2020 21:50:36 +0200 [thread overview] Message-ID: <20200928195037.22654-3-eric.auger@redhat.com> (raw) In-Reply-To: <20200928195037.22654-1-eric.auger@redhat.com> VFIO currently exposes the usable IOVA regions through the VFIO_IOMMU_GET_INFO ioctl. However it fails to take into account the dma_mask of the devices within the container. The top limit currently is defined by the iommu aperture. So, for instance, if the IOMMU supports up to 48bits, it may give the impression the max IOVA is 48b while a device may have a dma_mask of 42b. So this API cannot really be used to compute the max usable IOVA. This patch removes the IOVA region beyond the dma_mask's. As we start to expose this reserved region in the sysfs file /sys/kernel/iommu_groups/<n>/reserved_regions, we also need to expose the IOVA range beyond the IOMMU aperture to handle the case where the dma_mask would have a higher number of bits than the iommu max input address. Those out-of-reach regions get the IOMMU_RESV_RESERVED type. This is a change to the ABI as this reserved region was not yet exposed in sysfs /sys/kernel/iommu_groups/<n>/reserved_regions or through the VFIO ioctl. Document that change. Signed-off-by: Eric Auger <eric.auger@redhat.com> --- .../ABI/testing/sysfs-kernel-iommu_groups | 7 ++++ drivers/iommu/iommu.c | 39 +++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-kernel-iommu_groups b/Documentation/ABI/testing/sysfs-kernel-iommu_groups index 017f5bc3920c..2f316686c88b 100644 --- a/Documentation/ABI/testing/sysfs-kernel-iommu_groups +++ b/Documentation/ABI/testing/sysfs-kernel-iommu_groups @@ -33,3 +33,10 @@ Description: In case an RMRR is used only by graphics or USB devices it is now exposed as "direct-relaxable" instead of "direct". In device assignment use case, for instance, those RMRR are considered to be relaxable and safe. + +What: /sys/kernel/iommu_groups/reserved_regions +Date: Sept 2020 +KernelVersion: v5.11 +Contact: Eric Auger <eric.auger@redhat.com> +Description: Regions beyond the device dma_mask and the iommu aperture + now are exposed as IOMMU_RESV_RESERVED reserved regions. diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index dd8cda340e62..d797f07b3625 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2511,9 +2511,48 @@ EXPORT_SYMBOL_GPL(iommu_domain_set_attr); void iommu_get_resv_regions(struct device *dev, struct list_head *list) { const struct iommu_ops *ops = dev->bus->iommu_ops; + struct iommu_resv_region *region; + struct iommu_domain *domain; + + domain = iommu_get_domain_for_dev(dev); + + if (domain) { + struct iommu_domain_geometry geo; + + if (iommu_domain_get_attr(domain, DOMAIN_ATTR_GEOMETRY, &geo)) + return; + + if (geo.aperture_end < ULLONG_MAX && geo.aperture_end != geo.aperture_start) { + region = iommu_alloc_resv_region(geo.aperture_end + 1, + ULLONG_MAX - geo.aperture_end, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } + + if (geo.aperture_start > 0) { + region = iommu_alloc_resv_region(0, geo.aperture_start, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } + } if (ops && ops->get_resv_regions) ops->get_resv_regions(dev, list); + + if (!dev->dma_mask || *dev->dma_mask == ULLONG_MAX) + return; + + region = iommu_alloc_resv_region(*dev->dma_mask + 1, + ULLONG_MAX - *dev->dma_mask, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + + list_add_tail(®ion->list, list); } void iommu_put_resv_regions(struct device *dev, struct list_head *list) -- 2.21.3
WARNING: multiple messages have this Message-ID (diff)
From: Eric Auger <eric.auger@redhat.com> To: eric.auger.pro@gmail.com, eric.auger@redhat.com, joro@8bytes.org, iommu@lists.linux-foundation.org, linux-kernel@vger.kernel.org, will.deacon@arm.com, robin.murphy@arm.com, dwmw2@infradead.org, alex.williamson@redhat.com Cc: jean-philippe.brucker@arm.com Subject: [RFC 2/3] iommu: Account for dma_mask and iommu aperture in IOVA reserved regions Date: Mon, 28 Sep 2020 21:50:36 +0200 [thread overview] Message-ID: <20200928195037.22654-3-eric.auger@redhat.com> (raw) In-Reply-To: <20200928195037.22654-1-eric.auger@redhat.com> VFIO currently exposes the usable IOVA regions through the VFIO_IOMMU_GET_INFO ioctl. However it fails to take into account the dma_mask of the devices within the container. The top limit currently is defined by the iommu aperture. So, for instance, if the IOMMU supports up to 48bits, it may give the impression the max IOVA is 48b while a device may have a dma_mask of 42b. So this API cannot really be used to compute the max usable IOVA. This patch removes the IOVA region beyond the dma_mask's. As we start to expose this reserved region in the sysfs file /sys/kernel/iommu_groups/<n>/reserved_regions, we also need to expose the IOVA range beyond the IOMMU aperture to handle the case where the dma_mask would have a higher number of bits than the iommu max input address. Those out-of-reach regions get the IOMMU_RESV_RESERVED type. This is a change to the ABI as this reserved region was not yet exposed in sysfs /sys/kernel/iommu_groups/<n>/reserved_regions or through the VFIO ioctl. Document that change. Signed-off-by: Eric Auger <eric.auger@redhat.com> --- .../ABI/testing/sysfs-kernel-iommu_groups | 7 ++++ drivers/iommu/iommu.c | 39 +++++++++++++++++++ 2 files changed, 46 insertions(+) diff --git a/Documentation/ABI/testing/sysfs-kernel-iommu_groups b/Documentation/ABI/testing/sysfs-kernel-iommu_groups index 017f5bc3920c..2f316686c88b 100644 --- a/Documentation/ABI/testing/sysfs-kernel-iommu_groups +++ b/Documentation/ABI/testing/sysfs-kernel-iommu_groups @@ -33,3 +33,10 @@ Description: In case an RMRR is used only by graphics or USB devices it is now exposed as "direct-relaxable" instead of "direct". In device assignment use case, for instance, those RMRR are considered to be relaxable and safe. + +What: /sys/kernel/iommu_groups/reserved_regions +Date: Sept 2020 +KernelVersion: v5.11 +Contact: Eric Auger <eric.auger@redhat.com> +Description: Regions beyond the device dma_mask and the iommu aperture + now are exposed as IOMMU_RESV_RESERVED reserved regions. diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c index dd8cda340e62..d797f07b3625 100644 --- a/drivers/iommu/iommu.c +++ b/drivers/iommu/iommu.c @@ -2511,9 +2511,48 @@ EXPORT_SYMBOL_GPL(iommu_domain_set_attr); void iommu_get_resv_regions(struct device *dev, struct list_head *list) { const struct iommu_ops *ops = dev->bus->iommu_ops; + struct iommu_resv_region *region; + struct iommu_domain *domain; + + domain = iommu_get_domain_for_dev(dev); + + if (domain) { + struct iommu_domain_geometry geo; + + if (iommu_domain_get_attr(domain, DOMAIN_ATTR_GEOMETRY, &geo)) + return; + + if (geo.aperture_end < ULLONG_MAX && geo.aperture_end != geo.aperture_start) { + region = iommu_alloc_resv_region(geo.aperture_end + 1, + ULLONG_MAX - geo.aperture_end, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } + + if (geo.aperture_start > 0) { + region = iommu_alloc_resv_region(0, geo.aperture_start, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + list_add_tail(®ion->list, list); + } + } if (ops && ops->get_resv_regions) ops->get_resv_regions(dev, list); + + if (!dev->dma_mask || *dev->dma_mask == ULLONG_MAX) + return; + + region = iommu_alloc_resv_region(*dev->dma_mask + 1, + ULLONG_MAX - *dev->dma_mask, + 0, IOMMU_RESV_RESERVED); + if (!region) + return; + + list_add_tail(®ion->list, list); } void iommu_put_resv_regions(struct device *dev, struct list_head *list) -- 2.21.3 _______________________________________________ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2020-09-28 19:51 UTC|newest] Thread overview: 26+ messages / expand[flat|nested] mbox.gz Atom feed top 2020-09-28 19:50 [RFC 0/3] iommu: Reserved regions for IOVAs beyond dma_mask and iommu aperture Eric Auger 2020-09-28 19:50 ` Eric Auger 2020-09-28 19:50 ` [RFC 1/3] iommu: Fix merging in iommu_insert_resv_region Eric Auger 2020-09-28 19:50 ` Eric Auger 2020-09-28 19:50 ` Eric Auger [this message] 2020-09-28 19:50 ` [RFC 2/3] iommu: Account for dma_mask and iommu aperture in IOVA reserved regions Eric Auger 2020-09-29 6:03 ` Christoph Hellwig 2020-09-29 6:03 ` Christoph Hellwig 2020-09-29 7:20 ` Auger Eric 2020-09-29 7:20 ` Auger Eric 2020-09-28 19:50 ` [RFC 3/3] vfio/type1: Increase the version of VFIO_IOMMU_TYPE1_INFO_CAP_IOVA_RANGE Eric Auger 2020-09-28 19:50 ` Eric Auger 2020-09-28 22:42 ` [RFC 0/3] iommu: Reserved regions for IOVAs beyond dma_mask and iommu aperture Alex Williamson 2020-09-28 22:42 ` Alex Williamson 2020-09-29 7:18 ` Auger Eric 2020-09-29 7:18 ` Auger Eric 2020-09-29 18:18 ` Alex Williamson 2020-09-29 18:18 ` Alex Williamson 2020-09-30 9:59 ` Auger Eric 2020-09-30 9:59 ` Auger Eric 2020-10-05 10:44 ` Lorenzo Pieralisi 2020-10-05 10:44 ` Lorenzo Pieralisi 2020-10-05 13:08 ` Christoph Hellwig 2020-10-05 13:08 ` Christoph Hellwig 2020-10-06 15:41 ` Auger Eric 2020-10-06 15:41 ` Auger Eric
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20200928195037.22654-3-eric.auger@redhat.com \ --to=eric.auger@redhat.com \ --cc=alex.williamson@redhat.com \ --cc=baolu.lu@linux.intel.com \ --cc=dwmw2@infradead.org \ --cc=eric.auger.pro@gmail.com \ --cc=iommu@lists.linux-foundation.org \ --cc=jean-philippe.brucker@arm.com \ --cc=joro@8bytes.org \ --cc=linux-kernel@vger.kernel.org \ --cc=robin.murphy@arm.com \ --cc=shameerali.kolothum.thodi@huawei.com \ --cc=will.deacon@arm.com \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.