From: "Isaac J. Manjarres" <isaacm@codeaurora.org>
To: iommu@lists.linux-foundation.org, linux-arm-kernel@lists.infradead.org
Cc: "Isaac J . Manjarres" <isaacm@codeaurora.org>,
robin.murphy@arm.com, Will Deacon <will@kernel.org>,
pratikp@codeaurora.org
Subject: [RFC PATCH v5 07/15] iommu: Hook up '->unmap_pages' driver callback
Date: Thu, 8 Apr 2021 10:13:54 -0700 [thread overview]
Message-ID: <20210408171402.12607-8-isaacm@codeaurora.org> (raw)
In-Reply-To: <20210408171402.12607-1-isaacm@codeaurora.org>
From: Will Deacon <will@kernel.org>
Extend iommu_pgsize() to populate an optional 'count' parameter so that
we can direct unmapping operation to the ->unmap_pages callback if it
has been provided by the driver.
Signed-off-by: Will Deacon <will@kernel.org>
Signed-off-by: Isaac J. Manjarres <isaacm@codeaurora.org>
---
drivers/iommu/iommu.c | 60 ++++++++++++++++++++++++++++++++++++-------
1 file changed, 51 insertions(+), 9 deletions(-)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 624ce3c7ae33..1fc919ea95ac 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2358,11 +2358,11 @@ phys_addr_t iommu_iova_to_phys(struct iommu_domain *domain, dma_addr_t iova)
EXPORT_SYMBOL_GPL(iommu_iova_to_phys);
static size_t iommu_pgsize(struct iommu_domain *domain, unsigned long iova,
- phys_addr_t paddr, size_t size)
+ phys_addr_t paddr, size_t size, size_t *count)
{
- unsigned int pgsize_idx;
+ unsigned int pgsize_idx, pgsize_idx_next;
unsigned long pgsizes;
- size_t pgsize;
+ size_t offset, pgsize, pgsize_next;
unsigned long addr_merge = paddr | iova;
/* Page sizes supported by the hardware and small enough for @size */
@@ -2378,7 +2378,37 @@ static size_t iommu_pgsize(struct iommu_domain *domain, unsigned long iova,
/* Pick the biggest page size remaining */
pgsize_idx = __fls(pgsizes);
pgsize = BIT(pgsize_idx);
+ if (!count)
+ return pgsize;
+
+ /* Find the next biggest support page size, if it exists */
+ pgsizes = domain->pgsize_bitmap & ~GENMASK(pgsize_idx, 0);
+ if (!pgsizes)
+ goto out_set_count;
+
+ pgsize_idx_next = __ffs(pgsizes);
+ pgsize_next = BIT(pgsize_idx_next);
+
+ /*
+ * There's no point trying a bigger page size unless the virtual
+ * and physical addresses are similarly offset within the larger page.
+ */
+ if ((iova ^ paddr) & (pgsize_next - 1))
+ goto out_set_count;
+
+ /* Calculate the offset to the next page size alignment boundary */
+ offset = pgsize_next - (addr_merge & (pgsize_next - 1));
+
+ /*
+ * If size is big enough to accommodate the larger page, reduce
+ * the number of smaller pages.
+ */
+ if (offset + pgsize_next <= size)
+ size = offset;
+
+out_set_count:
+ *count = size >> pgsize_idx;
return pgsize;
}
@@ -2416,7 +2446,7 @@ static int __iommu_map(struct iommu_domain *domain, unsigned long iova,
pr_debug("map: iova 0x%lx pa %pa size 0x%zx\n", iova, &paddr, size);
while (size) {
- size_t pgsize = iommu_pgsize(domain, iova, paddr, size);
+ size_t pgsize = iommu_pgsize(domain, iova, paddr, size, NULL);
pr_debug("mapping: iova 0x%lx pa %pa pgsize 0x%zx\n",
iova, &paddr, pgsize);
@@ -2467,6 +2497,19 @@ int iommu_map_atomic(struct iommu_domain *domain, unsigned long iova,
}
EXPORT_SYMBOL_GPL(iommu_map_atomic);
+static size_t __iommu_unmap_pages(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ struct iommu_iotlb_gather *iotlb_gather)
+{
+ const struct iommu_ops *ops = domain->ops;
+ size_t pgsize, count;
+
+ pgsize = iommu_pgsize(domain, iova, iova, size, &count);
+ return ops->unmap_pages ?
+ ops->unmap_pages(domain, iova, pgsize, count, iotlb_gather) :
+ ops->unmap(domain, iova, pgsize, iotlb_gather);
+}
+
static size_t __iommu_unmap(struct iommu_domain *domain,
unsigned long iova, size_t size,
struct iommu_iotlb_gather *iotlb_gather)
@@ -2476,7 +2519,7 @@ static size_t __iommu_unmap(struct iommu_domain *domain,
unsigned long orig_iova = iova;
unsigned int min_pagesz;
- if (unlikely(ops->unmap == NULL ||
+ if (unlikely(!(ops->unmap || ops->unmap_pages) ||
domain->pgsize_bitmap == 0UL))
return 0;
@@ -2504,10 +2547,9 @@ static size_t __iommu_unmap(struct iommu_domain *domain,
* or we hit an area that isn't mapped.
*/
while (unmapped < size) {
- size_t pgsize;
-
- pgsize = iommu_pgsize(domain, iova, iova, size - unmapped);
- unmapped_page = ops->unmap(domain, iova, pgsize, iotlb_gather);
+ unmapped_page = __iommu_unmap_pages(domain, iova,
+ size - unmapped,
+ iotlb_gather);
if (!unmapped_page)
break;
--
The Qualcomm Innovation Center, Inc. is a member of the Code Aurora Forum,
a Linux Foundation Collaborative Project
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-04-08 17:14 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-08 17:13 [RFC PATCH v5 00/15] Optimizing iommu_[map/unmap] performance Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 01/15] iommu/io-pgtable: Introduce unmap_pages() as a page table op Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 02/15] iommu: Add an unmap_pages() op for IOMMU drivers Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 03/15] iommu/io-pgtable: Introduce map_pages() as a page table op Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 04/15] iommu: Add a map_pages() op for IOMMU drivers Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 05/15] iommu: Use bitmap to calculate page size in iommu_pgsize() Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 06/15] iommu: Split 'addr_merge' argument to iommu_pgsize() into separate parts Isaac J. Manjarres
2021-04-08 17:13 ` Isaac J. Manjarres [this message]
2021-04-08 17:13 ` [RFC PATCH v5 08/15] iommu: Add support for the map_pages() callback Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 09/15] iommu/io-pgtable-arm: Prepare PTE methods for handling multiple entries Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 10/15] iommu/io-pgtable-arm: Implement arm_lpae_unmap_pages() Isaac J. Manjarres
2021-04-08 17:13 ` [RFC PATCH v5 11/15] iommu/io-pgtable-arm: Implement arm_lpae_map_pages() Isaac J. Manjarres
2021-04-20 5:59 ` chenxiang (M)
2021-04-08 17:13 ` [RFC PATCH v5 12/15] iommu/io-pgtable-arm-v7s: Implement arm_v7s_unmap_pages() Isaac J. Manjarres
2021-04-08 17:14 ` [RFC PATCH v5 13/15] iommu/io-pgtable-arm-v7s: Implement arm_v7s_map_pages() Isaac J. Manjarres
2021-04-08 17:14 ` [RFC PATCH v5 14/15] iommu/arm-smmu: Implement the unmap_pages() IOMMU driver callback Isaac J. Manjarres
2021-04-08 17:14 ` [RFC PATCH v5 15/15] iommu/arm-smmu: Implement the map_pages() " Isaac J. Manjarres
2021-06-11 3:10 ` [RFC PATCH v5 00/15] Optimizing iommu_[map/unmap] performance Lu Baolu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210408171402.12607-8-isaacm@codeaurora.org \
--to=isaacm@codeaurora.org \
--cc=iommu@lists.linux-foundation.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=pratikp@codeaurora.org \
--cc=robin.murphy@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).