From: Keqian Zhu <zhukeqian1@huawei.com>
To: <linux-kernel@vger.kernel.org>,
<linux-arm-kernel@lists.infradead.org>,
<iommu@lists.linux-foundation.org>,
Robin Murphy <robin.murphy@arm.com>,
"Will Deacon" <will@kernel.org>, Joerg Roedel <joro@8bytes.org>,
Yi Sun <yi.y.sun@linux.intel.com>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
Jonathan Cameron <Jonathan.Cameron@huawei.com>,
Tian Kevin <kevin.tian@intel.com>,
Lu Baolu <baolu.lu@linux.intel.com>
Cc: Alex Williamson <alex.williamson@redhat.com>,
Cornelia Huck <cohuck@redhat.com>,
Kirti Wankhede <kwankhede@nvidia.com>,
<wanghaibin.wang@huawei.com>, <jiangkunkun@huawei.com>,
<yuzenghui@huawei.com>, <lushenming@huawei.com>
Subject: [PATCH v3 03/12] iommu: Add iommu_merge_page interface
Date: Tue, 13 Apr 2021 16:54:48 +0800 [thread overview]
Message-ID: <20210413085457.25400-4-zhukeqian1@huawei.com> (raw)
In-Reply-To: <20210413085457.25400-1-zhukeqian1@huawei.com>
If block(largepage) mappings are split during start dirty log, then
when stop dirty log, we need to recover them for better DMA performance.
This adds a new interface named iommu_merge_page in IOMMU base layer.
A specific IOMMU driver can invoke it during stop dirty log. If so, the
driver also need to realize the merge_page iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of iommu, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 12 +++++++
2 files changed, 87 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index bb413a927870..8f0d71bafb3a 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2762,6 +2762,81 @@ int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
}
EXPORT_SYMBOL_GPL(iommu_split_block);
+static int __iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->merge_page))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | paddr | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx pa %pa size 0x%zx min_pagesz 0x%x\n",
+ iova, &paddr, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova | paddr, size);
+
+ ret = ops->merge_page(domain, iova, paddr, pgsize, prot);
+ if (ret)
+ break;
+
+ pr_debug("merge handled: iova 0x%lx pa %pa size 0x%zx\n",
+ iova, &paddr, pgsize);
+
+ iova += pgsize;
+ paddr += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot)
+{
+ phys_addr_t phys;
+ dma_addr_t p, i;
+ size_t cont_size;
+ bool flush = false;
+ int ret = 0;
+
+ while (size) {
+ flush = true;
+
+ phys = iommu_iova_to_phys(domain, iova);
+ cont_size = PAGE_SIZE;
+ p = phys + cont_size;
+ i = iova + cont_size;
+
+ while (cont_size < size && p == iommu_iova_to_phys(domain, i)) {
+ p += PAGE_SIZE;
+ i += PAGE_SIZE;
+ cont_size += PAGE_SIZE;
+ }
+
+ ret = __iommu_merge_page(domain, iova, phys, cont_size, prot);
+ if (ret)
+ break;
+
+ iova += cont_size;
+ size -= cont_size;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_merge_page);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index c6c90ac069e3..fea3ecabff3d 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -209,6 +209,7 @@ struct iommu_iotlb_gather {
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
* @split_block: Split block mapping into page mapping
+ * @merge_page: Merge page mapping into block mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -270,6 +271,8 @@ struct iommu_ops {
/* Track dirty log */
int (*split_block)(struct iommu_domain *domain, unsigned long iova,
size_t size);
+ int (*merge_page)(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -534,6 +537,8 @@ extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
size_t size);
+extern int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -940,6 +945,13 @@ static inline int iommu_split_block(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ int prot)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
next prev parent reply other threads:[~2021-04-13 8:55 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-04-13 8:54 [PATCH v3 00/12] iommu/smmuv3: Implement hardware dirty log tracking Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 01/12] iommu: Introduce dirty log tracking framework Keqian Zhu
2021-04-14 7:00 ` Lu Baolu
2021-04-15 6:18 ` Keqian Zhu
2021-04-15 7:03 ` Lu Baolu
2021-04-15 7:43 ` Keqian Zhu
2021-04-15 10:21 ` Lu Baolu
2021-04-16 9:07 ` Keqian Zhu
2021-04-19 1:59 ` Lu Baolu
2021-04-13 8:54 ` [PATCH v3 02/12] iommu: Add iommu_split_block interface Keqian Zhu
2021-04-14 7:14 ` Lu Baolu
2021-04-19 9:32 ` Keqian Zhu
2021-04-19 13:33 ` Lu Baolu
2021-04-20 1:25 ` Keqian Zhu
2021-04-20 2:09 ` Lu Baolu
2021-04-20 7:32 ` Keqian Zhu
2021-04-20 7:53 ` Lu Baolu
2021-04-13 8:54 ` Keqian Zhu [this message]
2021-04-13 8:54 ` [PATCH v3 04/12] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 05/12] iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 06/12] iommu/arm-smmu-v3: Add feature detection for BBML Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 07/12] iommu/arm-smmu-v3: Realize split_block iommu ops Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 08/12] iommu/arm-smmu-v3: Realize merge_page " Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 09/12] iommu/arm-smmu-v3: Realize switch_dirty_log " Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 10/12] iommu/arm-smmu-v3: Realize sync_dirty_log " Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 11/12] iommu/arm-smmu-v3: Realize clear_dirty_log " Keqian Zhu
2021-04-13 8:54 ` [PATCH v3 12/12] iommu/arm-smmu-v3: Add HWDBM device feature reporting Keqian Zhu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210413085457.25400-4-zhukeqian1@huawei.com \
--to=zhukeqian1@huawei.com \
--cc=Jonathan.Cameron@huawei.com \
--cc=alex.williamson@redhat.com \
--cc=baolu.lu@linux.intel.com \
--cc=cohuck@redhat.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jean-philippe@linaro.org \
--cc=jiangkunkun@huawei.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kwankhede@nvidia.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lushenming@huawei.com \
--cc=robin.murphy@arm.com \
--cc=wanghaibin.wang@huawei.com \
--cc=will@kernel.org \
--cc=yi.y.sun@linux.intel.com \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).