* [PATCH v3 00/12] iommu/smmuv3: Implement hardware dirty log tracking
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
This patch series is split from the series[1] that containes both IOMMU part and
VFIO part. The new VFIO part will be sent out in another series.
[1] https://lore.kernel.org/linux-iommu/20210310090614.26668-1-zhukeqian1@huawei.com/
changelog:
v3:
- Merge start_dirty_log and stop_dirty_log into switch_dirty_log. (Yi Sun)
- Maintain the dirty log status in iommu_domain.
- Update commit message to make patch easier to review.
v2:
- Address all comments of RFC version, thanks for all of you ;-)
- Add a bugfix that start dirty log for newly added dma ranges and domain.
Hi everyone,
This patch series introduces a framework of iommu dirty log tracking, and smmuv3
realizes this framework. This new feature can be used by VFIO dma dirty tracking.
Intention：
Some types of IOMMU are capable of tracking DMA dirty log, such as
ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
dirty log tracking framework in the IOMMU base layer.
Three new essential interfaces are added, and we maintaince the status
of dirty log tracking in iommu_domain.
1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
About SMMU HTTU:
HTTU (Hardware Translation Table Update) is a feature of ARM SMMUv3, it can update
access flag or/and dirty state of the TTD (Translation Table Descriptor) by hardware.
With HTTU, stage1 TTD is classified into 3 types:
DBM bit AP[2](readonly bit)
1. writable_clean 1 1
2. writable_dirty 1 0
3. readonly 0 1
If HTTU_HD (manage dirty state) is enabled, smmu can change TTD from writable_clean to
writable_dirty. Then software can scan TTD to sync dirty state into dirty bitmap. With
this feature, we can track the dirty log of DMA continuously and precisely.
About this series:
Patch 1-3：Introduce dirty log tracking framework in the IOMMU base layer, and two common
interfaces that can be used by many types of iommu.
Patch 4-6: Add feature detection for smmu HTTU and enable HTTU for smmu stage1 mapping.
And add feature detection for smmu BBML. We need to split block mapping when
start dirty log tracking and merge page mapping when stop dirty log tracking,
which requires break-before-make procedure. But it might cause problems when the
TTD is alive. The I/O streams might not tolerate translation faults. So BBML
should be used.
Patch 7-12: We implement these interfaces for arm smmuv3.
Thanks,
Keqian
Jean-Philippe Brucker (1):
iommu/arm-smmu-v3: Add support for Hardware Translation Table Update
Keqian Zhu (3):
iommu: Introduce dirty log tracking framework
iommu: Add iommu_split_block interface
iommu: Add iommu_merge_page interface
Kunkun Jiang (8):
iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping
iommu/arm-smmu-v3: Add feature detection for BBML
iommu/arm-smmu-v3: Realize split_block iommu ops
iommu/arm-smmu-v3: Realize merge_page iommu ops
iommu/arm-smmu-v3: Realize switch_dirty_log iommu ops
iommu/arm-smmu-v3: Realize sync_dirty_log iommu ops
iommu/arm-smmu-v3: Realize clear_dirty_log iommu ops
iommu/arm-smmu-v3: Add HWDBM device feature reporting
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 217 +++++++++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 14 +
drivers/iommu/io-pgtable-arm.c | 392 +++++++++++++++++-
drivers/iommu/iommu.c | 266 ++++++++++++
include/linux/io-pgtable.h | 23 +
include/linux/iommu.h | 76 ++++
7 files changed, 988 insertions(+), 2 deletions(-)
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* [PATCH v3 00/12] iommu/smmuv3: Implement hardware dirty log tracking
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
This patch series is split from the series[1] that containes both IOMMU part and
VFIO part. The new VFIO part will be sent out in another series.
[1] https://lore.kernel.org/linux-iommu/20210310090614.26668-1-zhukeqian1@huawei.com/
changelog:
v3:
- Merge start_dirty_log and stop_dirty_log into switch_dirty_log. (Yi Sun)
- Maintain the dirty log status in iommu_domain.
- Update commit message to make patch easier to review.
v2:
- Address all comments of RFC version, thanks for all of you ;-)
- Add a bugfix that start dirty log for newly added dma ranges and domain.
Hi everyone,
This patch series introduces a framework of iommu dirty log tracking, and smmuv3
realizes this framework. This new feature can be used by VFIO dma dirty tracking.
Intention：
Some types of IOMMU are capable of tracking DMA dirty log, such as
ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
dirty log tracking framework in the IOMMU base layer.
Three new essential interfaces are added, and we maintaince the status
of dirty log tracking in iommu_domain.
1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
About SMMU HTTU:
HTTU (Hardware Translation Table Update) is a feature of ARM SMMUv3, it can update
access flag or/and dirty state of the TTD (Translation Table Descriptor) by hardware.
With HTTU, stage1 TTD is classified into 3 types:
DBM bit AP[2](readonly bit)
1. writable_clean 1 1
2. writable_dirty 1 0
3. readonly 0 1
If HTTU_HD (manage dirty state) is enabled, smmu can change TTD from writable_clean to
writable_dirty. Then software can scan TTD to sync dirty state into dirty bitmap. With
this feature, we can track the dirty log of DMA continuously and precisely.
About this series:
Patch 1-3：Introduce dirty log tracking framework in the IOMMU base layer, and two common
interfaces that can be used by many types of iommu.
Patch 4-6: Add feature detection for smmu HTTU and enable HTTU for smmu stage1 mapping.
And add feature detection for smmu BBML. We need to split block mapping when
start dirty log tracking and merge page mapping when stop dirty log tracking,
which requires break-before-make procedure. But it might cause problems when the
TTD is alive. The I/O streams might not tolerate translation faults. So BBML
should be used.
Patch 7-12: We implement these interfaces for arm smmuv3.
Thanks,
Keqian
Jean-Philippe Brucker (1):
iommu/arm-smmu-v3: Add support for Hardware Translation Table Update
Keqian Zhu (3):
iommu: Introduce dirty log tracking framework
iommu: Add iommu_split_block interface
iommu: Add iommu_merge_page interface
Kunkun Jiang (8):
iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping
iommu/arm-smmu-v3: Add feature detection for BBML
iommu/arm-smmu-v3: Realize split_block iommu ops
iommu/arm-smmu-v3: Realize merge_page iommu ops
iommu/arm-smmu-v3: Realize switch_dirty_log iommu ops
iommu/arm-smmu-v3: Realize sync_dirty_log iommu ops
iommu/arm-smmu-v3: Realize clear_dirty_log iommu ops
iommu/arm-smmu-v3: Add HWDBM device feature reporting
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 217 +++++++++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 14 +
drivers/iommu/io-pgtable-arm.c | 392 +++++++++++++++++-
drivers/iommu/iommu.c | 266 ++++++++++++
include/linux/io-pgtable.h | 23 +
include/linux/iommu.h | 76 ++++
7 files changed, 988 insertions(+), 2 deletions(-)
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Some types of IOMMU are capable of tracking DMA dirty log, such as
ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
dirty log tracking framework in the IOMMU base layer.
Three new essential interfaces are added, and we maintaince the status
of dirty log tracking in iommu_domain.
1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
A new dev feature are added to indicate whether a specific type of
iommu hardware supports and its driver realizes them.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 53 +++++++++++++++
2 files changed, 203 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index d0b0a15dba84..667b2d6d2fc0 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
domain->type = type;
/* Assume all sizes by default; the driver may override this later */
domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
+ mutex_init(&domain->switch_log_lock);
return domain;
}
@@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ int ret;
+
+ if (unlikely(!ops || !ops->switch_dirty_log))
+ return -ENODEV;
+
+ mutex_lock(&domain->switch_log_lock);
+ if (enable && domain->dirty_log_tracking) {
+ ret = -EBUSY;
+ goto out;
+ } else if (!enable && !domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
+ if (ret)
+ goto out;
+
+ domain->dirty_log_tracking = enable;
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
+
+int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t size, unsigned long *bitmap,
+ unsigned long base_iova, unsigned long bitmap_pgshift)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->sync_dirty_log))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
+ iova, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ mutex_lock(&domain->switch_log_lock);
+ if (!domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->sync_dirty_log(domain, iova, pgsize,
+ bitmap, base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+
+ pr_debug("dirty_log_sync handle: iova 0x%lx pagesz 0x%zx\n",
+ iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_sync_dirty_log);
+
+static int __iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ const struct iommu_ops *ops = domain->ops;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->clear_dirty_log))
+ return -ENODEV;
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->clear_dirty_log(domain, iova, pgsize, bitmap,
+ base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+
+ pr_debug("dirty_log_clear handled: iova 0x%lx pagesz 0x%zx\n",
+ iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ unsigned long riova, rsize;
+ unsigned int min_pagesz;
+ bool flush = false;
+ int rs, re, start, end;
+ int ret = 0;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx min_pagesz 0x%x\n",
+ iova, min_pagesz);
+ return -EINVAL;
+ }
+
+ mutex_lock(&domain->switch_log_lock);
+ if (!domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ start = (iova - base_iova) >> bitmap_pgshift;
+ end = start + (size >> bitmap_pgshift);
+ bitmap_for_each_set_region(bitmap, rs, re, start, end) {
+ flush = true;
+ riova = iova + (rs << bitmap_pgshift);
+ rsize = (re - rs) << bitmap_pgshift;
+ ret = __iommu_clear_dirty_log(domain, riova, rsize, bitmap,
+ base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_clear_dirty_log);
+
void iommu_get_resv_regions(struct device *dev, struct list_head *list)
{
const struct iommu_ops *ops = dev->bus->iommu_ops;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 5e7fe519430a..7f9ed9f520e2 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -87,6 +87,8 @@ struct iommu_domain {
void *handler_token;
struct iommu_domain_geometry geometry;
void *iova_cookie;
+ bool dirty_log_tracking;
+ struct mutex switch_log_lock;
};
enum iommu_cap {
@@ -160,6 +162,7 @@ struct iommu_resv_region {
enum iommu_dev_features {
IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */
IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */
+ IOMMU_DEV_FEAT_HWDBM, /* Hardware Dirty Bit Management */
};
#define IOMMU_PASID_INVALID (-1U)
@@ -205,6 +208,9 @@ struct iommu_iotlb_gather {
* @device_group: find iommu group for a particular device
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
+ * @switch_dirty_log: Perform actions to start|stop dirty log tracking
+ * @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
+ * @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
* @get_resv_regions: Request list of reserved regions for a device
* @put_resv_regions: Free list of reserved regions for a device
* @apply_resv_region: Temporary helper call-back for iova reserved ranges
@@ -260,6 +266,18 @@ struct iommu_ops {
int (*domain_set_attr)(struct iommu_domain *domain,
enum iommu_attr attr, void *data);
+ /* Track dirty log */
+ int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot);
+ int (*sync_dirty_log)(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+ int (*clear_dirty_log)(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+
/* Request/Free a list of reserved regions for a device */
void (*get_resv_regions)(struct device *dev, struct list_head *list);
void (*put_resv_regions)(struct device *dev, struct list_head *list);
@@ -511,6 +529,16 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
+extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot);
+extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t size, unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+extern int iommu_clear_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t dma_size, unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift);
/* Window handling function prototypes */
extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
@@ -901,6 +929,31 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
+ bool enable, unsigned long iova,
+ size_t size, int prot)
+{
+ return -EINVAL;
+}
+
+static inline int iommu_sync_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long pgshift)
+{
+ return -EINVAL;
+}
+
+static inline int iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long pgshift)
+{
+ return -EINVAL;
+}
+
static inline int iommu_device_register(struct iommu_device *iommu)
{
return -ENODEV;
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Some types of IOMMU are capable of tracking DMA dirty log, such as
ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
dirty log tracking framework in the IOMMU base layer.
Three new essential interfaces are added, and we maintaince the status
of dirty log tracking in iommu_domain.
1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
A new dev feature are added to indicate whether a specific type of
iommu hardware supports and its driver realizes them.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 53 +++++++++++++++
2 files changed, 203 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index d0b0a15dba84..667b2d6d2fc0 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
domain->type = type;
/* Assume all sizes by default; the driver may override this later */
domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
+ mutex_init(&domain->switch_log_lock);
return domain;
}
@@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ int ret;
+
+ if (unlikely(!ops || !ops->switch_dirty_log))
+ return -ENODEV;
+
+ mutex_lock(&domain->switch_log_lock);
+ if (enable && domain->dirty_log_tracking) {
+ ret = -EBUSY;
+ goto out;
+ } else if (!enable && !domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
+ if (ret)
+ goto out;
+
+ domain->dirty_log_tracking = enable;
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
+
+int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t size, unsigned long *bitmap,
+ unsigned long base_iova, unsigned long bitmap_pgshift)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->sync_dirty_log))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
+ iova, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ mutex_lock(&domain->switch_log_lock);
+ if (!domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->sync_dirty_log(domain, iova, pgsize,
+ bitmap, base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+
+ pr_debug("dirty_log_sync handle: iova 0x%lx pagesz 0x%zx\n",
+ iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_sync_dirty_log);
+
+static int __iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ const struct iommu_ops *ops = domain->ops;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->clear_dirty_log))
+ return -ENODEV;
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->clear_dirty_log(domain, iova, pgsize, bitmap,
+ base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+
+ pr_debug("dirty_log_clear handled: iova 0x%lx pagesz 0x%zx\n",
+ iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ unsigned long riova, rsize;
+ unsigned int min_pagesz;
+ bool flush = false;
+ int rs, re, start, end;
+ int ret = 0;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx min_pagesz 0x%x\n",
+ iova, min_pagesz);
+ return -EINVAL;
+ }
+
+ mutex_lock(&domain->switch_log_lock);
+ if (!domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ start = (iova - base_iova) >> bitmap_pgshift;
+ end = start + (size >> bitmap_pgshift);
+ bitmap_for_each_set_region(bitmap, rs, re, start, end) {
+ flush = true;
+ riova = iova + (rs << bitmap_pgshift);
+ rsize = (re - rs) << bitmap_pgshift;
+ ret = __iommu_clear_dirty_log(domain, riova, rsize, bitmap,
+ base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_clear_dirty_log);
+
void iommu_get_resv_regions(struct device *dev, struct list_head *list)
{
const struct iommu_ops *ops = dev->bus->iommu_ops;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 5e7fe519430a..7f9ed9f520e2 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -87,6 +87,8 @@ struct iommu_domain {
void *handler_token;
struct iommu_domain_geometry geometry;
void *iova_cookie;
+ bool dirty_log_tracking;
+ struct mutex switch_log_lock;
};
enum iommu_cap {
@@ -160,6 +162,7 @@ struct iommu_resv_region {
enum iommu_dev_features {
IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */
IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */
+ IOMMU_DEV_FEAT_HWDBM, /* Hardware Dirty Bit Management */
};
#define IOMMU_PASID_INVALID (-1U)
@@ -205,6 +208,9 @@ struct iommu_iotlb_gather {
* @device_group: find iommu group for a particular device
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
+ * @switch_dirty_log: Perform actions to start|stop dirty log tracking
+ * @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
+ * @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
* @get_resv_regions: Request list of reserved regions for a device
* @put_resv_regions: Free list of reserved regions for a device
* @apply_resv_region: Temporary helper call-back for iova reserved ranges
@@ -260,6 +266,18 @@ struct iommu_ops {
int (*domain_set_attr)(struct iommu_domain *domain,
enum iommu_attr attr, void *data);
+ /* Track dirty log */
+ int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot);
+ int (*sync_dirty_log)(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+ int (*clear_dirty_log)(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+
/* Request/Free a list of reserved regions for a device */
void (*get_resv_regions)(struct device *dev, struct list_head *list);
void (*put_resv_regions)(struct device *dev, struct list_head *list);
@@ -511,6 +529,16 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
+extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot);
+extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t size, unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+extern int iommu_clear_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t dma_size, unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift);
/* Window handling function prototypes */
extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
@@ -901,6 +929,31 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
+ bool enable, unsigned long iova,
+ size_t size, int prot)
+{
+ return -EINVAL;
+}
+
+static inline int iommu_sync_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long pgshift)
+{
+ return -EINVAL;
+}
+
+static inline int iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long pgshift)
+{
+ return -EINVAL;
+}
+
static inline int iommu_device_register(struct iommu_device *iommu)
{
return -ENODEV;
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Some types of IOMMU are capable of tracking DMA dirty log, such as
ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
dirty log tracking framework in the IOMMU base layer.
Three new essential interfaces are added, and we maintaince the status
of dirty log tracking in iommu_domain.
1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
A new dev feature are added to indicate whether a specific type of
iommu hardware supports and its driver realizes them.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 53 +++++++++++++++
2 files changed, 203 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index d0b0a15dba84..667b2d6d2fc0 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
domain->type = type;
/* Assume all sizes by default; the driver may override this later */
domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
+ mutex_init(&domain->switch_log_lock);
return domain;
}
@@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ int ret;
+
+ if (unlikely(!ops || !ops->switch_dirty_log))
+ return -ENODEV;
+
+ mutex_lock(&domain->switch_log_lock);
+ if (enable && domain->dirty_log_tracking) {
+ ret = -EBUSY;
+ goto out;
+ } else if (!enable && !domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
+ if (ret)
+ goto out;
+
+ domain->dirty_log_tracking = enable;
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
+
+int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t size, unsigned long *bitmap,
+ unsigned long base_iova, unsigned long bitmap_pgshift)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->sync_dirty_log))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
+ iova, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ mutex_lock(&domain->switch_log_lock);
+ if (!domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->sync_dirty_log(domain, iova, pgsize,
+ bitmap, base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+
+ pr_debug("dirty_log_sync handle: iova 0x%lx pagesz 0x%zx\n",
+ iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_sync_dirty_log);
+
+static int __iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ const struct iommu_ops *ops = domain->ops;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->clear_dirty_log))
+ return -ENODEV;
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->clear_dirty_log(domain, iova, pgsize, bitmap,
+ base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+
+ pr_debug("dirty_log_clear handled: iova 0x%lx pagesz 0x%zx\n",
+ iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ unsigned long riova, rsize;
+ unsigned int min_pagesz;
+ bool flush = false;
+ int rs, re, start, end;
+ int ret = 0;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx min_pagesz 0x%x\n",
+ iova, min_pagesz);
+ return -EINVAL;
+ }
+
+ mutex_lock(&domain->switch_log_lock);
+ if (!domain->dirty_log_tracking) {
+ ret = -EINVAL;
+ goto out;
+ }
+
+ start = (iova - base_iova) >> bitmap_pgshift;
+ end = start + (size >> bitmap_pgshift);
+ bitmap_for_each_set_region(bitmap, rs, re, start, end) {
+ flush = true;
+ riova = iova + (rs << bitmap_pgshift);
+ rsize = (re - rs) << bitmap_pgshift;
+ ret = __iommu_clear_dirty_log(domain, riova, rsize, bitmap,
+ base_iova, bitmap_pgshift);
+ if (ret)
+ break;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+out:
+ mutex_unlock(&domain->switch_log_lock);
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_clear_dirty_log);
+
void iommu_get_resv_regions(struct device *dev, struct list_head *list)
{
const struct iommu_ops *ops = dev->bus->iommu_ops;
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 5e7fe519430a..7f9ed9f520e2 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -87,6 +87,8 @@ struct iommu_domain {
void *handler_token;
struct iommu_domain_geometry geometry;
void *iova_cookie;
+ bool dirty_log_tracking;
+ struct mutex switch_log_lock;
};
enum iommu_cap {
@@ -160,6 +162,7 @@ struct iommu_resv_region {
enum iommu_dev_features {
IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */
IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */
+ IOMMU_DEV_FEAT_HWDBM, /* Hardware Dirty Bit Management */
};
#define IOMMU_PASID_INVALID (-1U)
@@ -205,6 +208,9 @@ struct iommu_iotlb_gather {
* @device_group: find iommu group for a particular device
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
+ * @switch_dirty_log: Perform actions to start|stop dirty log tracking
+ * @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
+ * @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
* @get_resv_regions: Request list of reserved regions for a device
* @put_resv_regions: Free list of reserved regions for a device
* @apply_resv_region: Temporary helper call-back for iova reserved ranges
@@ -260,6 +266,18 @@ struct iommu_ops {
int (*domain_set_attr)(struct iommu_domain *domain,
enum iommu_attr attr, void *data);
+ /* Track dirty log */
+ int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot);
+ int (*sync_dirty_log)(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+ int (*clear_dirty_log)(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+
/* Request/Free a list of reserved regions for a device */
void (*get_resv_regions)(struct device *dev, struct list_head *list);
void (*put_resv_regions)(struct device *dev, struct list_head *list);
@@ -511,6 +529,16 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
+extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot);
+extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t size, unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift);
+extern int iommu_clear_dirty_log(struct iommu_domain *domain, unsigned long iova,
+ size_t dma_size, unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift);
/* Window handling function prototypes */
extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
@@ -901,6 +929,31 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
+ bool enable, unsigned long iova,
+ size_t size, int prot)
+{
+ return -EINVAL;
+}
+
+static inline int iommu_sync_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long pgshift)
+{
+ return -EINVAL;
+}
+
+static inline int iommu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long pgshift)
+{
+ return -EINVAL;
+}
+
static inline int iommu_device_register(struct iommu_device *iommu)
{
return -ENODEV;
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-14 7:00 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-14 7:00 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 4/13/21 4:54 PM, Keqian Zhu wrote:
> Some types of IOMMU are capable of tracking DMA dirty log, such as
> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
> dirty log tracking framework in the IOMMU base layer.
>
> Three new essential interfaces are added, and we maintaince the status
> of dirty log tracking in iommu_domain.
> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>
> A new dev feature are added to indicate whether a specific type of
> iommu hardware supports and its driver realizes them.
>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
> ---
> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
> include/linux/iommu.h | 53 +++++++++++++++
> 2 files changed, 203 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index d0b0a15dba84..667b2d6d2fc0 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
> domain->type = type;
> /* Assume all sizes by default; the driver may override this later */
> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
> + mutex_init(&domain->switch_log_lock);
>
> return domain;
> }
> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
> }
> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>
> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + int ret;
> +
> + if (unlikely(!ops || !ops->switch_dirty_log))
> + return -ENODEV;
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (enable && domain->dirty_log_tracking) {
> + ret = -EBUSY;
> + goto out;
> + } else if (!enable && !domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
> + if (ret)
> + goto out;
> +
> + domain->dirty_log_tracking = enable;
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
difference between
iommu_switch_dirty_log(on) vs.
iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
iommu_switch_dirty_log(off) vs.
iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
> +
> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t size, unsigned long *bitmap,
> + unsigned long base_iova, unsigned long bitmap_pgshift)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + unsigned int min_pagesz;
> + size_t pgsize;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->sync_dirty_log))
> + return -ENODEV;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
> + iova, size, min_pagesz);
> + return -EINVAL;
> + }
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (!domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + while (size) {
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->sync_dirty_log(domain, iova, pgsize,
> + bitmap, base_iova, bitmap_pgshift);
Any reason why do you want to do this in a per-4K page manner? This can
lead to a lot of indirect calls and bad performance.
How about a sync_dirty_pages()?
The same comment applies to other places in this patch series.
> + if (ret)
> + break;
> +
> + pr_debug("dirty_log_sync handle: iova 0x%lx pagesz 0x%zx\n",
> + iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_sync_dirty_log);
> +
> +static int __iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + size_t pgsize;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->clear_dirty_log))
> + return -ENODEV;
> +
> + while (size) {
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->clear_dirty_log(domain, iova, pgsize, bitmap,
> + base_iova, bitmap_pgshift);
> + if (ret)
> + break;
> +
> + pr_debug("dirty_log_clear handled: iova 0x%lx pagesz 0x%zx\n",
> + iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +
> + return ret;
> +}
> +
> +int iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift)
> +{
> + unsigned long riova, rsize;
> + unsigned int min_pagesz;
> + bool flush = false;
> + int rs, re, start, end;
> + int ret = 0;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx min_pagesz 0x%x\n",
> + iova, min_pagesz);
> + return -EINVAL;
> + }
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (!domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + start = (iova - base_iova) >> bitmap_pgshift;
> + end = start + (size >> bitmap_pgshift);
> + bitmap_for_each_set_region(bitmap, rs, re, start, end) {
> + flush = true;
> + riova = iova + (rs << bitmap_pgshift);
> + rsize = (re - rs) << bitmap_pgshift;
> + ret = __iommu_clear_dirty_log(domain, riova, rsize, bitmap,
> + base_iova, bitmap_pgshift);
> + if (ret)
> + break;
> + }
> +
> + if (flush)
> + iommu_flush_iotlb_all(domain);
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_clear_dirty_log);
> +
> void iommu_get_resv_regions(struct device *dev, struct list_head *list)
> {
> const struct iommu_ops *ops = dev->bus->iommu_ops;
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 5e7fe519430a..7f9ed9f520e2 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -87,6 +87,8 @@ struct iommu_domain {
> void *handler_token;
> struct iommu_domain_geometry geometry;
> void *iova_cookie;
> + bool dirty_log_tracking;
> + struct mutex switch_log_lock;
> };
>
> enum iommu_cap {
> @@ -160,6 +162,7 @@ struct iommu_resv_region {
> enum iommu_dev_features {
> IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */
> IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */
> + IOMMU_DEV_FEAT_HWDBM, /* Hardware Dirty Bit Management */
> };
>
> #define IOMMU_PASID_INVALID (-1U)
> @@ -205,6 +208,9 @@ struct iommu_iotlb_gather {
> * @device_group: find iommu group for a particular device
> * @domain_get_attr: Query domain attributes
> * @domain_set_attr: Change domain attributes
> + * @switch_dirty_log: Perform actions to start|stop dirty log tracking
> + * @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
> + * @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
> * @get_resv_regions: Request list of reserved regions for a device
> * @put_resv_regions: Free list of reserved regions for a device
> * @apply_resv_region: Temporary helper call-back for iova reserved ranges
> @@ -260,6 +266,18 @@ struct iommu_ops {
> int (*domain_set_attr)(struct iommu_domain *domain,
> enum iommu_attr attr, void *data);
>
> + /* Track dirty log */
> + int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot);
> + int (*sync_dirty_log)(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> + int (*clear_dirty_log)(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> +
> /* Request/Free a list of reserved regions for a device */
> void (*get_resv_regions)(struct device *dev, struct list_head *list);
> void (*put_resv_regions)(struct device *dev, struct list_head *list);
> @@ -511,6 +529,16 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
> void *data);
> extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
> void *data);
> +extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot);
> +extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t size, unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> +extern int iommu_clear_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t dma_size, unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift);
>
> /* Window handling function prototypes */
> extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
> @@ -901,6 +929,31 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
> return -EINVAL;
> }
>
> +static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
> + bool enable, unsigned long iova,
> + size_t size, int prot)
> +{
> + return -EINVAL;
> +}
> +
> +static inline int iommu_sync_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long pgshift)
> +{
> + return -EINVAL;
> +}
> +
> +static inline int iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long pgshift)
> +{
> + return -EINVAL;
> +}
> +
> static inline int iommu_device_register(struct iommu_device *iommu)
> {
> return -ENODEV;
>
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-14 7:00 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-14 7:00 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 4/13/21 4:54 PM, Keqian Zhu wrote:
> Some types of IOMMU are capable of tracking DMA dirty log, such as
> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
> dirty log tracking framework in the IOMMU base layer.
>
> Three new essential interfaces are added, and we maintaince the status
> of dirty log tracking in iommu_domain.
> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>
> A new dev feature are added to indicate whether a specific type of
> iommu hardware supports and its driver realizes them.
>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
> ---
> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
> include/linux/iommu.h | 53 +++++++++++++++
> 2 files changed, 203 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index d0b0a15dba84..667b2d6d2fc0 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
> domain->type = type;
> /* Assume all sizes by default; the driver may override this later */
> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
> + mutex_init(&domain->switch_log_lock);
>
> return domain;
> }
> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
> }
> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>
> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + int ret;
> +
> + if (unlikely(!ops || !ops->switch_dirty_log))
> + return -ENODEV;
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (enable && domain->dirty_log_tracking) {
> + ret = -EBUSY;
> + goto out;
> + } else if (!enable && !domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
> + if (ret)
> + goto out;
> +
> + domain->dirty_log_tracking = enable;
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
difference between
iommu_switch_dirty_log(on) vs.
iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
iommu_switch_dirty_log(off) vs.
iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
> +
> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t size, unsigned long *bitmap,
> + unsigned long base_iova, unsigned long bitmap_pgshift)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + unsigned int min_pagesz;
> + size_t pgsize;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->sync_dirty_log))
> + return -ENODEV;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
> + iova, size, min_pagesz);
> + return -EINVAL;
> + }
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (!domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + while (size) {
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->sync_dirty_log(domain, iova, pgsize,
> + bitmap, base_iova, bitmap_pgshift);
Any reason why do you want to do this in a per-4K page manner? This can
lead to a lot of indirect calls and bad performance.
How about a sync_dirty_pages()?
The same comment applies to other places in this patch series.
> + if (ret)
> + break;
> +
> + pr_debug("dirty_log_sync handle: iova 0x%lx pagesz 0x%zx\n",
> + iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_sync_dirty_log);
> +
> +static int __iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + size_t pgsize;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->clear_dirty_log))
> + return -ENODEV;
> +
> + while (size) {
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->clear_dirty_log(domain, iova, pgsize, bitmap,
> + base_iova, bitmap_pgshift);
> + if (ret)
> + break;
> +
> + pr_debug("dirty_log_clear handled: iova 0x%lx pagesz 0x%zx\n",
> + iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +
> + return ret;
> +}
> +
> +int iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift)
> +{
> + unsigned long riova, rsize;
> + unsigned int min_pagesz;
> + bool flush = false;
> + int rs, re, start, end;
> + int ret = 0;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx min_pagesz 0x%x\n",
> + iova, min_pagesz);
> + return -EINVAL;
> + }
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (!domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + start = (iova - base_iova) >> bitmap_pgshift;
> + end = start + (size >> bitmap_pgshift);
> + bitmap_for_each_set_region(bitmap, rs, re, start, end) {
> + flush = true;
> + riova = iova + (rs << bitmap_pgshift);
> + rsize = (re - rs) << bitmap_pgshift;
> + ret = __iommu_clear_dirty_log(domain, riova, rsize, bitmap,
> + base_iova, bitmap_pgshift);
> + if (ret)
> + break;
> + }
> +
> + if (flush)
> + iommu_flush_iotlb_all(domain);
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_clear_dirty_log);
> +
> void iommu_get_resv_regions(struct device *dev, struct list_head *list)
> {
> const struct iommu_ops *ops = dev->bus->iommu_ops;
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 5e7fe519430a..7f9ed9f520e2 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -87,6 +87,8 @@ struct iommu_domain {
> void *handler_token;
> struct iommu_domain_geometry geometry;
> void *iova_cookie;
> + bool dirty_log_tracking;
> + struct mutex switch_log_lock;
> };
>
> enum iommu_cap {
> @@ -160,6 +162,7 @@ struct iommu_resv_region {
> enum iommu_dev_features {
> IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */
> IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */
> + IOMMU_DEV_FEAT_HWDBM, /* Hardware Dirty Bit Management */
> };
>
> #define IOMMU_PASID_INVALID (-1U)
> @@ -205,6 +208,9 @@ struct iommu_iotlb_gather {
> * @device_group: find iommu group for a particular device
> * @domain_get_attr: Query domain attributes
> * @domain_set_attr: Change domain attributes
> + * @switch_dirty_log: Perform actions to start|stop dirty log tracking
> + * @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
> + * @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
> * @get_resv_regions: Request list of reserved regions for a device
> * @put_resv_regions: Free list of reserved regions for a device
> * @apply_resv_region: Temporary helper call-back for iova reserved ranges
> @@ -260,6 +266,18 @@ struct iommu_ops {
> int (*domain_set_attr)(struct iommu_domain *domain,
> enum iommu_attr attr, void *data);
>
> + /* Track dirty log */
> + int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot);
> + int (*sync_dirty_log)(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> + int (*clear_dirty_log)(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> +
> /* Request/Free a list of reserved regions for a device */
> void (*get_resv_regions)(struct device *dev, struct list_head *list);
> void (*put_resv_regions)(struct device *dev, struct list_head *list);
> @@ -511,6 +529,16 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
> void *data);
> extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
> void *data);
> +extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot);
> +extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t size, unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> +extern int iommu_clear_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t dma_size, unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift);
>
> /* Window handling function prototypes */
> extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
> @@ -901,6 +929,31 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
> return -EINVAL;
> }
>
> +static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
> + bool enable, unsigned long iova,
> + size_t size, int prot)
> +{
> + return -EINVAL;
> +}
> +
> +static inline int iommu_sync_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long pgshift)
> +{
> + return -EINVAL;
> +}
> +
> +static inline int iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long pgshift)
> +{
> + return -EINVAL;
> +}
> +
> static inline int iommu_device_register(struct iommu_device *iommu)
> {
> return -ENODEV;
>
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-14 7:00 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-14 7:00 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Keqian,
On 4/13/21 4:54 PM, Keqian Zhu wrote:
> Some types of IOMMU are capable of tracking DMA dirty log, such as
> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
> dirty log tracking framework in the IOMMU base layer.
>
> Three new essential interfaces are added, and we maintaince the status
> of dirty log tracking in iommu_domain.
> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>
> A new dev feature are added to indicate whether a specific type of
> iommu hardware supports and its driver realizes them.
>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
> ---
> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
> include/linux/iommu.h | 53 +++++++++++++++
> 2 files changed, 203 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index d0b0a15dba84..667b2d6d2fc0 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
> domain->type = type;
> /* Assume all sizes by default; the driver may override this later */
> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
> + mutex_init(&domain->switch_log_lock);
>
> return domain;
> }
> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
> }
> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>
> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + int ret;
> +
> + if (unlikely(!ops || !ops->switch_dirty_log))
> + return -ENODEV;
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (enable && domain->dirty_log_tracking) {
> + ret = -EBUSY;
> + goto out;
> + } else if (!enable && !domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
> + if (ret)
> + goto out;
> +
> + domain->dirty_log_tracking = enable;
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
difference between
iommu_switch_dirty_log(on) vs.
iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
iommu_switch_dirty_log(off) vs.
iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
> +
> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t size, unsigned long *bitmap,
> + unsigned long base_iova, unsigned long bitmap_pgshift)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + unsigned int min_pagesz;
> + size_t pgsize;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->sync_dirty_log))
> + return -ENODEV;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
> + iova, size, min_pagesz);
> + return -EINVAL;
> + }
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (!domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + while (size) {
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->sync_dirty_log(domain, iova, pgsize,
> + bitmap, base_iova, bitmap_pgshift);
Any reason why do you want to do this in a per-4K page manner? This can
lead to a lot of indirect calls and bad performance.
How about a sync_dirty_pages()?
The same comment applies to other places in this patch series.
> + if (ret)
> + break;
> +
> + pr_debug("dirty_log_sync handle: iova 0x%lx pagesz 0x%zx\n",
> + iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_sync_dirty_log);
> +
> +static int __iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + size_t pgsize;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->clear_dirty_log))
> + return -ENODEV;
> +
> + while (size) {
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->clear_dirty_log(domain, iova, pgsize, bitmap,
> + base_iova, bitmap_pgshift);
> + if (ret)
> + break;
> +
> + pr_debug("dirty_log_clear handled: iova 0x%lx pagesz 0x%zx\n",
> + iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +
> + return ret;
> +}
> +
> +int iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift)
> +{
> + unsigned long riova, rsize;
> + unsigned int min_pagesz;
> + bool flush = false;
> + int rs, re, start, end;
> + int ret = 0;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx min_pagesz 0x%x\n",
> + iova, min_pagesz);
> + return -EINVAL;
> + }
> +
> + mutex_lock(&domain->switch_log_lock);
> + if (!domain->dirty_log_tracking) {
> + ret = -EINVAL;
> + goto out;
> + }
> +
> + start = (iova - base_iova) >> bitmap_pgshift;
> + end = start + (size >> bitmap_pgshift);
> + bitmap_for_each_set_region(bitmap, rs, re, start, end) {
> + flush = true;
> + riova = iova + (rs << bitmap_pgshift);
> + rsize = (re - rs) << bitmap_pgshift;
> + ret = __iommu_clear_dirty_log(domain, riova, rsize, bitmap,
> + base_iova, bitmap_pgshift);
> + if (ret)
> + break;
> + }
> +
> + if (flush)
> + iommu_flush_iotlb_all(domain);
> +out:
> + mutex_unlock(&domain->switch_log_lock);
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_clear_dirty_log);
> +
> void iommu_get_resv_regions(struct device *dev, struct list_head *list)
> {
> const struct iommu_ops *ops = dev->bus->iommu_ops;
> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
> index 5e7fe519430a..7f9ed9f520e2 100644
> --- a/include/linux/iommu.h
> +++ b/include/linux/iommu.h
> @@ -87,6 +87,8 @@ struct iommu_domain {
> void *handler_token;
> struct iommu_domain_geometry geometry;
> void *iova_cookie;
> + bool dirty_log_tracking;
> + struct mutex switch_log_lock;
> };
>
> enum iommu_cap {
> @@ -160,6 +162,7 @@ struct iommu_resv_region {
> enum iommu_dev_features {
> IOMMU_DEV_FEAT_AUX, /* Aux-domain feature */
> IOMMU_DEV_FEAT_SVA, /* Shared Virtual Addresses */
> + IOMMU_DEV_FEAT_HWDBM, /* Hardware Dirty Bit Management */
> };
>
> #define IOMMU_PASID_INVALID (-1U)
> @@ -205,6 +208,9 @@ struct iommu_iotlb_gather {
> * @device_group: find iommu group for a particular device
> * @domain_get_attr: Query domain attributes
> * @domain_set_attr: Change domain attributes
> + * @switch_dirty_log: Perform actions to start|stop dirty log tracking
> + * @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
> + * @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
> * @get_resv_regions: Request list of reserved regions for a device
> * @put_resv_regions: Free list of reserved regions for a device
> * @apply_resv_region: Temporary helper call-back for iova reserved ranges
> @@ -260,6 +266,18 @@ struct iommu_ops {
> int (*domain_set_attr)(struct iommu_domain *domain,
> enum iommu_attr attr, void *data);
>
> + /* Track dirty log */
> + int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot);
> + int (*sync_dirty_log)(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> + int (*clear_dirty_log)(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap, unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> +
> /* Request/Free a list of reserved regions for a device */
> void (*get_resv_regions)(struct device *dev, struct list_head *list);
> void (*put_resv_regions)(struct device *dev, struct list_head *list);
> @@ -511,6 +529,16 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
> void *data);
> extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
> void *data);
> +extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
> + unsigned long iova, size_t size, int prot);
> +extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t size, unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift);
> +extern int iommu_clear_dirty_log(struct iommu_domain *domain, unsigned long iova,
> + size_t dma_size, unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long bitmap_pgshift);
>
> /* Window handling function prototypes */
> extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
> @@ -901,6 +929,31 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
> return -EINVAL;
> }
>
> +static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
> + bool enable, unsigned long iova,
> + size_t size, int prot)
> +{
> + return -EINVAL;
> +}
> +
> +static inline int iommu_sync_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long pgshift)
> +{
> + return -EINVAL;
> +}
> +
> +static inline int iommu_clear_dirty_log(struct iommu_domain *domain,
> + unsigned long iova, size_t size,
> + unsigned long *bitmap,
> + unsigned long base_iova,
> + unsigned long pgshift)
> +{
> + return -EINVAL;
> +}
> +
> static inline int iommu_device_register(struct iommu_device *iommu)
> {
> return -ENODEV;
>
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-14 7:00 ` Lu Baolu
(?)
@ 2021-04-15 6:18 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-15 6:18 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
Thanks for the review!
On 2021/4/14 15:00, Lu Baolu wrote:
> Hi Keqian,
>
> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>> dirty log tracking framework in the IOMMU base layer.
>>
>> Three new essential interfaces are added, and we maintaince the status
>> of dirty log tracking in iommu_domain.
>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>
>> A new dev feature are added to indicate whether a specific type of
>> iommu hardware supports and its driver realizes them.
>>
>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 53 +++++++++++++++
>> 2 files changed, 203 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index d0b0a15dba84..667b2d6d2fc0 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>> domain->type = type;
>> /* Assume all sizes by default; the driver may override this later */
>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>> + mutex_init(&domain->switch_log_lock);
>> return domain;
>> }
>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>> }
>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>> + unsigned long iova, size_t size, int prot)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + int ret;
>> +
>> + if (unlikely(!ops || !ops->switch_dirty_log))
>> + return -ENODEV;
>> +
>> + mutex_lock(&domain->switch_log_lock);
>> + if (enable && domain->dirty_log_tracking) {
>> + ret = -EBUSY;
>> + goto out;
>> + } else if (!enable && !domain->dirty_log_tracking) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>> + if (ret)
>> + goto out;
>> +
>> + domain->dirty_log_tracking = enable;
>> +out:
>> + mutex_unlock(&domain->switch_log_lock);
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>
> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
> difference between
>
> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>
> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
interfaces for it.
IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
design it as not switchable. I will modify the commit message of patch#12, thanks!
>
>> +
>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>> + size_t size, unsigned long *bitmap,
>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + unsigned int min_pagesz;
>> + size_t pgsize;
>> + int ret = 0;
>> +
>> + if (unlikely(!ops || !ops->sync_dirty_log))
>> + return -ENODEV;
>> +
>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>> + iova, size, min_pagesz);
>> + return -EINVAL;
>> + }
>> +
>> + mutex_lock(&domain->switch_log_lock);
>> + if (!domain->dirty_log_tracking) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + while (size) {
>> + pgsize = iommu_pgsize(domain, iova, size);
>> +
>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>> + bitmap, base_iova, bitmap_pgshift);
>
> Any reason why do you want to do this in a per-4K page manner? This can
> lead to a lot of indirect calls and bad performance.
>
> How about a sync_dirty_pages()?
The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
compute the max size that fit into size, so the pgsize can be a large page size
even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
BRs,
Keqian
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 6:18 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-15 6:18 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
Thanks for the review!
On 2021/4/14 15:00, Lu Baolu wrote:
> Hi Keqian,
>
> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>> dirty log tracking framework in the IOMMU base layer.
>>
>> Three new essential interfaces are added, and we maintaince the status
>> of dirty log tracking in iommu_domain.
>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>
>> A new dev feature are added to indicate whether a specific type of
>> iommu hardware supports and its driver realizes them.
>>
>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 53 +++++++++++++++
>> 2 files changed, 203 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index d0b0a15dba84..667b2d6d2fc0 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>> domain->type = type;
>> /* Assume all sizes by default; the driver may override this later */
>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>> + mutex_init(&domain->switch_log_lock);
>> return domain;
>> }
>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>> }
>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>> + unsigned long iova, size_t size, int prot)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + int ret;
>> +
>> + if (unlikely(!ops || !ops->switch_dirty_log))
>> + return -ENODEV;
>> +
>> + mutex_lock(&domain->switch_log_lock);
>> + if (enable && domain->dirty_log_tracking) {
>> + ret = -EBUSY;
>> + goto out;
>> + } else if (!enable && !domain->dirty_log_tracking) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>> + if (ret)
>> + goto out;
>> +
>> + domain->dirty_log_tracking = enable;
>> +out:
>> + mutex_unlock(&domain->switch_log_lock);
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>
> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
> difference between
>
> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>
> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
interfaces for it.
IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
design it as not switchable. I will modify the commit message of patch#12, thanks!
>
>> +
>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>> + size_t size, unsigned long *bitmap,
>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + unsigned int min_pagesz;
>> + size_t pgsize;
>> + int ret = 0;
>> +
>> + if (unlikely(!ops || !ops->sync_dirty_log))
>> + return -ENODEV;
>> +
>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>> + iova, size, min_pagesz);
>> + return -EINVAL;
>> + }
>> +
>> + mutex_lock(&domain->switch_log_lock);
>> + if (!domain->dirty_log_tracking) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + while (size) {
>> + pgsize = iommu_pgsize(domain, iova, size);
>> +
>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>> + bitmap, base_iova, bitmap_pgshift);
>
> Any reason why do you want to do this in a per-4K page manner? This can
> lead to a lot of indirect calls and bad performance.
>
> How about a sync_dirty_pages()?
The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
compute the max size that fit into size, so the pgsize can be a large page size
even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
BRs,
Keqian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 6:18 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-15 6:18 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Baolu,
Thanks for the review!
On 2021/4/14 15:00, Lu Baolu wrote:
> Hi Keqian,
>
> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>> dirty log tracking framework in the IOMMU base layer.
>>
>> Three new essential interfaces are added, and we maintaince the status
>> of dirty log tracking in iommu_domain.
>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>
>> A new dev feature are added to indicate whether a specific type of
>> iommu hardware supports and its driver realizes them.
>>
>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 53 +++++++++++++++
>> 2 files changed, 203 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index d0b0a15dba84..667b2d6d2fc0 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>> domain->type = type;
>> /* Assume all sizes by default; the driver may override this later */
>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>> + mutex_init(&domain->switch_log_lock);
>> return domain;
>> }
>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>> }
>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>> + unsigned long iova, size_t size, int prot)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + int ret;
>> +
>> + if (unlikely(!ops || !ops->switch_dirty_log))
>> + return -ENODEV;
>> +
>> + mutex_lock(&domain->switch_log_lock);
>> + if (enable && domain->dirty_log_tracking) {
>> + ret = -EBUSY;
>> + goto out;
>> + } else if (!enable && !domain->dirty_log_tracking) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>> + if (ret)
>> + goto out;
>> +
>> + domain->dirty_log_tracking = enable;
>> +out:
>> + mutex_unlock(&domain->switch_log_lock);
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>
> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
> difference between
>
> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>
> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
interfaces for it.
IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
design it as not switchable. I will modify the commit message of patch#12, thanks!
>
>> +
>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>> + size_t size, unsigned long *bitmap,
>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + unsigned int min_pagesz;
>> + size_t pgsize;
>> + int ret = 0;
>> +
>> + if (unlikely(!ops || !ops->sync_dirty_log))
>> + return -ENODEV;
>> +
>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>> + iova, size, min_pagesz);
>> + return -EINVAL;
>> + }
>> +
>> + mutex_lock(&domain->switch_log_lock);
>> + if (!domain->dirty_log_tracking) {
>> + ret = -EINVAL;
>> + goto out;
>> + }
>> +
>> + while (size) {
>> + pgsize = iommu_pgsize(domain, iova, size);
>> +
>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>> + bitmap, base_iova, bitmap_pgshift);
>
> Any reason why do you want to do this in a per-4K page manner? This can
> lead to a lot of indirect calls and bad performance.
>
> How about a sync_dirty_pages()?
The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
compute the max size that fit into size, so the pgsize can be a large page size
even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
BRs,
Keqian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-15 6:18 ` Keqian Zhu
(?)
@ 2021-04-15 7:03 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-15 7:03 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
On 4/15/21 2:18 PM, Keqian Zhu wrote:
> Hi Baolu,
>
> Thanks for the review!
>
> On 2021/4/14 15:00, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>>> dirty log tracking framework in the IOMMU base layer.
>>>
>>> Three new essential interfaces are added, and we maintaince the status
>>> of dirty log tracking in iommu_domain.
>>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>>
>>> A new dev feature are added to indicate whether a specific type of
>>> iommu hardware supports and its driver realizes them.
>>>
>>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>>> ---
>>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>>> include/linux/iommu.h | 53 +++++++++++++++
>>> 2 files changed, 203 insertions(+)
>>>
>>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>>> index d0b0a15dba84..667b2d6d2fc0 100644
>>> --- a/drivers/iommu/iommu.c
>>> +++ b/drivers/iommu/iommu.c
>>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>>> domain->type = type;
>>> /* Assume all sizes by default; the driver may override this later */
>>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>>> + mutex_init(&domain->switch_log_lock);
>>> return domain;
>>> }
>>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>>> }
>>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>>> + unsigned long iova, size_t size, int prot)
>>> +{
>>> + const struct iommu_ops *ops = domain->ops;
>>> + int ret;
>>> +
>>> + if (unlikely(!ops || !ops->switch_dirty_log))
>>> + return -ENODEV;
>>> +
>>> + mutex_lock(&domain->switch_log_lock);
>>> + if (enable && domain->dirty_log_tracking) {
>>> + ret = -EBUSY;
>>> + goto out;
>>> + } else if (!enable && !domain->dirty_log_tracking) {
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>>> + if (ret)
>>> + goto out;
>>> +
>>> + domain->dirty_log_tracking = enable;
>>> +out:
>>> + mutex_unlock(&domain->switch_log_lock);
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>>
>> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
>> difference between
>>
>> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>>
>> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
> Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
> are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
> interfaces for it.
>
> IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
Previously we had iommu_dev_has_feature() and then was cleaned up due to
lack of real users. If you have a real case for it, why not bringing it
back?
> design it as not switchable. I will modify the commit message of patch#12, thanks!
I am not sure that I fully get your point. But I can't see any gaps of
using iommu_dev_enable/disable_feature() to switch dirty log on and off.
Probably I missed anything.
>
>>
>>> +
>>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>>> + size_t size, unsigned long *bitmap,
>>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>>> +{
>>> + const struct iommu_ops *ops = domain->ops;
>>> + unsigned int min_pagesz;
>>> + size_t pgsize;
>>> + int ret = 0;
>>> +
>>> + if (unlikely(!ops || !ops->sync_dirty_log))
>>> + return -ENODEV;
>>> +
>>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>>> + iova, size, min_pagesz);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + mutex_lock(&domain->switch_log_lock);
>>> + if (!domain->dirty_log_tracking) {
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> + while (size) {
>>> + pgsize = iommu_pgsize(domain, iova, size);
>>> +
>>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>>> + bitmap, base_iova, bitmap_pgshift);
>>
>> Any reason why do you want to do this in a per-4K page manner? This can
>> lead to a lot of indirect calls and bad performance.
>>
>> How about a sync_dirty_pages()?
> The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
> compute the max size that fit into size, so the pgsize can be a large page size
> even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
This series has some improvement on the iommu_pgsize() helper.
https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 7:03 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-15 7:03 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
On 4/15/21 2:18 PM, Keqian Zhu wrote:
> Hi Baolu,
>
> Thanks for the review!
>
> On 2021/4/14 15:00, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>>> dirty log tracking framework in the IOMMU base layer.
>>>
>>> Three new essential interfaces are added, and we maintaince the status
>>> of dirty log tracking in iommu_domain.
>>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>>
>>> A new dev feature are added to indicate whether a specific type of
>>> iommu hardware supports and its driver realizes them.
>>>
>>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>>> ---
>>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>>> include/linux/iommu.h | 53 +++++++++++++++
>>> 2 files changed, 203 insertions(+)
>>>
>>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>>> index d0b0a15dba84..667b2d6d2fc0 100644
>>> --- a/drivers/iommu/iommu.c
>>> +++ b/drivers/iommu/iommu.c
>>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>>> domain->type = type;
>>> /* Assume all sizes by default; the driver may override this later */
>>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>>> + mutex_init(&domain->switch_log_lock);
>>> return domain;
>>> }
>>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>>> }
>>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>>> + unsigned long iova, size_t size, int prot)
>>> +{
>>> + const struct iommu_ops *ops = domain->ops;
>>> + int ret;
>>> +
>>> + if (unlikely(!ops || !ops->switch_dirty_log))
>>> + return -ENODEV;
>>> +
>>> + mutex_lock(&domain->switch_log_lock);
>>> + if (enable && domain->dirty_log_tracking) {
>>> + ret = -EBUSY;
>>> + goto out;
>>> + } else if (!enable && !domain->dirty_log_tracking) {
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>>> + if (ret)
>>> + goto out;
>>> +
>>> + domain->dirty_log_tracking = enable;
>>> +out:
>>> + mutex_unlock(&domain->switch_log_lock);
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>>
>> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
>> difference between
>>
>> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>>
>> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
> Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
> are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
> interfaces for it.
>
> IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
Previously we had iommu_dev_has_feature() and then was cleaned up due to
lack of real users. If you have a real case for it, why not bringing it
back?
> design it as not switchable. I will modify the commit message of patch#12, thanks!
I am not sure that I fully get your point. But I can't see any gaps of
using iommu_dev_enable/disable_feature() to switch dirty log on and off.
Probably I missed anything.
>
>>
>>> +
>>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>>> + size_t size, unsigned long *bitmap,
>>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>>> +{
>>> + const struct iommu_ops *ops = domain->ops;
>>> + unsigned int min_pagesz;
>>> + size_t pgsize;
>>> + int ret = 0;
>>> +
>>> + if (unlikely(!ops || !ops->sync_dirty_log))
>>> + return -ENODEV;
>>> +
>>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>>> + iova, size, min_pagesz);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + mutex_lock(&domain->switch_log_lock);
>>> + if (!domain->dirty_log_tracking) {
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> + while (size) {
>>> + pgsize = iommu_pgsize(domain, iova, size);
>>> +
>>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>>> + bitmap, base_iova, bitmap_pgshift);
>>
>> Any reason why do you want to do this in a per-4K page manner? This can
>> lead to a lot of indirect calls and bad performance.
>>
>> How about a sync_dirty_pages()?
> The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
> compute the max size that fit into size, so the pgsize can be a large page size
> even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
This series has some improvement on the iommu_pgsize() helper.
https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 7:03 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-15 7:03 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
On 4/15/21 2:18 PM, Keqian Zhu wrote:
> Hi Baolu,
>
> Thanks for the review!
>
> On 2021/4/14 15:00, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>>> dirty log tracking framework in the IOMMU base layer.
>>>
>>> Three new essential interfaces are added, and we maintaince the status
>>> of dirty log tracking in iommu_domain.
>>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>>
>>> A new dev feature are added to indicate whether a specific type of
>>> iommu hardware supports and its driver realizes them.
>>>
>>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>>> ---
>>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>>> include/linux/iommu.h | 53 +++++++++++++++
>>> 2 files changed, 203 insertions(+)
>>>
>>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>>> index d0b0a15dba84..667b2d6d2fc0 100644
>>> --- a/drivers/iommu/iommu.c
>>> +++ b/drivers/iommu/iommu.c
>>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>>> domain->type = type;
>>> /* Assume all sizes by default; the driver may override this later */
>>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>>> + mutex_init(&domain->switch_log_lock);
>>> return domain;
>>> }
>>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>>> }
>>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>>> + unsigned long iova, size_t size, int prot)
>>> +{
>>> + const struct iommu_ops *ops = domain->ops;
>>> + int ret;
>>> +
>>> + if (unlikely(!ops || !ops->switch_dirty_log))
>>> + return -ENODEV;
>>> +
>>> + mutex_lock(&domain->switch_log_lock);
>>> + if (enable && domain->dirty_log_tracking) {
>>> + ret = -EBUSY;
>>> + goto out;
>>> + } else if (!enable && !domain->dirty_log_tracking) {
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>>> + if (ret)
>>> + goto out;
>>> +
>>> + domain->dirty_log_tracking = enable;
>>> +out:
>>> + mutex_unlock(&domain->switch_log_lock);
>>> + return ret;
>>> +}
>>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>>
>> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
>> difference between
>>
>> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>>
>> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
> Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
> are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
> interfaces for it.
>
> IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
Previously we had iommu_dev_has_feature() and then was cleaned up due to
lack of real users. If you have a real case for it, why not bringing it
back?
> design it as not switchable. I will modify the commit message of patch#12, thanks!
I am not sure that I fully get your point. But I can't see any gaps of
using iommu_dev_enable/disable_feature() to switch dirty log on and off.
Probably I missed anything.
>
>>
>>> +
>>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>>> + size_t size, unsigned long *bitmap,
>>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>>> +{
>>> + const struct iommu_ops *ops = domain->ops;
>>> + unsigned int min_pagesz;
>>> + size_t pgsize;
>>> + int ret = 0;
>>> +
>>> + if (unlikely(!ops || !ops->sync_dirty_log))
>>> + return -ENODEV;
>>> +
>>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>>> + iova, size, min_pagesz);
>>> + return -EINVAL;
>>> + }
>>> +
>>> + mutex_lock(&domain->switch_log_lock);
>>> + if (!domain->dirty_log_tracking) {
>>> + ret = -EINVAL;
>>> + goto out;
>>> + }
>>> +
>>> + while (size) {
>>> + pgsize = iommu_pgsize(domain, iova, size);
>>> +
>>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>>> + bitmap, base_iova, bitmap_pgshift);
>>
>> Any reason why do you want to do this in a per-4K page manner? This can
>> lead to a lot of indirect calls and bad performance.
>>
>> How about a sync_dirty_pages()?
> The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
> compute the max size that fit into size, so the pgsize can be a large page size
> even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
This series has some improvement on the iommu_pgsize() helper.
https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-15 7:03 ` Lu Baolu
(?)
@ 2021-04-15 7:43 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-15 7:43 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
On 2021/4/15 15:03, Lu Baolu wrote:
> On 4/15/21 2:18 PM, Keqian Zhu wrote:
>> Hi Baolu,
>>
>> Thanks for the review!
>>
>> On 2021/4/14 15:00, Lu Baolu wrote:
>>> Hi Keqian,
>>>
>>> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>>>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>>>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>>>> dirty log tracking framework in the IOMMU base layer.
>>>>
>>>> Three new essential interfaces are added, and we maintaince the status
>>>> of dirty log tracking in iommu_domain.
>>>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>>>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>>>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>>>
>>>> A new dev feature are added to indicate whether a specific type of
>>>> iommu hardware supports and its driver realizes them.
>>>>
>>>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>>>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>>>> ---
>>>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>>>> include/linux/iommu.h | 53 +++++++++++++++
>>>> 2 files changed, 203 insertions(+)
>>>>
>>>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>>>> index d0b0a15dba84..667b2d6d2fc0 100644
>>>> --- a/drivers/iommu/iommu.c
>>>> +++ b/drivers/iommu/iommu.c
>>>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>>>> domain->type = type;
>>>> /* Assume all sizes by default; the driver may override this later */
>>>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>>>> + mutex_init(&domain->switch_log_lock);
>>>> return domain;
>>>> }
>>>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>>>> }
>>>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>>>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>>>> + unsigned long iova, size_t size, int prot)
>>>> +{
>>>> + const struct iommu_ops *ops = domain->ops;
>>>> + int ret;
>>>> +
>>>> + if (unlikely(!ops || !ops->switch_dirty_log))
>>>> + return -ENODEV;
>>>> +
>>>> + mutex_lock(&domain->switch_log_lock);
>>>> + if (enable && domain->dirty_log_tracking) {
>>>> + ret = -EBUSY;
>>>> + goto out;
>>>> + } else if (!enable && !domain->dirty_log_tracking) {
>>>> + ret = -EINVAL;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>>>> + if (ret)
>>>> + goto out;
>>>> +
>>>> + domain->dirty_log_tracking = enable;
>>>> +out:
>>>> + mutex_unlock(&domain->switch_log_lock);
>>>> + return ret;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>>>
>>> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
>>> difference between
>>>
>>> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>>>
>>> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
>> Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
>> are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
>> interfaces for it.
>>
>> IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
>
> Previously we had iommu_dev_has_feature() and then was cleaned up due to
> lack of real users. If you have a real case for it, why not bringing it
> back?
Yep, good suggestion.
>
>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>
> I am not sure that I fully get your point. But I can't see any gaps of
> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
> Probably I missed anything.
IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
dirty tracking, it is not used to management the status of dirty log tracking.
The feature reporting is per device, but the status management is per iommu_domain.
Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
And I think we'd better not mix the feature reporting and status management. Thoughts?
>
>>
>>>
>>>> +
>>>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>>>> + size_t size, unsigned long *bitmap,
>>>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>>>> +{
>>>> + const struct iommu_ops *ops = domain->ops;
>>>> + unsigned int min_pagesz;
>>>> + size_t pgsize;
>>>> + int ret = 0;
>>>> +
>>>> + if (unlikely(!ops || !ops->sync_dirty_log))
>>>> + return -ENODEV;
>>>> +
>>>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>>>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>>>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>>>> + iova, size, min_pagesz);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + mutex_lock(&domain->switch_log_lock);
>>>> + if (!domain->dirty_log_tracking) {
>>>> + ret = -EINVAL;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + while (size) {
>>>> + pgsize = iommu_pgsize(domain, iova, size);
>>>> +
>>>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>>>> + bitmap, base_iova, bitmap_pgshift);
>>>
>>> Any reason why do you want to do this in a per-4K page manner? This can
>>> lead to a lot of indirect calls and bad performance.
>>>
>>> How about a sync_dirty_pages()?
>> The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
>> compute the max size that fit into size, so the pgsize can be a large page size
>> even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
>
> This series has some improvement on the iommu_pgsize() helper.
>
> https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/
OK, I get your idea. I will look into that, thanks!
BRs,
Keqian
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 7:43 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-15 7:43 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
On 2021/4/15 15:03, Lu Baolu wrote:
> On 4/15/21 2:18 PM, Keqian Zhu wrote:
>> Hi Baolu,
>>
>> Thanks for the review!
>>
>> On 2021/4/14 15:00, Lu Baolu wrote:
>>> Hi Keqian,
>>>
>>> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>>>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>>>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>>>> dirty log tracking framework in the IOMMU base layer.
>>>>
>>>> Three new essential interfaces are added, and we maintaince the status
>>>> of dirty log tracking in iommu_domain.
>>>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>>>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>>>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>>>
>>>> A new dev feature are added to indicate whether a specific type of
>>>> iommu hardware supports and its driver realizes them.
>>>>
>>>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>>>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>>>> ---
>>>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>>>> include/linux/iommu.h | 53 +++++++++++++++
>>>> 2 files changed, 203 insertions(+)
>>>>
>>>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>>>> index d0b0a15dba84..667b2d6d2fc0 100644
>>>> --- a/drivers/iommu/iommu.c
>>>> +++ b/drivers/iommu/iommu.c
>>>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>>>> domain->type = type;
>>>> /* Assume all sizes by default; the driver may override this later */
>>>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>>>> + mutex_init(&domain->switch_log_lock);
>>>> return domain;
>>>> }
>>>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>>>> }
>>>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>>>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>>>> + unsigned long iova, size_t size, int prot)
>>>> +{
>>>> + const struct iommu_ops *ops = domain->ops;
>>>> + int ret;
>>>> +
>>>> + if (unlikely(!ops || !ops->switch_dirty_log))
>>>> + return -ENODEV;
>>>> +
>>>> + mutex_lock(&domain->switch_log_lock);
>>>> + if (enable && domain->dirty_log_tracking) {
>>>> + ret = -EBUSY;
>>>> + goto out;
>>>> + } else if (!enable && !domain->dirty_log_tracking) {
>>>> + ret = -EINVAL;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>>>> + if (ret)
>>>> + goto out;
>>>> +
>>>> + domain->dirty_log_tracking = enable;
>>>> +out:
>>>> + mutex_unlock(&domain->switch_log_lock);
>>>> + return ret;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>>>
>>> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
>>> difference between
>>>
>>> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>>>
>>> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
>> Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
>> are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
>> interfaces for it.
>>
>> IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
>
> Previously we had iommu_dev_has_feature() and then was cleaned up due to
> lack of real users. If you have a real case for it, why not bringing it
> back?
Yep, good suggestion.
>
>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>
> I am not sure that I fully get your point. But I can't see any gaps of
> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
> Probably I missed anything.
IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
dirty tracking, it is not used to management the status of dirty log tracking.
The feature reporting is per device, but the status management is per iommu_domain.
Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
And I think we'd better not mix the feature reporting and status management. Thoughts?
>
>>
>>>
>>>> +
>>>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>>>> + size_t size, unsigned long *bitmap,
>>>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>>>> +{
>>>> + const struct iommu_ops *ops = domain->ops;
>>>> + unsigned int min_pagesz;
>>>> + size_t pgsize;
>>>> + int ret = 0;
>>>> +
>>>> + if (unlikely(!ops || !ops->sync_dirty_log))
>>>> + return -ENODEV;
>>>> +
>>>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>>>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>>>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>>>> + iova, size, min_pagesz);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + mutex_lock(&domain->switch_log_lock);
>>>> + if (!domain->dirty_log_tracking) {
>>>> + ret = -EINVAL;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + while (size) {
>>>> + pgsize = iommu_pgsize(domain, iova, size);
>>>> +
>>>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>>>> + bitmap, base_iova, bitmap_pgshift);
>>>
>>> Any reason why do you want to do this in a per-4K page manner? This can
>>> lead to a lot of indirect calls and bad performance.
>>>
>>> How about a sync_dirty_pages()?
>> The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
>> compute the max size that fit into size, so the pgsize can be a large page size
>> even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
>
> This series has some improvement on the iommu_pgsize() helper.
>
> https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/
OK, I get your idea. I will look into that, thanks!
BRs,
Keqian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 7:43 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-15 7:43 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
On 2021/4/15 15:03, Lu Baolu wrote:
> On 4/15/21 2:18 PM, Keqian Zhu wrote:
>> Hi Baolu,
>>
>> Thanks for the review!
>>
>> On 2021/4/14 15:00, Lu Baolu wrote:
>>> Hi Keqian,
>>>
>>> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>>>> Some types of IOMMU are capable of tracking DMA dirty log, such as
>>>> ARM SMMU with HTTU or Intel IOMMU with SLADE. This introduces the
>>>> dirty log tracking framework in the IOMMU base layer.
>>>>
>>>> Three new essential interfaces are added, and we maintaince the status
>>>> of dirty log tracking in iommu_domain.
>>>> 1. iommu_switch_dirty_log: Perform actions to start|stop dirty log tracking
>>>> 2. iommu_sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
>>>> 3. iommu_clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
>>>>
>>>> A new dev feature are added to indicate whether a specific type of
>>>> iommu hardware supports and its driver realizes them.
>>>>
>>>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>>>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>>>> ---
>>>> drivers/iommu/iommu.c | 150 ++++++++++++++++++++++++++++++++++++++++++
>>>> include/linux/iommu.h | 53 +++++++++++++++
>>>> 2 files changed, 203 insertions(+)
>>>>
>>>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>>>> index d0b0a15dba84..667b2d6d2fc0 100644
>>>> --- a/drivers/iommu/iommu.c
>>>> +++ b/drivers/iommu/iommu.c
>>>> @@ -1922,6 +1922,7 @@ static struct iommu_domain *__iommu_domain_alloc(struct bus_type *bus,
>>>> domain->type = type;
>>>> /* Assume all sizes by default; the driver may override this later */
>>>> domain->pgsize_bitmap = bus->iommu_ops->pgsize_bitmap;
>>>> + mutex_init(&domain->switch_log_lock);
>>>> return domain;
>>>> }
>>>> @@ -2720,6 +2721,155 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>>>> }
>>>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>>>> +int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
>>>> + unsigned long iova, size_t size, int prot)
>>>> +{
>>>> + const struct iommu_ops *ops = domain->ops;
>>>> + int ret;
>>>> +
>>>> + if (unlikely(!ops || !ops->switch_dirty_log))
>>>> + return -ENODEV;
>>>> +
>>>> + mutex_lock(&domain->switch_log_lock);
>>>> + if (enable && domain->dirty_log_tracking) {
>>>> + ret = -EBUSY;
>>>> + goto out;
>>>> + } else if (!enable && !domain->dirty_log_tracking) {
>>>> + ret = -EINVAL;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + ret = ops->switch_dirty_log(domain, enable, iova, size, prot);
>>>> + if (ret)
>>>> + goto out;
>>>> +
>>>> + domain->dirty_log_tracking = enable;
>>>> +out:
>>>> + mutex_unlock(&domain->switch_log_lock);
>>>> + return ret;
>>>> +}
>>>> +EXPORT_SYMBOL_GPL(iommu_switch_dirty_log);
>>>
>>> Since you also added IOMMU_DEV_FEAT_HWDBM, I am wondering what's the
>>> difference between
>>>
>>> iommu_switch_dirty_log(on) vs. iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
>>>
>>> iommu_switch_dirty_log(off) vs. iommu_dev_disable_feature(IOMMU_DEV_FEAT_HWDBM)
>> Indeed. As I can see, IOMMU_DEV_FEAT_AUX is not switchable, so enable/disable
>> are not applicable for it. IOMMU_DEV_FEAT_SVA is switchable, so we can use these
>> interfaces for it.
>>
>> IOMMU_DEV_FEAT_HWDBM is used to indicate whether hardware supports HWDBM, so we should
>
> Previously we had iommu_dev_has_feature() and then was cleaned up due to
> lack of real users. If you have a real case for it, why not bringing it
> back?
Yep, good suggestion.
>
>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>
> I am not sure that I fully get your point. But I can't see any gaps of
> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
> Probably I missed anything.
IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
dirty tracking, it is not used to management the status of dirty log tracking.
The feature reporting is per device, but the status management is per iommu_domain.
Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
And I think we'd better not mix the feature reporting and status management. Thoughts?
>
>>
>>>
>>>> +
>>>> +int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
>>>> + size_t size, unsigned long *bitmap,
>>>> + unsigned long base_iova, unsigned long bitmap_pgshift)
>>>> +{
>>>> + const struct iommu_ops *ops = domain->ops;
>>>> + unsigned int min_pagesz;
>>>> + size_t pgsize;
>>>> + int ret = 0;
>>>> +
>>>> + if (unlikely(!ops || !ops->sync_dirty_log))
>>>> + return -ENODEV;
>>>> +
>>>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>>>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>>>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>>>> + iova, size, min_pagesz);
>>>> + return -EINVAL;
>>>> + }
>>>> +
>>>> + mutex_lock(&domain->switch_log_lock);
>>>> + if (!domain->dirty_log_tracking) {
>>>> + ret = -EINVAL;
>>>> + goto out;
>>>> + }
>>>> +
>>>> + while (size) {
>>>> + pgsize = iommu_pgsize(domain, iova, size);
>>>> +
>>>> + ret = ops->sync_dirty_log(domain, iova, pgsize,
>>>> + bitmap, base_iova, bitmap_pgshift);
>>>
>>> Any reason why do you want to do this in a per-4K page manner? This can
>>> lead to a lot of indirect calls and bad performance.
>>>
>>> How about a sync_dirty_pages()?
>> The function name of iommu_pgsize() is a bit puzzling. Actually it will try to
>> compute the max size that fit into size, so the pgsize can be a large page size
>> even if the underlying mapping is 4K. The __iommu_unmap() also has a similar logic.
>
> This series has some improvement on the iommu_pgsize() helper.
>
> https://lore.kernel.org/linux-iommu/20210405191112.28192-1-isaacm@codeaurora.org/
OK, I get your idea. I will look into that, thanks!
BRs,
Keqian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-15 7:43 ` Keqian Zhu
(?)
@ 2021-04-15 10:21 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-15 10:21 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi,
On 2021/4/15 15:43, Keqian Zhu wrote:
>>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>> I am not sure that I fully get your point. But I can't see any gaps of
>> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
>> Probably I missed anything.
> IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
> dirty tracking, it is not used to management the status of dirty log tracking.
>
> The feature reporting is per device, but the status management is per iommu_domain.
> Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
So why not
for_each_device_attached_domain()
iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
?
>
> And I think we'd better not mix the feature reporting and status management. Thoughts?
>
I am worrying about having two sets of APIs for single purpose. From
vendor iommu driver's point of view, this feature is per device. Hence,
it still needs to do the same thing.
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 10:21 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-15 10:21 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi,
On 2021/4/15 15:43, Keqian Zhu wrote:
>>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>> I am not sure that I fully get your point. But I can't see any gaps of
>> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
>> Probably I missed anything.
> IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
> dirty tracking, it is not used to management the status of dirty log tracking.
>
> The feature reporting is per device, but the status management is per iommu_domain.
> Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
So why not
for_each_device_attached_domain()
iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
?
>
> And I think we'd better not mix the feature reporting and status management. Thoughts?
>
I am worrying about having two sets of APIs for single purpose. From
vendor iommu driver's point of view, this feature is per device. Hence,
it still needs to do the same thing.
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-15 10:21 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-15 10:21 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi,
On 2021/4/15 15:43, Keqian Zhu wrote:
>>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>> I am not sure that I fully get your point. But I can't see any gaps of
>> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
>> Probably I missed anything.
> IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
> dirty tracking, it is not used to management the status of dirty log tracking.
>
> The feature reporting is per device, but the status management is per iommu_domain.
> Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
So why not
for_each_device_attached_domain()
iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
?
>
> And I think we'd better not mix the feature reporting and status management. Thoughts?
>
I am worrying about having two sets of APIs for single purpose. From
vendor iommu driver's point of view, this feature is per device. Hence,
it still needs to do the same thing.
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-15 10:21 ` Lu Baolu
(?)
@ 2021-04-16 9:07 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-16 9:07 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
On 2021/4/15 18:21, Lu Baolu wrote:
> Hi,
>
> On 2021/4/15 15:43, Keqian Zhu wrote:
>>>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>>> I am not sure that I fully get your point. But I can't see any gaps of
>>> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
>>> Probably I missed anything.
>> IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
>> dirty tracking, it is not used to management the status of dirty log tracking.
>>
>> The feature reporting is per device, but the status management is per iommu_domain.
>> Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
>
> So why not
>
> for_each_device_attached_domain()
> iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
Looks reasonable, but the problem is that we just need to enable dirty log once per domain.
>
> ?
>>
>> And I think we'd better not mix the feature reporting and status management. Thoughts?
>>
>
> I am worrying about having two sets of APIs for single purpose. From
> vendor iommu driver's point of view, this feature is per device. Hence,
> it still needs to do the same thing.
Yes, we can unify the granule of feature reporting and status management.
The basic granule of dirty tracking is iommu_domain, I think it's very reasonable. We need an
interface to report the feature of iommu_domain, then the logic is much more clear.
Every time we add new device or remove device from the domain, we should update the feature (e.g.,
maintain a counter of unsupported devices).
What do you think about this idea?
Thanks,
Keqian
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-16 9:07 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-16 9:07 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
On 2021/4/15 18:21, Lu Baolu wrote:
> Hi,
>
> On 2021/4/15 15:43, Keqian Zhu wrote:
>>>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>>> I am not sure that I fully get your point. But I can't see any gaps of
>>> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
>>> Probably I missed anything.
>> IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
>> dirty tracking, it is not used to management the status of dirty log tracking.
>>
>> The feature reporting is per device, but the status management is per iommu_domain.
>> Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
>
> So why not
>
> for_each_device_attached_domain()
> iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
Looks reasonable, but the problem is that we just need to enable dirty log once per domain.
>
> ?
>>
>> And I think we'd better not mix the feature reporting and status management. Thoughts?
>>
>
> I am worrying about having two sets of APIs for single purpose. From
> vendor iommu driver's point of view, this feature is per device. Hence,
> it still needs to do the same thing.
Yes, we can unify the granule of feature reporting and status management.
The basic granule of dirty tracking is iommu_domain, I think it's very reasonable. We need an
interface to report the feature of iommu_domain, then the logic is much more clear.
Every time we add new device or remove device from the domain, we should update the feature (e.g.,
maintain a counter of unsupported devices).
What do you think about this idea?
Thanks,
Keqian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-16 9:07 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-16 9:07 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Baolu,
On 2021/4/15 18:21, Lu Baolu wrote:
> Hi,
>
> On 2021/4/15 15:43, Keqian Zhu wrote:
>>>> design it as not switchable. I will modify the commit message of patch#12, thanks!
>>> I am not sure that I fully get your point. But I can't see any gaps of
>>> using iommu_dev_enable/disable_feature() to switch dirty log on and off.
>>> Probably I missed anything.
>> IOMMU_DEV_FEAT_HWDBM just tells user whether underlying IOMMU driver supports
>> dirty tracking, it is not used to management the status of dirty log tracking.
>>
>> The feature reporting is per device, but the status management is per iommu_domain.
>> Only when all devices in a domain support HWDBM, we can start dirty log for the domain.
>
> So why not
>
> for_each_device_attached_domain()
> iommu_dev_enable_feature(IOMMU_DEV_FEAT_HWDBM)
Looks reasonable, but the problem is that we just need to enable dirty log once per domain.
>
> ?
>>
>> And I think we'd better not mix the feature reporting and status management. Thoughts?
>>
>
> I am worrying about having two sets of APIs for single purpose. From
> vendor iommu driver's point of view, this feature is per device. Hence,
> it still needs to do the same thing.
Yes, we can unify the granule of feature reporting and status management.
The basic granule of dirty tracking is iommu_domain, I think it's very reasonable. We need an
interface to report the feature of iommu_domain, then the logic is much more clear.
Every time we add new device or remove device from the domain, we should update the feature (e.g.,
maintain a counter of unsupported devices).
What do you think about this idea?
Thanks,
Keqian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
2021-04-16 9:07 ` Keqian Zhu
(?)
@ 2021-04-19 1:59 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-19 1:59 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 4/16/21 5:07 PM, Keqian Zhu wrote:
>> I am worrying about having two sets of APIs for single purpose. From
>> vendor iommu driver's point of view, this feature is per device. Hence,
>> it still needs to do the same thing.
> Yes, we can unify the granule of feature reporting and status management.
>
> The basic granule of dirty tracking is iommu_domain, I think it's very reasonable. We need an
> interface to report the feature of iommu_domain, then the logic is much more clear.
>
> Every time we add new device or remove device from the domain, we should update the feature (e.g.,
> maintain a counter of unsupported devices).
Yes. This looks cleaner.
>
> What do you think about this idea?
>
> Thanks,
> Keqian
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-19 1:59 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-19 1:59 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 4/16/21 5:07 PM, Keqian Zhu wrote:
>> I am worrying about having two sets of APIs for single purpose. From
>> vendor iommu driver's point of view, this feature is per device. Hence,
>> it still needs to do the same thing.
> Yes, we can unify the granule of feature reporting and status management.
>
> The basic granule of dirty tracking is iommu_domain, I think it's very reasonable. We need an
> interface to report the feature of iommu_domain, then the logic is much more clear.
>
> Every time we add new device or remove device from the domain, we should update the feature (e.g.,
> maintain a counter of unsupported devices).
Yes. This looks cleaner.
>
> What do you think about this idea?
>
> Thanks,
> Keqian
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 01/12] iommu: Introduce dirty log tracking framework
@ 2021-04-19 1:59 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-19 1:59 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Keqian,
On 4/16/21 5:07 PM, Keqian Zhu wrote:
>> I am worrying about having two sets of APIs for single purpose. From
>> vendor iommu driver's point of view, this feature is per device. Hence,
>> it still needs to do the same thing.
> Yes, we can unify the granule of feature reporting and status management.
>
> The basic granule of dirty tracking is iommu_domain, I think it's very reasonable. We need an
> interface to report the feature of iommu_domain, then the logic is much more clear.
>
> Every time we add new device or remove device from the domain, we should update the feature (e.g.,
> maintain a counter of unsupported devices).
Yes. This looks cleaner.
>
> What do you think about this idea?
>
> Thanks,
> Keqian
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Block(largepage) mapping is not a proper granule for dirty log tracking.
Take an extreme example, if DMA writes one byte, under 1G mapping, the
dirty amount reported is 1G, but under 4K mapping, the dirty amount is
just 4K.
This adds a new interface named iommu_split_block in IOMMU base layer.
A specific IOMMU driver can invoke it during start dirty log. If so, the
driver also need to realize the split_block iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of IOMMU, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 11 +++++++++++
2 files changed, 52 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 667b2d6d2fc0..bb413a927870 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ bool flush = false;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->split_block))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
+ iova, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ flush = true;
+
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->split_block(domain, iova, pgsize);
+ if (ret)
+ break;
+
+ pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_split_block);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 7f9ed9f520e2..c6c90ac069e3 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -208,6 +208,7 @@ struct iommu_iotlb_gather {
* @device_group: find iommu group for a particular device
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
+ * @split_block: Split block mapping into page mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -267,6 +268,8 @@ struct iommu_ops {
enum iommu_attr attr, void *data);
/* Track dirty log */
+ int (*split_block)(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -529,6 +532,8 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
+extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -929,6 +934,12 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_split_block(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Block(largepage) mapping is not a proper granule for dirty log tracking.
Take an extreme example, if DMA writes one byte, under 1G mapping, the
dirty amount reported is 1G, but under 4K mapping, the dirty amount is
just 4K.
This adds a new interface named iommu_split_block in IOMMU base layer.
A specific IOMMU driver can invoke it during start dirty log. If so, the
driver also need to realize the split_block iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of IOMMU, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 11 +++++++++++
2 files changed, 52 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 667b2d6d2fc0..bb413a927870 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ bool flush = false;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->split_block))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
+ iova, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ flush = true;
+
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->split_block(domain, iova, pgsize);
+ if (ret)
+ break;
+
+ pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_split_block);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 7f9ed9f520e2..c6c90ac069e3 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -208,6 +208,7 @@ struct iommu_iotlb_gather {
* @device_group: find iommu group for a particular device
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
+ * @split_block: Split block mapping into page mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -267,6 +268,8 @@ struct iommu_ops {
enum iommu_attr attr, void *data);
/* Track dirty log */
+ int (*split_block)(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -529,6 +532,8 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
+extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -929,6 +934,12 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_split_block(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Block(largepage) mapping is not a proper granule for dirty log tracking.
Take an extreme example, if DMA writes one byte, under 1G mapping, the
dirty amount reported is 1G, but under 4K mapping, the dirty amount is
just 4K.
This adds a new interface named iommu_split_block in IOMMU base layer.
A specific IOMMU driver can invoke it during start dirty log. If so, the
driver also need to realize the split_block iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of IOMMU, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 11 +++++++++++
2 files changed, 52 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index 667b2d6d2fc0..bb413a927870 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
}
EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
+int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
+ size_t size)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ bool flush = false;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->split_block))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
+ iova, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ flush = true;
+
+ pgsize = iommu_pgsize(domain, iova, size);
+
+ ret = ops->split_block(domain, iova, pgsize);
+ if (ret)
+ break;
+
+ pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
+
+ iova += pgsize;
+ size -= pgsize;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_split_block);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index 7f9ed9f520e2..c6c90ac069e3 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -208,6 +208,7 @@ struct iommu_iotlb_gather {
* @device_group: find iommu group for a particular device
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
+ * @split_block: Split block mapping into page mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -267,6 +268,8 @@ struct iommu_ops {
enum iommu_attr attr, void *data);
/* Track dirty log */
+ int (*split_block)(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -529,6 +532,8 @@ extern int iommu_domain_get_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
+extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
+ size_t size);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -929,6 +934,12 @@ static inline int iommu_domain_set_attr(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_split_block(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-14 7:14 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-14 7:14 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
On 4/13/21 4:54 PM, Keqian Zhu wrote:
> Block(largepage) mapping is not a proper granule for dirty log tracking.
> Take an extreme example, if DMA writes one byte, under 1G mapping, the
> dirty amount reported is 1G, but under 4K mapping, the dirty amount is
> just 4K.
>
> This adds a new interface named iommu_split_block in IOMMU base layer.
> A specific IOMMU driver can invoke it during start dirty log. If so, the
> driver also need to realize the split_block iommu ops.
>
> We flush all iotlbs after the whole procedure is completed to ease the
> pressure of IOMMU, as we will hanle a huge range of mapping in general.
>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
> ---
> drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
> include/linux/iommu.h | 11 +++++++++++
> 2 files changed, 52 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 667b2d6d2fc0..bb413a927870 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
> }
> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>
> +int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
> + size_t size)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + unsigned int min_pagesz;
> + size_t pgsize;
> + bool flush = false;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->split_block))
> + return -ENODEV;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
> + iova, size, min_pagesz);
> + return -EINVAL;
> + }
> +
> + while (size) {
> + flush = true;
> +
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->split_block(domain, iova, pgsize);
> + if (ret)
> + break;
> +
> + pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +
> + if (flush)
> + iommu_flush_iotlb_all(domain);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_split_block);
Do you really have any consumers of this interface other than the dirty
bit tracking? If not, I don't suggest to make this as a generic IOMMU
interface.
There is an implicit requirement for such interfaces. The
iommu_map/unmap(iova, size) shouldn't be called at the same time.
Currently there's no such sanity check in the iommu core. A poorly
written driver could mess up the kernel by misusing this interface.
This also applies to iommu_merge_page().
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-14 7:14 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-14 7:14 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
On 4/13/21 4:54 PM, Keqian Zhu wrote:
> Block(largepage) mapping is not a proper granule for dirty log tracking.
> Take an extreme example, if DMA writes one byte, under 1G mapping, the
> dirty amount reported is 1G, but under 4K mapping, the dirty amount is
> just 4K.
>
> This adds a new interface named iommu_split_block in IOMMU base layer.
> A specific IOMMU driver can invoke it during start dirty log. If so, the
> driver also need to realize the split_block iommu ops.
>
> We flush all iotlbs after the whole procedure is completed to ease the
> pressure of IOMMU, as we will hanle a huge range of mapping in general.
>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
> ---
> drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
> include/linux/iommu.h | 11 +++++++++++
> 2 files changed, 52 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 667b2d6d2fc0..bb413a927870 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
> }
> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>
> +int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
> + size_t size)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + unsigned int min_pagesz;
> + size_t pgsize;
> + bool flush = false;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->split_block))
> + return -ENODEV;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
> + iova, size, min_pagesz);
> + return -EINVAL;
> + }
> +
> + while (size) {
> + flush = true;
> +
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->split_block(domain, iova, pgsize);
> + if (ret)
> + break;
> +
> + pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +
> + if (flush)
> + iommu_flush_iotlb_all(domain);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_split_block);
Do you really have any consumers of this interface other than the dirty
bit tracking? If not, I don't suggest to make this as a generic IOMMU
interface.
There is an implicit requirement for such interfaces. The
iommu_map/unmap(iova, size) shouldn't be called at the same time.
Currently there's no such sanity check in the iommu core. A poorly
written driver could mess up the kernel by misusing this interface.
This also applies to iommu_merge_page().
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-14 7:14 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-14 7:14 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
On 4/13/21 4:54 PM, Keqian Zhu wrote:
> Block(largepage) mapping is not a proper granule for dirty log tracking.
> Take an extreme example, if DMA writes one byte, under 1G mapping, the
> dirty amount reported is 1G, but under 4K mapping, the dirty amount is
> just 4K.
>
> This adds a new interface named iommu_split_block in IOMMU base layer.
> A specific IOMMU driver can invoke it during start dirty log. If so, the
> driver also need to realize the split_block iommu ops.
>
> We flush all iotlbs after the whole procedure is completed to ease the
> pressure of IOMMU, as we will hanle a huge range of mapping in general.
>
> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
> ---
> drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
> include/linux/iommu.h | 11 +++++++++++
> 2 files changed, 52 insertions(+)
>
> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
> index 667b2d6d2fc0..bb413a927870 100644
> --- a/drivers/iommu/iommu.c
> +++ b/drivers/iommu/iommu.c
> @@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
> }
> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>
> +int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
> + size_t size)
> +{
> + const struct iommu_ops *ops = domain->ops;
> + unsigned int min_pagesz;
> + size_t pgsize;
> + bool flush = false;
> + int ret = 0;
> +
> + if (unlikely(!ops || !ops->split_block))
> + return -ENODEV;
> +
> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
> + iova, size, min_pagesz);
> + return -EINVAL;
> + }
> +
> + while (size) {
> + flush = true;
> +
> + pgsize = iommu_pgsize(domain, iova, size);
> +
> + ret = ops->split_block(domain, iova, pgsize);
> + if (ret)
> + break;
> +
> + pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
> +
> + iova += pgsize;
> + size -= pgsize;
> + }
> +
> + if (flush)
> + iommu_flush_iotlb_all(domain);
> +
> + return ret;
> +}
> +EXPORT_SYMBOL_GPL(iommu_split_block);
Do you really have any consumers of this interface other than the dirty
bit tracking? If not, I don't suggest to make this as a generic IOMMU
interface.
There is an implicit requirement for such interfaces. The
iommu_map/unmap(iova, size) shouldn't be called at the same time.
Currently there's no such sanity check in the iommu core. A poorly
written driver could mess up the kernel by misusing this interface.
This also applies to iommu_merge_page().
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-14 7:14 ` Lu Baolu
(?)
@ 2021-04-19 9:32 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-19 9:32 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
On 2021/4/14 15:14, Lu Baolu wrote:
> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>> Block(largepage) mapping is not a proper granule for dirty log tracking.
>> Take an extreme example, if DMA writes one byte, under 1G mapping, the
>> dirty amount reported is 1G, but under 4K mapping, the dirty amount is
>> just 4K.
>>
>> This adds a new interface named iommu_split_block in IOMMU base layer.
>> A specific IOMMU driver can invoke it during start dirty log. If so, the
>> driver also need to realize the split_block iommu ops.
>>
>> We flush all iotlbs after the whole procedure is completed to ease the
>> pressure of IOMMU, as we will hanle a huge range of mapping in general.
>>
>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 11 +++++++++++
>> 2 files changed, 52 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 667b2d6d2fc0..bb413a927870 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>> }
>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>> +int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
>> + size_t size)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + unsigned int min_pagesz;
>> + size_t pgsize;
>> + bool flush = false;
>> + int ret = 0;
>> +
>> + if (unlikely(!ops || !ops->split_block))
>> + return -ENODEV;
>> +
>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>> + iova, size, min_pagesz);
>> + return -EINVAL;
>> + }
>> +
>> + while (size) {
>> + flush = true;
>> +
>> + pgsize = iommu_pgsize(domain, iova, size);
>> +
>> + ret = ops->split_block(domain, iova, pgsize);
>> + if (ret)
>> + break;
>> +
>> + pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
>> +
>> + iova += pgsize;
>> + size -= pgsize;
>> + }
>> +
>> + if (flush)
>> + iommu_flush_iotlb_all(domain);
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>
> Do you really have any consumers of this interface other than the dirty
> bit tracking? If not, I don't suggest to make this as a generic IOMMU
> interface.
>
> There is an implicit requirement for such interfaces. The
> iommu_map/unmap(iova, size) shouldn't be called at the same time.
> Currently there's no such sanity check in the iommu core. A poorly
> written driver could mess up the kernel by misusing this interface.
Yes, I don't think up a scenario except dirty tracking.
Indeed, we'd better not make them as a generic interface.
Do you have any suggestion that underlying iommu drivers can share these code but
not make it as a generic iommu interface?
I have a not so good idea. Make the "split" interfaces as a static function, and
transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
On the other hand, if a driver calls map/unmap with split/merge at the same time,
it's a bug of driver, it should follow the rule.
>
> This also applies to iommu_merge_page().
>
Thanks,
Keqian
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-19 9:32 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-19 9:32 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
On 2021/4/14 15:14, Lu Baolu wrote:
> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>> Block(largepage) mapping is not a proper granule for dirty log tracking.
>> Take an extreme example, if DMA writes one byte, under 1G mapping, the
>> dirty amount reported is 1G, but under 4K mapping, the dirty amount is
>> just 4K.
>>
>> This adds a new interface named iommu_split_block in IOMMU base layer.
>> A specific IOMMU driver can invoke it during start dirty log. If so, the
>> driver also need to realize the split_block iommu ops.
>>
>> We flush all iotlbs after the whole procedure is completed to ease the
>> pressure of IOMMU, as we will hanle a huge range of mapping in general.
>>
>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 11 +++++++++++
>> 2 files changed, 52 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 667b2d6d2fc0..bb413a927870 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>> }
>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>> +int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
>> + size_t size)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + unsigned int min_pagesz;
>> + size_t pgsize;
>> + bool flush = false;
>> + int ret = 0;
>> +
>> + if (unlikely(!ops || !ops->split_block))
>> + return -ENODEV;
>> +
>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>> + iova, size, min_pagesz);
>> + return -EINVAL;
>> + }
>> +
>> + while (size) {
>> + flush = true;
>> +
>> + pgsize = iommu_pgsize(domain, iova, size);
>> +
>> + ret = ops->split_block(domain, iova, pgsize);
>> + if (ret)
>> + break;
>> +
>> + pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
>> +
>> + iova += pgsize;
>> + size -= pgsize;
>> + }
>> +
>> + if (flush)
>> + iommu_flush_iotlb_all(domain);
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>
> Do you really have any consumers of this interface other than the dirty
> bit tracking? If not, I don't suggest to make this as a generic IOMMU
> interface.
>
> There is an implicit requirement for such interfaces. The
> iommu_map/unmap(iova, size) shouldn't be called at the same time.
> Currently there's no such sanity check in the iommu core. A poorly
> written driver could mess up the kernel by misusing this interface.
Yes, I don't think up a scenario except dirty tracking.
Indeed, we'd better not make them as a generic interface.
Do you have any suggestion that underlying iommu drivers can share these code but
not make it as a generic iommu interface?
I have a not so good idea. Make the "split" interfaces as a static function, and
transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
On the other hand, if a driver calls map/unmap with split/merge at the same time,
it's a bug of driver, it should follow the rule.
>
> This also applies to iommu_merge_page().
>
Thanks,
Keqian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-19 9:32 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-19 9:32 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Baolu,
On 2021/4/14 15:14, Lu Baolu wrote:
> On 4/13/21 4:54 PM, Keqian Zhu wrote:
>> Block(largepage) mapping is not a proper granule for dirty log tracking.
>> Take an extreme example, if DMA writes one byte, under 1G mapping, the
>> dirty amount reported is 1G, but under 4K mapping, the dirty amount is
>> just 4K.
>>
>> This adds a new interface named iommu_split_block in IOMMU base layer.
>> A specific IOMMU driver can invoke it during start dirty log. If so, the
>> driver also need to realize the split_block iommu ops.
>>
>> We flush all iotlbs after the whole procedure is completed to ease the
>> pressure of IOMMU, as we will hanle a huge range of mapping in general.
>>
>> Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/iommu.c | 41 +++++++++++++++++++++++++++++++++++++++++
>> include/linux/iommu.h | 11 +++++++++++
>> 2 files changed, 52 insertions(+)
>>
>> diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
>> index 667b2d6d2fc0..bb413a927870 100644
>> --- a/drivers/iommu/iommu.c
>> +++ b/drivers/iommu/iommu.c
>> @@ -2721,6 +2721,47 @@ int iommu_domain_set_attr(struct iommu_domain *domain,
>> }
>> EXPORT_SYMBOL_GPL(iommu_domain_set_attr);
>> +int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
>> + size_t size)
>> +{
>> + const struct iommu_ops *ops = domain->ops;
>> + unsigned int min_pagesz;
>> + size_t pgsize;
>> + bool flush = false;
>> + int ret = 0;
>> +
>> + if (unlikely(!ops || !ops->split_block))
>> + return -ENODEV;
>> +
>> + min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
>> + if (!IS_ALIGNED(iova | size, min_pagesz)) {
>> + pr_err("unaligned: iova 0x%lx size 0x%zx min_pagesz 0x%x\n",
>> + iova, size, min_pagesz);
>> + return -EINVAL;
>> + }
>> +
>> + while (size) {
>> + flush = true;
>> +
>> + pgsize = iommu_pgsize(domain, iova, size);
>> +
>> + ret = ops->split_block(domain, iova, pgsize);
>> + if (ret)
>> + break;
>> +
>> + pr_debug("split handled: iova 0x%lx size 0x%zx\n", iova, pgsize);
>> +
>> + iova += pgsize;
>> + size -= pgsize;
>> + }
>> +
>> + if (flush)
>> + iommu_flush_iotlb_all(domain);
>> +
>> + return ret;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>
> Do you really have any consumers of this interface other than the dirty
> bit tracking? If not, I don't suggest to make this as a generic IOMMU
> interface.
>
> There is an implicit requirement for such interfaces. The
> iommu_map/unmap(iova, size) shouldn't be called at the same time.
> Currently there's no such sanity check in the iommu core. A poorly
> written driver could mess up the kernel by misusing this interface.
Yes, I don't think up a scenario except dirty tracking.
Indeed, we'd better not make them as a generic interface.
Do you have any suggestion that underlying iommu drivers can share these code but
not make it as a generic iommu interface?
I have a not so good idea. Make the "split" interfaces as a static function, and
transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
On the other hand, if a driver calls map/unmap with split/merge at the same time,
it's a bug of driver, it should follow the rule.
>
> This also applies to iommu_merge_page().
>
Thanks,
Keqian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-19 9:32 ` Keqian Zhu
(?)
@ 2021-04-19 13:33 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-19 13:33 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 2021/4/19 17:32, Keqian Zhu wrote:
>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>> Do you really have any consumers of this interface other than the dirty
>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>> interface.
>>
>> There is an implicit requirement for such interfaces. The
>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>> Currently there's no such sanity check in the iommu core. A poorly
>> written driver could mess up the kernel by misusing this interface.
> Yes, I don't think up a scenario except dirty tracking.
>
> Indeed, we'd better not make them as a generic interface.
>
> Do you have any suggestion that underlying iommu drivers can share these code but
> not make it as a generic iommu interface?
>
> I have a not so good idea. Make the "split" interfaces as a static function, and
> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
I understand splitting/merging super pages is an optimization, but not a
functional requirement. So is it possible to let the vendor iommu driver
decide whether splitting super pages when starting dirty bit tracking
and the opposite operation during when stopping it? The requirement for
upper layer is that starting/stopping dirty bit tracking and
mapping/unmapping are mutually exclusive.
>
> On the other hand, if a driver calls map/unmap with split/merge at the same time,
> it's a bug of driver, it should follow the rule.
>
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-19 13:33 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-19 13:33 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 2021/4/19 17:32, Keqian Zhu wrote:
>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>> Do you really have any consumers of this interface other than the dirty
>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>> interface.
>>
>> There is an implicit requirement for such interfaces. The
>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>> Currently there's no such sanity check in the iommu core. A poorly
>> written driver could mess up the kernel by misusing this interface.
> Yes, I don't think up a scenario except dirty tracking.
>
> Indeed, we'd better not make them as a generic interface.
>
> Do you have any suggestion that underlying iommu drivers can share these code but
> not make it as a generic iommu interface?
>
> I have a not so good idea. Make the "split" interfaces as a static function, and
> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
I understand splitting/merging super pages is an optimization, but not a
functional requirement. So is it possible to let the vendor iommu driver
decide whether splitting super pages when starting dirty bit tracking
and the opposite operation during when stopping it? The requirement for
upper layer is that starting/stopping dirty bit tracking and
mapping/unmapping are mutually exclusive.
>
> On the other hand, if a driver calls map/unmap with split/merge at the same time,
> it's a bug of driver, it should follow the rule.
>
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-19 13:33 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-19 13:33 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Keqian,
On 2021/4/19 17:32, Keqian Zhu wrote:
>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>> Do you really have any consumers of this interface other than the dirty
>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>> interface.
>>
>> There is an implicit requirement for such interfaces. The
>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>> Currently there's no such sanity check in the iommu core. A poorly
>> written driver could mess up the kernel by misusing this interface.
> Yes, I don't think up a scenario except dirty tracking.
>
> Indeed, we'd better not make them as a generic interface.
>
> Do you have any suggestion that underlying iommu drivers can share these code but
> not make it as a generic iommu interface?
>
> I have a not so good idea. Make the "split" interfaces as a static function, and
> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
I understand splitting/merging super pages is an optimization, but not a
functional requirement. So is it possible to let the vendor iommu driver
decide whether splitting super pages when starting dirty bit tracking
and the opposite operation during when stopping it? The requirement for
upper layer is that starting/stopping dirty bit tracking and
mapping/unmapping are mutually exclusive.
>
> On the other hand, if a driver calls map/unmap with split/merge at the same time,
> it's a bug of driver, it should follow the rule.
>
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-19 13:33 ` Lu Baolu
(?)
@ 2021-04-20 1:25 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-20 1:25 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
On 2021/4/19 21:33, Lu Baolu wrote:
> Hi Keqian,
>
> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>> Do you really have any consumers of this interface other than the dirty
>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>> interface.
>>>
>>> There is an implicit requirement for such interfaces. The
>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>> Currently there's no such sanity check in the iommu core. A poorly
>>> written driver could mess up the kernel by misusing this interface.
>> Yes, I don't think up a scenario except dirty tracking.
>>
>> Indeed, we'd better not make them as a generic interface.
>>
>> Do you have any suggestion that underlying iommu drivers can share these code but
>> not make it as a generic iommu interface?
>>
>> I have a not so good idea. Make the "split" interfaces as a static function, and
>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>
> I understand splitting/merging super pages is an optimization, but not a
> functional requirement. So is it possible to let the vendor iommu driver
> decide whether splitting super pages when starting dirty bit tracking
> and the opposite operation during when stopping it? The requirement for
Right. If I understand you correct, actually that is what this series does.
We realized split/merge in IOMMU core layer, but don't force vendor driver to use it.
The problem is that when we expose these interfaces to vendor IOMMU driver, will also
expose them to upper driver.
> upper layer is that starting/stopping dirty bit tracking and
> mapping/unmapping are mutually exclusive.
OK, I will explicitly add the hints. Thanks.
Thanks,
Keqian
>
>>
>> On the other hand, if a driver calls map/unmap with split/merge at the same time,
>> it's a bug of driver, it should follow the rule.
>>
>
> Best regards,
> baolu
> .
>
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 1:25 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-20 1:25 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
On 2021/4/19 21:33, Lu Baolu wrote:
> Hi Keqian,
>
> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>> Do you really have any consumers of this interface other than the dirty
>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>> interface.
>>>
>>> There is an implicit requirement for such interfaces. The
>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>> Currently there's no such sanity check in the iommu core. A poorly
>>> written driver could mess up the kernel by misusing this interface.
>> Yes, I don't think up a scenario except dirty tracking.
>>
>> Indeed, we'd better not make them as a generic interface.
>>
>> Do you have any suggestion that underlying iommu drivers can share these code but
>> not make it as a generic iommu interface?
>>
>> I have a not so good idea. Make the "split" interfaces as a static function, and
>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>
> I understand splitting/merging super pages is an optimization, but not a
> functional requirement. So is it possible to let the vendor iommu driver
> decide whether splitting super pages when starting dirty bit tracking
> and the opposite operation during when stopping it? The requirement for
Right. If I understand you correct, actually that is what this series does.
We realized split/merge in IOMMU core layer, but don't force vendor driver to use it.
The problem is that when we expose these interfaces to vendor IOMMU driver, will also
expose them to upper driver.
> upper layer is that starting/stopping dirty bit tracking and
> mapping/unmapping are mutually exclusive.
OK, I will explicitly add the hints. Thanks.
Thanks,
Keqian
>
>>
>> On the other hand, if a driver calls map/unmap with split/merge at the same time,
>> it's a bug of driver, it should follow the rule.
>>
>
> Best regards,
> baolu
> .
>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 1:25 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-20 1:25 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Baolu,
On 2021/4/19 21:33, Lu Baolu wrote:
> Hi Keqian,
>
> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>> Do you really have any consumers of this interface other than the dirty
>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>> interface.
>>>
>>> There is an implicit requirement for such interfaces. The
>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>> Currently there's no such sanity check in the iommu core. A poorly
>>> written driver could mess up the kernel by misusing this interface.
>> Yes, I don't think up a scenario except dirty tracking.
>>
>> Indeed, we'd better not make them as a generic interface.
>>
>> Do you have any suggestion that underlying iommu drivers can share these code but
>> not make it as a generic iommu interface?
>>
>> I have a not so good idea. Make the "split" interfaces as a static function, and
>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>
> I understand splitting/merging super pages is an optimization, but not a
> functional requirement. So is it possible to let the vendor iommu driver
> decide whether splitting super pages when starting dirty bit tracking
> and the opposite operation during when stopping it? The requirement for
Right. If I understand you correct, actually that is what this series does.
We realized split/merge in IOMMU core layer, but don't force vendor driver to use it.
The problem is that when we expose these interfaces to vendor IOMMU driver, will also
expose them to upper driver.
> upper layer is that starting/stopping dirty bit tracking and
> mapping/unmapping are mutually exclusive.
OK, I will explicitly add the hints. Thanks.
Thanks,
Keqian
>
>>
>> On the other hand, if a driver calls map/unmap with split/merge at the same time,
>> it's a bug of driver, it should follow the rule.
>>
>
> Best regards,
> baolu
> .
>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-20 1:25 ` Keqian Zhu
(?)
@ 2021-04-20 2:09 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-20 2:09 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 4/20/21 9:25 AM, Keqian Zhu wrote:
> Hi Baolu,
>
> On 2021/4/19 21:33, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>> Do you really have any consumers of this interface other than the dirty
>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>> interface.
>>>>
>>>> There is an implicit requirement for such interfaces. The
>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>> written driver could mess up the kernel by misusing this interface.
>>> Yes, I don't think up a scenario except dirty tracking.
>>>
>>> Indeed, we'd better not make them as a generic interface.
>>>
>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>> not make it as a generic iommu interface?
>>>
>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>
>> I understand splitting/merging super pages is an optimization, but not a
>> functional requirement. So is it possible to let the vendor iommu driver
>> decide whether splitting super pages when starting dirty bit tracking
>> and the opposite operation during when stopping it? The requirement for
> Right. If I understand you correct, actually that is what this series does.
I mean to say no generic APIs, jut do it by the iommu subsystem itself.
It's totally transparent to the upper level, just like what map() does.
The upper layer doesn't care about either super page or small page is
in use when do a mapping, right?
If you want to consolidate some code, how about putting them in
start/stop_tracking()?
Best regards,
baolu
> We realized split/merge in IOMMU core layer, but don't force vendor driver to use it.
>
> The problem is that when we expose these interfaces to vendor IOMMU driver, will also
> expose them to upper driver.
>
>> upper layer is that starting/stopping dirty bit tracking and
>> mapping/unmapping are mutually exclusive.
> OK, I will explicitly add the hints. Thanks.
>
> Thanks,
> Keqian
>>
>>>
>>> On the other hand, if a driver calls map/unmap with split/merge at the same time,
>>> it's a bug of driver, it should follow the rule.
>>>
>>
>> Best regards,
>> baolu
>> .
>>
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 2:09 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-20 2:09 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
Hi Keqian,
On 4/20/21 9:25 AM, Keqian Zhu wrote:
> Hi Baolu,
>
> On 2021/4/19 21:33, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>> Do you really have any consumers of this interface other than the dirty
>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>> interface.
>>>>
>>>> There is an implicit requirement for such interfaces. The
>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>> written driver could mess up the kernel by misusing this interface.
>>> Yes, I don't think up a scenario except dirty tracking.
>>>
>>> Indeed, we'd better not make them as a generic interface.
>>>
>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>> not make it as a generic iommu interface?
>>>
>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>
>> I understand splitting/merging super pages is an optimization, but not a
>> functional requirement. So is it possible to let the vendor iommu driver
>> decide whether splitting super pages when starting dirty bit tracking
>> and the opposite operation during when stopping it? The requirement for
> Right. If I understand you correct, actually that is what this series does.
I mean to say no generic APIs, jut do it by the iommu subsystem itself.
It's totally transparent to the upper level, just like what map() does.
The upper layer doesn't care about either super page or small page is
in use when do a mapping, right?
If you want to consolidate some code, how about putting them in
start/stop_tracking()?
Best regards,
baolu
> We realized split/merge in IOMMU core layer, but don't force vendor driver to use it.
>
> The problem is that when we expose these interfaces to vendor IOMMU driver, will also
> expose them to upper driver.
>
>> upper layer is that starting/stopping dirty bit tracking and
>> mapping/unmapping are mutually exclusive.
> OK, I will explicitly add the hints. Thanks.
>
> Thanks,
> Keqian
>>
>>>
>>> On the other hand, if a driver calls map/unmap with split/merge at the same time,
>>> it's a bug of driver, it should follow the rule.
>>>
>>
>> Best regards,
>> baolu
>> .
>>
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 2:09 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-20 2:09 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Keqian,
On 4/20/21 9:25 AM, Keqian Zhu wrote:
> Hi Baolu,
>
> On 2021/4/19 21:33, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>> Do you really have any consumers of this interface other than the dirty
>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>> interface.
>>>>
>>>> There is an implicit requirement for such interfaces. The
>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>> written driver could mess up the kernel by misusing this interface.
>>> Yes, I don't think up a scenario except dirty tracking.
>>>
>>> Indeed, we'd better not make them as a generic interface.
>>>
>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>> not make it as a generic iommu interface?
>>>
>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>
>> I understand splitting/merging super pages is an optimization, but not a
>> functional requirement. So is it possible to let the vendor iommu driver
>> decide whether splitting super pages when starting dirty bit tracking
>> and the opposite operation during when stopping it? The requirement for
> Right. If I understand you correct, actually that is what this series does.
I mean to say no generic APIs, jut do it by the iommu subsystem itself.
It's totally transparent to the upper level, just like what map() does.
The upper layer doesn't care about either super page or small page is
in use when do a mapping, right?
If you want to consolidate some code, how about putting them in
start/stop_tracking()?
Best regards,
baolu
> We realized split/merge in IOMMU core layer, but don't force vendor driver to use it.
>
> The problem is that when we expose these interfaces to vendor IOMMU driver, will also
> expose them to upper driver.
>
>> upper layer is that starting/stopping dirty bit tracking and
>> mapping/unmapping are mutually exclusive.
> OK, I will explicitly add the hints. Thanks.
>
> Thanks,
> Keqian
>>
>>>
>>> On the other hand, if a driver calls map/unmap with split/merge at the same time,
>>> it's a bug of driver, it should follow the rule.
>>>
>>
>> Best regards,
>> baolu
>> .
>>
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-20 2:09 ` Lu Baolu
(?)
@ 2021-04-20 7:32 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-20 7:32 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
Cheers for the your quick reply.
On 2021/4/20 10:09, Lu Baolu wrote:
> Hi Keqian,
>
> On 4/20/21 9:25 AM, Keqian Zhu wrote:
>> Hi Baolu,
>>
>> On 2021/4/19 21:33, Lu Baolu wrote:
>>> Hi Keqian,
>>>
>>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>>> Do you really have any consumers of this interface other than the dirty
>>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>>> interface.
>>>>>
>>>>> There is an implicit requirement for such interfaces. The
>>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>>> written driver could mess up the kernel by misusing this interface.
>>>> Yes, I don't think up a scenario except dirty tracking.
>>>>
>>>> Indeed, we'd better not make them as a generic interface.
>>>>
>>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>>> not make it as a generic iommu interface?
>>>>
>>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>>
>>> I understand splitting/merging super pages is an optimization, but not a
>>> functional requirement. So is it possible to let the vendor iommu driver
>>> decide whether splitting super pages when starting dirty bit tracking
>>> and the opposite operation during when stopping it? The requirement for
>> Right. If I understand you correct, actually that is what this series does.
>
> I mean to say no generic APIs, jut do it by the iommu subsystem itself.
> It's totally transparent to the upper level, just like what map() does.
> The upper layer doesn't care about either super page or small page is
> in use when do a mapping, right?
>
> If you want to consolidate some code, how about putting them in
> start/stop_tracking()?
Yep, this reminds me. What we want to reuse is the logic of "chunk by chunk" in split().
We can implement switch_dirty_log to be "chunk by chunk" too (just the same as sync/clear),
then the vendor iommu driver can invoke it's own private implementation of split().
So we can completely remove split() in the IOMMU core layer.
example code logic
iommu.c:
switch_dirty_log(big range) {
for_each_iommu_page(big range) {
ops->switch_dirty_log(iommu_pgsize)
}
}
vendor iommu driver:
switch_dirty_log(iommu_pgsize) {
if (enable) {
ops->split_block(iommu_pgsize)
/* And other actions, such as enable hardware capability */
} else {
for_each_continuous_physical_address(iommu_pgsize)
ops->merge_page()
}
}
Besides, vendor iommu driver can invoke split() in clear_dirty_log instead of in switch_dirty_log.
The benefit is that we usually clear dirty log gradually during dirty tracking, then we can split
large page mapping gradually, which speedup start_dirty_log and make less side effect on DMA performance.
Does it looks good for you?
Thanks,
Keqian
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 7:32 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-20 7:32 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
Hi Baolu,
Cheers for the your quick reply.
On 2021/4/20 10:09, Lu Baolu wrote:
> Hi Keqian,
>
> On 4/20/21 9:25 AM, Keqian Zhu wrote:
>> Hi Baolu,
>>
>> On 2021/4/19 21:33, Lu Baolu wrote:
>>> Hi Keqian,
>>>
>>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>>> Do you really have any consumers of this interface other than the dirty
>>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>>> interface.
>>>>>
>>>>> There is an implicit requirement for such interfaces. The
>>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>>> written driver could mess up the kernel by misusing this interface.
>>>> Yes, I don't think up a scenario except dirty tracking.
>>>>
>>>> Indeed, we'd better not make them as a generic interface.
>>>>
>>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>>> not make it as a generic iommu interface?
>>>>
>>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>>
>>> I understand splitting/merging super pages is an optimization, but not a
>>> functional requirement. So is it possible to let the vendor iommu driver
>>> decide whether splitting super pages when starting dirty bit tracking
>>> and the opposite operation during when stopping it? The requirement for
>> Right. If I understand you correct, actually that is what this series does.
>
> I mean to say no generic APIs, jut do it by the iommu subsystem itself.
> It's totally transparent to the upper level, just like what map() does.
> The upper layer doesn't care about either super page or small page is
> in use when do a mapping, right?
>
> If you want to consolidate some code, how about putting them in
> start/stop_tracking()?
Yep, this reminds me. What we want to reuse is the logic of "chunk by chunk" in split().
We can implement switch_dirty_log to be "chunk by chunk" too (just the same as sync/clear),
then the vendor iommu driver can invoke it's own private implementation of split().
So we can completely remove split() in the IOMMU core layer.
example code logic
iommu.c:
switch_dirty_log(big range) {
for_each_iommu_page(big range) {
ops->switch_dirty_log(iommu_pgsize)
}
}
vendor iommu driver:
switch_dirty_log(iommu_pgsize) {
if (enable) {
ops->split_block(iommu_pgsize)
/* And other actions, such as enable hardware capability */
} else {
for_each_continuous_physical_address(iommu_pgsize)
ops->merge_page()
}
}
Besides, vendor iommu driver can invoke split() in clear_dirty_log instead of in switch_dirty_log.
The benefit is that we usually clear dirty log gradually during dirty tracking, then we can split
large page mapping gradually, which speedup start_dirty_log and make less side effect on DMA performance.
Does it looks good for you?
Thanks,
Keqian
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 7:32 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-20 7:32 UTC (permalink / raw)
To: Lu Baolu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
Hi Baolu,
Cheers for the your quick reply.
On 2021/4/20 10:09, Lu Baolu wrote:
> Hi Keqian,
>
> On 4/20/21 9:25 AM, Keqian Zhu wrote:
>> Hi Baolu,
>>
>> On 2021/4/19 21:33, Lu Baolu wrote:
>>> Hi Keqian,
>>>
>>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>>> Do you really have any consumers of this interface other than the dirty
>>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>>> interface.
>>>>>
>>>>> There is an implicit requirement for such interfaces. The
>>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>>> written driver could mess up the kernel by misusing this interface.
>>>> Yes, I don't think up a scenario except dirty tracking.
>>>>
>>>> Indeed, we'd better not make them as a generic interface.
>>>>
>>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>>> not make it as a generic iommu interface?
>>>>
>>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>>
>>> I understand splitting/merging super pages is an optimization, but not a
>>> functional requirement. So is it possible to let the vendor iommu driver
>>> decide whether splitting super pages when starting dirty bit tracking
>>> and the opposite operation during when stopping it? The requirement for
>> Right. If I understand you correct, actually that is what this series does.
>
> I mean to say no generic APIs, jut do it by the iommu subsystem itself.
> It's totally transparent to the upper level, just like what map() does.
> The upper layer doesn't care about either super page or small page is
> in use when do a mapping, right?
>
> If you want to consolidate some code, how about putting them in
> start/stop_tracking()?
Yep, this reminds me. What we want to reuse is the logic of "chunk by chunk" in split().
We can implement switch_dirty_log to be "chunk by chunk" too (just the same as sync/clear),
then the vendor iommu driver can invoke it's own private implementation of split().
So we can completely remove split() in the IOMMU core layer.
example code logic
iommu.c:
switch_dirty_log(big range) {
for_each_iommu_page(big range) {
ops->switch_dirty_log(iommu_pgsize)
}
}
vendor iommu driver:
switch_dirty_log(iommu_pgsize) {
if (enable) {
ops->split_block(iommu_pgsize)
/* And other actions, such as enable hardware capability */
} else {
for_each_continuous_physical_address(iommu_pgsize)
ops->merge_page()
}
}
Besides, vendor iommu driver can invoke split() in clear_dirty_log instead of in switch_dirty_log.
The benefit is that we usually clear dirty log gradually during dirty tracking, then we can split
large page mapping gradually, which speedup start_dirty_log and make less side effect on DMA performance.
Does it looks good for you?
Thanks,
Keqian
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
2021-04-20 7:32 ` Keqian Zhu
(?)
@ 2021-04-20 7:53 ` Lu Baolu
-1 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-20 7:53 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
On 4/20/21 3:32 PM, Keqian Zhu wrote:
> Hi Baolu,
>
> Cheers for the your quick reply.
>
> On 2021/4/20 10:09, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 4/20/21 9:25 AM, Keqian Zhu wrote:
>>> Hi Baolu,
>>>
>>> On 2021/4/19 21:33, Lu Baolu wrote:
>>>> Hi Keqian,
>>>>
>>>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>>>> Do you really have any consumers of this interface other than the dirty
>>>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>>>> interface.
>>>>>>
>>>>>> There is an implicit requirement for such interfaces. The
>>>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>>>> written driver could mess up the kernel by misusing this interface.
>>>>> Yes, I don't think up a scenario except dirty tracking.
>>>>>
>>>>> Indeed, we'd better not make them as a generic interface.
>>>>>
>>>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>>>> not make it as a generic iommu interface?
>>>>>
>>>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>>>
>>>> I understand splitting/merging super pages is an optimization, but not a
>>>> functional requirement. So is it possible to let the vendor iommu driver
>>>> decide whether splitting super pages when starting dirty bit tracking
>>>> and the opposite operation during when stopping it? The requirement for
>>> Right. If I understand you correct, actually that is what this series does.
>>
>> I mean to say no generic APIs, jut do it by the iommu subsystem itself.
>> It's totally transparent to the upper level, just like what map() does.
>> The upper layer doesn't care about either super page or small page is
>> in use when do a mapping, right?
>>
>> If you want to consolidate some code, how about putting them in
>> start/stop_tracking()?
>
> Yep, this reminds me. What we want to reuse is the logic of "chunk by chunk" in split().
> We can implement switch_dirty_log to be "chunk by chunk" too (just the same as sync/clear),
> then the vendor iommu driver can invoke it's own private implementation of split().
> So we can completely remove split() in the IOMMU core layer.
>
> example code logic
>
> iommu.c:
> switch_dirty_log(big range) {
> for_each_iommu_page(big range) {
> ops->switch_dirty_log(iommu_pgsize)
> }
> }
>
> vendor iommu driver:
> switch_dirty_log(iommu_pgsize) {
>
> if (enable) {
> ops->split_block(iommu_pgsize)
> /* And other actions, such as enable hardware capability */
> } else {
> for_each_continuous_physical_address(iommu_pgsize)
> ops->merge_page()
> }
> }
>
> Besides, vendor iommu driver can invoke split() in clear_dirty_log instead of in switch_dirty_log.
> The benefit is that we usually clear dirty log gradually during dirty tracking, then we can split
> large page mapping gradually, which speedup start_dirty_log and make less side effect on DMA performance.
>
> Does it looks good for you?
Yes. It's clearer now.
>
> Thanks,
> Keqian
>
Best regards,
baolu
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 7:53 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-20 7:53 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: baolu.lu, Alex Williamson, Cornelia Huck, Kirti Wankhede,
wanghaibin.wang, jiangkunkun, yuzenghui, lushenming
On 4/20/21 3:32 PM, Keqian Zhu wrote:
> Hi Baolu,
>
> Cheers for the your quick reply.
>
> On 2021/4/20 10:09, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 4/20/21 9:25 AM, Keqian Zhu wrote:
>>> Hi Baolu,
>>>
>>> On 2021/4/19 21:33, Lu Baolu wrote:
>>>> Hi Keqian,
>>>>
>>>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>>>> Do you really have any consumers of this interface other than the dirty
>>>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>>>> interface.
>>>>>>
>>>>>> There is an implicit requirement for such interfaces. The
>>>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>>>> written driver could mess up the kernel by misusing this interface.
>>>>> Yes, I don't think up a scenario except dirty tracking.
>>>>>
>>>>> Indeed, we'd better not make them as a generic interface.
>>>>>
>>>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>>>> not make it as a generic iommu interface?
>>>>>
>>>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>>>
>>>> I understand splitting/merging super pages is an optimization, but not a
>>>> functional requirement. So is it possible to let the vendor iommu driver
>>>> decide whether splitting super pages when starting dirty bit tracking
>>>> and the opposite operation during when stopping it? The requirement for
>>> Right. If I understand you correct, actually that is what this series does.
>>
>> I mean to say no generic APIs, jut do it by the iommu subsystem itself.
>> It's totally transparent to the upper level, just like what map() does.
>> The upper layer doesn't care about either super page or small page is
>> in use when do a mapping, right?
>>
>> If you want to consolidate some code, how about putting them in
>> start/stop_tracking()?
>
> Yep, this reminds me. What we want to reuse is the logic of "chunk by chunk" in split().
> We can implement switch_dirty_log to be "chunk by chunk" too (just the same as sync/clear),
> then the vendor iommu driver can invoke it's own private implementation of split().
> So we can completely remove split() in the IOMMU core layer.
>
> example code logic
>
> iommu.c:
> switch_dirty_log(big range) {
> for_each_iommu_page(big range) {
> ops->switch_dirty_log(iommu_pgsize)
> }
> }
>
> vendor iommu driver:
> switch_dirty_log(iommu_pgsize) {
>
> if (enable) {
> ops->split_block(iommu_pgsize)
> /* And other actions, such as enable hardware capability */
> } else {
> for_each_continuous_physical_address(iommu_pgsize)
> ops->merge_page()
> }
> }
>
> Besides, vendor iommu driver can invoke split() in clear_dirty_log instead of in switch_dirty_log.
> The benefit is that we usually clear dirty log gradually during dirty tracking, then we can split
> large page mapping gradually, which speedup start_dirty_log and make less side effect on DMA performance.
>
> Does it looks good for you?
Yes. It's clearer now.
>
> Thanks,
> Keqian
>
Best regards,
baolu
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply [flat|nested] 81+ messages in thread* Re: [PATCH v3 02/12] iommu: Add iommu_split_block interface
@ 2021-04-20 7:53 ` Lu Baolu
0 siblings, 0 replies; 81+ messages in thread
From: Lu Baolu @ 2021-04-20 7:53 UTC (permalink / raw)
To: Keqian Zhu, linux-kernel, linux-arm-kernel, iommu, Robin Murphy,
Will Deacon, Joerg Roedel, Yi Sun, Jean-Philippe Brucker,
Jonathan Cameron, Tian Kevin
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
On 4/20/21 3:32 PM, Keqian Zhu wrote:
> Hi Baolu,
>
> Cheers for the your quick reply.
>
> On 2021/4/20 10:09, Lu Baolu wrote:
>> Hi Keqian,
>>
>> On 4/20/21 9:25 AM, Keqian Zhu wrote:
>>> Hi Baolu,
>>>
>>> On 2021/4/19 21:33, Lu Baolu wrote:
>>>> Hi Keqian,
>>>>
>>>> On 2021/4/19 17:32, Keqian Zhu wrote:
>>>>>>> +EXPORT_SYMBOL_GPL(iommu_split_block);
>>>>>> Do you really have any consumers of this interface other than the dirty
>>>>>> bit tracking? If not, I don't suggest to make this as a generic IOMMU
>>>>>> interface.
>>>>>>
>>>>>> There is an implicit requirement for such interfaces. The
>>>>>> iommu_map/unmap(iova, size) shouldn't be called at the same time.
>>>>>> Currently there's no such sanity check in the iommu core. A poorly
>>>>>> written driver could mess up the kernel by misusing this interface.
>>>>> Yes, I don't think up a scenario except dirty tracking.
>>>>>
>>>>> Indeed, we'd better not make them as a generic interface.
>>>>>
>>>>> Do you have any suggestion that underlying iommu drivers can share these code but
>>>>> not make it as a generic iommu interface?
>>>>>
>>>>> I have a not so good idea. Make the "split" interfaces as a static function, and
>>>>> transfer the function pointer to start_dirty_log. But it looks weird and inflexible.
>>>>
>>>> I understand splitting/merging super pages is an optimization, but not a
>>>> functional requirement. So is it possible to let the vendor iommu driver
>>>> decide whether splitting super pages when starting dirty bit tracking
>>>> and the opposite operation during when stopping it? The requirement for
>>> Right. If I understand you correct, actually that is what this series does.
>>
>> I mean to say no generic APIs, jut do it by the iommu subsystem itself.
>> It's totally transparent to the upper level, just like what map() does.
>> The upper layer doesn't care about either super page or small page is
>> in use when do a mapping, right?
>>
>> If you want to consolidate some code, how about putting them in
>> start/stop_tracking()?
>
> Yep, this reminds me. What we want to reuse is the logic of "chunk by chunk" in split().
> We can implement switch_dirty_log to be "chunk by chunk" too (just the same as sync/clear),
> then the vendor iommu driver can invoke it's own private implementation of split().
> So we can completely remove split() in the IOMMU core layer.
>
> example code logic
>
> iommu.c:
> switch_dirty_log(big range) {
> for_each_iommu_page(big range) {
> ops->switch_dirty_log(iommu_pgsize)
> }
> }
>
> vendor iommu driver:
> switch_dirty_log(iommu_pgsize) {
>
> if (enable) {
> ops->split_block(iommu_pgsize)
> /* And other actions, such as enable hardware capability */
> } else {
> for_each_continuous_physical_address(iommu_pgsize)
> ops->merge_page()
> }
> }
>
> Besides, vendor iommu driver can invoke split() in clear_dirty_log instead of in switch_dirty_log.
> The benefit is that we usually clear dirty log gradually during dirty tracking, then we can split
> large page mapping gradually, which speedup start_dirty_log and make less side effect on DMA performance.
>
> Does it looks good for you?
Yes. It's clearer now.
>
> Thanks,
> Keqian
>
Best regards,
baolu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply [flat|nested] 81+ messages in thread
* [PATCH v3 03/12] iommu: Add iommu_merge_page interface
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
If block(largepage) mappings are split during start dirty log, then
when stop dirty log, we need to recover them for better DMA performance.
This adds a new interface named iommu_merge_page in IOMMU base layer.
A specific IOMMU driver can invoke it during stop dirty log. If so, the
driver also need to realize the merge_page iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of iommu, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 12 +++++++
2 files changed, 87 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index bb413a927870..8f0d71bafb3a 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2762,6 +2762,81 @@ int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
}
EXPORT_SYMBOL_GPL(iommu_split_block);
+static int __iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->merge_page))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | paddr | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx pa %pa size 0x%zx min_pagesz 0x%x\n",
+ iova, &paddr, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova | paddr, size);
+
+ ret = ops->merge_page(domain, iova, paddr, pgsize, prot);
+ if (ret)
+ break;
+
+ pr_debug("merge handled: iova 0x%lx pa %pa size 0x%zx\n",
+ iova, &paddr, pgsize);
+
+ iova += pgsize;
+ paddr += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot)
+{
+ phys_addr_t phys;
+ dma_addr_t p, i;
+ size_t cont_size;
+ bool flush = false;
+ int ret = 0;
+
+ while (size) {
+ flush = true;
+
+ phys = iommu_iova_to_phys(domain, iova);
+ cont_size = PAGE_SIZE;
+ p = phys + cont_size;
+ i = iova + cont_size;
+
+ while (cont_size < size && p == iommu_iova_to_phys(domain, i)) {
+ p += PAGE_SIZE;
+ i += PAGE_SIZE;
+ cont_size += PAGE_SIZE;
+ }
+
+ ret = __iommu_merge_page(domain, iova, phys, cont_size, prot);
+ if (ret)
+ break;
+
+ iova += cont_size;
+ size -= cont_size;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_merge_page);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index c6c90ac069e3..fea3ecabff3d 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -209,6 +209,7 @@ struct iommu_iotlb_gather {
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
* @split_block: Split block mapping into page mapping
+ * @merge_page: Merge page mapping into block mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -270,6 +271,8 @@ struct iommu_ops {
/* Track dirty log */
int (*split_block)(struct iommu_domain *domain, unsigned long iova,
size_t size);
+ int (*merge_page)(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -534,6 +537,8 @@ extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
size_t size);
+extern int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -940,6 +945,13 @@ static inline int iommu_split_block(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ int prot)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 03/12] iommu: Add iommu_merge_page interface
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
If block(largepage) mappings are split during start dirty log, then
when stop dirty log, we need to recover them for better DMA performance.
This adds a new interface named iommu_merge_page in IOMMU base layer.
A specific IOMMU driver can invoke it during stop dirty log. If so, the
driver also need to realize the merge_page iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of iommu, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 12 +++++++
2 files changed, 87 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index bb413a927870..8f0d71bafb3a 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2762,6 +2762,81 @@ int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
}
EXPORT_SYMBOL_GPL(iommu_split_block);
+static int __iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->merge_page))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | paddr | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx pa %pa size 0x%zx min_pagesz 0x%x\n",
+ iova, &paddr, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova | paddr, size);
+
+ ret = ops->merge_page(domain, iova, paddr, pgsize, prot);
+ if (ret)
+ break;
+
+ pr_debug("merge handled: iova 0x%lx pa %pa size 0x%zx\n",
+ iova, &paddr, pgsize);
+
+ iova += pgsize;
+ paddr += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot)
+{
+ phys_addr_t phys;
+ dma_addr_t p, i;
+ size_t cont_size;
+ bool flush = false;
+ int ret = 0;
+
+ while (size) {
+ flush = true;
+
+ phys = iommu_iova_to_phys(domain, iova);
+ cont_size = PAGE_SIZE;
+ p = phys + cont_size;
+ i = iova + cont_size;
+
+ while (cont_size < size && p == iommu_iova_to_phys(domain, i)) {
+ p += PAGE_SIZE;
+ i += PAGE_SIZE;
+ cont_size += PAGE_SIZE;
+ }
+
+ ret = __iommu_merge_page(domain, iova, phys, cont_size, prot);
+ if (ret)
+ break;
+
+ iova += cont_size;
+ size -= cont_size;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_merge_page);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index c6c90ac069e3..fea3ecabff3d 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -209,6 +209,7 @@ struct iommu_iotlb_gather {
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
* @split_block: Split block mapping into page mapping
+ * @merge_page: Merge page mapping into block mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -270,6 +271,8 @@ struct iommu_ops {
/* Track dirty log */
int (*split_block)(struct iommu_domain *domain, unsigned long iova,
size_t size);
+ int (*merge_page)(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -534,6 +537,8 @@ extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
size_t size);
+extern int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -940,6 +945,13 @@ static inline int iommu_split_block(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ int prot)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 03/12] iommu: Add iommu_merge_page interface
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
If block(largepage) mappings are split during start dirty log, then
when stop dirty log, we need to recover them for better DMA performance.
This adds a new interface named iommu_merge_page in IOMMU base layer.
A specific IOMMU driver can invoke it during stop dirty log. If so, the
driver also need to realize the merge_page iommu ops.
We flush all iotlbs after the whole procedure is completed to ease the
pressure of iommu, as we will hanle a huge range of mapping in general.
Signed-off-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/iommu.c | 75 +++++++++++++++++++++++++++++++++++++++++++
include/linux/iommu.h | 12 +++++++
2 files changed, 87 insertions(+)
diff --git a/drivers/iommu/iommu.c b/drivers/iommu/iommu.c
index bb413a927870..8f0d71bafb3a 100644
--- a/drivers/iommu/iommu.c
+++ b/drivers/iommu/iommu.c
@@ -2762,6 +2762,81 @@ int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
}
EXPORT_SYMBOL_GPL(iommu_split_block);
+static int __iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ const struct iommu_ops *ops = domain->ops;
+ unsigned int min_pagesz;
+ size_t pgsize;
+ int ret = 0;
+
+ if (unlikely(!ops || !ops->merge_page))
+ return -ENODEV;
+
+ min_pagesz = 1 << __ffs(domain->pgsize_bitmap);
+ if (!IS_ALIGNED(iova | paddr | size, min_pagesz)) {
+ pr_err("unaligned: iova 0x%lx pa %pa size 0x%zx min_pagesz 0x%x\n",
+ iova, &paddr, size, min_pagesz);
+ return -EINVAL;
+ }
+
+ while (size) {
+ pgsize = iommu_pgsize(domain, iova | paddr, size);
+
+ ret = ops->merge_page(domain, iova, paddr, pgsize, prot);
+ if (ret)
+ break;
+
+ pr_debug("merge handled: iova 0x%lx pa %pa size 0x%zx\n",
+ iova, &paddr, pgsize);
+
+ iova += pgsize;
+ paddr += pgsize;
+ size -= pgsize;
+ }
+
+ return ret;
+}
+
+int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot)
+{
+ phys_addr_t phys;
+ dma_addr_t p, i;
+ size_t cont_size;
+ bool flush = false;
+ int ret = 0;
+
+ while (size) {
+ flush = true;
+
+ phys = iommu_iova_to_phys(domain, iova);
+ cont_size = PAGE_SIZE;
+ p = phys + cont_size;
+ i = iova + cont_size;
+
+ while (cont_size < size && p == iommu_iova_to_phys(domain, i)) {
+ p += PAGE_SIZE;
+ i += PAGE_SIZE;
+ cont_size += PAGE_SIZE;
+ }
+
+ ret = __iommu_merge_page(domain, iova, phys, cont_size, prot);
+ if (ret)
+ break;
+
+ iova += cont_size;
+ size -= cont_size;
+ }
+
+ if (flush)
+ iommu_flush_iotlb_all(domain);
+
+ return ret;
+}
+EXPORT_SYMBOL_GPL(iommu_merge_page);
+
int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot)
{
diff --git a/include/linux/iommu.h b/include/linux/iommu.h
index c6c90ac069e3..fea3ecabff3d 100644
--- a/include/linux/iommu.h
+++ b/include/linux/iommu.h
@@ -209,6 +209,7 @@ struct iommu_iotlb_gather {
* @domain_get_attr: Query domain attributes
* @domain_set_attr: Change domain attributes
* @split_block: Split block mapping into page mapping
+ * @merge_page: Merge page mapping into block mapping
* @switch_dirty_log: Perform actions to start|stop dirty log tracking
* @sync_dirty_log: Sync dirty log from IOMMU into a dirty bitmap
* @clear_dirty_log: Clear dirty log of IOMMU by a mask bitmap
@@ -270,6 +271,8 @@ struct iommu_ops {
/* Track dirty log */
int (*split_block)(struct iommu_domain *domain, unsigned long iova,
size_t size);
+ int (*merge_page)(struct iommu_domain *domain, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
int (*switch_dirty_log)(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
int (*sync_dirty_log)(struct iommu_domain *domain,
@@ -534,6 +537,8 @@ extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
void *data);
extern int iommu_split_block(struct iommu_domain *domain, unsigned long iova,
size_t size);
+extern int iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
+ size_t size, int prot);
extern int iommu_switch_dirty_log(struct iommu_domain *domain, bool enable,
unsigned long iova, size_t size, int prot);
extern int iommu_sync_dirty_log(struct iommu_domain *domain, unsigned long iova,
@@ -940,6 +945,13 @@ static inline int iommu_split_block(struct iommu_domain *domain,
return -EINVAL;
}
+static inline int iommu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ int prot)
+{
+ return -EINVAL;
+}
+
static inline int iommu_switch_dirty_log(struct iommu_domain *domain,
bool enable, unsigned long iova,
size_t size, int prot)
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 04/12] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Jean-Philippe Brucker <jean-philippe@linaro.org>
If the SMMU supports it and the kernel was built with HTTU support,
enable hardware update of access and dirty flags. This is essential for
shared page tables, to reduce the number of access faults on the fault
queue. Normal DMA with io-pgtables doesn't currently use the access or
dirty flags.
We can enable HTTU even if CPUs don't support it, because the kernel
always checks for HW dirty bit and updates the PTE flags atomically.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 41 ++++++++++++++++++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 8 ++++
3 files changed, 50 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index bb251cab61f3..ae075e675892 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -121,10 +121,12 @@ static struct arm_smmu_ctx_desc *arm_smmu_alloc_shared_cd(struct mm_struct *mm)
if (err)
goto out_free_asid;
+ /* HA and HD will be filtered out later if not supported by the SMMU */
tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, 64ULL - vabits_actual) |
FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, ARM_LPAE_TCR_RGN_WBWA) |
FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, ARM_LPAE_TCR_RGN_WBWA) |
FIELD_PREP(CTXDESC_CD_0_TCR_SH0, ARM_LPAE_TCR_SH_IS) |
+ CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD |
CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64;
switch (PAGE_SIZE) {
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 8594b4a83043..b6d965504f44 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1012,10 +1012,17 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
* this substream's traffic
*/
} else { /* (1) and (2) */
+ u64 tcr = cd->tcr;
+
cdptr[1] = cpu_to_le64(cd->ttbr & CTXDESC_CD_1_TTB0_MASK);
cdptr[2] = 0;
cdptr[3] = cpu_to_le64(cd->mair);
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ tcr &= ~CTXDESC_CD_0_TCR_HD;
+ if (!(smmu->features & ARM_SMMU_FEAT_HA))
+ tcr &= ~CTXDESC_CD_0_TCR_HA;
+
/*
* STE is live, and the SMMU might read dwords of this CD in any
* order. Ensure that it observes valid values before reading
@@ -1023,7 +1030,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
*/
arm_smmu_sync_cd(smmu_domain, ssid, true);
- val = cd->tcr |
+ val = tcr |
#ifdef __BIG_ENDIAN
CTXDESC_CD_0_ENDI |
#endif
@@ -3196,6 +3203,28 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
return 0;
}
+static void arm_smmu_get_httu(struct arm_smmu_device *smmu, u32 reg)
+{
+ u32 fw_features = smmu->features & (ARM_SMMU_FEAT_HA | ARM_SMMU_FEAT_HD);
+ u32 features = 0;
+
+ switch (FIELD_GET(IDR0_HTTU, reg)) {
+ case IDR0_HTTU_ACCESS_DIRTY:
+ features |= ARM_SMMU_FEAT_HD;
+ fallthrough;
+ case IDR0_HTTU_ACCESS:
+ features |= ARM_SMMU_FEAT_HA;
+ }
+
+ if (smmu->dev->of_node)
+ smmu->features |= features;
+ else if (features != fw_features)
+ /* ACPI IORT sets the HTTU bits */
+ dev_warn(smmu->dev,
+ "IDR0.HTTU overridden by FW configuration (0x%x)\n",
+ fw_features);
+}
+
static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
{
u32 reg;
@@ -3256,6 +3285,8 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
smmu->features |= ARM_SMMU_FEAT_E2H;
}
+ arm_smmu_get_httu(smmu, reg);
+
/*
* The coherency feature as set by FW is used in preference to the ID
* register, but warn on mismatch.
@@ -3441,6 +3472,14 @@ static int arm_smmu_device_acpi_probe(struct platform_device *pdev,
if (iort_smmu->flags & ACPI_IORT_SMMU_V3_COHACC_OVERRIDE)
smmu->features |= ARM_SMMU_FEAT_COHERENCY;
+ switch (FIELD_GET(ACPI_IORT_SMMU_V3_HTTU_OVERRIDE, iort_smmu->flags)) {
+ case IDR0_HTTU_ACCESS_DIRTY:
+ smmu->features |= ARM_SMMU_FEAT_HD;
+ fallthrough;
+ case IDR0_HTTU_ACCESS:
+ smmu->features |= ARM_SMMU_FEAT_HA;
+ }
+
return 0;
}
#else
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index f985817c967a..26d6b935b383 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -33,6 +33,9 @@
#define IDR0_ASID16 (1 << 12)
#define IDR0_ATS (1 << 10)
#define IDR0_HYP (1 << 9)
+#define IDR0_HTTU GENMASK(7, 6)
+#define IDR0_HTTU_ACCESS 1
+#define IDR0_HTTU_ACCESS_DIRTY 2
#define IDR0_COHACC (1 << 4)
#define IDR0_TTF GENMASK(3, 2)
#define IDR0_TTF_AARCH64 2
@@ -285,6 +288,9 @@
#define CTXDESC_CD_0_TCR_IPS GENMASK_ULL(34, 32)
#define CTXDESC_CD_0_TCR_TBI0 (1ULL << 38)
+#define CTXDESC_CD_0_TCR_HA (1UL << 43)
+#define CTXDESC_CD_0_TCR_HD (1UL << 42)
+
#define CTXDESC_CD_0_AA64 (1UL << 41)
#define CTXDESC_CD_0_S (1UL << 44)
#define CTXDESC_CD_0_R (1UL << 45)
@@ -607,6 +613,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_BTM (1 << 16)
#define ARM_SMMU_FEAT_SVA (1 << 17)
#define ARM_SMMU_FEAT_E2H (1 << 18)
+#define ARM_SMMU_FEAT_HA (1 << 19)
+#define ARM_SMMU_FEAT_HD (1 << 20)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 04/12] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Jean-Philippe Brucker <jean-philippe@linaro.org>
If the SMMU supports it and the kernel was built with HTTU support,
enable hardware update of access and dirty flags. This is essential for
shared page tables, to reduce the number of access faults on the fault
queue. Normal DMA with io-pgtables doesn't currently use the access or
dirty flags.
We can enable HTTU even if CPUs don't support it, because the kernel
always checks for HW dirty bit and updates the PTE flags atomically.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 41 ++++++++++++++++++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 8 ++++
3 files changed, 50 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index bb251cab61f3..ae075e675892 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -121,10 +121,12 @@ static struct arm_smmu_ctx_desc *arm_smmu_alloc_shared_cd(struct mm_struct *mm)
if (err)
goto out_free_asid;
+ /* HA and HD will be filtered out later if not supported by the SMMU */
tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, 64ULL - vabits_actual) |
FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, ARM_LPAE_TCR_RGN_WBWA) |
FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, ARM_LPAE_TCR_RGN_WBWA) |
FIELD_PREP(CTXDESC_CD_0_TCR_SH0, ARM_LPAE_TCR_SH_IS) |
+ CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD |
CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64;
switch (PAGE_SIZE) {
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 8594b4a83043..b6d965504f44 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1012,10 +1012,17 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
* this substream's traffic
*/
} else { /* (1) and (2) */
+ u64 tcr = cd->tcr;
+
cdptr[1] = cpu_to_le64(cd->ttbr & CTXDESC_CD_1_TTB0_MASK);
cdptr[2] = 0;
cdptr[3] = cpu_to_le64(cd->mair);
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ tcr &= ~CTXDESC_CD_0_TCR_HD;
+ if (!(smmu->features & ARM_SMMU_FEAT_HA))
+ tcr &= ~CTXDESC_CD_0_TCR_HA;
+
/*
* STE is live, and the SMMU might read dwords of this CD in any
* order. Ensure that it observes valid values before reading
@@ -1023,7 +1030,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
*/
arm_smmu_sync_cd(smmu_domain, ssid, true);
- val = cd->tcr |
+ val = tcr |
#ifdef __BIG_ENDIAN
CTXDESC_CD_0_ENDI |
#endif
@@ -3196,6 +3203,28 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
return 0;
}
+static void arm_smmu_get_httu(struct arm_smmu_device *smmu, u32 reg)
+{
+ u32 fw_features = smmu->features & (ARM_SMMU_FEAT_HA | ARM_SMMU_FEAT_HD);
+ u32 features = 0;
+
+ switch (FIELD_GET(IDR0_HTTU, reg)) {
+ case IDR0_HTTU_ACCESS_DIRTY:
+ features |= ARM_SMMU_FEAT_HD;
+ fallthrough;
+ case IDR0_HTTU_ACCESS:
+ features |= ARM_SMMU_FEAT_HA;
+ }
+
+ if (smmu->dev->of_node)
+ smmu->features |= features;
+ else if (features != fw_features)
+ /* ACPI IORT sets the HTTU bits */
+ dev_warn(smmu->dev,
+ "IDR0.HTTU overridden by FW configuration (0x%x)\n",
+ fw_features);
+}
+
static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
{
u32 reg;
@@ -3256,6 +3285,8 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
smmu->features |= ARM_SMMU_FEAT_E2H;
}
+ arm_smmu_get_httu(smmu, reg);
+
/*
* The coherency feature as set by FW is used in preference to the ID
* register, but warn on mismatch.
@@ -3441,6 +3472,14 @@ static int arm_smmu_device_acpi_probe(struct platform_device *pdev,
if (iort_smmu->flags & ACPI_IORT_SMMU_V3_COHACC_OVERRIDE)
smmu->features |= ARM_SMMU_FEAT_COHERENCY;
+ switch (FIELD_GET(ACPI_IORT_SMMU_V3_HTTU_OVERRIDE, iort_smmu->flags)) {
+ case IDR0_HTTU_ACCESS_DIRTY:
+ smmu->features |= ARM_SMMU_FEAT_HD;
+ fallthrough;
+ case IDR0_HTTU_ACCESS:
+ smmu->features |= ARM_SMMU_FEAT_HA;
+ }
+
return 0;
}
#else
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index f985817c967a..26d6b935b383 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -33,6 +33,9 @@
#define IDR0_ASID16 (1 << 12)
#define IDR0_ATS (1 << 10)
#define IDR0_HYP (1 << 9)
+#define IDR0_HTTU GENMASK(7, 6)
+#define IDR0_HTTU_ACCESS 1
+#define IDR0_HTTU_ACCESS_DIRTY 2
#define IDR0_COHACC (1 << 4)
#define IDR0_TTF GENMASK(3, 2)
#define IDR0_TTF_AARCH64 2
@@ -285,6 +288,9 @@
#define CTXDESC_CD_0_TCR_IPS GENMASK_ULL(34, 32)
#define CTXDESC_CD_0_TCR_TBI0 (1ULL << 38)
+#define CTXDESC_CD_0_TCR_HA (1UL << 43)
+#define CTXDESC_CD_0_TCR_HD (1UL << 42)
+
#define CTXDESC_CD_0_AA64 (1UL << 41)
#define CTXDESC_CD_0_S (1UL << 44)
#define CTXDESC_CD_0_R (1UL << 45)
@@ -607,6 +613,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_BTM (1 << 16)
#define ARM_SMMU_FEAT_SVA (1 << 17)
#define ARM_SMMU_FEAT_E2H (1 << 18)
+#define ARM_SMMU_FEAT_HA (1 << 19)
+#define ARM_SMMU_FEAT_HD (1 << 20)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 04/12] iommu/arm-smmu-v3: Add support for Hardware Translation Table Update
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Jean-Philippe Brucker <jean-philippe@linaro.org>
If the SMMU supports it and the kernel was built with HTTU support,
enable hardware update of access and dirty flags. This is essential for
shared page tables, to reduce the number of access faults on the fault
queue. Normal DMA with io-pgtables doesn't currently use the access or
dirty flags.
We can enable HTTU even if CPUs don't support it, because the kernel
always checks for HW dirty bit and updates the PTE flags atomically.
Signed-off-by: Jean-Philippe Brucker <jean-philippe@linaro.org>
---
.../iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c | 2 +
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 41 ++++++++++++++++++-
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 8 ++++
3 files changed, 50 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
index bb251cab61f3..ae075e675892 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3-sva.c
@@ -121,10 +121,12 @@ static struct arm_smmu_ctx_desc *arm_smmu_alloc_shared_cd(struct mm_struct *mm)
if (err)
goto out_free_asid;
+ /* HA and HD will be filtered out later if not supported by the SMMU */
tcr = FIELD_PREP(CTXDESC_CD_0_TCR_T0SZ, 64ULL - vabits_actual) |
FIELD_PREP(CTXDESC_CD_0_TCR_IRGN0, ARM_LPAE_TCR_RGN_WBWA) |
FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, ARM_LPAE_TCR_RGN_WBWA) |
FIELD_PREP(CTXDESC_CD_0_TCR_SH0, ARM_LPAE_TCR_SH_IS) |
+ CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD |
CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64;
switch (PAGE_SIZE) {
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 8594b4a83043..b6d965504f44 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1012,10 +1012,17 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
* this substream's traffic
*/
} else { /* (1) and (2) */
+ u64 tcr = cd->tcr;
+
cdptr[1] = cpu_to_le64(cd->ttbr & CTXDESC_CD_1_TTB0_MASK);
cdptr[2] = 0;
cdptr[3] = cpu_to_le64(cd->mair);
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ tcr &= ~CTXDESC_CD_0_TCR_HD;
+ if (!(smmu->features & ARM_SMMU_FEAT_HA))
+ tcr &= ~CTXDESC_CD_0_TCR_HA;
+
/*
* STE is live, and the SMMU might read dwords of this CD in any
* order. Ensure that it observes valid values before reading
@@ -1023,7 +1030,7 @@ int arm_smmu_write_ctx_desc(struct arm_smmu_domain *smmu_domain, int ssid,
*/
arm_smmu_sync_cd(smmu_domain, ssid, true);
- val = cd->tcr |
+ val = tcr |
#ifdef __BIG_ENDIAN
CTXDESC_CD_0_ENDI |
#endif
@@ -3196,6 +3203,28 @@ static int arm_smmu_device_reset(struct arm_smmu_device *smmu, bool bypass)
return 0;
}
+static void arm_smmu_get_httu(struct arm_smmu_device *smmu, u32 reg)
+{
+ u32 fw_features = smmu->features & (ARM_SMMU_FEAT_HA | ARM_SMMU_FEAT_HD);
+ u32 features = 0;
+
+ switch (FIELD_GET(IDR0_HTTU, reg)) {
+ case IDR0_HTTU_ACCESS_DIRTY:
+ features |= ARM_SMMU_FEAT_HD;
+ fallthrough;
+ case IDR0_HTTU_ACCESS:
+ features |= ARM_SMMU_FEAT_HA;
+ }
+
+ if (smmu->dev->of_node)
+ smmu->features |= features;
+ else if (features != fw_features)
+ /* ACPI IORT sets the HTTU bits */
+ dev_warn(smmu->dev,
+ "IDR0.HTTU overridden by FW configuration (0x%x)\n",
+ fw_features);
+}
+
static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
{
u32 reg;
@@ -3256,6 +3285,8 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
smmu->features |= ARM_SMMU_FEAT_E2H;
}
+ arm_smmu_get_httu(smmu, reg);
+
/*
* The coherency feature as set by FW is used in preference to the ID
* register, but warn on mismatch.
@@ -3441,6 +3472,14 @@ static int arm_smmu_device_acpi_probe(struct platform_device *pdev,
if (iort_smmu->flags & ACPI_IORT_SMMU_V3_COHACC_OVERRIDE)
smmu->features |= ARM_SMMU_FEAT_COHERENCY;
+ switch (FIELD_GET(ACPI_IORT_SMMU_V3_HTTU_OVERRIDE, iort_smmu->flags)) {
+ case IDR0_HTTU_ACCESS_DIRTY:
+ smmu->features |= ARM_SMMU_FEAT_HD;
+ fallthrough;
+ case IDR0_HTTU_ACCESS:
+ smmu->features |= ARM_SMMU_FEAT_HA;
+ }
+
return 0;
}
#else
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index f985817c967a..26d6b935b383 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -33,6 +33,9 @@
#define IDR0_ASID16 (1 << 12)
#define IDR0_ATS (1 << 10)
#define IDR0_HYP (1 << 9)
+#define IDR0_HTTU GENMASK(7, 6)
+#define IDR0_HTTU_ACCESS 1
+#define IDR0_HTTU_ACCESS_DIRTY 2
#define IDR0_COHACC (1 << 4)
#define IDR0_TTF GENMASK(3, 2)
#define IDR0_TTF_AARCH64 2
@@ -285,6 +288,9 @@
#define CTXDESC_CD_0_TCR_IPS GENMASK_ULL(34, 32)
#define CTXDESC_CD_0_TCR_TBI0 (1ULL << 38)
+#define CTXDESC_CD_0_TCR_HA (1UL << 43)
+#define CTXDESC_CD_0_TCR_HD (1UL << 42)
+
#define CTXDESC_CD_0_AA64 (1UL << 41)
#define CTXDESC_CD_0_S (1UL << 44)
#define CTXDESC_CD_0_R (1UL << 45)
@@ -607,6 +613,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_BTM (1 << 16)
#define ARM_SMMU_FEAT_SVA (1 << 17)
#define ARM_SMMU_FEAT_E2H (1 << 18)
+#define ARM_SMMU_FEAT_HA (1 << 19)
+#define ARM_SMMU_FEAT_HD (1 << 20)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 05/12] iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
As nested mode is not upstreamed now, we just aim to support dirty
log tracking for stage1 with io-pgtable mapping (means not support
SVA mapping). If HTTU is supported, we enable HA/HD bits in the SMMU
CD, and set DBM bit for writable TTD.
The dirty state information is encoded using the access permission
bits AP[2] (stage 1) or S2AP[1] (stage 2) in conjunction with the
DBM (Dirty Bit Modifier) bit, where DBM means writable and AP[2]/
S2AP[1] means dirty.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
drivers/iommu/io-pgtable-arm.c | 7 ++++++-
include/linux/io-pgtable.h | 3 +++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index b6d965504f44..369c0ea7a104 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1921,6 +1921,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,
FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) |
FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) |
FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) |
+ CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD |
CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64;
cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair;
@@ -2026,6 +2027,8 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
if (smmu_domain->non_strict)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT;
+ if (smmu->features & ARM_SMMU_FEAT_HD)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD;
pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain);
if (!pgtbl_ops)
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 87def58e79b5..94d790b8ed27 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -72,6 +72,7 @@
#define ARM_LPAE_PTE_NSTABLE (((arm_lpae_iopte)1) << 63)
#define ARM_LPAE_PTE_XN (((arm_lpae_iopte)3) << 53)
+#define ARM_LPAE_PTE_DBM (((arm_lpae_iopte)1) << 51)
#define ARM_LPAE_PTE_AF (((arm_lpae_iopte)1) << 10)
#define ARM_LPAE_PTE_SH_NS (((arm_lpae_iopte)0) << 8)
#define ARM_LPAE_PTE_SH_OS (((arm_lpae_iopte)2) << 8)
@@ -81,7 +82,7 @@
#define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2)
/* Ignore the contiguous bit for block splitting */
-#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)6) << 52)
+#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)13) << 51)
#define ARM_LPAE_PTE_ATTR_MASK (ARM_LPAE_PTE_ATTR_LO_MASK | \
ARM_LPAE_PTE_ATTR_HI_MASK)
/* Software bit for solving coherency races */
@@ -379,6 +380,7 @@ static int __arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova,
static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
int prot)
{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
arm_lpae_iopte pte;
if (data->iop.fmt == ARM_64_LPAE_S1 ||
@@ -386,6 +388,9 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
pte = ARM_LPAE_PTE_nG;
if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ))
pte |= ARM_LPAE_PTE_AP_RDONLY;
+ else if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_HD)
+ pte |= ARM_LPAE_PTE_DBM;
+
if (!(prot & IOMMU_PRIV))
pte |= ARM_LPAE_PTE_AP_UNPRIV;
} else {
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index a4c9ca2c31f1..64cee6831c97 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -82,6 +82,8 @@ struct io_pgtable_cfg {
*
* IO_PGTABLE_QUIRK_ARM_OUTER_WBWA: Override the outer-cacheability
* attributes set in the TCR for a non-coherent page-table walker.
+ *
+ * IO_PGTABLE_QUIRK_ARM_HD: Support hardware management of dirty status.
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -89,6 +91,7 @@ struct io_pgtable_cfg {
#define IO_PGTABLE_QUIRK_NON_STRICT BIT(4)
#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5)
#define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6)
+ #define IO_PGTABLE_QUIRK_ARM_HD BIT(7)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 05/12] iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
As nested mode is not upstreamed now, we just aim to support dirty
log tracking for stage1 with io-pgtable mapping (means not support
SVA mapping). If HTTU is supported, we enable HA/HD bits in the SMMU
CD, and set DBM bit for writable TTD.
The dirty state information is encoded using the access permission
bits AP[2] (stage 1) or S2AP[1] (stage 2) in conjunction with the
DBM (Dirty Bit Modifier) bit, where DBM means writable and AP[2]/
S2AP[1] means dirty.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
drivers/iommu/io-pgtable-arm.c | 7 ++++++-
include/linux/io-pgtable.h | 3 +++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index b6d965504f44..369c0ea7a104 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1921,6 +1921,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,
FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) |
FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) |
FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) |
+ CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD |
CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64;
cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair;
@@ -2026,6 +2027,8 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
if (smmu_domain->non_strict)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT;
+ if (smmu->features & ARM_SMMU_FEAT_HD)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD;
pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain);
if (!pgtbl_ops)
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 87def58e79b5..94d790b8ed27 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -72,6 +72,7 @@
#define ARM_LPAE_PTE_NSTABLE (((arm_lpae_iopte)1) << 63)
#define ARM_LPAE_PTE_XN (((arm_lpae_iopte)3) << 53)
+#define ARM_LPAE_PTE_DBM (((arm_lpae_iopte)1) << 51)
#define ARM_LPAE_PTE_AF (((arm_lpae_iopte)1) << 10)
#define ARM_LPAE_PTE_SH_NS (((arm_lpae_iopte)0) << 8)
#define ARM_LPAE_PTE_SH_OS (((arm_lpae_iopte)2) << 8)
@@ -81,7 +82,7 @@
#define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2)
/* Ignore the contiguous bit for block splitting */
-#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)6) << 52)
+#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)13) << 51)
#define ARM_LPAE_PTE_ATTR_MASK (ARM_LPAE_PTE_ATTR_LO_MASK | \
ARM_LPAE_PTE_ATTR_HI_MASK)
/* Software bit for solving coherency races */
@@ -379,6 +380,7 @@ static int __arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova,
static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
int prot)
{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
arm_lpae_iopte pte;
if (data->iop.fmt == ARM_64_LPAE_S1 ||
@@ -386,6 +388,9 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
pte = ARM_LPAE_PTE_nG;
if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ))
pte |= ARM_LPAE_PTE_AP_RDONLY;
+ else if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_HD)
+ pte |= ARM_LPAE_PTE_DBM;
+
if (!(prot & IOMMU_PRIV))
pte |= ARM_LPAE_PTE_AP_UNPRIV;
} else {
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index a4c9ca2c31f1..64cee6831c97 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -82,6 +82,8 @@ struct io_pgtable_cfg {
*
* IO_PGTABLE_QUIRK_ARM_OUTER_WBWA: Override the outer-cacheability
* attributes set in the TCR for a non-coherent page-table walker.
+ *
+ * IO_PGTABLE_QUIRK_ARM_HD: Support hardware management of dirty status.
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -89,6 +91,7 @@ struct io_pgtable_cfg {
#define IO_PGTABLE_QUIRK_NON_STRICT BIT(4)
#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5)
#define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6)
+ #define IO_PGTABLE_QUIRK_ARM_HD BIT(7)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 05/12] iommu/arm-smmu-v3: Enable HTTU for stage1 with io-pgtable mapping
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
As nested mode is not upstreamed now, we just aim to support dirty
log tracking for stage1 with io-pgtable mapping (means not support
SVA mapping). If HTTU is supported, we enable HA/HD bits in the SMMU
CD, and set DBM bit for writable TTD.
The dirty state information is encoded using the access permission
bits AP[2] (stage 1) or S2AP[1] (stage 2) in conjunction with the
DBM (Dirty Bit Modifier) bit, where DBM means writable and AP[2]/
S2AP[1] means dirty.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 3 +++
drivers/iommu/io-pgtable-arm.c | 7 ++++++-
include/linux/io-pgtable.h | 3 +++
3 files changed, 12 insertions(+), 1 deletion(-)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index b6d965504f44..369c0ea7a104 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -1921,6 +1921,7 @@ static int arm_smmu_domain_finalise_s1(struct arm_smmu_domain *smmu_domain,
FIELD_PREP(CTXDESC_CD_0_TCR_ORGN0, tcr->orgn) |
FIELD_PREP(CTXDESC_CD_0_TCR_SH0, tcr->sh) |
FIELD_PREP(CTXDESC_CD_0_TCR_IPS, tcr->ips) |
+ CTXDESC_CD_0_TCR_HA | CTXDESC_CD_0_TCR_HD |
CTXDESC_CD_0_TCR_EPD1 | CTXDESC_CD_0_AA64;
cfg->cd.mair = pgtbl_cfg->arm_lpae_s1_cfg.mair;
@@ -2026,6 +2027,8 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
if (smmu_domain->non_strict)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_NON_STRICT;
+ if (smmu->features & ARM_SMMU_FEAT_HD)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD;
pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain);
if (!pgtbl_ops)
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 87def58e79b5..94d790b8ed27 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -72,6 +72,7 @@
#define ARM_LPAE_PTE_NSTABLE (((arm_lpae_iopte)1) << 63)
#define ARM_LPAE_PTE_XN (((arm_lpae_iopte)3) << 53)
+#define ARM_LPAE_PTE_DBM (((arm_lpae_iopte)1) << 51)
#define ARM_LPAE_PTE_AF (((arm_lpae_iopte)1) << 10)
#define ARM_LPAE_PTE_SH_NS (((arm_lpae_iopte)0) << 8)
#define ARM_LPAE_PTE_SH_OS (((arm_lpae_iopte)2) << 8)
@@ -81,7 +82,7 @@
#define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2)
/* Ignore the contiguous bit for block splitting */
-#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)6) << 52)
+#define ARM_LPAE_PTE_ATTR_HI_MASK (((arm_lpae_iopte)13) << 51)
#define ARM_LPAE_PTE_ATTR_MASK (ARM_LPAE_PTE_ATTR_LO_MASK | \
ARM_LPAE_PTE_ATTR_HI_MASK)
/* Software bit for solving coherency races */
@@ -379,6 +380,7 @@ static int __arm_lpae_map(struct arm_lpae_io_pgtable *data, unsigned long iova,
static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
int prot)
{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
arm_lpae_iopte pte;
if (data->iop.fmt == ARM_64_LPAE_S1 ||
@@ -386,6 +388,9 @@ static arm_lpae_iopte arm_lpae_prot_to_pte(struct arm_lpae_io_pgtable *data,
pte = ARM_LPAE_PTE_nG;
if (!(prot & IOMMU_WRITE) && (prot & IOMMU_READ))
pte |= ARM_LPAE_PTE_AP_RDONLY;
+ else if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_HD)
+ pte |= ARM_LPAE_PTE_DBM;
+
if (!(prot & IOMMU_PRIV))
pte |= ARM_LPAE_PTE_AP_UNPRIV;
} else {
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index a4c9ca2c31f1..64cee6831c97 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -82,6 +82,8 @@ struct io_pgtable_cfg {
*
* IO_PGTABLE_QUIRK_ARM_OUTER_WBWA: Override the outer-cacheability
* attributes set in the TCR for a non-coherent page-table walker.
+ *
+ * IO_PGTABLE_QUIRK_ARM_HD: Support hardware management of dirty status.
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -89,6 +91,7 @@ struct io_pgtable_cfg {
#define IO_PGTABLE_QUIRK_NON_STRICT BIT(4)
#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5)
#define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6)
+ #define IO_PGTABLE_QUIRK_ARM_HD BIT(7)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 06/12] iommu/arm-smmu-v3: Add feature detection for BBML
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
When altering a translation table descriptor of some specific reasons,
we require break-before-make procedure. But it might cause problems when
the TTD is alive. The I/O streams might not tolerate translation faults.
If the SMMU supports BBM level 1 or BBM level 2, we can change the block
size without using break-before-make sequence.
This adds feature detection for BBML, none functional change expected.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 19 +++++++++++++++++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 ++++++
include/linux/io-pgtable.h | 8 ++++++++
3 files changed, 33 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 369c0ea7a104..443ac19c6da9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2030,6 +2030,11 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
if (smmu->features & ARM_SMMU_FEAT_HD)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD;
+ if (smmu->features & ARM_SMMU_FEAT_BBML1)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_BBML1;
+ else if (smmu->features & ARM_SMMU_FEAT_BBML2)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_BBML2;
+
pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain);
if (!pgtbl_ops)
return -ENOMEM;
@@ -3373,6 +3378,20 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
/* IDR3 */
reg = readl_relaxed(smmu->base + ARM_SMMU_IDR3);
+ switch (FIELD_GET(IDR3_BBML, reg)) {
+ case IDR3_BBML0:
+ break;
+ case IDR3_BBML1:
+ smmu->features |= ARM_SMMU_FEAT_BBML1;
+ break;
+ case IDR3_BBML2:
+ smmu->features |= ARM_SMMU_FEAT_BBML2;
+ break;
+ default:
+ dev_err(smmu->dev, "unknown/unsupported BBM behavior level\n");
+ return -ENXIO;
+ }
+
if (FIELD_GET(IDR3_RIL, reg))
smmu->features |= ARM_SMMU_FEAT_RANGE_INV;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 26d6b935b383..a74125675544 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -54,6 +54,10 @@
#define IDR1_SIDSIZE GENMASK(5, 0)
#define ARM_SMMU_IDR3 0xc
+#define IDR3_BBML GENMASK(12, 11)
+#define IDR3_BBML0 0
+#define IDR3_BBML1 1
+#define IDR3_BBML2 2
#define IDR3_RIL (1 << 10)
#define ARM_SMMU_IDR5 0x14
@@ -615,6 +619,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_E2H (1 << 18)
#define ARM_SMMU_FEAT_HA (1 << 19)
#define ARM_SMMU_FEAT_HD (1 << 20)
+#define ARM_SMMU_FEAT_BBML1 (1 << 21)
+#define ARM_SMMU_FEAT_BBML2 (1 << 22)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 64cee6831c97..9e7163ec9447 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -84,6 +84,12 @@ struct io_pgtable_cfg {
* attributes set in the TCR for a non-coherent page-table walker.
*
* IO_PGTABLE_QUIRK_ARM_HD: Support hardware management of dirty status.
+ *
+ * IO_PGTABLE_QUIRK_ARM_BBML1: ARM SMMU supports BBM Level 1 behavior
+ * when changing block size.
+ *
+ * IO_PGTABLE_QUIRK_ARM_BBML2: ARM SMMU supports BBM Level 2 behavior
+ * when changing block size.
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -92,6 +98,8 @@ struct io_pgtable_cfg {
#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5)
#define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6)
#define IO_PGTABLE_QUIRK_ARM_HD BIT(7)
+ #define IO_PGTABLE_QUIRK_ARM_BBML1 BIT(8)
+ #define IO_PGTABLE_QUIRK_ARM_BBML2 BIT(9)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 06/12] iommu/arm-smmu-v3: Add feature detection for BBML
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
When altering a translation table descriptor of some specific reasons,
we require break-before-make procedure. But it might cause problems when
the TTD is alive. The I/O streams might not tolerate translation faults.
If the SMMU supports BBM level 1 or BBM level 2, we can change the block
size without using break-before-make sequence.
This adds feature detection for BBML, none functional change expected.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 19 +++++++++++++++++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 ++++++
include/linux/io-pgtable.h | 8 ++++++++
3 files changed, 33 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 369c0ea7a104..443ac19c6da9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2030,6 +2030,11 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
if (smmu->features & ARM_SMMU_FEAT_HD)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD;
+ if (smmu->features & ARM_SMMU_FEAT_BBML1)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_BBML1;
+ else if (smmu->features & ARM_SMMU_FEAT_BBML2)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_BBML2;
+
pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain);
if (!pgtbl_ops)
return -ENOMEM;
@@ -3373,6 +3378,20 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
/* IDR3 */
reg = readl_relaxed(smmu->base + ARM_SMMU_IDR3);
+ switch (FIELD_GET(IDR3_BBML, reg)) {
+ case IDR3_BBML0:
+ break;
+ case IDR3_BBML1:
+ smmu->features |= ARM_SMMU_FEAT_BBML1;
+ break;
+ case IDR3_BBML2:
+ smmu->features |= ARM_SMMU_FEAT_BBML2;
+ break;
+ default:
+ dev_err(smmu->dev, "unknown/unsupported BBM behavior level\n");
+ return -ENXIO;
+ }
+
if (FIELD_GET(IDR3_RIL, reg))
smmu->features |= ARM_SMMU_FEAT_RANGE_INV;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 26d6b935b383..a74125675544 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -54,6 +54,10 @@
#define IDR1_SIDSIZE GENMASK(5, 0)
#define ARM_SMMU_IDR3 0xc
+#define IDR3_BBML GENMASK(12, 11)
+#define IDR3_BBML0 0
+#define IDR3_BBML1 1
+#define IDR3_BBML2 2
#define IDR3_RIL (1 << 10)
#define ARM_SMMU_IDR5 0x14
@@ -615,6 +619,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_E2H (1 << 18)
#define ARM_SMMU_FEAT_HA (1 << 19)
#define ARM_SMMU_FEAT_HD (1 << 20)
+#define ARM_SMMU_FEAT_BBML1 (1 << 21)
+#define ARM_SMMU_FEAT_BBML2 (1 << 22)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 64cee6831c97..9e7163ec9447 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -84,6 +84,12 @@ struct io_pgtable_cfg {
* attributes set in the TCR for a non-coherent page-table walker.
*
* IO_PGTABLE_QUIRK_ARM_HD: Support hardware management of dirty status.
+ *
+ * IO_PGTABLE_QUIRK_ARM_BBML1: ARM SMMU supports BBM Level 1 behavior
+ * when changing block size.
+ *
+ * IO_PGTABLE_QUIRK_ARM_BBML2: ARM SMMU supports BBM Level 2 behavior
+ * when changing block size.
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -92,6 +98,8 @@ struct io_pgtable_cfg {
#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5)
#define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6)
#define IO_PGTABLE_QUIRK_ARM_HD BIT(7)
+ #define IO_PGTABLE_QUIRK_ARM_BBML1 BIT(8)
+ #define IO_PGTABLE_QUIRK_ARM_BBML2 BIT(9)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 06/12] iommu/arm-smmu-v3: Add feature detection for BBML
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
When altering a translation table descriptor of some specific reasons,
we require break-before-make procedure. But it might cause problems when
the TTD is alive. The I/O streams might not tolerate translation faults.
If the SMMU supports BBM level 1 or BBM level 2, we can change the block
size without using break-before-make sequence.
This adds feature detection for BBML, none functional change expected.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 19 +++++++++++++++++++
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 6 ++++++
include/linux/io-pgtable.h | 8 ++++++++
3 files changed, 33 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 369c0ea7a104..443ac19c6da9 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2030,6 +2030,11 @@ static int arm_smmu_domain_finalise(struct iommu_domain *domain,
if (smmu->features & ARM_SMMU_FEAT_HD)
pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_HD;
+ if (smmu->features & ARM_SMMU_FEAT_BBML1)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_BBML1;
+ else if (smmu->features & ARM_SMMU_FEAT_BBML2)
+ pgtbl_cfg.quirks |= IO_PGTABLE_QUIRK_ARM_BBML2;
+
pgtbl_ops = alloc_io_pgtable_ops(fmt, &pgtbl_cfg, smmu_domain);
if (!pgtbl_ops)
return -ENOMEM;
@@ -3373,6 +3378,20 @@ static int arm_smmu_device_hw_probe(struct arm_smmu_device *smmu)
/* IDR3 */
reg = readl_relaxed(smmu->base + ARM_SMMU_IDR3);
+ switch (FIELD_GET(IDR3_BBML, reg)) {
+ case IDR3_BBML0:
+ break;
+ case IDR3_BBML1:
+ smmu->features |= ARM_SMMU_FEAT_BBML1;
+ break;
+ case IDR3_BBML2:
+ smmu->features |= ARM_SMMU_FEAT_BBML2;
+ break;
+ default:
+ dev_err(smmu->dev, "unknown/unsupported BBM behavior level\n");
+ return -ENXIO;
+ }
+
if (FIELD_GET(IDR3_RIL, reg))
smmu->features |= ARM_SMMU_FEAT_RANGE_INV;
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
index 26d6b935b383..a74125675544 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h
@@ -54,6 +54,10 @@
#define IDR1_SIDSIZE GENMASK(5, 0)
#define ARM_SMMU_IDR3 0xc
+#define IDR3_BBML GENMASK(12, 11)
+#define IDR3_BBML0 0
+#define IDR3_BBML1 1
+#define IDR3_BBML2 2
#define IDR3_RIL (1 << 10)
#define ARM_SMMU_IDR5 0x14
@@ -615,6 +619,8 @@ struct arm_smmu_device {
#define ARM_SMMU_FEAT_E2H (1 << 18)
#define ARM_SMMU_FEAT_HA (1 << 19)
#define ARM_SMMU_FEAT_HD (1 << 20)
+#define ARM_SMMU_FEAT_BBML1 (1 << 21)
+#define ARM_SMMU_FEAT_BBML2 (1 << 22)
u32 features;
#define ARM_SMMU_OPT_SKIP_PREFETCH (1 << 0)
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 64cee6831c97..9e7163ec9447 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -84,6 +84,12 @@ struct io_pgtable_cfg {
* attributes set in the TCR for a non-coherent page-table walker.
*
* IO_PGTABLE_QUIRK_ARM_HD: Support hardware management of dirty status.
+ *
+ * IO_PGTABLE_QUIRK_ARM_BBML1: ARM SMMU supports BBM Level 1 behavior
+ * when changing block size.
+ *
+ * IO_PGTABLE_QUIRK_ARM_BBML2: ARM SMMU supports BBM Level 2 behavior
+ * when changing block size.
*/
#define IO_PGTABLE_QUIRK_ARM_NS BIT(0)
#define IO_PGTABLE_QUIRK_NO_PERMS BIT(1)
@@ -92,6 +98,8 @@ struct io_pgtable_cfg {
#define IO_PGTABLE_QUIRK_ARM_TTBR1 BIT(5)
#define IO_PGTABLE_QUIRK_ARM_OUTER_WBWA BIT(6)
#define IO_PGTABLE_QUIRK_ARM_HD BIT(7)
+ #define IO_PGTABLE_QUIRK_ARM_BBML1 BIT(8)
+ #define IO_PGTABLE_QUIRK_ARM_BBML2 BIT(9)
unsigned long quirks;
unsigned long pgsize_bitmap;
unsigned int ias;
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 07/12] iommu/arm-smmu-v3: Realize split_block iommu ops
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
This splits block descriptor to an span of page descriptors. BBML1
or BBML2 feature is required.
Spliting block does not simultaneously work with other pgtable ops,
as the only designed user is vfio, which always hold a lock, so race
condition is not considered in the pgtable ops.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 27 +++++
drivers/iommu/io-pgtable-arm.c | 122 ++++++++++++++++++++
include/linux/io-pgtable.h | 2 +
3 files changed, 151 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 443ac19c6da9..cfa83fa03c89 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2537,6 +2537,32 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain,
return ret;
}
+static int arm_smmu_split_block(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ size_t handled_size;
+
+ if (!(smmu->features & (ARM_SMMU_FEAT_BBML1 | ARM_SMMU_FEAT_BBML2))) {
+ dev_err(smmu->dev, "don't support BBML1/2, can't split block\n");
+ return -ENODEV;
+ }
+ if (!ops || !ops->split_block) {
+ pr_err("io-pgtable don't realize split block\n");
+ return -ENODEV;
+ }
+
+ handled_size = ops->split_block(ops, iova, size);
+ if (handled_size != size) {
+ pr_err("split block failed\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2636,6 +2662,7 @@ static struct iommu_ops arm_smmu_ops = {
.device_group = arm_smmu_device_group,
.domain_get_attr = arm_smmu_domain_get_attr,
.domain_set_attr = arm_smmu_domain_set_attr,
+ .split_block = arm_smmu_split_block,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 94d790b8ed27..4c4eec3c0698 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -79,6 +79,8 @@
#define ARM_LPAE_PTE_SH_IS (((arm_lpae_iopte)3) << 8)
#define ARM_LPAE_PTE_NS (((arm_lpae_iopte)1) << 5)
#define ARM_LPAE_PTE_VALID (((arm_lpae_iopte)1) << 0)
+/* Block descriptor bits */
+#define ARM_LPAE_PTE_NT (((arm_lpae_iopte)1) << 16)
#define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2)
/* Ignore the contiguous bit for block splitting */
@@ -679,6 +681,125 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops,
return iopte_to_paddr(pte, data) | iova;
}
+static size_t __arm_lpae_split_block(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size, int lvl,
+ arm_lpae_iopte *ptep);
+
+static size_t arm_lpae_do_split_blk(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ arm_lpae_iopte blk_pte, int lvl,
+ arm_lpae_iopte *ptep)
+{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ arm_lpae_iopte pte, *tablep;
+ phys_addr_t blk_paddr;
+ size_t tablesz = ARM_LPAE_GRANULE(data);
+ size_t split_sz = ARM_LPAE_BLOCK_SIZE(lvl, data);
+ int i;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ tablep = __arm_lpae_alloc_pages(tablesz, GFP_ATOMIC, cfg);
+ if (!tablep)
+ return 0;
+
+ blk_paddr = iopte_to_paddr(blk_pte, data);
+ pte = iopte_prot(blk_pte);
+ for (i = 0; i < tablesz / sizeof(pte); i++, blk_paddr += split_sz)
+ __arm_lpae_init_pte(data, blk_paddr, pte, lvl, &tablep[i]);
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_BBML1) {
+ /* Race does not exist */
+ blk_pte |= ARM_LPAE_PTE_NT;
+ __arm_lpae_set_pte(ptep, blk_pte, cfg);
+ io_pgtable_tlb_flush_walk(&data->iop, iova, size, size);
+ }
+ /* Race does not exist */
+ pte = arm_lpae_install_table(tablep, ptep, blk_pte, cfg);
+
+ /* Have splited it into page? */
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 1))
+ return size;
+
+ /* Go back to lvl - 1 */
+ ptep -= ARM_LPAE_LVL_IDX(iova, lvl - 1, data);
+ return __arm_lpae_split_block(data, iova, size, lvl - 1, ptep);
+}
+
+static size_t __arm_lpae_split_block(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size, int lvl,
+ arm_lpae_iopte *ptep)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ size_t base, next_size, total_size;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return 0;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 1) ||
+ (pte & ARM_LPAE_PTE_AP_RDONLY))
+ return size;
+
+ /* We find a writable block, split it. */
+ return arm_lpae_do_split_blk(data, iova, size, pte,
+ lvl + 1, ptep);
+ } else {
+ /* If it is the last table level, then nothing to do */
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 2))
+ return size;
+
+ total_size = 0;
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size)
+ total_size += __arm_lpae_split_block(data,
+ iova + base, next_size, lvl + 1,
+ ptep);
+ return total_size;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ WARN(1, "Can't split behind a block.\n");
+ return 0;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_split_block(data, iova, size, lvl + 1, ptep);
+}
+
+static size_t arm_lpae_split_block(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ int lvl = data->start_level;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return 0;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return 0;
+
+ /* If it is smallest granule, then nothing to do */
+ if (size == ARM_LPAE_BLOCK_SIZE(ARM_LPAE_MAX_LEVELS - 1, data))
+ return size;
+
+ return __arm_lpae_split_block(data, iova, size, lvl, ptep);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -757,6 +878,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.map = arm_lpae_map,
.unmap = arm_lpae_unmap,
.iova_to_phys = arm_lpae_iova_to_phys,
+ .split_block = arm_lpae_split_block,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 9e7163ec9447..47aa8f20849b 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -167,6 +167,8 @@ struct io_pgtable_ops {
size_t size, struct iommu_iotlb_gather *gather);
phys_addr_t (*iova_to_phys)(struct io_pgtable_ops *ops,
unsigned long iova);
+ size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
+ size_t size);
};
/**
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 07/12] iommu/arm-smmu-v3: Realize split_block iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
This splits block descriptor to an span of page descriptors. BBML1
or BBML2 feature is required.
Spliting block does not simultaneously work with other pgtable ops,
as the only designed user is vfio, which always hold a lock, so race
condition is not considered in the pgtable ops.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 27 +++++
drivers/iommu/io-pgtable-arm.c | 122 ++++++++++++++++++++
include/linux/io-pgtable.h | 2 +
3 files changed, 151 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 443ac19c6da9..cfa83fa03c89 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2537,6 +2537,32 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain,
return ret;
}
+static int arm_smmu_split_block(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ size_t handled_size;
+
+ if (!(smmu->features & (ARM_SMMU_FEAT_BBML1 | ARM_SMMU_FEAT_BBML2))) {
+ dev_err(smmu->dev, "don't support BBML1/2, can't split block\n");
+ return -ENODEV;
+ }
+ if (!ops || !ops->split_block) {
+ pr_err("io-pgtable don't realize split block\n");
+ return -ENODEV;
+ }
+
+ handled_size = ops->split_block(ops, iova, size);
+ if (handled_size != size) {
+ pr_err("split block failed\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2636,6 +2662,7 @@ static struct iommu_ops arm_smmu_ops = {
.device_group = arm_smmu_device_group,
.domain_get_attr = arm_smmu_domain_get_attr,
.domain_set_attr = arm_smmu_domain_set_attr,
+ .split_block = arm_smmu_split_block,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 94d790b8ed27..4c4eec3c0698 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -79,6 +79,8 @@
#define ARM_LPAE_PTE_SH_IS (((arm_lpae_iopte)3) << 8)
#define ARM_LPAE_PTE_NS (((arm_lpae_iopte)1) << 5)
#define ARM_LPAE_PTE_VALID (((arm_lpae_iopte)1) << 0)
+/* Block descriptor bits */
+#define ARM_LPAE_PTE_NT (((arm_lpae_iopte)1) << 16)
#define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2)
/* Ignore the contiguous bit for block splitting */
@@ -679,6 +681,125 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops,
return iopte_to_paddr(pte, data) | iova;
}
+static size_t __arm_lpae_split_block(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size, int lvl,
+ arm_lpae_iopte *ptep);
+
+static size_t arm_lpae_do_split_blk(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ arm_lpae_iopte blk_pte, int lvl,
+ arm_lpae_iopte *ptep)
+{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ arm_lpae_iopte pte, *tablep;
+ phys_addr_t blk_paddr;
+ size_t tablesz = ARM_LPAE_GRANULE(data);
+ size_t split_sz = ARM_LPAE_BLOCK_SIZE(lvl, data);
+ int i;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ tablep = __arm_lpae_alloc_pages(tablesz, GFP_ATOMIC, cfg);
+ if (!tablep)
+ return 0;
+
+ blk_paddr = iopte_to_paddr(blk_pte, data);
+ pte = iopte_prot(blk_pte);
+ for (i = 0; i < tablesz / sizeof(pte); i++, blk_paddr += split_sz)
+ __arm_lpae_init_pte(data, blk_paddr, pte, lvl, &tablep[i]);
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_BBML1) {
+ /* Race does not exist */
+ blk_pte |= ARM_LPAE_PTE_NT;
+ __arm_lpae_set_pte(ptep, blk_pte, cfg);
+ io_pgtable_tlb_flush_walk(&data->iop, iova, size, size);
+ }
+ /* Race does not exist */
+ pte = arm_lpae_install_table(tablep, ptep, blk_pte, cfg);
+
+ /* Have splited it into page? */
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 1))
+ return size;
+
+ /* Go back to lvl - 1 */
+ ptep -= ARM_LPAE_LVL_IDX(iova, lvl - 1, data);
+ return __arm_lpae_split_block(data, iova, size, lvl - 1, ptep);
+}
+
+static size_t __arm_lpae_split_block(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size, int lvl,
+ arm_lpae_iopte *ptep)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ size_t base, next_size, total_size;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return 0;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 1) ||
+ (pte & ARM_LPAE_PTE_AP_RDONLY))
+ return size;
+
+ /* We find a writable block, split it. */
+ return arm_lpae_do_split_blk(data, iova, size, pte,
+ lvl + 1, ptep);
+ } else {
+ /* If it is the last table level, then nothing to do */
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 2))
+ return size;
+
+ total_size = 0;
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size)
+ total_size += __arm_lpae_split_block(data,
+ iova + base, next_size, lvl + 1,
+ ptep);
+ return total_size;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ WARN(1, "Can't split behind a block.\n");
+ return 0;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_split_block(data, iova, size, lvl + 1, ptep);
+}
+
+static size_t arm_lpae_split_block(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ int lvl = data->start_level;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return 0;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return 0;
+
+ /* If it is smallest granule, then nothing to do */
+ if (size == ARM_LPAE_BLOCK_SIZE(ARM_LPAE_MAX_LEVELS - 1, data))
+ return size;
+
+ return __arm_lpae_split_block(data, iova, size, lvl, ptep);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -757,6 +878,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.map = arm_lpae_map,
.unmap = arm_lpae_unmap,
.iova_to_phys = arm_lpae_iova_to_phys,
+ .split_block = arm_lpae_split_block,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 9e7163ec9447..47aa8f20849b 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -167,6 +167,8 @@ struct io_pgtable_ops {
size_t size, struct iommu_iotlb_gather *gather);
phys_addr_t (*iova_to_phys)(struct io_pgtable_ops *ops,
unsigned long iova);
+ size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
+ size_t size);
};
/**
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 07/12] iommu/arm-smmu-v3: Realize split_block iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
This splits block descriptor to an span of page descriptors. BBML1
or BBML2 feature is required.
Spliting block does not simultaneously work with other pgtable ops,
as the only designed user is vfio, which always hold a lock, so race
condition is not considered in the pgtable ops.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 27 +++++
drivers/iommu/io-pgtable-arm.c | 122 ++++++++++++++++++++
include/linux/io-pgtable.h | 2 +
3 files changed, 151 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 443ac19c6da9..cfa83fa03c89 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2537,6 +2537,32 @@ static int arm_smmu_domain_set_attr(struct iommu_domain *domain,
return ret;
}
+static int arm_smmu_split_block(struct iommu_domain *domain,
+ unsigned long iova, size_t size)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ size_t handled_size;
+
+ if (!(smmu->features & (ARM_SMMU_FEAT_BBML1 | ARM_SMMU_FEAT_BBML2))) {
+ dev_err(smmu->dev, "don't support BBML1/2, can't split block\n");
+ return -ENODEV;
+ }
+ if (!ops || !ops->split_block) {
+ pr_err("io-pgtable don't realize split block\n");
+ return -ENODEV;
+ }
+
+ handled_size = ops->split_block(ops, iova, size);
+ if (handled_size != size) {
+ pr_err("split block failed\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2636,6 +2662,7 @@ static struct iommu_ops arm_smmu_ops = {
.device_group = arm_smmu_device_group,
.domain_get_attr = arm_smmu_domain_get_attr,
.domain_set_attr = arm_smmu_domain_set_attr,
+ .split_block = arm_smmu_split_block,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 94d790b8ed27..4c4eec3c0698 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -79,6 +79,8 @@
#define ARM_LPAE_PTE_SH_IS (((arm_lpae_iopte)3) << 8)
#define ARM_LPAE_PTE_NS (((arm_lpae_iopte)1) << 5)
#define ARM_LPAE_PTE_VALID (((arm_lpae_iopte)1) << 0)
+/* Block descriptor bits */
+#define ARM_LPAE_PTE_NT (((arm_lpae_iopte)1) << 16)
#define ARM_LPAE_PTE_ATTR_LO_MASK (((arm_lpae_iopte)0x3ff) << 2)
/* Ignore the contiguous bit for block splitting */
@@ -679,6 +681,125 @@ static phys_addr_t arm_lpae_iova_to_phys(struct io_pgtable_ops *ops,
return iopte_to_paddr(pte, data) | iova;
}
+static size_t __arm_lpae_split_block(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size, int lvl,
+ arm_lpae_iopte *ptep);
+
+static size_t arm_lpae_do_split_blk(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ arm_lpae_iopte blk_pte, int lvl,
+ arm_lpae_iopte *ptep)
+{
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ arm_lpae_iopte pte, *tablep;
+ phys_addr_t blk_paddr;
+ size_t tablesz = ARM_LPAE_GRANULE(data);
+ size_t split_sz = ARM_LPAE_BLOCK_SIZE(lvl, data);
+ int i;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ tablep = __arm_lpae_alloc_pages(tablesz, GFP_ATOMIC, cfg);
+ if (!tablep)
+ return 0;
+
+ blk_paddr = iopte_to_paddr(blk_pte, data);
+ pte = iopte_prot(blk_pte);
+ for (i = 0; i < tablesz / sizeof(pte); i++, blk_paddr += split_sz)
+ __arm_lpae_init_pte(data, blk_paddr, pte, lvl, &tablep[i]);
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_BBML1) {
+ /* Race does not exist */
+ blk_pte |= ARM_LPAE_PTE_NT;
+ __arm_lpae_set_pte(ptep, blk_pte, cfg);
+ io_pgtable_tlb_flush_walk(&data->iop, iova, size, size);
+ }
+ /* Race does not exist */
+ pte = arm_lpae_install_table(tablep, ptep, blk_pte, cfg);
+
+ /* Have splited it into page? */
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 1))
+ return size;
+
+ /* Go back to lvl - 1 */
+ ptep -= ARM_LPAE_LVL_IDX(iova, lvl - 1, data);
+ return __arm_lpae_split_block(data, iova, size, lvl - 1, ptep);
+}
+
+static size_t __arm_lpae_split_block(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size, int lvl,
+ arm_lpae_iopte *ptep)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ size_t base, next_size, total_size;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return 0;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 1) ||
+ (pte & ARM_LPAE_PTE_AP_RDONLY))
+ return size;
+
+ /* We find a writable block, split it. */
+ return arm_lpae_do_split_blk(data, iova, size, pte,
+ lvl + 1, ptep);
+ } else {
+ /* If it is the last table level, then nothing to do */
+ if (lvl == (ARM_LPAE_MAX_LEVELS - 2))
+ return size;
+
+ total_size = 0;
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size)
+ total_size += __arm_lpae_split_block(data,
+ iova + base, next_size, lvl + 1,
+ ptep);
+ return total_size;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ WARN(1, "Can't split behind a block.\n");
+ return 0;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_split_block(data, iova, size, lvl + 1, ptep);
+}
+
+static size_t arm_lpae_split_block(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ int lvl = data->start_level;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return 0;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return 0;
+
+ /* If it is smallest granule, then nothing to do */
+ if (size == ARM_LPAE_BLOCK_SIZE(ARM_LPAE_MAX_LEVELS - 1, data))
+ return size;
+
+ return __arm_lpae_split_block(data, iova, size, lvl, ptep);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -757,6 +878,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.map = arm_lpae_map,
.unmap = arm_lpae_unmap,
.iova_to_phys = arm_lpae_iova_to_phys,
+ .split_block = arm_lpae_split_block,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 9e7163ec9447..47aa8f20849b 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -167,6 +167,8 @@ struct io_pgtable_ops {
size_t size, struct iommu_iotlb_gather *gather);
phys_addr_t (*iova_to_phys)(struct io_pgtable_ops *ops,
unsigned long iova);
+ size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
+ size_t size);
};
/**
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 08/12] iommu/arm-smmu-v3: Realize merge_page iommu ops
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
This reinstalls block mappings and unmap the span of page mappings.
BBML1 or BBML2 feature is required.
Merging page does not simultaneously work with other pgtable ops,
as the only designed user is vfio, which always hold a lock, so race
condition is not considered in the pgtable ops.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 ++++++++
drivers/iommu/io-pgtable-arm.c | 78 +++++++++++++++++++++
include/linux/io-pgtable.h | 2 +
3 files changed, 108 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index cfa83fa03c89..4d8495d88be2 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2563,6 +2563,33 @@ static int arm_smmu_split_block(struct iommu_domain *domain,
return 0;
}
+static int arm_smmu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ size_t handled_size;
+
+ if (!(smmu->features & (ARM_SMMU_FEAT_BBML1 | ARM_SMMU_FEAT_BBML2))) {
+ dev_err(smmu->dev, "don't support BBML1/2, can't merge page\n");
+ return -ENODEV;
+ }
+ if (!ops || !ops->merge_page) {
+ pr_err("io-pgtable don't realize merge page\n");
+ return -ENODEV;
+ }
+
+ handled_size = ops->merge_page(ops, iova, paddr, size, prot);
+ if (handled_size != size) {
+ pr_err("merge page failed\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2663,6 +2690,7 @@ static struct iommu_ops arm_smmu_ops = {
.domain_get_attr = arm_smmu_domain_get_attr,
.domain_set_attr = arm_smmu_domain_set_attr,
.split_block = arm_smmu_split_block,
+ .merge_page = arm_smmu_merge_page,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 4c4eec3c0698..9028328b99b0 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -800,6 +800,83 @@ static size_t arm_lpae_split_block(struct io_pgtable_ops *ops,
return __arm_lpae_split_block(data, iova, size, lvl, ptep);
}
+static size_t __arm_lpae_merge_page(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int lvl, arm_lpae_iopte *ptep,
+ arm_lpae_iopte prot)
+{
+ arm_lpae_iopte pte, *tablep;
+ struct io_pgtable *iop = &data->iop;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return 0;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt))
+ return size;
+
+ /* Race does not exist */
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_BBML1) {
+ prot |= ARM_LPAE_PTE_NT;
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ io_pgtable_tlb_flush_walk(iop, iova, size,
+ ARM_LPAE_GRANULE(data));
+
+ prot &= ~(ARM_LPAE_PTE_NT);
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ } else {
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ }
+
+ tablep = iopte_deref(pte, data);
+ __arm_lpae_free_pgtable(data, lvl + 1, tablep);
+ return size;
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ /* The size is too small, already merged */
+ return size;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_merge_page(data, iova, paddr, size, lvl + 1, ptep, prot);
+}
+
+static size_t arm_lpae_merge_page(struct io_pgtable_ops *ops, unsigned long iova,
+ phys_addr_t paddr, size_t size, int iommu_prot)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ arm_lpae_iopte prot;
+ long iaext = (s64)iova >> cfg->ias;
+
+ /* If no access, then nothing to do */
+ if (!(iommu_prot & (IOMMU_READ | IOMMU_WRITE)))
+ return size;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return 0;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext || paddr >> cfg->oas))
+ return 0;
+
+ /* If it is smallest granule, then nothing to do */
+ if (size == ARM_LPAE_BLOCK_SIZE(ARM_LPAE_MAX_LEVELS - 1, data))
+ return size;
+
+ prot = arm_lpae_prot_to_pte(data, iommu_prot);
+ return __arm_lpae_merge_page(data, iova, paddr, size, lvl, ptep, prot);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -879,6 +956,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.unmap = arm_lpae_unmap,
.iova_to_phys = arm_lpae_iova_to_phys,
.split_block = arm_lpae_split_block,
+ .merge_page = arm_lpae_merge_page,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 47aa8f20849b..dbee6170de7f 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -169,6 +169,8 @@ struct io_pgtable_ops {
unsigned long iova);
size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
size_t size);
+ size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
};
/**
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 08/12] iommu/arm-smmu-v3: Realize merge_page iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
This reinstalls block mappings and unmap the span of page mappings.
BBML1 or BBML2 feature is required.
Merging page does not simultaneously work with other pgtable ops,
as the only designed user is vfio, which always hold a lock, so race
condition is not considered in the pgtable ops.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 ++++++++
drivers/iommu/io-pgtable-arm.c | 78 +++++++++++++++++++++
include/linux/io-pgtable.h | 2 +
3 files changed, 108 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index cfa83fa03c89..4d8495d88be2 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2563,6 +2563,33 @@ static int arm_smmu_split_block(struct iommu_domain *domain,
return 0;
}
+static int arm_smmu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ size_t handled_size;
+
+ if (!(smmu->features & (ARM_SMMU_FEAT_BBML1 | ARM_SMMU_FEAT_BBML2))) {
+ dev_err(smmu->dev, "don't support BBML1/2, can't merge page\n");
+ return -ENODEV;
+ }
+ if (!ops || !ops->merge_page) {
+ pr_err("io-pgtable don't realize merge page\n");
+ return -ENODEV;
+ }
+
+ handled_size = ops->merge_page(ops, iova, paddr, size, prot);
+ if (handled_size != size) {
+ pr_err("merge page failed\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2663,6 +2690,7 @@ static struct iommu_ops arm_smmu_ops = {
.domain_get_attr = arm_smmu_domain_get_attr,
.domain_set_attr = arm_smmu_domain_set_attr,
.split_block = arm_smmu_split_block,
+ .merge_page = arm_smmu_merge_page,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 4c4eec3c0698..9028328b99b0 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -800,6 +800,83 @@ static size_t arm_lpae_split_block(struct io_pgtable_ops *ops,
return __arm_lpae_split_block(data, iova, size, lvl, ptep);
}
+static size_t __arm_lpae_merge_page(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int lvl, arm_lpae_iopte *ptep,
+ arm_lpae_iopte prot)
+{
+ arm_lpae_iopte pte, *tablep;
+ struct io_pgtable *iop = &data->iop;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return 0;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt))
+ return size;
+
+ /* Race does not exist */
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_BBML1) {
+ prot |= ARM_LPAE_PTE_NT;
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ io_pgtable_tlb_flush_walk(iop, iova, size,
+ ARM_LPAE_GRANULE(data));
+
+ prot &= ~(ARM_LPAE_PTE_NT);
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ } else {
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ }
+
+ tablep = iopte_deref(pte, data);
+ __arm_lpae_free_pgtable(data, lvl + 1, tablep);
+ return size;
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ /* The size is too small, already merged */
+ return size;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_merge_page(data, iova, paddr, size, lvl + 1, ptep, prot);
+}
+
+static size_t arm_lpae_merge_page(struct io_pgtable_ops *ops, unsigned long iova,
+ phys_addr_t paddr, size_t size, int iommu_prot)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ arm_lpae_iopte prot;
+ long iaext = (s64)iova >> cfg->ias;
+
+ /* If no access, then nothing to do */
+ if (!(iommu_prot & (IOMMU_READ | IOMMU_WRITE)))
+ return size;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return 0;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext || paddr >> cfg->oas))
+ return 0;
+
+ /* If it is smallest granule, then nothing to do */
+ if (size == ARM_LPAE_BLOCK_SIZE(ARM_LPAE_MAX_LEVELS - 1, data))
+ return size;
+
+ prot = arm_lpae_prot_to_pte(data, iommu_prot);
+ return __arm_lpae_merge_page(data, iova, paddr, size, lvl, ptep, prot);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -879,6 +956,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.unmap = arm_lpae_unmap,
.iova_to_phys = arm_lpae_iova_to_phys,
.split_block = arm_lpae_split_block,
+ .merge_page = arm_lpae_merge_page,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 47aa8f20849b..dbee6170de7f 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -169,6 +169,8 @@ struct io_pgtable_ops {
unsigned long iova);
size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
size_t size);
+ size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
};
/**
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 08/12] iommu/arm-smmu-v3: Realize merge_page iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
This reinstalls block mappings and unmap the span of page mappings.
BBML1 or BBML2 feature is required.
Merging page does not simultaneously work with other pgtable ops,
as the only designed user is vfio, which always hold a lock, so race
condition is not considered in the pgtable ops.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 28 ++++++++
drivers/iommu/io-pgtable-arm.c | 78 +++++++++++++++++++++
include/linux/io-pgtable.h | 2 +
3 files changed, 108 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index cfa83fa03c89..4d8495d88be2 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2563,6 +2563,33 @@ static int arm_smmu_split_block(struct iommu_domain *domain,
return 0;
}
+static int arm_smmu_merge_page(struct iommu_domain *domain,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ size_t handled_size;
+
+ if (!(smmu->features & (ARM_SMMU_FEAT_BBML1 | ARM_SMMU_FEAT_BBML2))) {
+ dev_err(smmu->dev, "don't support BBML1/2, can't merge page\n");
+ return -ENODEV;
+ }
+ if (!ops || !ops->merge_page) {
+ pr_err("io-pgtable don't realize merge page\n");
+ return -ENODEV;
+ }
+
+ handled_size = ops->merge_page(ops, iova, paddr, size, prot);
+ if (handled_size != size) {
+ pr_err("merge page failed\n");
+ return -EFAULT;
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2663,6 +2690,7 @@ static struct iommu_ops arm_smmu_ops = {
.domain_get_attr = arm_smmu_domain_get_attr,
.domain_set_attr = arm_smmu_domain_set_attr,
.split_block = arm_smmu_split_block,
+ .merge_page = arm_smmu_merge_page,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 4c4eec3c0698..9028328b99b0 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -800,6 +800,83 @@ static size_t arm_lpae_split_block(struct io_pgtable_ops *ops,
return __arm_lpae_split_block(data, iova, size, lvl, ptep);
}
+static size_t __arm_lpae_merge_page(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, phys_addr_t paddr,
+ size_t size, int lvl, arm_lpae_iopte *ptep,
+ arm_lpae_iopte prot)
+{
+ arm_lpae_iopte pte, *tablep;
+ struct io_pgtable *iop = &data->iop;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return 0;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return 0;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt))
+ return size;
+
+ /* Race does not exist */
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_BBML1) {
+ prot |= ARM_LPAE_PTE_NT;
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ io_pgtable_tlb_flush_walk(iop, iova, size,
+ ARM_LPAE_GRANULE(data));
+
+ prot &= ~(ARM_LPAE_PTE_NT);
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ } else {
+ __arm_lpae_init_pte(data, paddr, prot, lvl, ptep);
+ }
+
+ tablep = iopte_deref(pte, data);
+ __arm_lpae_free_pgtable(data, lvl + 1, tablep);
+ return size;
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ /* The size is too small, already merged */
+ return size;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_merge_page(data, iova, paddr, size, lvl + 1, ptep, prot);
+}
+
+static size_t arm_lpae_merge_page(struct io_pgtable_ops *ops, unsigned long iova,
+ phys_addr_t paddr, size_t size, int iommu_prot)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ arm_lpae_iopte prot;
+ long iaext = (s64)iova >> cfg->ias;
+
+ /* If no access, then nothing to do */
+ if (!(iommu_prot & (IOMMU_READ | IOMMU_WRITE)))
+ return size;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return 0;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext || paddr >> cfg->oas))
+ return 0;
+
+ /* If it is smallest granule, then nothing to do */
+ if (size == ARM_LPAE_BLOCK_SIZE(ARM_LPAE_MAX_LEVELS - 1, data))
+ return size;
+
+ prot = arm_lpae_prot_to_pte(data, iommu_prot);
+ return __arm_lpae_merge_page(data, iova, paddr, size, lvl, ptep, prot);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -879,6 +956,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.unmap = arm_lpae_unmap,
.iova_to_phys = arm_lpae_iova_to_phys,
.split_block = arm_lpae_split_block,
+ .merge_page = arm_lpae_merge_page,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 47aa8f20849b..dbee6170de7f 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -169,6 +169,8 @@ struct io_pgtable_ops {
unsigned long iova);
size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
size_t size);
+ size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
+ phys_addr_t phys, size_t size, int prot);
};
/**
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 09/12] iommu/arm-smmu-v3: Realize switch_dirty_log iommu ops
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
This realizes switch_dirty_log by invoking iommu_split_block() and
iommu_merge_page(). HTTU HD feature is required.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 38 +++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 4d8495d88be2..52c6f3e74d6f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2590,6 +2590,43 @@ static int arm_smmu_merge_page(struct iommu_domain *domain,
return 0;
}
+static int arm_smmu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (enable) {
+ /*
+ * For SMMU, the hardware dirty management is always enabled if
+ * hardware supports HTTU HD. The action to start dirty log is
+ * spliting block mapping.
+ *
+ * We don't return error even if the split operation fail, as we
+ * can still track dirty at block granule, which is still a much
+ * better choice compared to full dirty policy.
+ */
+ iommu_split_block(domain, iova, size);
+ } else {
+ /*
+ * For SMMU, the hardware dirty management is always enabled if
+ * hardware supports HTTU HD. The action to start dirty log is
+ * merging page mapping.
+ *
+ * We don't return error even if the merge operation fail, as it
+ * just effects performace of DMA transaction.
+ */
+ iommu_merge_page(domain, iova, size, prot);
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2691,6 +2728,7 @@ static struct iommu_ops arm_smmu_ops = {
.domain_set_attr = arm_smmu_domain_set_attr,
.split_block = arm_smmu_split_block,
.merge_page = arm_smmu_merge_page,
+ .switch_dirty_log = arm_smmu_switch_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 09/12] iommu/arm-smmu-v3: Realize switch_dirty_log iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
This realizes switch_dirty_log by invoking iommu_split_block() and
iommu_merge_page(). HTTU HD feature is required.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 38 +++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 4d8495d88be2..52c6f3e74d6f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2590,6 +2590,43 @@ static int arm_smmu_merge_page(struct iommu_domain *domain,
return 0;
}
+static int arm_smmu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (enable) {
+ /*
+ * For SMMU, the hardware dirty management is always enabled if
+ * hardware supports HTTU HD. The action to start dirty log is
+ * spliting block mapping.
+ *
+ * We don't return error even if the split operation fail, as we
+ * can still track dirty at block granule, which is still a much
+ * better choice compared to full dirty policy.
+ */
+ iommu_split_block(domain, iova, size);
+ } else {
+ /*
+ * For SMMU, the hardware dirty management is always enabled if
+ * hardware supports HTTU HD. The action to start dirty log is
+ * merging page mapping.
+ *
+ * We don't return error even if the merge operation fail, as it
+ * just effects performace of DMA transaction.
+ */
+ iommu_merge_page(domain, iova, size, prot);
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2691,6 +2728,7 @@ static struct iommu_ops arm_smmu_ops = {
.domain_set_attr = arm_smmu_domain_set_attr,
.split_block = arm_smmu_split_block,
.merge_page = arm_smmu_merge_page,
+ .switch_dirty_log = arm_smmu_switch_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 09/12] iommu/arm-smmu-v3: Realize switch_dirty_log iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
This realizes switch_dirty_log by invoking iommu_split_block() and
iommu_merge_page(). HTTU HD feature is required.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 38 +++++++++++++++++++++
1 file changed, 38 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 4d8495d88be2..52c6f3e74d6f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2590,6 +2590,43 @@ static int arm_smmu_merge_page(struct iommu_domain *domain,
return 0;
}
+static int arm_smmu_switch_dirty_log(struct iommu_domain *domain, bool enable,
+ unsigned long iova, size_t size, int prot)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (enable) {
+ /*
+ * For SMMU, the hardware dirty management is always enabled if
+ * hardware supports HTTU HD. The action to start dirty log is
+ * spliting block mapping.
+ *
+ * We don't return error even if the split operation fail, as we
+ * can still track dirty at block granule, which is still a much
+ * better choice compared to full dirty policy.
+ */
+ iommu_split_block(domain, iova, size);
+ } else {
+ /*
+ * For SMMU, the hardware dirty management is always enabled if
+ * hardware supports HTTU HD. The action to start dirty log is
+ * merging page mapping.
+ *
+ * We don't return error even if the merge operation fail, as it
+ * just effects performace of DMA transaction.
+ */
+ iommu_merge_page(domain, iova, size, prot);
+ }
+
+ return 0;
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2691,6 +2728,7 @@ static struct iommu_ops arm_smmu_ops = {
.domain_set_attr = arm_smmu_domain_set_attr,
.split_block = arm_smmu_split_block,
.merge_page = arm_smmu_merge_page,
+ .switch_dirty_log = arm_smmu_switch_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 10/12] iommu/arm-smmu-v3: Realize sync_dirty_log iommu ops
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
During dirty log tracking, user will try to retrieve dirty log from
iommu if it supports hardware dirty log. Scan leaf TTD and treat it
is dirty if it's writable. As we just enable HTTU for stage1, so
check whether AP[2] is not set.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 30 +++++++
drivers/iommu/io-pgtable-arm.c | 90 +++++++++++++++++++++
include/linux/io-pgtable.h | 4 +
3 files changed, 124 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 52c6f3e74d6f..9eb209a07acc 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2627,6 +2627,35 @@ static int arm_smmu_switch_dirty_log(struct iommu_domain *domain, bool enable,
return 0;
}
+static int arm_smmu_sync_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!ops || !ops->sync_dirty_log) {
+ pr_err("io-pgtable don't realize sync dirty log\n");
+ return -ENODEV;
+ }
+
+ /*
+ * Flush iotlb to ensure all inflight transactions are completed.
+ * See doc IHI0070Da 3.13.4 "HTTU behavior summary".
+ */
+ arm_smmu_flush_iotlb_all(domain);
+ return ops->sync_dirty_log(ops, iova, size, bitmap, base_iova,
+ bitmap_pgshift);
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2729,6 +2758,7 @@ static struct iommu_ops arm_smmu_ops = {
.split_block = arm_smmu_split_block,
.merge_page = arm_smmu_merge_page,
.switch_dirty_log = arm_smmu_switch_dirty_log,
+ .sync_dirty_log = arm_smmu_sync_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 9028328b99b0..67a208a05ab2 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -877,6 +877,95 @@ static size_t arm_lpae_merge_page(struct io_pgtable_ops *ops, unsigned long iova
return __arm_lpae_merge_page(data, iova, paddr, size, lvl, ptep, prot);
}
+static int __arm_lpae_sync_dirty_log(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ int lvl, arm_lpae_iopte *ptep,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ size_t base, next_size;
+ unsigned long offset;
+ int nbits, ret;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return -EINVAL;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* It is writable, set the bitmap */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ bitmap_set(bitmap, offset, nbits);
+ return 0;
+ } else {
+ /* To traverse next level */
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size) {
+ ret = __arm_lpae_sync_dirty_log(data,
+ iova + base, next_size, lvl + 1,
+ ptep, bitmap, base_iova, bitmap_pgshift);
+ if (ret)
+ return ret;
+ }
+ return 0;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* Though the size is too small, also set bitmap */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ bitmap_set(bitmap, offset, nbits);
+ return 0;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_sync_dirty_log(data, iova, size, lvl + 1, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
+static int arm_lpae_sync_dirty_log(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return -EINVAL;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return -EINVAL;
+
+ if (data->iop.fmt != ARM_64_LPAE_S1 &&
+ data->iop.fmt != ARM_32_LPAE_S1)
+ return -EINVAL;
+
+ return __arm_lpae_sync_dirty_log(data, iova, size, lvl, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -957,6 +1046,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.iova_to_phys = arm_lpae_iova_to_phys,
.split_block = arm_lpae_split_block,
.merge_page = arm_lpae_merge_page,
+ .sync_dirty_log = arm_lpae_sync_dirty_log,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index dbee6170de7f..9e2757d12066 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -171,6 +171,10 @@ struct io_pgtable_ops {
size_t size);
size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
phys_addr_t phys, size_t size, int prot);
+ int (*sync_dirty_log)(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
};
/**
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 10/12] iommu/arm-smmu-v3: Realize sync_dirty_log iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
During dirty log tracking, user will try to retrieve dirty log from
iommu if it supports hardware dirty log. Scan leaf TTD and treat it
is dirty if it's writable. As we just enable HTTU for stage1, so
check whether AP[2] is not set.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 30 +++++++
drivers/iommu/io-pgtable-arm.c | 90 +++++++++++++++++++++
include/linux/io-pgtable.h | 4 +
3 files changed, 124 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 52c6f3e74d6f..9eb209a07acc 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2627,6 +2627,35 @@ static int arm_smmu_switch_dirty_log(struct iommu_domain *domain, bool enable,
return 0;
}
+static int arm_smmu_sync_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!ops || !ops->sync_dirty_log) {
+ pr_err("io-pgtable don't realize sync dirty log\n");
+ return -ENODEV;
+ }
+
+ /*
+ * Flush iotlb to ensure all inflight transactions are completed.
+ * See doc IHI0070Da 3.13.4 "HTTU behavior summary".
+ */
+ arm_smmu_flush_iotlb_all(domain);
+ return ops->sync_dirty_log(ops, iova, size, bitmap, base_iova,
+ bitmap_pgshift);
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2729,6 +2758,7 @@ static struct iommu_ops arm_smmu_ops = {
.split_block = arm_smmu_split_block,
.merge_page = arm_smmu_merge_page,
.switch_dirty_log = arm_smmu_switch_dirty_log,
+ .sync_dirty_log = arm_smmu_sync_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 9028328b99b0..67a208a05ab2 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -877,6 +877,95 @@ static size_t arm_lpae_merge_page(struct io_pgtable_ops *ops, unsigned long iova
return __arm_lpae_merge_page(data, iova, paddr, size, lvl, ptep, prot);
}
+static int __arm_lpae_sync_dirty_log(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ int lvl, arm_lpae_iopte *ptep,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ size_t base, next_size;
+ unsigned long offset;
+ int nbits, ret;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return -EINVAL;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* It is writable, set the bitmap */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ bitmap_set(bitmap, offset, nbits);
+ return 0;
+ } else {
+ /* To traverse next level */
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size) {
+ ret = __arm_lpae_sync_dirty_log(data,
+ iova + base, next_size, lvl + 1,
+ ptep, bitmap, base_iova, bitmap_pgshift);
+ if (ret)
+ return ret;
+ }
+ return 0;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* Though the size is too small, also set bitmap */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ bitmap_set(bitmap, offset, nbits);
+ return 0;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_sync_dirty_log(data, iova, size, lvl + 1, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
+static int arm_lpae_sync_dirty_log(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return -EINVAL;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return -EINVAL;
+
+ if (data->iop.fmt != ARM_64_LPAE_S1 &&
+ data->iop.fmt != ARM_32_LPAE_S1)
+ return -EINVAL;
+
+ return __arm_lpae_sync_dirty_log(data, iova, size, lvl, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -957,6 +1046,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.iova_to_phys = arm_lpae_iova_to_phys,
.split_block = arm_lpae_split_block,
.merge_page = arm_lpae_merge_page,
+ .sync_dirty_log = arm_lpae_sync_dirty_log,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index dbee6170de7f..9e2757d12066 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -171,6 +171,10 @@ struct io_pgtable_ops {
size_t size);
size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
phys_addr_t phys, size_t size, int prot);
+ int (*sync_dirty_log)(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
};
/**
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 10/12] iommu/arm-smmu-v3: Realize sync_dirty_log iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
During dirty log tracking, user will try to retrieve dirty log from
iommu if it supports hardware dirty log. Scan leaf TTD and treat it
is dirty if it's writable. As we just enable HTTU for stage1, so
check whether AP[2] is not set.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 30 +++++++
drivers/iommu/io-pgtable-arm.c | 90 +++++++++++++++++++++
include/linux/io-pgtable.h | 4 +
3 files changed, 124 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 52c6f3e74d6f..9eb209a07acc 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2627,6 +2627,35 @@ static int arm_smmu_switch_dirty_log(struct iommu_domain *domain, bool enable,
return 0;
}
+static int arm_smmu_sync_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!ops || !ops->sync_dirty_log) {
+ pr_err("io-pgtable don't realize sync dirty log\n");
+ return -ENODEV;
+ }
+
+ /*
+ * Flush iotlb to ensure all inflight transactions are completed.
+ * See doc IHI0070Da 3.13.4 "HTTU behavior summary".
+ */
+ arm_smmu_flush_iotlb_all(domain);
+ return ops->sync_dirty_log(ops, iova, size, bitmap, base_iova,
+ bitmap_pgshift);
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2729,6 +2758,7 @@ static struct iommu_ops arm_smmu_ops = {
.split_block = arm_smmu_split_block,
.merge_page = arm_smmu_merge_page,
.switch_dirty_log = arm_smmu_switch_dirty_log,
+ .sync_dirty_log = arm_smmu_sync_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 9028328b99b0..67a208a05ab2 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -877,6 +877,95 @@ static size_t arm_lpae_merge_page(struct io_pgtable_ops *ops, unsigned long iova
return __arm_lpae_merge_page(data, iova, paddr, size, lvl, ptep, prot);
}
+static int __arm_lpae_sync_dirty_log(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ int lvl, arm_lpae_iopte *ptep,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ size_t base, next_size;
+ unsigned long offset;
+ int nbits, ret;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return -EINVAL;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* It is writable, set the bitmap */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ bitmap_set(bitmap, offset, nbits);
+ return 0;
+ } else {
+ /* To traverse next level */
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size) {
+ ret = __arm_lpae_sync_dirty_log(data,
+ iova + base, next_size, lvl + 1,
+ ptep, bitmap, base_iova, bitmap_pgshift);
+ if (ret)
+ return ret;
+ }
+ return 0;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* Though the size is too small, also set bitmap */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ bitmap_set(bitmap, offset, nbits);
+ return 0;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_sync_dirty_log(data, iova, size, lvl + 1, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
+static int arm_lpae_sync_dirty_log(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return -EINVAL;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return -EINVAL;
+
+ if (data->iop.fmt != ARM_64_LPAE_S1 &&
+ data->iop.fmt != ARM_32_LPAE_S1)
+ return -EINVAL;
+
+ return __arm_lpae_sync_dirty_log(data, iova, size, lvl, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -957,6 +1046,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.iova_to_phys = arm_lpae_iova_to_phys,
.split_block = arm_lpae_split_block,
.merge_page = arm_lpae_merge_page,
+ .sync_dirty_log = arm_lpae_sync_dirty_log,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index dbee6170de7f..9e2757d12066 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -171,6 +171,10 @@ struct io_pgtable_ops {
size_t size);
size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
phys_addr_t phys, size_t size, int prot);
+ int (*sync_dirty_log)(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
};
/**
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 11/12] iommu/arm-smmu-v3: Realize clear_dirty_log iommu ops
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
After dirty log is retrieved, user should clear dirty log to re-enable
dirty log tracking for these dirtied pages. This clears the dirty state
(As we just enable HTTU for stage1, so set the AP[2] bit) of these leaf
TTDs that are specified by the user provided bitmap.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 25 ++++++
drivers/iommu/io-pgtable-arm.c | 95 +++++++++++++++++++++
include/linux/io-pgtable.h | 4 +
3 files changed, 124 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9eb209a07acc..59bb1d198631 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2656,6 +2656,30 @@ static int arm_smmu_sync_dirty_log(struct iommu_domain *domain,
bitmap_pgshift);
}
+static int arm_smmu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!ops || !ops->clear_dirty_log) {
+ pr_err("io-pgtable don't realize clear dirty log\n");
+ return -ENODEV;
+ }
+
+ return ops->clear_dirty_log(ops, iova, size, bitmap, base_iova,
+ bitmap_pgshift);
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2759,6 +2783,7 @@ static struct iommu_ops arm_smmu_ops = {
.merge_page = arm_smmu_merge_page,
.switch_dirty_log = arm_smmu_switch_dirty_log,
.sync_dirty_log = arm_smmu_sync_dirty_log,
+ .clear_dirty_log = arm_smmu_clear_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 67a208a05ab2..e3ef0f50611c 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -966,6 +966,100 @@ static int arm_lpae_sync_dirty_log(struct io_pgtable_ops *ops,
bitmap, base_iova, bitmap_pgshift);
}
+static int __arm_lpae_clear_dirty_log(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ int lvl, arm_lpae_iopte *ptep,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ unsigned long offset;
+ size_t base, next_size;
+ int nbits, ret, i;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return -EINVAL;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* Ensure all corresponding bits are set */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ for (i = offset; i < offset + nbits; i++) {
+ if (!test_bit(i, bitmap))
+ return 0;
+ }
+
+ /* Race does not exist */
+ pte |= ARM_LPAE_PTE_AP_RDONLY;
+ __arm_lpae_set_pte(ptep, pte, &iop->cfg);
+ return 0;
+ } else {
+ /* To traverse next level */
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size) {
+ ret = __arm_lpae_clear_dirty_log(data,
+ iova + base, next_size, lvl + 1,
+ ptep, bitmap, base_iova,
+ bitmap_pgshift);
+ if (ret)
+ return ret;
+ }
+ return 0;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ /* Though the size is too small, it is already clean */
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ return -EINVAL;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_clear_dirty_log(data, iova, size, lvl + 1, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
+static int arm_lpae_clear_dirty_log(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return -EINVAL;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return -EINVAL;
+
+ if (data->iop.fmt != ARM_64_LPAE_S1 &&
+ data->iop.fmt != ARM_32_LPAE_S1)
+ return -EINVAL;
+
+ return __arm_lpae_clear_dirty_log(data, iova, size, lvl, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -1047,6 +1141,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.split_block = arm_lpae_split_block,
.merge_page = arm_lpae_merge_page,
.sync_dirty_log = arm_lpae_sync_dirty_log,
+ .clear_dirty_log = arm_lpae_clear_dirty_log,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 9e2757d12066..ad65b9099be6 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -175,6 +175,10 @@ struct io_pgtable_ops {
unsigned long iova, size_t size,
unsigned long *bitmap, unsigned long base_iova,
unsigned long bitmap_pgshift);
+ int (*clear_dirty_log)(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
};
/**
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 11/12] iommu/arm-smmu-v3: Realize clear_dirty_log iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
After dirty log is retrieved, user should clear dirty log to re-enable
dirty log tracking for these dirtied pages. This clears the dirty state
(As we just enable HTTU for stage1, so set the AP[2] bit) of these leaf
TTDs that are specified by the user provided bitmap.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 25 ++++++
drivers/iommu/io-pgtable-arm.c | 95 +++++++++++++++++++++
include/linux/io-pgtable.h | 4 +
3 files changed, 124 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9eb209a07acc..59bb1d198631 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2656,6 +2656,30 @@ static int arm_smmu_sync_dirty_log(struct iommu_domain *domain,
bitmap_pgshift);
}
+static int arm_smmu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!ops || !ops->clear_dirty_log) {
+ pr_err("io-pgtable don't realize clear dirty log\n");
+ return -ENODEV;
+ }
+
+ return ops->clear_dirty_log(ops, iova, size, bitmap, base_iova,
+ bitmap_pgshift);
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2759,6 +2783,7 @@ static struct iommu_ops arm_smmu_ops = {
.merge_page = arm_smmu_merge_page,
.switch_dirty_log = arm_smmu_switch_dirty_log,
.sync_dirty_log = arm_smmu_sync_dirty_log,
+ .clear_dirty_log = arm_smmu_clear_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 67a208a05ab2..e3ef0f50611c 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -966,6 +966,100 @@ static int arm_lpae_sync_dirty_log(struct io_pgtable_ops *ops,
bitmap, base_iova, bitmap_pgshift);
}
+static int __arm_lpae_clear_dirty_log(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ int lvl, arm_lpae_iopte *ptep,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ unsigned long offset;
+ size_t base, next_size;
+ int nbits, ret, i;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return -EINVAL;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* Ensure all corresponding bits are set */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ for (i = offset; i < offset + nbits; i++) {
+ if (!test_bit(i, bitmap))
+ return 0;
+ }
+
+ /* Race does not exist */
+ pte |= ARM_LPAE_PTE_AP_RDONLY;
+ __arm_lpae_set_pte(ptep, pte, &iop->cfg);
+ return 0;
+ } else {
+ /* To traverse next level */
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size) {
+ ret = __arm_lpae_clear_dirty_log(data,
+ iova + base, next_size, lvl + 1,
+ ptep, bitmap, base_iova,
+ bitmap_pgshift);
+ if (ret)
+ return ret;
+ }
+ return 0;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ /* Though the size is too small, it is already clean */
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ return -EINVAL;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_clear_dirty_log(data, iova, size, lvl + 1, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
+static int arm_lpae_clear_dirty_log(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return -EINVAL;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return -EINVAL;
+
+ if (data->iop.fmt != ARM_64_LPAE_S1 &&
+ data->iop.fmt != ARM_32_LPAE_S1)
+ return -EINVAL;
+
+ return __arm_lpae_clear_dirty_log(data, iova, size, lvl, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -1047,6 +1141,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.split_block = arm_lpae_split_block,
.merge_page = arm_lpae_merge_page,
.sync_dirty_log = arm_lpae_sync_dirty_log,
+ .clear_dirty_log = arm_lpae_clear_dirty_log,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 9e2757d12066..ad65b9099be6 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -175,6 +175,10 @@ struct io_pgtable_ops {
unsigned long iova, size_t size,
unsigned long *bitmap, unsigned long base_iova,
unsigned long bitmap_pgshift);
+ int (*clear_dirty_log)(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
};
/**
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 11/12] iommu/arm-smmu-v3: Realize clear_dirty_log iommu ops
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
After dirty log is retrieved, user should clear dirty log to re-enable
dirty log tracking for these dirtied pages. This clears the dirty state
(As we just enable HTTU for stage1, so set the AP[2] bit) of these leaf
TTDs that are specified by the user provided bitmap.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 25 ++++++
drivers/iommu/io-pgtable-arm.c | 95 +++++++++++++++++++++
include/linux/io-pgtable.h | 4 +
3 files changed, 124 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 9eb209a07acc..59bb1d198631 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2656,6 +2656,30 @@ static int arm_smmu_sync_dirty_log(struct iommu_domain *domain,
bitmap_pgshift);
}
+static int arm_smmu_clear_dirty_log(struct iommu_domain *domain,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
+ struct io_pgtable_ops *ops = smmu_domain->pgtbl_ops;
+ struct arm_smmu_device *smmu = smmu_domain->smmu;
+
+ if (!(smmu->features & ARM_SMMU_FEAT_HD))
+ return -ENODEV;
+ if (smmu_domain->stage != ARM_SMMU_DOMAIN_S1)
+ return -EINVAL;
+
+ if (!ops || !ops->clear_dirty_log) {
+ pr_err("io-pgtable don't realize clear dirty log\n");
+ return -ENODEV;
+ }
+
+ return ops->clear_dirty_log(ops, iova, size, bitmap, base_iova,
+ bitmap_pgshift);
+}
+
static int arm_smmu_of_xlate(struct device *dev, struct of_phandle_args *args)
{
return iommu_fwspec_add_ids(dev, args->args, 1);
@@ -2759,6 +2783,7 @@ static struct iommu_ops arm_smmu_ops = {
.merge_page = arm_smmu_merge_page,
.switch_dirty_log = arm_smmu_switch_dirty_log,
.sync_dirty_log = arm_smmu_sync_dirty_log,
+ .clear_dirty_log = arm_smmu_clear_dirty_log,
.of_xlate = arm_smmu_of_xlate,
.get_resv_regions = arm_smmu_get_resv_regions,
.put_resv_regions = generic_iommu_put_resv_regions,
diff --git a/drivers/iommu/io-pgtable-arm.c b/drivers/iommu/io-pgtable-arm.c
index 67a208a05ab2..e3ef0f50611c 100644
--- a/drivers/iommu/io-pgtable-arm.c
+++ b/drivers/iommu/io-pgtable-arm.c
@@ -966,6 +966,100 @@ static int arm_lpae_sync_dirty_log(struct io_pgtable_ops *ops,
bitmap, base_iova, bitmap_pgshift);
}
+static int __arm_lpae_clear_dirty_log(struct arm_lpae_io_pgtable *data,
+ unsigned long iova, size_t size,
+ int lvl, arm_lpae_iopte *ptep,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ arm_lpae_iopte pte;
+ struct io_pgtable *iop = &data->iop;
+ unsigned long offset;
+ size_t base, next_size;
+ int nbits, ret, i;
+
+ if (WARN_ON(lvl == ARM_LPAE_MAX_LEVELS))
+ return -EINVAL;
+
+ ptep += ARM_LPAE_LVL_IDX(iova, lvl, data);
+ pte = READ_ONCE(*ptep);
+ if (WARN_ON(!pte))
+ return -EINVAL;
+
+ if (size == ARM_LPAE_BLOCK_SIZE(lvl, data)) {
+ if (iopte_leaf(pte, lvl, iop->fmt)) {
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ /* Ensure all corresponding bits are set */
+ nbits = size >> bitmap_pgshift;
+ offset = (iova - base_iova) >> bitmap_pgshift;
+ for (i = offset; i < offset + nbits; i++) {
+ if (!test_bit(i, bitmap))
+ return 0;
+ }
+
+ /* Race does not exist */
+ pte |= ARM_LPAE_PTE_AP_RDONLY;
+ __arm_lpae_set_pte(ptep, pte, &iop->cfg);
+ return 0;
+ } else {
+ /* To traverse next level */
+ next_size = ARM_LPAE_BLOCK_SIZE(lvl + 1, data);
+ ptep = iopte_deref(pte, data);
+ for (base = 0; base < size; base += next_size) {
+ ret = __arm_lpae_clear_dirty_log(data,
+ iova + base, next_size, lvl + 1,
+ ptep, bitmap, base_iova,
+ bitmap_pgshift);
+ if (ret)
+ return ret;
+ }
+ return 0;
+ }
+ } else if (iopte_leaf(pte, lvl, iop->fmt)) {
+ /* Though the size is too small, it is already clean */
+ if (pte & ARM_LPAE_PTE_AP_RDONLY)
+ return 0;
+
+ return -EINVAL;
+ }
+
+ /* Keep on walkin */
+ ptep = iopte_deref(pte, data);
+ return __arm_lpae_clear_dirty_log(data, iova, size, lvl + 1, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
+static int arm_lpae_clear_dirty_log(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap,
+ unsigned long base_iova,
+ unsigned long bitmap_pgshift)
+{
+ struct arm_lpae_io_pgtable *data = io_pgtable_ops_to_data(ops);
+ arm_lpae_iopte *ptep = data->pgd;
+ int lvl = data->start_level;
+ struct io_pgtable_cfg *cfg = &data->iop.cfg;
+ long iaext = (s64)iova >> cfg->ias;
+
+ if (WARN_ON(!size || (size & cfg->pgsize_bitmap) != size))
+ return -EINVAL;
+
+ if (cfg->quirks & IO_PGTABLE_QUIRK_ARM_TTBR1)
+ iaext = ~iaext;
+ if (WARN_ON(iaext))
+ return -EINVAL;
+
+ if (data->iop.fmt != ARM_64_LPAE_S1 &&
+ data->iop.fmt != ARM_32_LPAE_S1)
+ return -EINVAL;
+
+ return __arm_lpae_clear_dirty_log(data, iova, size, lvl, ptep,
+ bitmap, base_iova, bitmap_pgshift);
+}
+
static void arm_lpae_restrict_pgsizes(struct io_pgtable_cfg *cfg)
{
unsigned long granule, page_sizes;
@@ -1047,6 +1141,7 @@ arm_lpae_alloc_pgtable(struct io_pgtable_cfg *cfg)
.split_block = arm_lpae_split_block,
.merge_page = arm_lpae_merge_page,
.sync_dirty_log = arm_lpae_sync_dirty_log,
+ .clear_dirty_log = arm_lpae_clear_dirty_log,
};
return data;
diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
index 9e2757d12066..ad65b9099be6 100644
--- a/include/linux/io-pgtable.h
+++ b/include/linux/io-pgtable.h
@@ -175,6 +175,10 @@ struct io_pgtable_ops {
unsigned long iova, size_t size,
unsigned long *bitmap, unsigned long base_iova,
unsigned long bitmap_pgshift);
+ int (*clear_dirty_log)(struct io_pgtable_ops *ops,
+ unsigned long iova, size_t size,
+ unsigned long *bitmap, unsigned long base_iova,
+ unsigned long bitmap_pgshift);
};
/**
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread
* [PATCH v3 12/12] iommu/arm-smmu-v3: Add HWDBM device feature reporting
2021-04-13 8:54 ` Keqian Zhu
(?)
@ 2021-04-13 8:54 ` Keqian Zhu
-1 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
We have implemented these interfaces required to support iommu
dirty log tracking. The last step is reporting this feature to
upper user, then the user can perform higher policy base on it.
There is a new dev feature named IOMMU_DEV_FEAT_HWDBM in iommu
layer. For arm smmuv3, it is equal to ARM_SMMU_FEAT_HD and it is
enabled by default if supported. Other types of IOMMU can enable
it by default or when user invokes enable_feature.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 59bb1d198631..2d716ee5621f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2712,6 +2712,9 @@ static bool arm_smmu_dev_has_feature(struct device *dev,
switch (feat) {
case IOMMU_DEV_FEAT_SVA:
return arm_smmu_master_sva_supported(master);
+ case IOMMU_DEV_FEAT_HWDBM:
+ /* No requirement for device, require HTTU HD of smmu */
+ return !!(master->smmu->features & ARM_SMMU_FEAT_HD);
default:
return false;
}
@@ -2728,6 +2731,9 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev,
switch (feat) {
case IOMMU_DEV_FEAT_SVA:
return arm_smmu_master_sva_enabled(master);
+ case IOMMU_DEV_FEAT_HWDBM:
+ /* HTTU HD is enabled if supported */
+ return arm_smmu_dev_has_feature(dev, feat);
default:
return false;
}
--
2.19.1
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 12/12] iommu/arm-smmu-v3: Add HWDBM device feature reporting
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: Alex Williamson, Cornelia Huck, Kirti Wankhede, wanghaibin.wang,
jiangkunkun, yuzenghui, lushenming
From: Kunkun Jiang <jiangkunkun@huawei.com>
We have implemented these interfaces required to support iommu
dirty log tracking. The last step is reporting this feature to
upper user, then the user can perform higher policy base on it.
There is a new dev feature named IOMMU_DEV_FEAT_HWDBM in iommu
layer. For arm smmuv3, it is equal to ARM_SMMU_FEAT_HD and it is
enabled by default if supported. Other types of IOMMU can enable
it by default or when user invokes enable_feature.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 59bb1d198631..2d716ee5621f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2712,6 +2712,9 @@ static bool arm_smmu_dev_has_feature(struct device *dev,
switch (feat) {
case IOMMU_DEV_FEAT_SVA:
return arm_smmu_master_sva_supported(master);
+ case IOMMU_DEV_FEAT_HWDBM:
+ /* No requirement for device, require HTTU HD of smmu */
+ return !!(master->smmu->features & ARM_SMMU_FEAT_HD);
default:
return false;
}
@@ -2728,6 +2731,9 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev,
switch (feat) {
case IOMMU_DEV_FEAT_SVA:
return arm_smmu_master_sva_enabled(master);
+ case IOMMU_DEV_FEAT_HWDBM:
+ /* HTTU HD is enabled if supported */
+ return arm_smmu_dev_has_feature(dev, feat);
default:
return false;
}
--
2.19.1
_______________________________________________
linux-arm-kernel mailing list
linux-arm-kernel@lists.infradead.org
http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
^ permalink raw reply related [flat|nested] 81+ messages in thread* [PATCH v3 12/12] iommu/arm-smmu-v3: Add HWDBM device feature reporting
@ 2021-04-13 8:54 ` Keqian Zhu
0 siblings, 0 replies; 81+ messages in thread
From: Keqian Zhu @ 2021-04-13 8:54 UTC (permalink / raw)
To: linux-kernel, linux-arm-kernel, iommu, Robin Murphy, Will Deacon,
Joerg Roedel, Yi Sun, Jean-Philippe Brucker, Jonathan Cameron,
Tian Kevin, Lu Baolu
Cc: jiangkunkun, Cornelia Huck, Kirti Wankhede, lushenming,
Alex Williamson, wanghaibin.wang
From: Kunkun Jiang <jiangkunkun@huawei.com>
We have implemented these interfaces required to support iommu
dirty log tracking. The last step is reporting this feature to
upper user, then the user can perform higher policy base on it.
There is a new dev feature named IOMMU_DEV_FEAT_HWDBM in iommu
layer. For arm smmuv3, it is equal to ARM_SMMU_FEAT_HD and it is
enabled by default if supported. Other types of IOMMU can enable
it by default or when user invokes enable_feature.
Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
---
drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 6 ++++++
1 file changed, 6 insertions(+)
diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
index 59bb1d198631..2d716ee5621f 100644
--- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
+++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
@@ -2712,6 +2712,9 @@ static bool arm_smmu_dev_has_feature(struct device *dev,
switch (feat) {
case IOMMU_DEV_FEAT_SVA:
return arm_smmu_master_sva_supported(master);
+ case IOMMU_DEV_FEAT_HWDBM:
+ /* No requirement for device, require HTTU HD of smmu */
+ return !!(master->smmu->features & ARM_SMMU_FEAT_HD);
default:
return false;
}
@@ -2728,6 +2731,9 @@ static bool arm_smmu_dev_feature_enabled(struct device *dev,
switch (feat) {
case IOMMU_DEV_FEAT_SVA:
return arm_smmu_master_sva_enabled(master);
+ case IOMMU_DEV_FEAT_HWDBM:
+ /* HTTU HD is enabled if supported */
+ return arm_smmu_dev_has_feature(dev, feat);
default:
return false;
}
--
2.19.1
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
^ permalink raw reply related [flat|nested] 81+ messages in thread