From: Keqian Zhu <zhukeqian1@huawei.com>
To: Robin Murphy <robin.murphy@arm.com>,
<linux-kernel@vger.kernel.org>,
<linux-arm-kernel@lists.infradead.org>, <kvm@vger.kernel.org>,
<kvmarm@lists.cs.columbia.edu>,
<iommu@lists.linux-foundation.org>,
"Will Deacon" <will@kernel.org>,
Alex Williamson <alex.williamson@redhat.com>,
"Marc Zyngier" <maz@kernel.org>,
Catalin Marinas <catalin.marinas@arm.com>
Cc: Kirti Wankhede <kwankhede@nvidia.com>,
Cornelia Huck <cohuck@redhat.com>,
Mark Rutland <mark.rutland@arm.com>,
James Morse <james.morse@arm.com>,
Suzuki K Poulose <suzuki.poulose@arm.com>,
<wanghaibin.wang@huawei.com>, <jiangkunkun@huawei.com>,
<yuzenghui@huawei.com>, <lushenming@huawei.com>
Subject: Re: [RFC PATCH 05/11] iommu/arm-smmu-v3: Merge a span of page to block descriptor
Date: Sun, 7 Feb 2021 20:13:18 +0800 [thread overview]
Message-ID: <682f6b24-ecfb-d462-0d1f-5c3ba53136c0@huawei.com> (raw)
In-Reply-To: <af03aa13-9fd1-d7d1-6e55-fd59ff9d0688@arm.com>
Hi Robin,
On 2021/2/5 3:52, Robin Murphy wrote:
> On 2021-01-28 15:17, Keqian Zhu wrote:
>> From: jiangkunkun <jiangkunkun@huawei.com>
>>
>> When stop dirty log tracking, we need to recover all block descriptors
>> which are splited when start dirty log tracking. This adds a new
>> interface named merge_page in iommu layer and arm smmuv3 implements it,
>> which reinstall block mappings and unmap the span of page mappings.
>>
>> It's caller's duty to find contiuous physical memory.
>>
>> During merging page, other interfaces are not expected to be working,
>> so race condition does not exist. And we flush all iotlbs after the merge
>> procedure is completed to ease the pressure of iommu, as we will merge a
>> huge range of page mappings in general.
>
> Again, I think we need better reasoning than "race conditions don't exist because we don't expect them to exist".
Sure, because they can't. ;-)
>
>> Co-developed-by: Keqian Zhu <zhukeqian1@huawei.com>
>> Signed-off-by: Kunkun Jiang <jiangkunkun@huawei.com>
>> ---
>> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 20 ++++++
>> drivers/iommu/io-pgtable-arm.c | 78 +++++++++++++++++++++
>> drivers/iommu/iommu.c | 75 ++++++++++++++++++++
>> include/linux/io-pgtable.h | 2 +
>> include/linux/iommu.h | 10 +++
>> 5 files changed, 185 insertions(+)
>>
>> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> index 5469f4fca820..2434519e4bb6 100644
>> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
>> @@ -2529,6 +2529,25 @@ static size_t arm_smmu_split_block(struct iommu_domain *domain,
>> return ops->split_block(ops, iova, size);
>> }
[...]
>> +
>> +size_t iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
>> + size_t size, int prot)
>> +{
>> + phys_addr_t phys;
>> + dma_addr_t p, i;
>> + size_t cont_size, merged_size;
>> + size_t merged = 0;
>> +
>> + while (size) {
>> + phys = iommu_iova_to_phys(domain, iova);
>> + cont_size = PAGE_SIZE;
>> + p = phys + cont_size;
>> + i = iova + cont_size;
>> +
>> + while (cont_size < size && p == iommu_iova_to_phys(domain, i)) {
>> + p += PAGE_SIZE;
>> + i += PAGE_SIZE;
>> + cont_size += PAGE_SIZE;
>> + }
>> +
>> + merged_size = __iommu_merge_page(domain, iova, phys, cont_size,
>> + prot);
>
> This is incredibly silly. The amount of time you'll spend just on walking the tables in all those iova_to_phys() calls is probably significantly more than it would take the low-level pagetable code to do the entire operation for itself. At this level, any knowledge of how mappings are actually constructed is lost once __iommu_map() returns, so we just don't know, and for this operation in particular there seems little point in trying to guess - the driver backend still has to figure out whether something we *think* might me mergeable actually is, so it's better off doing the entire operation in a single pass by itself.
>
> There's especially little point in starting all this work *before* checking that it's even possible...
>
> Robin.
Well, this looks silly indeed. But the iova->phys info is only stored in pgtable. It seems that there is no other method to find continuous physical address :-( (actually, the vfio_iommu_replay() has similar logic).
We put the finding procedure of continuous physical address in common iommu layer, because this is a common logic for all types of iommu driver.
If a vendor iommu driver thinks (iova, phys, cont_size) is not merge-able, it can make its own decision to map them. This keeps same as iommu_map(), which provides (iova, paddr, pgsize) to vendor driver, and vendor driver can make its own decision to map them.
Do I understand your idea correctly?
Thanks,
Keqian
>
>> + iova += merged_size;
>> + size -= merged_size;
>> + merged += merged_size;
>> +
>> + if (merged_size != cont_size)
>> + break;
>> + }
>> + iommu_flush_iotlb_all(domain);
>> +
>> + return merged;
>> +}
>> +EXPORT_SYMBOL_GPL(iommu_merge_page);
>> +
>> void iommu_get_resv_regions(struct device *dev, struct list_head *list)
>> {
>> const struct iommu_ops *ops = dev->bus->iommu_ops;
>> diff --git a/include/linux/io-pgtable.h b/include/linux/io-pgtable.h
>> index b87c6f4ecaa2..754b62a1bbaf 100644
>> --- a/include/linux/io-pgtable.h
>> +++ b/include/linux/io-pgtable.h
>> @@ -164,6 +164,8 @@ struct io_pgtable_ops {
>> unsigned long iova);
>> size_t (*split_block)(struct io_pgtable_ops *ops, unsigned long iova,
>> size_t size);
>> + size_t (*merge_page)(struct io_pgtable_ops *ops, unsigned long iova,
>> + phys_addr_t phys, size_t size, int prot);
>> };
>> /**
>> diff --git a/include/linux/iommu.h b/include/linux/iommu.h
>> index abeb811098a5..ac2b0b1bce0f 100644
>> --- a/include/linux/iommu.h
>> +++ b/include/linux/iommu.h
>> @@ -260,6 +260,8 @@ struct iommu_ops {
>> enum iommu_attr attr, void *data);
>> size_t (*split_block)(struct iommu_domain *domain, unsigned long iova,
>> size_t size);
>> + size_t (*merge_page)(struct iommu_domain *domain, unsigned long iova,
>> + phys_addr_t phys, size_t size, int prot);
>> /* Request/Free a list of reserved regions for a device */
>> void (*get_resv_regions)(struct device *dev, struct list_head *list);
>> @@ -513,6 +515,8 @@ extern int iommu_domain_set_attr(struct iommu_domain *domain, enum iommu_attr,
>> void *data);
>> extern size_t iommu_split_block(struct iommu_domain *domain, unsigned long iova,
>> size_t size);
>> +extern size_t iommu_merge_page(struct iommu_domain *domain, unsigned long iova,
>> + size_t size, int prot);
>> /* Window handling function prototypes */
>> extern int iommu_domain_window_enable(struct iommu_domain *domain, u32 wnd_nr,
>> @@ -913,6 +917,12 @@ static inline size_t iommu_split_block(struct iommu_domain *domain,
>> return 0;
>> }
>> +static inline size_t iommu_merge_page(struct iommu_domain *domain,
>> + unsigned long iova, size_t size, int prot)
>> +{
>> + return -EINVAL;
>> +}
>> +
>> static inline int iommu_device_register(struct iommu_device *iommu)
>> {
>> return -ENODEV;
>>
> .
>
next prev parent reply other threads:[~2021-02-07 12:14 UTC|newest]
Thread overview: 35+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-01-28 15:17 [RFC PATCH 00/11] vfio/iommu_type1: Implement dirty log tracking based on smmuv3 HTTU Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 01/11] iommu/arm-smmu-v3: Add feature detection for HTTU Keqian Zhu
2021-02-04 19:50 ` Robin Murphy
2021-02-05 9:13 ` Keqian Zhu
2021-02-05 9:51 ` Jean-Philippe Brucker
2021-02-07 1:42 ` Keqian Zhu
2021-02-05 11:48 ` Robin Murphy
2021-02-05 16:11 ` Robin Murphy
2021-02-07 1:56 ` Keqian Zhu
2021-02-07 2:19 ` Keqian Zhu
2021-03-02 7:42 ` Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 02/11] iommu/arm-smmu-v3: Enable HTTU for SMMU stage1 mapping Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 03/11] iommu/arm-smmu-v3: Add feature detection for BBML Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 04/11] iommu/arm-smmu-v3: Split block descriptor to a span of page Keqian Zhu
2021-02-04 19:51 ` Robin Murphy
2021-02-07 8:18 ` Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 05/11] iommu/arm-smmu-v3: Merge a span of page to block descriptor Keqian Zhu
2021-02-04 19:52 ` Robin Murphy
2021-02-07 12:13 ` Keqian Zhu [this message]
2021-01-28 15:17 ` [RFC PATCH 06/11] iommu/arm-smmu-v3: Scan leaf TTD to sync hardware dirty log Keqian Zhu
2021-02-04 19:52 ` Robin Murphy
2021-02-07 12:41 ` Keqian Zhu
2021-02-08 1:17 ` Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 07/11] iommu/arm-smmu-v3: Clear dirty log according to bitmap Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 08/11] iommu/arm-smmu-v3: Add HWDBM device feature reporting Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 09/11] vfio/iommu_type1: Add HWDBM status maintanance Keqian Zhu
2021-01-28 15:17 ` [RFC PATCH 10/11] vfio/iommu_type1: Optimize dirty bitmap population based on iommu HWDBM Keqian Zhu
2021-02-07 9:56 ` Yi Sun
2021-02-07 10:40 ` Keqian Zhu
2021-02-09 11:57 ` Yi Sun
2021-02-09 12:08 ` Robin Murphy
2021-02-18 1:17 ` Keqian Zhu
2021-02-09 11:16 ` Robin Murphy
2021-02-09 12:02 ` Yi Sun
2021-01-28 15:17 ` [RFC PATCH 11/11] vfio/iommu_type1: Add support for manual dirty log clear Keqian Zhu
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=682f6b24-ecfb-d462-0d1f-5c3ba53136c0@huawei.com \
--to=zhukeqian1@huawei.com \
--cc=alex.williamson@redhat.com \
--cc=catalin.marinas@arm.com \
--cc=cohuck@redhat.com \
--cc=iommu@lists.linux-foundation.org \
--cc=james.morse@arm.com \
--cc=jiangkunkun@huawei.com \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=kwankhede@nvidia.com \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=linux-kernel@vger.kernel.org \
--cc=lushenming@huawei.com \
--cc=mark.rutland@arm.com \
--cc=maz@kernel.org \
--cc=robin.murphy@arm.com \
--cc=suzuki.poulose@arm.com \
--cc=wanghaibin.wang@huawei.com \
--cc=will@kernel.org \
--cc=yuzenghui@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).