From: "chenxiang (M)" <chenxiang66@hisilicon.com>
To: <iommu@lists.linux-foundation.org>
Cc: linuxarm@openeuler.org
Subject: Re: [PATCH v13 08/15] iommu/smmuv3: Implement cache_invalidate
Date: Sat, 16 Jan 2021 10:24:00 +0800 [thread overview]
Message-ID: <c405d689-aea5-c9eb-d418-ec5a30a97fa6@hisilicon.com> (raw)
In-Reply-To: <20201118112151.25412-9-eric.auger@redhat.com>
Hi Eric,
在 2020/11/18 19:21, Eric Auger 写道:
> Implement domain-selective and page-selective IOTLB invalidations.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
> v7 -> v8:
> - ASID based invalidation using iommu_inv_pasid_info
> - check ARCHID/PASID flags in addr based invalidation
> - use __arm_smmu_tlb_inv_context and __arm_smmu_tlb_inv_range_nosync
>
> v6 -> v7
> - check the uapi version
>
> v3 -> v4:
> - adapt to changes in the uapi
> - add support for leaf parameter
> - do not use arm_smmu_tlb_inv_range_nosync or arm_smmu_tlb_inv_context
> anymore
>
> v2 -> v3:
> - replace __arm_smmu_tlb_sync by arm_smmu_cmdq_issue_sync
>
> v1 -> v2:
> - properly pass the asid
> ---
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 53 +++++++++++++++++++++
> 1 file changed, 53 insertions(+)
>
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index fdecc9f17b36..24124361dd3b 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -2771,6 +2771,58 @@ static void arm_smmu_detach_pasid_table(struct iommu_domain *domain)
> mutex_unlock(&smmu_domain->init_mutex);
> }
>
> +static int
> +arm_smmu_cache_invalidate(struct iommu_domain *domain, struct device *dev,
> + struct iommu_cache_invalidate_info *inv_info)
> +{
> + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> + struct arm_smmu_device *smmu = smmu_domain->smmu;
> +
> + if (smmu_domain->stage != ARM_SMMU_DOMAIN_NESTED)
> + return -EINVAL;
> +
> + if (!smmu)
> + return -EINVAL;
> +
> + if (inv_info->version != IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)
> + return -EINVAL;
> +
> + if (inv_info->cache & IOMMU_CACHE_INV_TYPE_IOTLB) {
> + if (inv_info->granularity == IOMMU_INV_GRANU_PASID) {
> + struct iommu_inv_pasid_info *info =
> + &inv_info->granu.pasid_info;
> +
> + if (!(info->flags & IOMMU_INV_PASID_FLAGS_ARCHID) ||
> + (info->flags & IOMMU_INV_PASID_FLAGS_PASID))
> + return -EINVAL;
> +
> + __arm_smmu_tlb_inv_context(smmu_domain, info->archid);
> +
> + } else if (inv_info->granularity == IOMMU_INV_GRANU_ADDR) {
> + struct iommu_inv_addr_info *info = &inv_info->granu.addr_info;
> + size_t size = info->nb_granules * info->granule_size;
> + bool leaf = info->flags & IOMMU_INV_ADDR_FLAGS_LEAF;
> +
> + if (!(info->flags & IOMMU_INV_ADDR_FLAGS_ARCHID) ||
> + (info->flags & IOMMU_INV_ADDR_FLAGS_PASID))
> + return -EINVAL;
> +
> + __arm_smmu_tlb_inv_range(info->addr, size,
> + info->granule_size, leaf,
> + smmu_domain, info->archid);
When debugging with vSMMU on ARM64 huawei platform, there is a issue:
RIL feature is enabled on guest OS while it is not supported on host
OS, with some operations
(such as rmmod driver during iperf between guest and host), SMMU
translation error occurs frequently, and it works well if RIL feature is
disabled on guest OS.
We find that in function vfio_iommu_unmap_notify() (qemu code) it passes
total size of tlb (num_pages * granule_size) as granule size to host OS :
addr_info->granules_size = size
addr_info->nb_granule = 1
So total size (num_pages * granule_size) is passed as granule size to
function __arm_smmu_inv_range(), if RIL feature is not supported, then
total size may be not
the granule size that host OS can recongize, i will only invalidate part
of tlb it wants to invalidate (for example, 4K/2M/1G pagesize is
supported on host OS, if total_size = 8K, then addr_info->granule_size
= 8K,
addr_info->nb_granule = 1, as RIL feature is not supported, it
invalidates just 4K one time but it thought it had invalidated 8K).
Please have a check of the issue, and we have a temporary fix as follows
on the issue.
hw/arm/smmuv3.c | 3 ++-
hw/vfio/common.c | 6 +++---
include/exec/memory.h | 1 +
3 files changed, 6 insertions(+), 4 deletions(-)
diff --git a/hw/arm/smmuv3.c b/hw/arm/smmuv3.c
index 6725019..891a65d 100644
--- a/hw/arm/smmuv3.c
+++ b/hw/arm/smmuv3.c
@@ -821,7 +821,8 @@ static void smmuv3_notify_iova(IOMMUMemoryRegion *mr,
entry.target_as = &address_space_memory;
entry.iova = iova;
- entry.addr_mask = num_pages * (1 << granule) - 1;
+ entry.addr_mask = (1 << granule) - 1;
+ entry.num_pages = num_pages;
entry.perm = IOMMU_NONE;
entry.arch_id = asid;
entry.flags = IOMMU_INV_GRANU_ADDR;
diff --git a/hw/vfio/common.c b/hw/vfio/common.c
index 5d365e0..c0164b1 100644
--- a/hw/vfio/common.c
+++ b/hw/vfio/common.c
@@ -604,7 +604,7 @@ static void vfio_iommu_unmap_notify(IOMMUNotifier
*n, IOMMUTLBEntry *iotlb)
hwaddr start = iotlb->iova + giommu->iommu_offset;
VFIOContainer *container = giommu->container;
struct vfio_iommu_type1_cache_invalidate ustruct = {};
- size_t size = iotlb->addr_mask + 1;
+ size_t size = iotlb->num_pages * (iotlb->addr_mask + 1);
int ret;
assert(iotlb->perm == IOMMU_NONE);
@@ -632,8 +632,8 @@ static void vfio_iommu_unmap_notify(IOMMUNotifier
*n, IOMMUTLBEntry *iotlb)
}
addr_info->archid = iotlb->arch_id;
addr_info->addr = start;
- addr_info->granule_size = size;
- addr_info->nb_granules = 1;
+ addr_info->granule_size = iotlb->addr_mask + 1;
+ addr_info->nb_granules = iotlb->num_pages;
}
trace_vfio_iommu_addr_inv_iotlb(iotlb->arch_id, start, size, 1,
iotlb->leaf);
diff --git a/include/exec/memory.h b/include/exec/memory.h
index 21959a7..aa8d43e 100644
--- a/include/exec/memory.h
+++ b/include/exec/memory.h
@@ -103,6 +103,7 @@ struct IOMMUTLBEntry {
hwaddr iova;
hwaddr translated_addr;
hwaddr addr_mask;
+ uint64_t num_pages;
IOMMUAccessFlags perm;
uint32_t arch_id;
uint32_t flags;
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-01-16 2:24 UTC|newest]
Thread overview: 175+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-11-18 11:21 [PATCH v13 00/15] SMMUv3 Nested Stage Setup (IOMMU part) Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 01/15] iommu: Introduce attach/detach_pasid_table API Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 16:19 ` Jacob Pan
2020-11-18 16:19 ` Jacob Pan
2020-11-18 16:19 ` Jacob Pan
2020-11-19 17:02 ` Auger Eric
2020-11-19 17:02 ` Auger Eric
2020-11-19 17:02 ` Auger Eric
2021-02-01 11:27 ` Keqian Zhu
2021-02-01 11:27 ` Keqian Zhu
2021-02-01 11:27 ` Keqian Zhu
2021-02-01 17:18 ` Auger Eric
2021-02-01 17:18 ` Auger Eric
2021-02-01 17:18 ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 02/15] iommu: Introduce bind/unbind_guest_msi Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2021-02-01 11:52 ` Keqian Zhu
2021-02-01 11:52 ` Keqian Zhu
2021-02-01 11:52 ` Keqian Zhu
2021-02-12 8:55 ` Auger Eric
2021-02-12 8:55 ` Auger Eric
2021-02-12 8:55 ` Auger Eric
2021-02-18 8:43 ` Keqian Zhu
2021-02-18 8:43 ` Keqian Zhu
2021-02-18 8:43 ` Keqian Zhu
2021-02-18 10:35 ` Auger Eric
2021-02-18 10:35 ` Auger Eric
2021-02-18 10:35 ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 03/15] iommu/arm-smmu-v3: Maintain a SID->device structure Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2021-02-01 12:26 ` Keqian Zhu
2021-02-01 12:26 ` Keqian Zhu
2021-02-01 12:26 ` Keqian Zhu
2021-02-01 15:15 ` Jean-Philippe Brucker
2021-02-01 15:15 ` Jean-Philippe Brucker
2021-02-01 15:15 ` Jean-Philippe Brucker
2021-02-02 6:39 ` Keqian Zhu
2021-02-02 6:39 ` Keqian Zhu
2021-02-02 6:39 ` Keqian Zhu
2021-02-01 17:19 ` Auger Eric
2021-02-01 17:19 ` Auger Eric
2021-02-01 17:19 ` Auger Eric
2021-02-02 7:20 ` Keqian Zhu
2021-02-02 7:20 ` Keqian Zhu
2021-02-02 7:20 ` Keqian Zhu
2020-11-18 11:21 ` [PATCH v13 04/15] iommu/smmuv3: Allow s1 and s2 configs to coexist Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2021-02-01 12:35 ` Keqian Zhu
2021-02-01 12:35 ` Keqian Zhu
2021-02-01 12:35 ` Keqian Zhu
2020-11-18 11:21 ` [PATCH v13 05/15] iommu/smmuv3: Get prepared for nested stage support Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-19 3:59 ` kernel test robot
2020-11-19 3:59 ` kernel test robot
2020-11-19 3:59 ` kernel test robot
2020-11-19 3:59 ` kernel test robot
2020-12-03 12:32 ` Kunkun Jiang
2020-12-03 12:32 ` Kunkun Jiang
2020-12-03 13:01 ` Auger Eric
2020-12-03 13:01 ` Auger Eric
2020-12-03 13:01 ` Auger Eric
2020-12-03 13:23 ` Kunkun Jiang
2020-12-03 13:23 ` Kunkun Jiang
2020-12-03 13:23 ` Kunkun Jiang
2020-12-09 14:26 ` Shameerali Kolothum Thodi
2020-12-09 14:26 ` Shameerali Kolothum Thodi
2020-12-09 14:26 ` Shameerali Kolothum Thodi
2021-02-02 7:14 ` Keqian Zhu
2021-02-02 7:14 ` Keqian Zhu
2021-02-02 7:14 ` Keqian Zhu
2021-02-11 17:36 ` Auger Eric
2021-02-11 17:36 ` Auger Eric
2021-02-11 17:36 ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 06/15] iommu/smmuv3: Implement attach/detach_pasid_table Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2021-02-02 8:03 ` Keqian Zhu
2021-02-02 8:03 ` Keqian Zhu
2021-02-02 8:03 ` Keqian Zhu
2021-02-11 17:35 ` Auger Eric
2021-02-11 17:35 ` Auger Eric
2021-02-11 17:35 ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 07/15] iommu/smmuv3: Allow stage 1 invalidation with unmanaged ASIDs Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-12-01 13:33 ` Xingang Wang
2020-12-01 13:33 ` Xingang Wang
2020-12-01 13:33 ` Xingang Wang
2020-12-01 13:58 ` Auger Eric
2020-12-01 13:58 ` Auger Eric
2020-12-01 13:58 ` Auger Eric
2020-12-02 12:59 ` Wang Xingang
2020-12-02 12:59 ` Wang Xingang
2020-12-02 12:59 ` Wang Xingang
2020-12-03 18:42 ` Shameerali Kolothum Thodi
2020-12-03 18:42 ` Shameerali Kolothum Thodi
2020-12-03 18:42 ` Shameerali Kolothum Thodi
2020-12-04 9:53 ` Jean-Philippe Brucker
2020-12-04 9:53 ` Jean-Philippe Brucker
2020-12-04 9:53 ` Jean-Philippe Brucker
2020-12-04 10:20 ` Shameerali Kolothum Thodi
2020-12-04 10:20 ` Shameerali Kolothum Thodi
2020-12-04 10:20 ` Shameerali Kolothum Thodi
2020-12-04 10:23 ` Auger Eric
2020-12-04 10:23 ` Auger Eric
2020-12-04 10:23 ` Auger Eric
2021-01-14 16:58 ` Auger Eric
2021-01-14 16:58 ` Auger Eric
2021-01-14 16:58 ` Auger Eric
2021-01-14 17:09 ` Shameerali Kolothum Thodi
2021-01-14 17:09 ` Shameerali Kolothum Thodi
2021-01-14 17:09 ` Shameerali Kolothum Thodi
2021-01-14 17:33 ` Jean-Philippe Brucker
2021-01-14 17:33 ` Jean-Philippe Brucker
2021-01-14 17:33 ` Jean-Philippe Brucker
2021-01-14 18:00 ` Auger Eric
2021-01-14 18:00 ` Auger Eric
2021-01-14 18:00 ` Auger Eric
2021-02-15 13:17 ` Auger Eric
2021-02-15 13:17 ` Auger Eric
2021-02-15 13:17 ` Auger Eric
2020-11-18 11:21 ` [PATCH v13 08/15] iommu/smmuv3: Implement cache_invalidate Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2021-01-16 2:24 ` chenxiang (M) [this message]
2020-11-18 11:21 ` [PATCH v13 09/15] dma-iommu: Implement NESTED_MSI cookie Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 10/15] iommu/smmuv3: Nested mode single MSI doorbell per domain enforcement Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 11/15] iommu/smmuv3: Enforce incompatibility between nested mode and HW MSI regions Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 12/15] iommu/smmuv3: Implement bind/unbind_guest_msi Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 13/15] iommu/smmuv3: Report non recoverable faults Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 14/15] iommu/smmuv3: Accept configs with more than one context descriptor Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` [PATCH v13 15/15] iommu/smmuv3: Add PASID cache invalidation per PASID Eric Auger
2020-11-18 11:21 ` Eric Auger
2020-11-18 11:21 ` Eric Auger
2021-01-08 17:05 ` [PATCH v13 00/15] SMMUv3 Nested Stage Setup (IOMMU part) Shameerali Kolothum Thodi
2021-01-08 17:05 ` Shameerali Kolothum Thodi
2021-01-08 17:05 ` Shameerali Kolothum Thodi
2021-01-13 15:37 ` Auger Eric
2021-01-13 15:37 ` Auger Eric
2021-01-13 15:37 ` Auger Eric
2021-02-21 18:21 ` Auger Eric
2021-02-21 18:21 ` Auger Eric
2021-02-21 18:21 ` Auger Eric
2021-02-22 8:56 ` Shameerali Kolothum Thodi
2021-02-22 8:56 ` Shameerali Kolothum Thodi
2021-02-22 8:56 ` Shameerali Kolothum Thodi
2021-03-15 18:04 ` Krishna Reddy
2021-03-15 18:04 ` Krishna Reddy
2021-03-15 18:04 ` Krishna Reddy
2021-03-16 8:22 ` Auger Eric
2021-03-16 8:22 ` Auger Eric
2021-03-16 8:22 ` Auger Eric
2021-03-16 18:10 ` Krishna Reddy
2021-03-16 18:10 ` Krishna Reddy
2021-03-16 18:10 ` Krishna Reddy
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c405d689-aea5-c9eb-d418-ec5a30a97fa6@hisilicon.com \
--to=chenxiang66@hisilicon.com \
--cc=iommu@lists.linux-foundation.org \
--cc=linuxarm@openeuler.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.