From: "chenxiang (M)" <chenxiang66@hisilicon.com>
To: Eric Auger <eric.auger@redhat.com>, <eric.auger.pro@gmail.com>,
<iommu@lists.linux-foundation.org>,
<linux-kernel@vger.kernel.org>, <kvm@vger.kernel.org>,
<kvmarm@lists.cs.columbia.edu>, <will@kernel.org>,
<maz@kernel.org>, <robin.murphy@arm.com>, <joro@8bytes.org>,
<alex.williamson@redhat.com>, <tn@semihalf.com>,
<zhukeqian1@huawei.com>
Cc: jean-philippe@linaro.org, wangxingang5@huawei.com,
lushenming@huawei.com, jiangkunkun@huawei.com,
linuxarm@openeuler.org, vsethi@nvidia.com, vivek.gautam@arm.com,
zhangfei.gao@linaro.org
Subject: Re: [PATCH v14 07/13] iommu/smmuv3: Implement cache_invalidate
Date: Thu, 4 Mar 2021 15:55:53 +0800 [thread overview]
Message-ID: <c10c6405-5efe-5a41-2b3a-f3af85a528ba@hisilicon.com> (raw)
In-Reply-To: <20210223205634.604221-8-eric.auger@redhat.com>
[-- Attachment #1.1: Type: text/plain, Size: 5264 bytes --]
Hi Eric,
在 2021/2/24 4:56, Eric Auger 写道:
> Implement domain-selective, pasid selective and page-selective
> IOTLB invalidations.
>
> Signed-off-by: Eric Auger <eric.auger@redhat.com>
>
> ---
>
> v13 -> v14:
> - Add domain invalidation
> - do global inval when asid is not provided with addr
> granularity
>
> v7 -> v8:
> - ASID based invalidation using iommu_inv_pasid_info
> - check ARCHID/PASID flags in addr based invalidation
> - use __arm_smmu_tlb_inv_context and __arm_smmu_tlb_inv_range_nosync
>
> v6 -> v7
> - check the uapi version
>
> v3 -> v4:
> - adapt to changes in the uapi
> - add support for leaf parameter
> - do not use arm_smmu_tlb_inv_range_nosync or arm_smmu_tlb_inv_context
> anymore
>
> v2 -> v3:
> - replace __arm_smmu_tlb_sync by arm_smmu_cmdq_issue_sync
>
> v1 -> v2:
> - properly pass the asid
> ---
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 74 +++++++++++++++++++++
> 1 file changed, 74 insertions(+)
>
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index 4c19a1114de4..df3adc49111c 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -2949,6 +2949,79 @@ static void arm_smmu_detach_pasid_table(struct iommu_domain *domain)
> mutex_unlock(&smmu_domain->init_mutex);
> }
>
> +static int
> +arm_smmu_cache_invalidate(struct iommu_domain *domain, struct device *dev,
> + struct iommu_cache_invalidate_info *inv_info)
> +{
> + struct arm_smmu_cmdq_ent cmd = {.opcode = CMDQ_OP_TLBI_NSNH_ALL};
> + struct arm_smmu_domain *smmu_domain = to_smmu_domain(domain);
> + struct arm_smmu_device *smmu = smmu_domain->smmu;
> +
> + if (smmu_domain->stage != ARM_SMMU_DOMAIN_NESTED)
> + return -EINVAL;
> +
> + if (!smmu)
> + return -EINVAL;
> +
> + if (inv_info->version != IOMMU_CACHE_INVALIDATE_INFO_VERSION_1)
> + return -EINVAL;
> +
> + if (inv_info->cache & IOMMU_CACHE_INV_TYPE_PASID ||
> + inv_info->cache & IOMMU_CACHE_INV_TYPE_DEV_IOTLB) {
> + return -ENOENT;
> + }
> +
> + if (!(inv_info->cache & IOMMU_CACHE_INV_TYPE_IOTLB))
> + return -EINVAL;
> +
> + /* IOTLB invalidation */
> +
> + switch (inv_info->granularity) {
> + case IOMMU_INV_GRANU_PASID:
> + {
> + struct iommu_inv_pasid_info *info =
> + &inv_info->granu.pasid_info;
> +
> + if (info->flags & IOMMU_INV_ADDR_FLAGS_PASID)
> + return -ENOENT;
> + if (!(info->flags & IOMMU_INV_PASID_FLAGS_ARCHID))
> + return -EINVAL;
> +
> + __arm_smmu_tlb_inv_context(smmu_domain, info->archid);
> + return 0;
> + }
> + case IOMMU_INV_GRANU_ADDR:
> + {
> + struct iommu_inv_addr_info *info = &inv_info->granu.addr_info;
> + size_t size = info->nb_granules * info->granule_size;
> + bool leaf = info->flags & IOMMU_INV_ADDR_FLAGS_LEAF;
> +
> + if (info->flags & IOMMU_INV_ADDR_FLAGS_PASID)
> + return -ENOENT;
> +
> + if (!(info->flags & IOMMU_INV_ADDR_FLAGS_ARCHID))
> + break;
> +
> + arm_smmu_tlb_inv_range_domain(info->addr, size,
> + info->granule_size, leaf,
> + info->archid, smmu_domain);
Is it possible to add a check before the function to make sure that
info->granule_size can be recognized by SMMU?
There is a scenario which will cause TLBI issue: RIL feature is enabled
on guest but is disabled on host, and then on
host it just invalidate 4K/2M/1G once a time, but from QEMU,
info->nb_granules is always 1 and info->granule_size = size,
if size is not equal to 4K or 2M or 1G (for example size = granule_size
is 5M), it will only part of the size it wants to invalidate.
I think maybe we can add a check here: if RIL is not enabled and also
size is not the granule_size (4K/2M/1G) supported by
SMMU hardware, can we just simply use the smallest granule_size
supported by hardware all the time?
> +
> + arm_smmu_cmdq_issue_sync(smmu);
> + return 0;
> + }
> + case IOMMU_INV_GRANU_DOMAIN:
> + break;
I check the qemu code
(https://github.com/eauger/qemu/tree/v5.2.0-2stage-rfcv8), for opcode
CMD_TLBI_NH_ALL or CMD_TLBI_NSNH_ALL from guest OS
it calls smmu_inv_notifiers_all() to unamp all notifiers of all mr's,
but it seems not set event.entry.granularity which i think it should set
IOMMU_INV_GRAN_ADDR.
BTW, for opcode CMD_TLBI_NH_ALL or CMD_TLBI_NSNH_ALL, it needs to unmap
size = 0x1000000000000 on 48bit system (it may spend much time), maybe
it is better
to set it as IOMMU_INV_GRANU_DOMAIN, then in host OS, send TLBI_NH_ALL
directly when IOMMU_INV_GRANU_DOMAIN.
> + default:
> + return -EINVAL;
> + }
> +
> + /* Global S1 invalidation */
> + cmd.tlbi.vmid = smmu_domain->s2_cfg.vmid;
> + arm_smmu_cmdq_issue_cmd(smmu, &cmd);
> + arm_smmu_cmdq_issue_sync(smmu);
> + return 0;
> +}
> +
> static bool arm_smmu_dev_has_feature(struct device *dev,
> enum iommu_dev_features feat)
> {
> @@ -3048,6 +3121,7 @@ static struct iommu_ops arm_smmu_ops = {
> .put_resv_regions = generic_iommu_put_resv_regions,
> .attach_pasid_table = arm_smmu_attach_pasid_table,
> .detach_pasid_table = arm_smmu_detach_pasid_table,
> + .cache_invalidate = arm_smmu_cache_invalidate,
> .dev_has_feat = arm_smmu_dev_has_feature,
> .dev_feat_enabled = arm_smmu_dev_feature_enabled,
> .dev_enable_feat = arm_smmu_dev_enable_feature,
[-- Attachment #1.2: Type: text/html, Size: 6615 bytes --]
[-- Attachment #2: Type: text/plain, Size: 156 bytes --]
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
next prev parent reply other threads:[~2021-03-04 7:56 UTC|newest]
Thread overview: 47+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-02-23 20:56 [PATCH v14 00/13] SMMUv3 Nested Stage Setup (IOMMU part) Eric Auger
2021-02-23 20:56 ` [PATCH v14 01/13] iommu: Introduce attach/detach_pasid_table API Eric Auger
2021-02-23 20:56 ` [PATCH v14 02/13] iommu: Introduce bind/unbind_guest_msi Eric Auger
2021-02-23 20:56 ` [PATCH v14 03/13] iommu/smmuv3: Allow s1 and s2 configs to coexist Eric Auger
2021-02-23 20:56 ` [PATCH v14 04/13] iommu/smmuv3: Get prepared for nested stage support Eric Auger
2021-02-23 20:56 ` [PATCH v14 05/13] iommu/smmuv3: Implement attach/detach_pasid_table Eric Auger
2021-03-02 8:35 ` Keqian Zhu
2021-03-19 13:15 ` Auger Eric
2021-03-22 6:23 ` Keqian Zhu
2021-02-23 20:56 ` [PATCH v14 06/13] iommu/smmuv3: Allow stage 1 invalidation with unmanaged ASIDs Eric Auger
2021-03-30 9:17 ` Zenghui Yu
2021-04-01 9:38 ` Auger Eric
2021-04-01 12:37 ` Kunkun Jiang
2021-04-08 12:30 ` Auger Eric
2021-04-09 4:48 ` Kunkun Jiang
2021-04-09 8:31 ` Auger Eric
2021-04-09 9:43 ` Kunkun Jiang
2021-02-23 20:56 ` [PATCH v14 07/13] iommu/smmuv3: Implement cache_invalidate Eric Auger
2021-03-04 7:55 ` chenxiang (M) [this message]
2021-03-19 17:36 ` Auger Eric
2021-03-22 6:40 ` [Linuxarm] " chenxiang (M)
2021-03-22 9:05 ` Auger Eric
2021-03-23 1:28 ` chenxiang (M)
2021-04-01 6:11 ` Zenghui Yu
2021-04-01 12:06 ` Auger Eric
2021-02-23 20:56 ` [PATCH v14 08/13] dma-iommu: Implement NESTED_MSI cookie Eric Auger
2021-04-07 7:39 ` Zenghui Yu
2021-04-10 13:34 ` Auger Eric
2021-02-23 20:56 ` [PATCH v14 09/13] iommu/smmuv3: Nested mode single MSI doorbell per domain enforcement Eric Auger
2021-02-23 20:56 ` [PATCH v14 10/13] iommu/smmuv3: Enforce incompatibility between nested mode and HW MSI regions Eric Auger
2021-02-23 20:56 ` [PATCH v14 11/13] iommu/smmuv3: Implement bind/unbind_guest_msi Eric Auger
2021-02-23 20:56 ` [PATCH v14 12/13] iommu/smmuv3: report additional recoverable faults Eric Auger
2021-02-23 20:56 ` [PATCH v14 13/13] iommu/smmuv3: Accept configs with more than one context descriptor Eric Auger
2021-03-30 9:23 ` Zenghui Yu
2021-04-01 11:48 ` Auger Eric
2021-04-01 12:38 ` Shameerali Kolothum Thodi
2021-04-01 13:20 ` Auger Eric
2021-02-25 16:06 ` [PATCH v14 00/13] SMMUv3 Nested Stage Setup (IOMMU part) Auger Eric
2021-04-22 15:04 ` Sumit Gupta
2021-04-23 13:00 ` Jean-Philippe Brucker
2021-04-23 17:16 ` Sumit Gupta
2021-04-23 17:58 ` Krishna Reddy
2021-04-23 18:19 ` Sumit Gupta
2021-04-24 9:06 ` Marc Zyngier
2021-04-24 11:29 ` Sumit Gupta
2021-03-18 0:16 ` Krishna Reddy
2021-03-19 13:17 ` Auger Eric
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=c10c6405-5efe-5a41-2b3a-f3af85a528ba@hisilicon.com \
--to=chenxiang66@hisilicon.com \
--cc=alex.williamson@redhat.com \
--cc=eric.auger.pro@gmail.com \
--cc=eric.auger@redhat.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jean-philippe@linaro.org \
--cc=jiangkunkun@huawei.com \
--cc=joro@8bytes.org \
--cc=kvm@vger.kernel.org \
--cc=kvmarm@lists.cs.columbia.edu \
--cc=linux-kernel@vger.kernel.org \
--cc=linuxarm@openeuler.org \
--cc=lushenming@huawei.com \
--cc=maz@kernel.org \
--cc=robin.murphy@arm.com \
--cc=tn@semihalf.com \
--cc=vivek.gautam@arm.com \
--cc=vsethi@nvidia.com \
--cc=wangxingang5@huawei.com \
--cc=will@kernel.org \
--cc=zhangfei.gao@linaro.org \
--cc=zhukeqian1@huawei.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).