From: Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>
To: Jason Gunthorpe <jgg@nvidia.com>,
"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
Joerg Roedel <joro@8bytes.org>,
"linux-arm-kernel@lists.infradead.org"
<linux-arm-kernel@lists.infradead.org>,
Robin Murphy <robin.murphy@arm.com>,
Will Deacon <will@kernel.org>
Cc: Moritz Fischer <mdf@kernel.org>,
Moritz Fischer <moritzf@google.com>,
Michael Shavit <mshavit@google.com>,
Nicolin Chen <nicolinc@nvidia.com>,
"patches@lists.linux.dev" <patches@lists.linux.dev>
Subject: RE: [PATCH v4 12/16] iommu/arm-smmu-v3: Add a global static IDENTITY domain
Date: Mon, 29 Jan 2024 18:11:48 +0000 [thread overview]
Message-ID: <2a828e481416405fb3a4cceb9e075a59@huawei.com> (raw)
In-Reply-To: <12-v4-c93b774edcc4+42d2b-smmuv3_newapi_p1_jgg@nvidia.com>
> -----Original Message-----
> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Thursday, January 25, 2024 11:57 PM
> To: iommu@lists.linux.dev; Joerg Roedel <joro@8bytes.org>; linux-arm-
> kernel@lists.infradead.org; Robin Murphy <robin.murphy@arm.com>; Will
> Deacon <will@kernel.org>
> Cc: Moritz Fischer <mdf@kernel.org>; Moritz Fischer <moritzf@google.com>;
> Michael Shavit <mshavit@google.com>; Nicolin Chen <nicolinc@nvidia.com>;
> patches@lists.linux.dev; Shameerali Kolothum Thodi
> <shameerali.kolothum.thodi@huawei.com>
> Subject: [PATCH v4 12/16] iommu/arm-smmu-v3: Add a global static
> IDENTITY domain
>
> Move to the new static global for identity domains. Move all the logic out
> of arm_smmu_attach_dev into an identity only function.
>
> Reviewed-by: Michael Shavit <mshavit@google.com>
> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com>
> Tested-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com>
> Tested-by: Nicolin Chen <nicolinc@nvidia.com>
> Tested-by: Moritz Fischer <moritzf@google.com>
> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 82 +++++++++++++++--
> ----
> drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.h | 1 -
> 2 files changed, 58 insertions(+), 25 deletions(-)
>
> diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> index f08cfa9b90b3eb..d35bf9655c9b1b 100644
> --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c
> @@ -2226,8 +2226,7 @@ static struct iommu_domain
> *arm_smmu_domain_alloc(unsigned type)
> return arm_smmu_sva_domain_alloc();
>
> if (type != IOMMU_DOMAIN_UNMANAGED &&
> - type != IOMMU_DOMAIN_DMA &&
> - type != IOMMU_DOMAIN_IDENTITY)
> + type != IOMMU_DOMAIN_DMA)
> return NULL;
>
> /*
> @@ -2335,11 +2334,6 @@ static int arm_smmu_domain_finalise(struct
> iommu_domain *domain)
> struct arm_smmu_domain *smmu_domain =
> to_smmu_domain(domain);
> struct arm_smmu_device *smmu = smmu_domain->smmu;
>
> - if (domain->type == IOMMU_DOMAIN_IDENTITY) {
> - smmu_domain->stage = ARM_SMMU_DOMAIN_BYPASS;
> - return 0;
> - }
> -
> /* Restrict the stage to what we can actually support */
> if (!(smmu->features & ARM_SMMU_FEAT_TRANS_S1))
> smmu_domain->stage = ARM_SMMU_DOMAIN_S2;
> @@ -2537,7 +2531,7 @@ static void arm_smmu_detach_dev(struct
> arm_smmu_master *master)
> struct arm_smmu_domain *smmu_domain;
> unsigned long flags;
>
> - if (!domain)
> + if (!domain || !(domain->type & __IOMMU_DOMAIN_PAGING))
> return;
>
> smmu_domain = to_smmu_domain(domain);
> @@ -2600,15 +2594,7 @@ static int arm_smmu_attach_dev(struct
> iommu_domain *domain, struct device *dev)
>
> arm_smmu_detach_dev(master);
>
> - /*
> - * The SMMU does not support enabling ATS with bypass. When the
> STE is
> - * in bypass (STE.Config[2:0] == 0b100), ATS Translation Requests and
> - * Translated transactions are denied as though ATS is disabled for
> the
> - * stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and
> - * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry).
> - */
> - if (smmu_domain->stage != ARM_SMMU_DOMAIN_BYPASS)
> - master->ats_enabled = arm_smmu_ats_supported(master);
> + master->ats_enabled = arm_smmu_ats_supported(master);
>
> spin_lock_irqsave(&smmu_domain->devices_lock, flags);
> list_add(&master->domain_head, &smmu_domain->devices);
> @@ -2645,13 +2631,6 @@ static int arm_smmu_attach_dev(struct
> iommu_domain *domain, struct device *dev)
> arm_smmu_write_ctx_desc(master,
> IOMMU_NO_PASID,
> NULL);
> break;
> - case ARM_SMMU_DOMAIN_BYPASS:
> - arm_smmu_make_bypass_ste(&target);
> - arm_smmu_install_ste_for_dev(master, &target);
> - if (master->cd_table.cdtab)
> - arm_smmu_write_ctx_desc(master,
> IOMMU_NO_PASID,
> - NULL);
> - break;
> }
>
> arm_smmu_enable_ats(master, smmu_domain);
> @@ -2667,6 +2646,60 @@ static int arm_smmu_attach_dev(struct
> iommu_domain *domain, struct device *dev)
> return ret;
> }
>
> +static int arm_smmu_attach_dev_ste(struct device *dev,
> + struct arm_smmu_ste *ste)
> +{
> + struct arm_smmu_master *master = dev_iommu_priv_get(dev);
> +
> + if (arm_smmu_master_sva_enabled(master))
> + return -EBUSY;
> +
> + /*
> + * Do not allow any ASID to be changed while are working on the STE,
> + * otherwise we could miss invalidations.
> + */
> + mutex_lock(&arm_smmu_asid_lock);
> +
> + /*
> + * The SMMU does not support enabling ATS with bypass/abort.
> When the
> + * STE is in bypass (STE.Config[2:0] == 0b100), ATS Translation
> Requests
> + * and Translated transactions are denied as though ATS is disabled
> for
> + * the stream (STE.EATS == 0b00), causing F_BAD_ATS_TREQ and
> + * F_TRANSL_FORBIDDEN events (IHI0070Ea 5.2 Stream Table Entry).
> + */
> + arm_smmu_detach_dev(master);
> +
> + arm_smmu_install_ste_for_dev(master, ste);
> + mutex_unlock(&arm_smmu_asid_lock);
> +
> + /*
> + * This has to be done after removing the master from the
> + * arm_smmu_domain->devices to avoid races updating the same
> context
> + * descriptor from arm_smmu_share_asid().
> + */
> + if (master->cd_table.cdtab)
> + arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID,
> NULL);
> + return 0;
> +}
> +
> +static int arm_smmu_attach_dev_identity(struct iommu_domain *domain,
> + struct device *dev)
> +{
> + struct arm_smmu_ste ste;
> +
> + arm_smmu_make_bypass_ste(&ste);
> + return arm_smmu_attach_dev_ste(dev, &ste);
> +}
> +
> +static const struct iommu_domain_ops arm_smmu_identity_ops = {
> + .attach_dev = arm_smmu_attach_dev_identity,
> +};
> +
> +static struct iommu_domain arm_smmu_identity_domain = {
> + .type = IOMMU_DOMAIN_IDENTITY,
> + .ops = &arm_smmu_identity_ops,
> +};
> +
> static int arm_smmu_map_pages(struct iommu_domain *domain, unsigned
> long iova,
> phys_addr_t paddr, size_t pgsize, size_t pgcount,
> int prot, gfp_t gfp, size_t *mapped)
> @@ -3056,6 +3089,7 @@ static void arm_smmu_remove_dev_pasid(struct
> device *dev, ioasid_t pasid)
> }
>
> static struct iommu_ops arm_smmu_ops = {
> + .identity_domain = &arm_smmu_identity_domain,
This seems to create a problem when we have set the identity domain and
try to enable sva for the device. Since there is no smmu_domain for this case
and there is no specific domain type checking in iommu_sva_bind_device() path,
it eventually crashes(hangs in my test) in,
iommu_sva_bind_device()
...
arm_smmu_sva_set_dev_pasid()
__arm_smmu_sva_bind()
arm_smmu_mmu_notifier_get(smmu_domain, ..) --> never exit the mmu notifier list loop.
I think we should check for the domain type in iommu_sva_bind_device() or later
before trying to use smmu_domain. At present(ie, without this series) it returns error
while we are trying to write the CD. But that looks too late as well.
Thanks,
Shameer
next prev parent reply other threads:[~2024-01-29 18:11 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2024-01-25 23:57 [PATCH v4 00/16] Update SMMUv3 to the modern iommu API (part 1/3) Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 01/16] iommu/arm-smmu-v3: Make STE programming independent of the callers Jason Gunthorpe
2024-01-26 4:03 ` Michael Shavit
2024-01-29 19:53 ` Moritz Fischer
2024-01-30 22:42 ` Mostafa Saleh
2024-01-30 23:56 ` Jason Gunthorpe
2024-01-31 14:34 ` Mostafa Saleh
2024-01-31 14:40 ` Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 02/16] iommu/arm-smmu-v3: Consolidate the STE generation for abort/bypass Jason Gunthorpe
2024-01-31 14:40 ` Mostafa Saleh
2024-01-31 14:47 ` Jason Gunthorpe
2024-02-01 11:32 ` Mostafa Saleh
2024-02-01 13:02 ` Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 03/16] iommu/arm-smmu-v3: Move arm_smmu_rmr_install_bypass_ste() Jason Gunthorpe
2024-01-29 15:07 ` Shameerali Kolothum Thodi
2024-01-29 15:43 ` Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 04/16] iommu/arm-smmu-v3: Move the STE generation for S1 and S2 domains into functions Jason Gunthorpe
2024-01-31 14:50 ` Mostafa Saleh
2024-01-31 15:05 ` Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 05/16] iommu/arm-smmu-v3: Build the whole STE in arm_smmu_make_s2_domain_ste() Jason Gunthorpe
2024-02-01 11:34 ` Mostafa Saleh
2024-01-25 23:57 ` [PATCH v4 06/16] iommu/arm-smmu-v3: Hold arm_smmu_asid_lock during all of attach_dev Jason Gunthorpe
2024-02-01 12:15 ` Mostafa Saleh
2024-02-01 13:24 ` Jason Gunthorpe
2024-02-13 13:30 ` Mostafa Saleh
2024-01-25 23:57 ` [PATCH v4 07/16] iommu/arm-smmu-v3: Compute the STE only once for each master Jason Gunthorpe
2024-02-01 12:18 ` Mostafa Saleh
2024-01-25 23:57 ` [PATCH v4 08/16] iommu/arm-smmu-v3: Do not change the STE twice during arm_smmu_attach_dev() Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 09/16] iommu/arm-smmu-v3: Put writing the context descriptor in the right order Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 10/16] iommu/arm-smmu-v3: Pass smmu_domain to arm_enable/disable_ats() Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 11/16] iommu/arm-smmu-v3: Remove arm_smmu_master->domain Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 12/16] iommu/arm-smmu-v3: Add a global static IDENTITY domain Jason Gunthorpe
2024-01-29 18:11 ` Shameerali Kolothum Thodi [this message]
2024-01-29 18:37 ` Jason Gunthorpe
2024-01-30 8:35 ` Shameerali Kolothum Thodi
2024-01-25 23:57 ` [PATCH v4 13/16] iommu/arm-smmu-v3: Add a global static BLOCKED domain Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 14/16] iommu/arm-smmu-v3: Use the identity/blocked domain during release Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 15/16] iommu/arm-smmu-v3: Pass arm_smmu_domain and arm_smmu_device to finalize Jason Gunthorpe
2024-01-25 23:57 ` [PATCH v4 16/16] iommu/arm-smmu-v3: Convert to domain_alloc_paging() Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=2a828e481416405fb3a4cceb9e075a59@huawei.com \
--to=shameerali.kolothum.thodi@huawei.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=linux-arm-kernel@lists.infradead.org \
--cc=mdf@kernel.org \
--cc=moritzf@google.com \
--cc=mshavit@google.com \
--cc=nicolinc@nvidia.com \
--cc=patches@lists.linux.dev \
--cc=robin.murphy@arm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).