From: Jason Gunthorpe <jgg@nvidia.com> To: iommu@lists.linux.dev, Joerg Roedel <joro@8bytes.org>, linux-arm-kernel@lists.infradead.org, Robin Murphy <robin.murphy@arm.com>, Will Deacon <will@kernel.org> Cc: Lu Baolu <baolu.lu@linux.intel.com>, Jean-Philippe Brucker <jean-philippe@linaro.org>, Joerg Roedel <jroedel@suse.de>, Moritz Fischer <mdf@kernel.org>, Moritz Fischer <moritzf@google.com>, Michael Shavit <mshavit@google.com>, Nicolin Chen <nicolinc@nvidia.com>, patches@lists.linux.dev, Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>, Mostafa Saleh <smostafa@google.com>, Zhangfei Gao <zhangfei.gao@linaro.org> Subject: [PATCH v6 05/16] iommu/arm-smmu-v3: Hold arm_smmu_asid_lock during all of attach_dev Date: Mon, 26 Feb 2024 13:07:16 -0400 [thread overview] Message-ID: <5-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com> (raw) In-Reply-To: <0-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com> The BTM support wants to be able to change the ASID of any smmu_domain. When it goes to do this it holds the arm_smmu_asid_lock and iterates over the target domain's devices list. During attach of a S1 domain we must ensure that the devices list and CD are in sync, otherwise we could miss CD updates or a parallel CD update could push an out of date CD. This is pretty complicated, and almost works today because arm_smmu_detach_dev() removes the master from the linked list before working on the CD entries, preventing parallel update of the CD. However, it does have an issue where the CD can remain programed while the domain appears to be unattached. arm_smmu_share_asid() will then not clear any CD entriess and install its own CD entry with the same ASID concurrently. This creates a small race window where the IOMMU can see two ASIDs pointing to different translations. CPU0 CPU1 arm_smmu_attach_dev() arm_smmu_detach_dev() spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_del(&master->domain_head); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_mmu_notifier_get() arm_smmu_alloc_shared_cd() arm_smmu_share_asid(): // Does nothing due to list_del above arm_smmu_update_ctx_desc_devices() arm_smmu_tlb_inv_asid() arm_smmu_write_ctx_desc() ** Now the ASID is in two CDs with different translation arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); Solve this by wrapping most of the attach flow in the arm_smmu_asid_lock. This locks more than strictly needed to prepare for the next patch which will reorganize the order of the linked list, STE and CD changes. Move arm_smmu_detach_dev() till after we have initialized the domain so the lock can be held for less time. Reviewed-by: Michael Shavit <mshavit@google.com> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com> Reviewed-by: Mostafa Saleh <smostafa@google.com> Tested-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Tested-by: Nicolin Chen <nicolinc@nvidia.com> Tested-by: Moritz Fischer <moritzf@google.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 22 ++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b81e621a8e5921..d2fc609fab60ab 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2586,8 +2586,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - mutex_lock(&smmu_domain->init_mutex); if (!smmu_domain->smmu) { @@ -2602,6 +2600,16 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (ret) return ret; + /* + * Prevent arm_smmu_share_asid() from trying to change the ASID + * of either the old or new domain while we are working on it. + * This allows the STE and the smmu_domain->devices list to + * be inconsistent during this routine. + */ + mutex_lock(&arm_smmu_asid_lock); + + arm_smmu_detach_dev(master); + master->domain = smmu_domain; /* @@ -2627,13 +2635,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } } - /* - * Prevent SVA from concurrently modifying the CD or writing to - * the CD entry - */ - mutex_lock(&arm_smmu_asid_lock); ret = arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, &smmu_domain->cd); - mutex_unlock(&arm_smmu_asid_lock); if (ret) { master->domain = NULL; goto out_list_del; @@ -2643,13 +2645,15 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_install_ste_for_dev(master); arm_smmu_enable_ats(master); - return 0; + goto out_unlock; out_list_del: spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_del(&master->domain_head); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +out_unlock: + mutex_unlock(&arm_smmu_asid_lock); return ret; } -- 2.43.2
WARNING: multiple messages have this Message-ID (diff)
From: Jason Gunthorpe <jgg@nvidia.com> To: iommu@lists.linux.dev, Joerg Roedel <joro@8bytes.org>, linux-arm-kernel@lists.infradead.org, Robin Murphy <robin.murphy@arm.com>, Will Deacon <will@kernel.org> Cc: Lu Baolu <baolu.lu@linux.intel.com>, Jean-Philippe Brucker <jean-philippe@linaro.org>, Joerg Roedel <jroedel@suse.de>, Moritz Fischer <mdf@kernel.org>, Moritz Fischer <moritzf@google.com>, Michael Shavit <mshavit@google.com>, Nicolin Chen <nicolinc@nvidia.com>, patches@lists.linux.dev, Shameerali Kolothum Thodi <shameerali.kolothum.thodi@huawei.com>, Mostafa Saleh <smostafa@google.com>, Zhangfei Gao <zhangfei.gao@linaro.org> Subject: [PATCH v6 05/16] iommu/arm-smmu-v3: Hold arm_smmu_asid_lock during all of attach_dev Date: Mon, 26 Feb 2024 13:07:16 -0400 [thread overview] Message-ID: <5-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com> (raw) In-Reply-To: <0-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com> The BTM support wants to be able to change the ASID of any smmu_domain. When it goes to do this it holds the arm_smmu_asid_lock and iterates over the target domain's devices list. During attach of a S1 domain we must ensure that the devices list and CD are in sync, otherwise we could miss CD updates or a parallel CD update could push an out of date CD. This is pretty complicated, and almost works today because arm_smmu_detach_dev() removes the master from the linked list before working on the CD entries, preventing parallel update of the CD. However, it does have an issue where the CD can remain programed while the domain appears to be unattached. arm_smmu_share_asid() will then not clear any CD entriess and install its own CD entry with the same ASID concurrently. This creates a small race window where the IOMMU can see two ASIDs pointing to different translations. CPU0 CPU1 arm_smmu_attach_dev() arm_smmu_detach_dev() spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_del(&master->domain_head); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); arm_smmu_mmu_notifier_get() arm_smmu_alloc_shared_cd() arm_smmu_share_asid(): // Does nothing due to list_del above arm_smmu_update_ctx_desc_devices() arm_smmu_tlb_inv_asid() arm_smmu_write_ctx_desc() ** Now the ASID is in two CDs with different translation arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, NULL); Solve this by wrapping most of the attach flow in the arm_smmu_asid_lock. This locks more than strictly needed to prepare for the next patch which will reorganize the order of the linked list, STE and CD changes. Move arm_smmu_detach_dev() till after we have initialized the domain so the lock can be held for less time. Reviewed-by: Michael Shavit <mshavit@google.com> Reviewed-by: Nicolin Chen <nicolinc@nvidia.com> Reviewed-by: Mostafa Saleh <smostafa@google.com> Tested-by: Shameer Kolothum <shameerali.kolothum.thodi@huawei.com> Tested-by: Nicolin Chen <nicolinc@nvidia.com> Tested-by: Moritz Fischer <moritzf@google.com> Signed-off-by: Jason Gunthorpe <jgg@nvidia.com> --- drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c | 22 ++++++++++++--------- 1 file changed, 13 insertions(+), 9 deletions(-) diff --git a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c index b81e621a8e5921..d2fc609fab60ab 100644 --- a/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c +++ b/drivers/iommu/arm/arm-smmu-v3/arm-smmu-v3.c @@ -2586,8 +2586,6 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) return -EBUSY; } - arm_smmu_detach_dev(master); - mutex_lock(&smmu_domain->init_mutex); if (!smmu_domain->smmu) { @@ -2602,6 +2600,16 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) if (ret) return ret; + /* + * Prevent arm_smmu_share_asid() from trying to change the ASID + * of either the old or new domain while we are working on it. + * This allows the STE and the smmu_domain->devices list to + * be inconsistent during this routine. + */ + mutex_lock(&arm_smmu_asid_lock); + + arm_smmu_detach_dev(master); + master->domain = smmu_domain; /* @@ -2627,13 +2635,7 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) } } - /* - * Prevent SVA from concurrently modifying the CD or writing to - * the CD entry - */ - mutex_lock(&arm_smmu_asid_lock); ret = arm_smmu_write_ctx_desc(master, IOMMU_NO_PASID, &smmu_domain->cd); - mutex_unlock(&arm_smmu_asid_lock); if (ret) { master->domain = NULL; goto out_list_del; @@ -2643,13 +2645,15 @@ static int arm_smmu_attach_dev(struct iommu_domain *domain, struct device *dev) arm_smmu_install_ste_for_dev(master); arm_smmu_enable_ats(master); - return 0; + goto out_unlock; out_list_del: spin_lock_irqsave(&smmu_domain->devices_lock, flags); list_del(&master->domain_head); spin_unlock_irqrestore(&smmu_domain->devices_lock, flags); +out_unlock: + mutex_unlock(&arm_smmu_asid_lock); return ret; } -- 2.43.2 _______________________________________________ linux-arm-kernel mailing list linux-arm-kernel@lists.infradead.org http://lists.infradead.org/mailman/listinfo/linux-arm-kernel
next prev parent reply other threads:[~2024-02-26 17:07 UTC|newest] Thread overview: 46+ messages / expand[flat|nested] mbox.gz Atom feed top 2024-02-26 17:07 [PATCH v6 00/16] Update SMMUv3 to the modern iommu API (part 1/3) Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 01/16] iommu/arm-smmu-v3: Make STE programming independent of the callers Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-27 12:47 ` Will Deacon 2024-02-27 12:47 ` Will Deacon 2024-02-29 14:07 ` Jason Gunthorpe 2024-02-29 14:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 02/16] iommu/arm-smmu-v3: Consolidate the STE generation for abort/bypass Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 03/16] iommu/arm-smmu-v3: Move the STE generation for S1 and S2 domains into functions Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 04/16] iommu/arm-smmu-v3: Build the whole STE in arm_smmu_make_s2_domain_ste() Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe [this message] 2024-02-26 17:07 ` [PATCH v6 05/16] iommu/arm-smmu-v3: Hold arm_smmu_asid_lock during all of attach_dev Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 06/16] iommu/arm-smmu-v3: Compute the STE only once for each master Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 07/16] iommu/arm-smmu-v3: Do not change the STE twice during arm_smmu_attach_dev() Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 08/16] iommu/arm-smmu-v3: Put writing the context descriptor in the right order Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 09/16] iommu/arm-smmu-v3: Pass smmu_domain to arm_enable/disable_ats() Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 10/16] iommu/arm-smmu-v3: Remove arm_smmu_master->domain Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 11/16] iommu/arm-smmu-v3: Check that the RID domain is S1 in SVA Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 12/16] iommu/arm-smmu-v3: Add a global static IDENTITY domain Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 13/16] iommu/arm-smmu-v3: Add a global static BLOCKED domain Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 14/16] iommu/arm-smmu-v3: Use the identity/blocked domain during release Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 15/16] iommu/arm-smmu-v3: Pass arm_smmu_domain and arm_smmu_device to finalize Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-26 17:07 ` [PATCH v6 16/16] iommu/arm-smmu-v3: Convert to domain_alloc_paging() Jason Gunthorpe 2024-02-26 17:07 ` Jason Gunthorpe 2024-02-29 16:34 ` [PATCH v6 00/16] Update SMMUv3 to the modern iommu API (part 1/3) Will Deacon 2024-02-29 16:34 ` Will Deacon 2024-02-29 20:23 ` Jason Gunthorpe 2024-02-29 20:23 ` Jason Gunthorpe 2024-02-29 20:47 ` Nicolin Chen 2024-02-29 20:47 ` Nicolin Chen 2024-03-01 8:01 ` Will Deacon 2024-03-01 8:01 ` Will Deacon
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=5-v6-96275f25c39d+2d4-smmuv3_newapi_p1_jgg@nvidia.com \ --to=jgg@nvidia.com \ --cc=baolu.lu@linux.intel.com \ --cc=iommu@lists.linux.dev \ --cc=jean-philippe@linaro.org \ --cc=joro@8bytes.org \ --cc=jroedel@suse.de \ --cc=linux-arm-kernel@lists.infradead.org \ --cc=mdf@kernel.org \ --cc=moritzf@google.com \ --cc=mshavit@google.com \ --cc=nicolinc@nvidia.com \ --cc=patches@lists.linux.dev \ --cc=robin.murphy@arm.com \ --cc=shameerali.kolothum.thodi@huawei.com \ --cc=smostafa@google.com \ --cc=will@kernel.org \ --cc=zhangfei.gao@linaro.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.