From: Baolu Lu <baolu.lu@linux.intel.com>
To: Jason Gunthorpe <jgg@nvidia.com>,
Niklas Schnelle <schnelle@linux.ibm.com>
Cc: baolu.lu@linux.intel.com, Robin Murphy <robin.murphy@arm.com>,
Matthew Rosato <mjrosato@linux.ibm.com>,
Gerd Bayer <gbayer@linux.ibm.com>,
iommu@lists.linux.dev, Joerg Roedel <joro@8bytes.org>,
Will Deacon <will@kernel.org>,
Wenjia Zhang <wenjia@linux.ibm.com>,
Pierre Morel <pmorel@linux.ibm.com>,
linux-s390@vger.kernel.org, borntraeger@linux.ibm.com,
hca@linux.ibm.com, gor@linux.ibm.com,
gerald.schaefer@linux.ibm.com, agordeev@linux.ibm.com,
svens@linux.ibm.com, linux-kernel@vger.kernel.org,
Julian Ruess <julianr@linux.ibm.com>
Subject: Re: [PATCH v2 4/7] iommu: Let iommu.strict override ops->def_domain_type
Date: Wed, 7 Dec 2022 21:18:19 +0800 [thread overview]
Message-ID: <f9433a12-eacd-9e6b-a517-c3be438193d1@linux.intel.com> (raw)
In-Reply-To: <Y4/LsZKmR3iWFphU@nvidia.com>
On 2022/12/7 7:09, Jason Gunthorpe wrote:
> static ssize_t iommu_group_store_type(struct iommu_group *group,
> const char *buf, size_t count)
> {
> - struct group_device *grp_dev;
> - struct device *dev;
> - int ret, req_type;
> + enum dma_api_policy policy;
> + int ret;
>
> if (!capable(CAP_SYS_ADMIN) || !capable(CAP_SYS_RAWIO))
> return -EACCES;
> @@ -2977,77 +2907,30 @@ static ssize_t iommu_group_store_type(struct iommu_group *group,
> return -EINVAL;
>
> if (sysfs_streq(buf, "identity"))
> - req_type = IOMMU_DOMAIN_IDENTITY;
> + policy = DMA_API_POLICY_IDENTITY;
> else if (sysfs_streq(buf, "DMA"))
> - req_type = IOMMU_DOMAIN_DMA;
> + policy = DMA_API_POLICY_STRICT;
> else if (sysfs_streq(buf, "DMA-FQ"))
> - req_type = IOMMU_DOMAIN_DMA_FQ;
> + policy = DMA_API_POLICY_LAZY;
> else if (sysfs_streq(buf, "auto"))
> - req_type = 0;
> + policy = DMA_API_POLICY_ANY;
> else
> return -EINVAL;
>
> /*
> - * Lock/Unlock the group mutex here before device lock to
> - * 1. Make sure that the iommu group has only one device (this is a
> - * prerequisite for step 2)
> - * 2. Get struct *dev which is needed to lock device
> - */
> - mutex_lock(&group->mutex);
> - if (iommu_group_device_count(group) != 1) {
> - mutex_unlock(&group->mutex);
> - pr_err_ratelimited("Cannot change default domain: Group has more than one device\n");
> - return -EINVAL;
> - }
> -
> - /* Since group has only one device */
> - grp_dev = list_first_entry(&group->devices, struct group_device, list);
> - dev = grp_dev->dev;
> - get_device(dev);
> -
> - /*
> - * Don't hold the group mutex because taking group mutex first and then
> - * the device lock could potentially cause a deadlock as below. Assume
> - * two threads T1 and T2. T1 is trying to change default domain of an
> - * iommu group and T2 is trying to hot unplug a device or release [1] VF
> - * of a PCIe device which is in the same iommu group. T1 takes group
> - * mutex and before it could take device lock assume T2 has taken device
> - * lock and is yet to take group mutex. Now, both the threads will be
> - * waiting for the other thread to release lock. Below, lock order was
> - * suggested.
> - * device_lock(dev);
> - * mutex_lock(&group->mutex);
> - * iommu_change_dev_def_domain();
> - * mutex_unlock(&group->mutex);
> - * device_unlock(dev);
> - *
> - * [1] Typical device release path
> - * device_lock() from device/driver core code
> - * -> bus_notifier()
> - * -> iommu_bus_notifier()
> - * -> iommu_release_device()
> - * -> ops->release_device() vendor driver calls back iommu core code
> - * -> mutex_lock() from iommu core code
> + * Taking ownership disables the DMA API domain, prevents drivers from
> + * being attached, and switches to a blocking domain. Releasing
> + * ownership will put back the new or original DMA API domain.
> */
> - mutex_unlock(&group->mutex);
> -
> - /* Check if the device in the group still has a driver bound to it */
> - device_lock(dev);
With device_lock() removed, this probably races with the
iommu_release_device() path? group->mutex seems insufficient to avoid
the race. Perhaps I missed anything.
> - if (device_is_bound(dev) && !(req_type == IOMMU_DOMAIN_DMA_FQ &&
> - group->default_domain->type == IOMMU_DOMAIN_DMA)) {
> - pr_err_ratelimited("Device is still bound to driver\n");
> - ret = -EBUSY;
> - goto out;
> - }
> -
> - ret = iommu_change_dev_def_domain(group, dev, req_type);
> - ret = ret ?: count;
> -
> -out:
> - device_unlock(dev);
> - put_device(dev);
> + ret = iommu_group_claim_dma_owner(group, &ret);
> + if (ret)
> + return ret;
>
> - return ret;
> + ret = iommu_change_group_dma_api_policy(group, policy);
> + iommu_group_release_dma_owner(group);
> + if (ret)
> + return ret;
> + return count;
> }
>
> static bool iommu_is_default_domain(struct iommu_group *group)
Best regards,
baolu
next prev parent reply other threads:[~2022-12-07 13:18 UTC|newest]
Thread overview: 39+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-11-16 17:16 [PATCH v2 0/7] iommu/dma: s390 DMA API conversion and optimized IOTLB flushing Niklas Schnelle
2022-11-16 17:16 ` [PATCH v2 1/7] s390/ism: Set DMA coherent mask Niklas Schnelle
2022-11-16 17:16 ` [PATCH v2 2/7] s390/pci: prepare is_passed_through() for dma-iommu Niklas Schnelle
2022-11-16 17:16 ` [PATCH v2 3/7] s390/pci: Use dma-iommu layer Niklas Schnelle
2022-11-28 18:03 ` Robin Murphy
2022-12-19 15:17 ` Niklas Schnelle
2022-11-16 17:16 ` [PATCH v2 4/7] iommu: Let iommu.strict override ops->def_domain_type Niklas Schnelle
2022-11-17 1:55 ` Baolu Lu
2022-11-28 11:10 ` Niklas Schnelle
2022-11-28 13:00 ` Baolu Lu
2022-11-28 13:29 ` Jason Gunthorpe
2022-11-28 15:54 ` Niklas Schnelle
2022-11-28 16:35 ` Jason Gunthorpe
2022-11-28 21:01 ` Robin Murphy
2022-11-29 17:33 ` Jason Gunthorpe
2022-11-29 18:41 ` Robin Murphy
2022-11-29 20:09 ` Jason Gunthorpe
2022-11-30 1:28 ` Baolu Lu
2022-12-05 15:34 ` Niklas Schnelle
2022-12-06 23:09 ` Jason Gunthorpe
2022-12-07 13:18 ` Baolu Lu [this message]
2022-12-07 13:23 ` Jason Gunthorpe
2022-12-07 14:18 ` Robin Murphy
2022-12-07 14:30 ` Jason Gunthorpe
2022-11-28 16:56 ` Robin Murphy
2022-11-16 17:16 ` [PATCH v2 5/7] iommu/dma: Allow a single FQ in addition to per-CPU FQs Niklas Schnelle
2022-11-16 17:16 ` [PATCH v2 6/7] iommu/dma: Enable variable queue size and use larger single queue Niklas Schnelle
2022-11-16 17:16 ` [PATCH v2 7/7] iommu/s390: flush queued IOVAs on RPCIT out of resource indication Niklas Schnelle
2022-11-28 14:52 ` Robin Murphy
2022-11-29 12:00 ` Niklas Schnelle
2022-11-29 12:53 ` Robin Murphy
2022-11-29 14:40 ` Niklas Schnelle
2022-12-02 14:29 ` Niklas Schnelle
2022-12-02 14:42 ` Jason Gunthorpe
2022-12-02 15:12 ` Niklas Schnelle
2022-12-02 15:24 ` Jason Gunthorpe
2022-12-05 18:24 ` Robin Murphy
2022-12-06 10:13 ` Niklas Schnelle
2022-11-29 13:51 ` Matthew Rosato
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=f9433a12-eacd-9e6b-a517-c3be438193d1@linux.intel.com \
--to=baolu.lu@linux.intel.com \
--cc=agordeev@linux.ibm.com \
--cc=borntraeger@linux.ibm.com \
--cc=gbayer@linux.ibm.com \
--cc=gerald.schaefer@linux.ibm.com \
--cc=gor@linux.ibm.com \
--cc=hca@linux.ibm.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=julianr@linux.ibm.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-s390@vger.kernel.org \
--cc=mjrosato@linux.ibm.com \
--cc=pmorel@linux.ibm.com \
--cc=robin.murphy@arm.com \
--cc=schnelle@linux.ibm.com \
--cc=svens@linux.ibm.com \
--cc=wenjia@linux.ibm.com \
--cc=will@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).