From: Nicolin Chen <nicolinc@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: "jgg@nvidia.com" <jgg@nvidia.com>,
"joro@8bytes.org" <joro@8bytes.org>,
"will@kernel.org" <will@kernel.org>,
"robin.murphy@arm.com" <robin.murphy@arm.com>,
"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"shuah@kernel.org" <shuah@kernel.org>,
"Liu, Yi L" <yi.l.liu@intel.com>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kselftest@vger.kernel.org"
<linux-kselftest@vger.kernel.org>,
"baolu.lu@linux.intel.com" <baolu.lu@linux.intel.com>
Subject: Re: [PATCH v2 08/10] iommufd/device: Use iommu_group_replace_domain()
Date: Tue, 14 Feb 2023 02:59:30 -0800 [thread overview]
Message-ID: <Y+tpkpNYil3duTIP@Asurada-Nvidia> (raw)
In-Reply-To: <BN9PR11MB5276268D3ED0360913A05C368CDE9@BN9PR11MB5276.namprd11.prod.outlook.com>
On Fri, Feb 10, 2023 at 02:11:23AM +0000, Tian, Kevin wrote:
> My confusion is that we have different flows between detach/attach
> and replace.
>
> today with separate detach+attach we have following flow:
>
> Remove device from current hwpt;
> if (last_device in hwpt) {
> Remove hwpt domain from current iopt;
> if (last_device in group)
> detach group from hwpt domain;
> }
>
> if (first device in group) {
> attach group to new hwpt domain;
> if (first_device in hwpt)
> Add hwpt domain to new iopt;
> Add device to new hwpt;
>
> but replace flow is different on the detach part:
>
> if (first device in group) {
> replace group's domain from current hwpt to new hwpt;
> if (first_device in hwpt)
> Add hwpt domain to new iopt;
> }
>
> Remove device from old hwpt;
> if (last_device in old hwpt)
> Remove hwpt domain from old iopt;
>
> Add device to new hwpt;
>
> I'm yet to figure out whether we have sufficient lock protection to
> prevent other paths from using old iopt/hwpt to find the device
> which is already attached to a different domain.
With Jason's new series, the detach() routine is lighter now.
I wonder if it'd be safer now to do the detach() call after
iommu_group_replace_domain()?
Thanks
Nic
@@ -196,10 +198,41 @@ static bool iommufd_hw_pagetable_has_group(struct iommufd_hw_pagetable *hwpt,
return false;
}
+/**
+ * __iommufd_device_detach - Detach a device from idev->hwpt
+ * @idev: device to detach
+ * @detach_group: flag to call iommu_detach_group
+ *
+ * This is a cleanup helper shared by the replace and detach routines. Comparing
+ * to a detach routine, a replace call does not need the iommu_detach_group().
+ */
+static void __iommufd_device_detach(struct iommufd_device *idev,
+ bool detach_group)
+{
+ struct iommufd_hw_pagetable *hwpt = idev->hwpt;
+
+ mutex_lock(&hwpt->devices_lock);
+ list_del(&idev->devices_item);
+ if (detach_group && !iommufd_hw_pagetable_has_group(hwpt, idev->group))
+ iommu_detach_group(hwpt->domain, idev->group);
+ iopt_remove_reserved_iova(&hwpt->ioas->iopt, idev->dev);
+ mutex_unlock(&hwpt->devices_lock);
+
+ if (hwpt->auto_domain)
+ iommufd_object_destroy_user(idev->ictx, &hwpt->obj);
+ else
+ refcount_dec(&hwpt->obj.users);
+
+ idev->hwpt = NULL;
+
+ refcount_dec(&idev->obj.users);
+}
+
/* On success this consumes a hwpt reference from the caller */
static int iommufd_device_do_attach(struct iommufd_device *idev,
struct iommufd_hw_pagetable *hwpt)
{
+ struct iommufd_hw_pagetable *cur_hwpt = idev->hwpt;
phys_addr_t sw_msi_start = PHYS_ADDR_MAX;
int rc;
@@ -237,7 +270,7 @@ static int iommufd_device_do_attach(struct iommufd_device *idev,
* the group once for the first device that is in the group.
*/
if (!iommufd_hw_pagetable_has_group(hwpt, idev->group)) {
- rc = iommu_attach_group(hwpt->domain, idev->group);
+ rc = iommu_group_replace_domain(idev->group, hwpt->domain);
if (rc)
goto out_iova;
@@ -249,6 +282,10 @@ static int iommufd_device_do_attach(struct iommufd_device *idev,
}
}
+ /* Detach from the cur_hwpt without iommu_detach_group() */
+ if (cur_hwpt)
+ __iommufd_device_detach(idev, false);
+
idev->hwpt = hwpt;
/* The HWPT reference from the caller is moved to this list */
list_add(&idev->devices_item, &hwpt->devices);
next prev parent reply other threads:[~2023-02-14 10:59 UTC|newest]
Thread overview: 71+ messages / expand[flat|nested] mbox.gz Atom feed top
2023-02-07 21:17 [PATCH v2 00/10] Add IO page table replacement support Nicolin Chen
2023-02-07 21:17 ` [PATCH v2 01/10] iommu: Move dev_iommu_ops() to private header Nicolin Chen
2023-02-09 2:49 ` Tian, Kevin
2023-02-07 21:17 ` [PATCH v2 02/10] iommu: Introduce a new iommu_group_replace_domain() API Nicolin Chen
2023-02-09 2:55 ` Tian, Kevin
2023-02-09 13:23 ` Jason Gunthorpe
2023-02-10 1:34 ` Tian, Kevin
2023-02-10 23:51 ` Alex Williamson
2023-02-11 0:44 ` Jason Gunthorpe
2023-02-13 2:24 ` Tian, Kevin
2023-02-13 8:34 ` Baolu Lu
2023-02-13 14:45 ` Jason Gunthorpe
2023-02-14 3:29 ` Tian, Kevin
2023-02-15 6:10 ` Tian, Kevin
2023-02-15 12:52 ` Jason Gunthorpe
2023-02-22 2:11 ` Tian, Kevin
2023-02-24 0:57 ` Jason Gunthorpe
2023-02-24 8:07 ` Tian, Kevin
2023-02-07 21:17 ` [PATCH v2 03/10] iommufd: Create access in vfio_iommufd_emulated_bind() Nicolin Chen
2023-02-09 2:56 ` Tian, Kevin
2023-02-09 16:15 ` Nicolin Chen
2023-02-09 18:58 ` Eric Farman
2023-02-09 19:54 ` Nicolin Chen
2023-02-07 21:17 ` [PATCH v2 04/10] iommufd/selftest: Add IOMMU_TEST_OP_ACCESS_SET_IOAS coverage Nicolin Chen
2023-02-09 2:59 ` Tian, Kevin
2023-02-07 21:17 ` [PATCH v2 05/10] iommufd: Add replace support in iommufd_access_set_ioas() Nicolin Chen
2023-02-09 3:13 ` Tian, Kevin
2023-02-09 20:28 ` Nicolin Chen
2023-02-09 20:49 ` Jason Gunthorpe
2023-02-09 22:18 ` Nicolin Chen
2023-02-07 21:17 ` [PATCH v2 06/10] iommufd/selftest: Add coverage for access->ioas replacement Nicolin Chen
2023-02-07 21:17 ` [PATCH v2 07/10] iommufd/device: Make hwpt_list list_add/del symmetric Nicolin Chen
2023-02-09 3:23 ` Tian, Kevin
2023-02-09 13:24 ` Jason Gunthorpe
2023-02-10 1:46 ` Tian, Kevin
2023-02-10 21:17 ` Jason Gunthorpe
2023-02-13 2:12 ` Tian, Kevin
2023-02-07 21:18 ` [PATCH v2 08/10] iommufd/device: Use iommu_group_replace_domain() Nicolin Chen
2023-02-08 8:08 ` Liu, Yi L
2023-02-09 20:55 ` Nicolin Chen
2023-02-08 8:12 ` Liu, Yi L
2023-02-09 20:56 ` Nicolin Chen
2023-02-09 4:00 ` Tian, Kevin
2023-02-09 21:13 ` Nicolin Chen
2023-02-10 0:01 ` Jason Gunthorpe
2023-02-10 20:50 ` Nicolin Chen
2023-02-10 2:11 ` Tian, Kevin
2023-02-11 0:10 ` Nicolin Chen
2023-02-13 2:34 ` Tian, Kevin
2023-02-13 7:48 ` Nicolin Chen
2023-02-13 8:27 ` Tian, Kevin
2023-02-13 14:49 ` Jason Gunthorpe
2023-02-14 10:54 ` Nicolin Chen
2023-02-15 1:37 ` Tian, Kevin
2023-02-15 1:58 ` Nicolin Chen
2023-02-15 2:15 ` Tian, Kevin
2023-02-15 7:15 ` Nicolin Chen
2023-02-15 7:24 ` Tian, Kevin
2023-02-15 12:51 ` Jason Gunthorpe
2023-02-14 10:59 ` Nicolin Chen [this message]
2023-02-15 1:38 ` Tian, Kevin
2023-02-15 7:16 ` Nicolin Chen
2023-02-07 21:18 ` [PATCH v2 09/10] vfio: Support IO page table replacement Nicolin Chen
2023-02-09 4:06 ` Tian, Kevin
2023-02-07 21:18 ` [PATCH v2 10/10] vfio: Do not allow !ops->dma_unmap in vfio_pin/unpin_pages() Nicolin Chen
2023-02-09 4:10 ` Tian, Kevin
2023-02-09 13:26 ` Jason Gunthorpe
2023-02-09 16:19 ` Nicolin Chen
2023-02-09 2:50 ` [PATCH v2 00/10] Add IO page table replacement support Tian, Kevin
2023-02-09 16:13 ` Nicolin Chen
2023-02-10 1:34 ` Tian, Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=Y+tpkpNYil3duTIP@Asurada-Nvidia \
--to=nicolinc@nvidia.com \
--cc=alex.williamson@redhat.com \
--cc=baolu.lu@linux.intel.com \
--cc=iommu@lists.linux.dev \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-kselftest@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=shuah@kernel.org \
--cc=will@kernel.org \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).