Re: [PATCH v4 00/17] Add iommufd physical device operations for replace and alloc hwpt

From: Nicolin Chen <nicolinc@nvidia.com>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: <iommu@lists.linux.dev>, Kevin Tian <kevin.tian@intel.com>,
	<linux-kselftest@vger.kernel.org>,
	Lu Baolu <baolu.lu@linux.intel.com>, <kvm@vger.kernel.org>,
	Yi Liu <yi.l.liu@intel.com>
Subject: Re: [PATCH v4 00/17] Add iommufd physical device operations for replace and alloc hwpt
Date: Mon, 27 Mar 2023 13:05:39 -0700	[thread overview]
Message-ID: <ZCH3E+VnJilogZfi@Asurada-Nvidia> (raw)
In-Reply-To: <0-v4-9cd79ad52ee8+13f5-iommufd_alloc_jgg@nvidia.com>

On Mon, Mar 27, 2023 at 03:37:54PM -0300, Jason Gunthorpe wrote:
> This is the basic functionality for iommufd to support
> iommufd_device_replace() and IOMMU_HWPT_ALLOC for physical devices.
> 
> iommufd_device_replace() allows changing the HWPT associated with the
> device to a new IOAS or HWPT. Replace does this in way that failure leaves
> things unchanged, and utilizes the iommu iommu_group_replace_domain() API
> to allow the iommu driver to perform an optional non-disruptive change.
> 
> IOMMU_HWPT_ALLOC allows HWPTs to be explicitly allocated by the user and
> used by attach or replace. At this point it isn't very useful since the
> HWPT is the same as the automatically managed HWPT from the IOAS. However
> a following series will allow userspace to customize the created HWPT.
> 
> The implementation is complicated because we have to introduce some
> per-iommu_group memory in iommufd and redo how we think about multi-device
> groups to be more explicit. This solves all the locking problems in the
> prior attempts.
> 
> This series is infrastructure work for the following series which:
>  - Add replace for attach
>  - Expose replace through VFIO APIs
>  - Implement driver parameters for HWPT creation (nesting)
> 
> Once review of this is complete I will keep it on a side branch and
> accumulate the following series when they are ready so we can have a
> stable base and make more incremental progress. When we have all the parts
> together to get a full implementation it can go to Linus.
> 
> This is on github: https://github.com/jgunthorpe/linux/commits/iommufd_hwpt
> 
> v4:
>  - Refine comments and commit messages
>  - Move the group lock into iommufd_hw_pagetable_attach()
>  - Fix error unwind in iommufd_device_do_replace()

This runs well with iommufd selftest on x86 and QEMU sanity on
ARM64, applying nesting series on top of this and cdev series:
https://github.com/nicolinc/iommufd/commits/wip/iommufd_nesting-03272023

Tested-by: Nicolin Chen <nicolinc@nvidia.com>