Re: [PATCH v4 10/17] iommufd: Support IOMMU_HWPT_ALLOC allocation with user data

From: Jason Gunthorpe <jgg@nvidia.com>
To: Nicolin Chen <nicolinc@nvidia.com>
Cc: Yi Liu <yi.l.liu@intel.com>,
	joro@8bytes.org, alex.williamson@redhat.com,
	kevin.tian@intel.com, robin.murphy@arm.com,
	baolu.lu@linux.intel.com, cohuck@redhat.com,
	eric.auger@redhat.com, kvm@vger.kernel.org,
	mjrosato@linux.ibm.com, chao.p.peng@linux.intel.com,
	yi.y.sun@linux.intel.com, peterx@redhat.com, jasowang@redhat.com,
	shameerali.kolothum.thodi@huawei.com, lulu@redhat.com,
	suravee.suthikulpanit@amd.com, iommu@lists.linux.dev,
	linux-kernel@vger.kernel.org, linux-kselftest@vger.kernel.org,
	zhenzhong.duan@intel.com, joao.m.martins@oracle.com
Subject: Re: [PATCH v4 10/17] iommufd: Support IOMMU_HWPT_ALLOC allocation with user data
Date: Fri, 13 Oct 2023 21:07:09 -0300	[thread overview]
Message-ID: <20231014000709.GL3952@nvidia.com> (raw)
In-Reply-To: <ZSmvkxuEq7M13KYE@Asurada-Nvidia>

On Fri, Oct 13, 2023 at 01:58:59PM -0700, Nicolin Chen wrote:
> On Fri, Oct 13, 2023 at 12:19:23PM -0300, Jason Gunthorpe wrote:
> > On Thu, Sep 21, 2023 at 12:51:31AM -0700, Yi Liu wrote:
> > > IOMMU_HWPT_ALLOC already supports iommu_domain allocation for usersapce.
> > > But it can only allocate a hw_pagetable that associates to a given IOAS,
> > > i.e. only a kernel-managed hw_pagetable of IOMMU_HWPT_TYPE_DEFAULT type.
> > > 
> > > IOMMU drivers can now support user-managed hw_pagetables, for two-stage
> > > translation use cases, that require user data input from the user space.
> > > 
> > > Extend the IOMMU_HWPT_ALLOC ioctl to accept non-default hwpt_type with a
> > > type specified user data. Also, update the @pt_id to accept hwpt_id too
> > > besides an ioas_id. Then, pass them to the downstream alloc_fn().
> > > 
> > > Co-developed-by: Nicolin Chen <nicolinc@nvidia.com>
> > > Signed-off-by: Nicolin Chen <nicolinc@nvidia.com>
> > > Signed-off-by: Yi Liu <yi.l.liu@intel.com>
> > > ---
> > >  drivers/iommu/iommufd/hw_pagetable.c | 19 ++++++++++++++++++-
> > >  include/uapi/linux/iommufd.h         | 23 +++++++++++++++++++++--
> > >  2 files changed, 39 insertions(+), 3 deletions(-)
> > 
> > Can we also come with a small vt-d patch that does implement an op for
> > this? Or is it too big?
> > 
> > It would be nice if we could wrap IOMMU_HWPT_ALLOC into one
> > self-contained series and another series for invalidate.
> 
> We now only use IOMMU_HWPT_ALLOC for nested domain allocations,
> which won't be supported until the cache_invalidate_user ops is
> preset?
> 
> /* e.g. the following piece is in iommufd_user_managed_hwpt_alloc */
> 
> +       /* Driver is buggy by missing cache_invalidate_user in domain_ops */
> +       if (WARN_ON_ONCE(!hwpt->domain->ops->cache_invalidate_user)) {
> +               rc = -EINVAL;
> +               goto out_abort;
> +       }
>

Hm. That hunk could migrate to the invalidate series. 

I'm just leeary of doing the invalidation too considering how
complicated it is

Jason