RE: [PATCH v1 1/5] iommufd: Create access in vfio_iommufd_emulated_bind()

From: "Tian, Kevin" <kevin.tian@intel.com>
To: Nicolin Chen <nicolinc@nvidia.com>
Cc: "mjrosato@linux.ibm.com" <mjrosato@linux.ibm.com>,
	"jasowang@redhat.com" <jasowang@redhat.com>,
	"Hao, Xudong" <xudong.hao@intel.com>,
	"peterx@redhat.com" <peterx@redhat.com>,
	"Xu, Terrence" <terrence.xu@intel.com>,
	"chao.p.peng@linux.intel.com" <chao.p.peng@linux.intel.com>,
	"linux-s390@vger.kernel.org" <linux-s390@vger.kernel.org>,
	"Liu, Yi L" <yi.l.liu@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	"lulu@redhat.com" <lulu@redhat.com>,
	"joro@8bytes.org" <joro@8bytes.org>,
	Jason Gunthorpe <jgg@nvidia.com>,
	"Zhao, Yan Y" <yan.y.zhao@intel.com>,
	"intel-gfx@lists.freedesktop.org"
	<intel-gfx@lists.freedesktop.org>,
	"eric.auger@redhat.com" <eric.auger@redhat.com>,
	"alex.williamson@redhat.com" <alex.williamson@redhat.com>,
	"intel-gvt-dev@lists.freedesktop.org" 
	<intel-gvt-dev@lists.freedesktop.org>,
	"yi.y.sun@linux.intel.com" <yi.y.sun@linux.intel.com>,
	"cohuck@redhat.com" <cohuck@redhat.com>,
	"shameerali.kolothum.thodi@huawei.com" 
	<shameerali.kolothum.thodi@huawei.com>,
	"suravee.suthikulpanit@amd.com" <suravee.suthikulpanit@amd.com>,
	"robin.murphy@arm.com" <robin.murphy@arm.com>
Subject: RE: [PATCH v1 1/5] iommufd: Create access in vfio_iommufd_emulated_bind()
Date: Wed, 15 Mar 2023 06:50:53 +0000	[thread overview]
Message-ID: <BN9PR11MB52765D7977F987960072482E8CBF9@BN9PR11MB5276.namprd11.prod.outlook.com> (raw)
In-Reply-To: <ZBFmh2iAqGGb8CjK@Asurada-Nvidia>

> From: Nicolin Chen
> Sent: Wednesday, March 15, 2023 2:33 PM
> 
> On Wed, Mar 15, 2023 at 06:15:23AM +0000, Tian, Kevin wrote:
> > External email: Use caution opening links or attachments
> >
> >
> > > From: Nicolin Chen <nicolinc@nvidia.com>
> > > Sent: Wednesday, March 15, 2023 9:01 AM
> > >
> > > Hi Jason/Kevin,
> > >
> > > >
> > > > Perhaps we can have iommufd_access_attach/detach in this series
> > > > along with a vfio_iommufd_emulated_detach_ioas, and the locking
> > > > will come with another patch in replace series?
> > >
> > > I recall that we previously concluded that the unbind() is safe
> > > to go without doing access->ops->unmap, because close_device()
> > > would be called prior to the unbind().
> > >
> > > But, to add the vfio_iommufd_emulated_detach_ioas() in the cdev
> > > series, we'd need the access->ops->unmap call, and the locking
> > > and "ioas_unpin" too in the detach and attach APIs, right?
> >
> > yes. We need locking since detach can happen any time with
> > cdev while driver is conducting pinning.
> 
> So, this preparatory series will add a pair of simple attach()
> and detach() APIs. Then the cdev series will add the locking
> and the ioas_unpin stuff as a rework of the detach() API.
> 
> > > I tried a bit of some update, across this series, cdev series,
> > > and the replace series. Though we might be able to simplify a
> > > bit of this patch/series, yet it doesn't seem to simplify the
> > > changes overall, particularly in the following cdev series for
> > > having an unmap() call and the locking.
> > >
> > > Then the replace API would mostly overlap with the attach API,
> > > by only having an additional detach(), which means actually we
> > > only need an iommufd_access_attach API to cover both cases...
> >
> > there is a subtle difference. to match the physical path:
> >
> > for attach we expect the existing access->ioas is either NULL or
> > same as the new ioas.
> >
> > for replace access->ioas must exist.
> >
> > they need different condition checks.
> 
> I think they can be something mingled... the sample code that
> I sent previously could take care of those conditions. But, I
> am also thinking a bit that maybe attach() does not need the
> locking? I can do a separate replace() function in this case.
> 

w/o locking then you need smp_store_release() and its pair.

anyway it's not in perf critical path. Keeping lock for attach
is simpler and safe.