RE: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops

From: "Tian, Kevin" <kevin.tian@intel.com>
To: Jacob Pan <jacob.jun.pan@linux.intel.com>
Cc: "Luck, Tony" <tony.luck@intel.com>,
	"Jiang, Dave" <dave.jiang@intel.com>,
	"Raj, Ashok" <ashok.raj@intel.com>,
	"Zanussi, Tom" <tom.zanussi@intel.com>,
	"Kumar, Sanjay K" <sanjay.k.kumar@intel.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Christoph Hellwig <hch@infradead.org>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
	Jason Gunthorpe <jgg@nvidia.com>,
	"Williams, Dan J" <dan.j.williams@intel.com>,
	Jean-Philippe Brucker <jean-philippe@linaro.com>
Subject: RE: [PATCH v2 3/8] iommu/vt-d: Implement device_pasid domain attach ops
Date: Fri, 18 Mar 2022 05:33:38 +0000	[thread overview]
Message-ID: <BN9PR11MB5276FF347A54098F469936978C139@BN9PR11MB5276.namprd11.prod.outlook.com> (raw)
In-Reply-To: <20220316140140.76bb24c6@jacob-builder>

> From: Jacob Pan
> Sent: Thursday, March 17, 2022 5:02 AM
> 
> Hi Kevin,
> 
> On Wed, 16 Mar 2022 07:41:34 +0000, "Tian, Kevin" <kevin.tian@intel.com>
> wrote:
> 
> > > From: Jason Gunthorpe <jgg@nvidia.com>
> > > Sent: Tuesday, March 15, 2022 10:33 PM
> > >
> > > On Mon, Mar 14, 2022 at 10:07:07PM -0700, Jacob Pan wrote:
> > > > +	/*
> > > > +	 * Each domain could have multiple devices attached with
> > > > shared or
> > > per
> > > > +	 * device PASIDs. At the domain level, we keep track of
> > > > unique PASIDs
> > > and
> > > > +	 * device user count.
> > > > +	 * E.g. If a domain has two devices attached, device A has
> > > > PASID 0, 1;
> > > > +	 * device B has PASID 0, 2. Then the domain would have PASID
> > > > 0, 1, 2.
> > > > +	 */
> > >
> > > A 2d array of xarray's seems like a poor data structure for this task.
> >
> Perhaps i mis-presented here, I am not using 2D array. It is an 1D xarray
> for domain PASIDs only. Then I use the existing device list in each domain,
> adding another xa to track per-device-domain PASIDs.
> > besides that it also doesn't work when we support per-device PASID
> > allocation in the future. In that case merging device PASIDs together is
> > conceptually wrong.
> >
> Sorry, could you elaborate? If we do per-dev PASID allocation, we could use
> the ioasid_set for each pdev, right?

My point is simply about the comment above which says the domain
will have PASID 0, 1, 2 when there is [devA, PASID0] and [devB, PASID0].
You can maintain a single  PASID list only when it's globally allocated cross
devices. otherwise this has to be a tuple including device and PASID.

Thanks
Kevin