Re: [PATCH 3/6] vfio: remove the unused mdev iommu hook

From: Liu Yi L <yi.l.liu@linux.intel.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: " <iommu@lists.linux-foundation.org>,
	"@osuosl.org, " <kvm@vger.kernel.org>"@osuosl.org, "Tian,
	Kevin" <kevin.tian@intel.com>,
	kvm@vger.kernel.org, Will Deacon <will@kernel.org>,
	Kirti Wankhede <kwankhede@nvidia.com>,
	iommu@lists.linux-foundation.org,
	Alex Williamson <alex.williamson@redhat.com>,
	"linux-arm-kernel@lists.infradead.org\"
	<linux-arm-kernel@lists.infradead.org>,
	"@osuosl.org, David Woodhouse <dwmw2@infradead.org>,
	Christoph Hellwig <hch@lst.de>
Subject: Re: [PATCH 3/6] vfio: remove the unused mdev iommu hook
Date: Fri, 14 May 2021 22:08:45 +0800	[thread overview]
Message-ID: <20210514220845.6cd053c3@yiliu-dev> (raw)
In-Reply-To: <20210514133939.GL1096940@ziepe.ca>

Morning Jason,

On Fri, 14 May 2021 10:39:39 -0300, Jason Gunthorpe wrote:

> On Fri, May 14, 2021 at 01:17:23PM +0000, Tian, Kevin wrote:
> > > From: Jason Gunthorpe <jgg@ziepe.ca>
> > > Sent: Thursday, May 13, 2021 8:01 PM
> > > 
> > > On Thu, May 13, 2021 at 03:28:52AM +0000, Tian, Kevin wrote:
> > >   
> > > > Are you specially concerned about this iommu_device hack which
> > > > directly connects mdev_device to iommu layer or the entire removed
> > > > logic including the aux domain concept? For the former we are now
> > > > following up the referred thread to find a clean way. But for the latter
> > > > we feel it's still necessary regardless of how iommu interface is redesigned
> > > > to support device connection from the upper level driver. The reason is
> > > > that with mdev or subdevice one physical device could be attached to
> > > > multiple domains now. there could be a primary domain with DOMAIN_
> > > > DMA type for DMA_API use by parent driver itself, and multiple auxiliary
> > > > domains with DOMAIN_UNMANAGED types for subdevices assigned to
> > > > different VMs.  
> > > 
> > > Why do we need more domains than just the physical domain for the
> > > parent? How does auxdomain appear in /dev/ioasid?
> > >   
> > 
> > Another simple reason. Say you have 4 mdevs each from a different 
> > parent are attached to an ioasid. If only using physical domain of the 
> > parent + PASID it means there are 4 domains (thus 4 page tables) under 
> > this IOASID thus every dma map operation must be replicated in all
> > 4 domains which is really unnecessary. Having the domain created
> > with ioasid and allow a device attaching to multiple domains is much
> > cleaner for the upper-layer drivers to work with iommu interface.  
> 
> Eh? That sounds messed up.
> 
> The IOASID is the page table. If you have one IOASID and you attach it
> to 4 IOMMU routings (be it pasid, rid, whatever) then there should
> only ever by one page table.

yes, ioasid is the page table. But if want to let the 4 mdevs share the
same page table, it would be natural to let them share a domain. Since
mdev_device is not hw device, we should not let it participate in the
IOMMU. Therefore we got the aux-domain concept. mdev(RID#+PASID) is
attached to aux-domain. Such solution also fits the hybrid cases. e.g.
When there are both PF(RID#1) and mdev(RID#2+PASID) assigned to an ioasid,
they should share a page table as well. right? Surely we cannot attach the
PF(RID#1) to the domain of mdev's parent device(RID#2). Good way is PF(RID#1)
and the mdev (RID#2+PASID) attached to a single domain. This domain is
the primary domain for the PF(RID#1) but an aux-domain mdev's paretn(RID#2).

-- 
Regards,
Yi Liu
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu