Re: [PATCH v5 09/14] PCI: portdrv: Suppress kernel DMA ownership auto-claiming

From: Lu Baolu <baolu.lu@linux.intel.com>
To: Bjorn Helgaas <helgaas@kernel.org>
Cc: baolu.lu@linux.intel.com,
	Greg Kroah-Hartman <gregkh@linuxfoundation.org>,
	Joerg Roedel <joro@8bytes.org>,
	Alex Williamson <alex.williamson@redhat.com>,
	Bjorn Helgaas <bhelgaas@google.com>,
	Jason Gunthorpe <jgg@nvidia.com>,
	Christoph Hellwig <hch@infradead.org>,
	Kevin Tian <kevin.tian@intel.com>,
	Ashok Raj <ashok.raj@intel.com>, Will Deacon <will@kernel.org>,
	Robin Murphy <robin.murphy@arm.com>,
	Dan Williams <dan.j.williams@intel.com>,
	rafael@kernel.org, Diana Craciun <diana.craciun@oss.nxp.com>,
	Cornelia Huck <cohuck@redhat.com>,
	Eric Auger <eric.auger@redhat.com>, Liu Yi L <yi.l.liu@intel.com>,
	Jacob jun Pan <jacob.jun.pan@intel.com>,
	Chaitanya Kulkarni <kch@nvidia.com>,
	Stuart Yoder <stuyoder@gmail.com>,
	Laurentiu Tudor <laurentiu.tudor@nxp.com>,
	Thierry Reding <thierry.reding@gmail.com>,
	David Airlie <airlied@linux.ie>, Daniel Vetter <daniel@ffwll.ch>,
	Jonathan Hunter <jonathanh@nvidia.com>,
	Li Yang <leoyang.li@nxp.com>, Dmitry Osipenko <digetx@gmail.com>,
	iommu@lists.linux-foundation.org, linux-pci@vger.kernel.org,
	kvm@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: Re: [PATCH v5 09/14] PCI: portdrv: Suppress kernel DMA ownership auto-claiming
Date: Thu, 6 Jan 2022 12:12:35 +0800	[thread overview]
Message-ID: <039bbcf3-ccc6-f3b0-172e-9caa0866bb9e@linux.intel.com> (raw)
In-Reply-To: <20220104170631.GA99771@bhelgaas>

Hi Bjorn,

On 1/5/22 1:06 AM, Bjorn Helgaas wrote:
> On Tue, Jan 04, 2022 at 09:56:39AM +0800, Lu Baolu wrote:
>> If a switch lacks ACS P2P Request Redirect, a device below the switch can
>> bypass the IOMMU and DMA directly to other devices below the switch, so
>> all the downstream devices must be in the same IOMMU group as the switch
>> itself.
> Help me think through what's going on here.  IIUC, we put devices in
> the same IOMMU group when they can interfere with each other in any
> way (DMA, config access, etc).
> 
> (We said "DMA" above, but I guess this would also apply to config
> requests, right?)

I am not sure whether devices could interfere each other through config
space access. The IOMMU hardware only protects and isolates DMA
accesses, so that userspace could control DMA directly. The config
accesses will always be intercepted by VFIO. Hence, I don't see a
problem.

> 
> *This*  patch doesn't check for any ACS features.  Can you connect the
> dots for me?  I guess the presence or absence of P2P Request Redirect
> determines the size of the IOMMU group.  And the following says

It's done in iommu core (drivers/iommu/iommu.c):

/*
  * Use standard PCI bus topology, isolation features, and DMA alias quirks
  * to find or create an IOMMU group for a device.
  */
struct iommu_group *pci_device_group(struct device *dev)

> something about what is allowed in the group?  And .no_kernel_api_dma
> allows an exception to the general rule?
> 

Yes.

>> The pci_dma_configure() marks the iommu_group as containing only devices
>> with kernel drivers that manage DMA. Avoid this default behavior for the
>> portdrv driver in order for compatibility with the current vfio policy.
> I assume "IOMMU group" means the same as "iommu_group"; maybe we can
> use one of them consistently?

Sure.

Best regards,
baolu