From: Jason Gunthorpe <jgg@nvidia.com>
To: "Tian, Kevin" <kevin.tian@intel.com>
Cc: Baolu Lu <baolu.lu@linux.intel.com>,
Joerg Roedel <joro@8bytes.org>,
Christoph Hellwig <hch@infradead.org>,
"Raj, Ashok" <ashok.raj@intel.com>, Will Deacon <will@kernel.org>,
Robin Murphy <robin.murphy@arm.com>,
Jean-Philippe Brucker <jean-philippe@linaro.com>,
"Jiang, Dave" <dave.jiang@intel.com>,
Vinod Koul <vkoul@kernel.org>, Eric Auger <eric.auger@redhat.com>,
"Liu, Yi L" <yi.l.liu@intel.com>,
"Pan, Jacob jun" <jacob.jun.pan@intel.com>,
Zhangfei Gao <zhangfei.gao@linaro.org>,
"Zhu, Tony" <tony.zhu@intel.com>,
"iommu@lists.linux.dev" <iommu@lists.linux.dev>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
Jean-Philippe Brucker <jean-philippe@linaro.org>
Subject: Re: [PATCH v10 04/12] iommu: Add attach/detach_dev_pasid iommu interface
Date: Wed, 27 Jul 2022 08:53:39 -0300 [thread overview]
Message-ID: <20220727115339.GM4438@nvidia.com> (raw)
In-Reply-To: <BN9PR11MB5276974ABA5981A7361953708C979@BN9PR11MB5276.namprd11.prod.outlook.com>
On Wed, Jul 27, 2022 at 03:20:25AM +0000, Tian, Kevin wrote:
> > From: Jason Gunthorpe <jgg@nvidia.com>
> > Sent: Tuesday, July 26, 2022 9:57 PM
> >
> > On Tue, Jul 26, 2022 at 02:23:26PM +0800, Baolu Lu wrote:
> > > On 2022/7/25 22:40, Jason Gunthorpe wrote:
> > > > On Sun, Jul 24, 2022 at 03:03:16PM +0800, Baolu Lu wrote:
> > > >
> > > > > How about rephrasing this part of commit message like below:
> > > > >
> > > > > Some buses, like PCI, route packets without considering the PASID value.
> > > > > Thus a DMA target address with PASID might be treated as P2P if the
> > > > > address falls into the MMIO BAR of other devices in the group. To make
> > > > > things simple, these interfaces only apply to devices belonging to the
> > > > > singleton groups.
> > > >
> > > > > Considering that the PCI bus supports hot-plug, even a device boots
> > with
> > > > > a singleton group, a later hot-added device is still possible to share
> > > > > the group, which breaks the singleton group assumption. In order to
> > > > > avoid this situation, this interface requires that the ACS is enabled on
> > > > > all devices on the path from the device to the host-PCI bridge.
> > > >
> > > > But ACS directly fixes the routing issue above
> > > >
> > > > This entire explanation can be recast as saying we block PASID
> > > > attachment in all cases where the PCI fabric is routing based on
> > > > address. ACS disables that.
> > > >
> > > > Not sure it even has anything to do with hotplug or singleton??
> > >
> > > Yes, agreed. I polished this patch like below. Does it look good to you?
> > >
> > > iommu: Add attach/detach_dev_pasid iommu interface
> > >
> > > Attaching an IOMMU domain to a PASID of a device is a generic operation
> > > for modern IOMMU drivers which support PASID-granular DMA address
> > > translation. Currently visible usage scenarios include (but not limited):
> > >
> > > - SVA (Shared Virtual Address)
> > > - kernel DMA with PASID
> > > - hardware-assist mediated device
> > >
> > > This adds a pair of domain ops for this purpose and adds the interfaces
> > > for device drivers to attach/detach a domain to/from a {device,
> > > PASID}.
> >
> > > The PCI bus routes packets without considering the PASID value.
> >
> > More like:
> >
> > Some configurations of the PCI fabric will route device originated TLP
> > packets based on memory address, and these configurations are
> > incompatible with PASID as the PASID packets form a distinct address
> > space. For instance any configuration where switches are present
> > without ACS is incompatible with PASID.
>
> This description reads like ACS enables PASID-based routing...
Well, that is kind of what it is.
> In reality PCI fabric always route TLP based on memory address.
> ACS just provides a way to redirect the packet to RC, with or
> without PASID.
Always except in all the cases it doesn't, like ACS :)
> > > + * Block PASID attachment in all cases where the PCI fabric is
> > > + * routing based on address. ACS disables it.
> > > + */
> > > + if (dev_is_pci(dev) &&
> > > + !pci_acs_path_enabled(to_pci_dev(dev), NULL, REQ_ACS_FLAGS))
> > > + return -ENODEV;
> >
> > I would probably still put this in a function just to be clear, and
> > probably even a PCI layer funcion 'pci_is_pasid_supported' that
> > clearly indicates that the fabric path can route a PASID packet
> > without mis-routing it.
>
> But there is no single line in above check related to PASID...
The question to answer here is if the device/fabric supports PASID,
and on PCI that requires ACS on any switches. IMHO that is a PCI layer
question and perhaps we shouldn't even succeed pci_enable_pasid() if
ACS isn't on.
Then we don't need this weirdo check in the core iommu code at all.
Jason
next prev parent reply other threads:[~2022-07-27 11:53 UTC|newest]
Thread overview: 90+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-07-05 5:06 [PATCH v10 00/12] iommu: SVA and IOPF refactoring Lu Baolu
2022-07-05 5:06 ` [PATCH v10 01/12] iommu: Add max_pasids field in struct iommu_device Lu Baolu
2022-07-23 13:57 ` Jason Gunthorpe
2022-07-31 11:54 ` Yi Liu
2022-07-31 13:33 ` Baolu Lu
2022-07-05 5:07 ` [PATCH v10 02/12] iommu: Add max_pasids field in struct dev_iommu Lu Baolu
2022-07-23 14:02 ` Jason Gunthorpe
2022-07-31 11:55 ` Yi Liu
2022-07-05 5:07 ` [PATCH v10 03/12] iommu: Remove SVM_FLAG_SUPERVISOR_MODE support Lu Baolu
2022-07-31 12:01 ` Yi Liu
2022-07-31 13:39 ` Baolu Lu
2022-07-05 5:07 ` [PATCH v10 04/12] iommu: Add attach/detach_dev_pasid iommu interface Lu Baolu
2022-07-07 1:51 ` Tian, Kevin
2022-07-23 14:11 ` Jason Gunthorpe
2022-07-24 7:03 ` Baolu Lu
2022-07-25 14:40 ` Jason Gunthorpe
2022-07-26 6:23 ` Baolu Lu
2022-07-26 13:57 ` Jason Gunthorpe
2022-07-27 3:20 ` Tian, Kevin
2022-07-27 11:53 ` Jason Gunthorpe [this message]
2022-07-28 3:06 ` Tian, Kevin
2022-07-28 11:59 ` Jason Gunthorpe
2022-07-29 2:49 ` Baolu Lu
2022-07-29 2:56 ` Tian, Kevin
2022-07-29 3:20 ` Baolu Lu
2022-07-29 4:22 ` Tian, Kevin
2022-07-30 6:17 ` Baolu Lu
2022-07-29 2:51 ` Tian, Kevin
2022-07-29 12:22 ` Jason Gunthorpe
2022-07-30 6:23 ` Baolu Lu
2022-07-28 2:44 ` Baolu Lu
2022-07-28 6:27 ` Baolu Lu
2022-08-02 2:19 ` Baolu Lu
2022-08-02 12:37 ` Jason Gunthorpe
2022-08-03 13:07 ` Baolu Lu
2022-08-03 19:03 ` Jason Gunthorpe
2022-08-04 2:03 ` Tian, Kevin
2022-08-04 2:42 ` Baolu Lu
2022-07-24 7:23 ` Baolu Lu
2022-07-24 8:39 ` Baolu Lu
2022-07-24 9:13 ` Baolu Lu
2022-07-25 7:46 ` Tian, Kevin
2022-07-25 10:11 ` Baolu Lu
2022-07-31 12:10 ` Yi Liu
2022-07-05 5:07 ` [PATCH v10 05/12] iommu: Add IOMMU SVA domain support Lu Baolu
2022-07-07 1:52 ` Tian, Kevin
2022-07-07 3:01 ` Baolu Lu
2022-07-23 14:15 ` Jason Gunthorpe
2022-07-31 12:19 ` Yi Liu
2022-07-05 5:07 ` [PATCH v10 06/12] iommu/vt-d: Add " Lu Baolu
2022-07-23 14:16 ` Jason Gunthorpe
2022-07-31 12:20 ` Yi Liu
2022-07-05 5:07 ` [PATCH v10 07/12] arm-smmu-v3/sva: " Lu Baolu
2022-07-23 14:20 ` Jason Gunthorpe
2022-07-24 11:58 ` Baolu Lu
2022-07-05 5:07 ` [PATCH v10 08/12] iommu/sva: Refactoring iommu_sva_bind/unbind_device() Lu Baolu
2022-07-05 17:43 ` Jean-Philippe Brucker
2022-07-07 1:56 ` Tian, Kevin
2022-07-07 3:02 ` Baolu Lu
2022-07-23 14:26 ` Jason Gunthorpe
2022-07-24 13:48 ` Baolu Lu
2022-07-25 7:39 ` Jean-Philippe Brucker
2022-07-25 8:02 ` Tian, Kevin
2022-07-25 8:47 ` Jean-Philippe Brucker
2022-07-25 9:33 ` Baolu Lu
2022-07-25 9:52 ` Jean-Philippe Brucker
2022-07-25 14:46 ` Jason Gunthorpe
2022-07-25 7:50 ` Tian, Kevin
2022-07-25 10:22 ` Baolu Lu
2022-07-25 14:47 ` Jason Gunthorpe
2022-07-26 8:47 ` Baolu Lu
2022-07-31 12:36 ` Yi Liu
2022-07-31 13:45 ` Baolu Lu
2022-07-31 12:55 ` Yi Liu
2022-07-31 13:51 ` Baolu Lu
2022-07-05 5:07 ` [PATCH v10 09/12] iommu: Remove SVA related callbacks from iommu ops Lu Baolu
2022-07-23 14:27 ` Jason Gunthorpe
2022-07-31 12:38 ` Yi Liu
2022-07-05 5:07 ` [PATCH v10 10/12] iommu: Prepare IOMMU domain for IOPF Lu Baolu
2022-07-07 1:58 ` Tian, Kevin
2022-07-23 14:33 ` Jason Gunthorpe
2022-07-24 14:04 ` Baolu Lu
2022-07-25 14:49 ` Jason Gunthorpe
2022-07-31 12:50 ` Yi Liu
2022-07-31 13:47 ` Baolu Lu
2022-07-05 5:07 ` [PATCH v10 11/12] iommu: Per-domain I/O page fault handling Lu Baolu
2022-07-07 2:06 ` Tian, Kevin
2022-07-23 14:34 ` Jason Gunthorpe
2022-07-05 5:07 ` [PATCH v10 12/12] iommu: Rename iommu-sva-lib.{c,h} Lu Baolu
2022-07-23 14:34 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20220727115339.GM4438@nvidia.com \
--to=jgg@nvidia.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@linux.intel.com \
--cc=dave.jiang@intel.com \
--cc=eric.auger@redhat.com \
--cc=hch@infradead.org \
--cc=iommu@lists.linux.dev \
--cc=jacob.jun.pan@intel.com \
--cc=jean-philippe@linaro.com \
--cc=jean-philippe@linaro.org \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=linux-kernel@vger.kernel.org \
--cc=robin.murphy@arm.com \
--cc=tony.zhu@intel.com \
--cc=vkoul@kernel.org \
--cc=will@kernel.org \
--cc=yi.l.liu@intel.com \
--cc=zhangfei.gao@linaro.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).