From: David Gibson <david@gibson.dropbear.id.au>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Liu Yi L <yi.l.liu@intel.com>,
alex.williamson@redhat.com, hch@lst.de, jasowang@redhat.com,
joro@8bytes.org, jean-philippe@linaro.org, kevin.tian@intel.com,
parav@mellanox.com, lkml@metux.net, pbonzini@redhat.com,
lushenming@huawei.com, eric.auger@redhat.com, corbet@lwn.net,
ashok.raj@intel.com, yi.l.liu@linux.intel.com,
jun.j.tian@intel.com, hao.wu@intel.com, dave.jiang@intel.com,
jacob.jun.pan@linux.intel.com, kwankhede@nvidia.com,
robin.murphy@arm.com, kvm@vger.kernel.org,
iommu@lists.linux-foundation.org, dwmw2@infradead.org,
linux-kernel@vger.kernel.org, baolu.lu@linux.intel.com,
nicolinc@nvidia.com
Subject: Re: [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE
Date: Mon, 18 Oct 2021 14:50:54 +1100 [thread overview]
Message-ID: <YWzvHsm7YMYS2sP3@yekko> (raw)
In-Reply-To: <20211014145208.GR2744544@nvidia.com>
[-- Attachment #1: Type: text/plain, Size: 4816 bytes --]
On Thu, Oct 14, 2021 at 11:52:08AM -0300, Jason Gunthorpe wrote:
> On Thu, Oct 14, 2021 at 03:53:33PM +1100, David Gibson wrote:
>
> > > My feeling is that qemu should be dealing with the host != target
> > > case, not the kernel.
> > >
> > > The kernel's job should be to expose the IOMMU HW it has, with all
> > > features accessible, to userspace.
> >
> > See... to me this is contrary to the point we agreed on above.
>
> I'm not thinking of these as exclusive ideas.
>
> The IOCTL interface in iommu can quite happily expose:
> Create IOAS generically
> Manipulate IOAS generically
> Create IOAS with IOMMU driver specific attributes
> HW specific Manipulate IOAS
>
> IOCTL commands all together.
>
> So long as everything is focused on a generic in-kernel IOAS object it
> is fine to have multiple ways in the uAPI to create and manipulate the
> objects.
>
> When I speak about a generic interface I mean "Create IOAS
> generically" - ie a set of IOCTLs that work on most IOMMU HW and can
> be relied upon by things like DPDK/etc to always work and be portable.
> This is why I like "hints" to provide some limited widely applicable
> micro-optimization.
>
> When I said "expose the IOMMU HW it has with all features accessible"
> I mean also providing "Create IOAS with IOMMU driver specific
> attributes".
>
> These other IOCTLs would allow the IOMMU driver to expose every
> configuration knob its HW has, in a natural HW centric language.
> There is no pretense of genericness here, no crazy foo=A, foo=B hidden
> device specific interface.
>
> Think of it as a high level/low level interface to the same thing.
Ok, I see what you mean.
> > Those are certainly wrong, but they came about explicitly by *not*
> > being generic rather than by being too generic. So I'm really
> > confused aso to what you're arguing for / against.
>
> IMHO it is not having a PPC specific interface that was the problem,
> it was making the PPC specific interface exclusive to the type 1
> interface. If type 1 continued to work on PPC then DPDK/etc would
> never learned PPC specific code.
Ok, but the reason this happened is that the initial version of type 1
*could not* be used on PPC. The original Type 1 implicitly promised a
"large" IOVA range beginning at IOVA 0 without any real way of
specifying or discovering how large that range was. Since ppc could
typically only give a 2GiB range at IOVA 0, that wasn't usable.
That's why I say the problem was not making type1 generic enough. I
believe the current version of Type1 has addressed this - at least
enough to be usable in common cases. But by this time the ppc backend
is already out there, so no-one's had the capacity to go back and make
ppc work with Type1.
> For iommufd with the high/low interface each IOMMU HW should ask basic
> questions:
>
> - What should the generic high level interface do on this HW?
> For instance what should 'Create IOAS generically' do for PPC?
> It should not fail, it should create *something*
> What is the best thing for DPDK?
> I guess the 64 bit window is most broadly useful.
Right, which means the kernel must (at least in the common case) have
the capcity to choose and report a non-zero base-IOVA.
Hrm... which makes me think... if we allow this for the common
kernel-managed case, do we even need to have capcity in the high-level
interface for reporting IO holes? If the kernel can choose a non-zero
base, it could just choose on x86 to place it's advertised window
above the IO hole.
> - How to accurately describe the HW in terms of standard IOAS objects
> and where to put HW specific structs to support this.
>
> This is where PPC would decide how best to expose a control over
> its low/high window (eg 1,2,3 IOAS). Whatever the IOMMU driver
> wants, so long as it fits into the kernel IOAS model facing the
> connected device driver.
>
> QEMU would have IOMMU userspace drivers. One would be the "generic
> driver" using only the high level generic interface. It should work as
> best it can on all HW devices. This is the fallback path you talked
> of.
>
> QEMU would also have HW specific IOMMU userspace drivers that know how
> to operate the exact HW. eg these drivers would know how to use
> userspace page tables, how to form IOPTEs and how to access the
> special features.
>
> This is how QEMU could use an optimzed path with nested page tables,
> for instance.
The concept makes sense in general. The devil's in the details, as usual.
--
David Gibson | I'll have my music baroque, and my code
david AT gibson.dropbear.id.au | minimalist, thank you. NOT _the_ _other_
| _way_ _around_!
http://www.ozlabs.org/~dgibson
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
next prev parent reply other threads:[~2021-10-18 4:14 UTC|newest]
Thread overview: 274+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-19 6:38 [RFC 00/20] Introduce /dev/iommu for userspace I/O address space management Liu Yi L
2021-09-19 6:38 ` [RFC 01/20] iommu/iommufd: Add /dev/iommu core Liu Yi L
2021-09-21 15:41 ` Jason Gunthorpe
2021-09-22 1:51 ` Tian, Kevin
2021-09-22 12:40 ` Jason Gunthorpe
2021-09-22 13:59 ` Tian, Kevin
2021-09-22 14:10 ` Jason Gunthorpe
2021-10-15 9:18 ` Liu, Yi L
2021-10-15 11:18 ` Jason Gunthorpe
2021-10-15 11:29 ` Liu, Yi L
2021-10-19 16:57 ` Jacob Pan
2021-10-19 16:57 ` Jason Gunthorpe
2021-10-19 17:11 ` Jacob Pan
2021-10-19 17:12 ` Jason Gunthorpe
2021-09-19 6:38 ` [RFC 02/20] vfio: Add device class for /dev/vfio/devices Liu Yi L
2021-09-21 15:57 ` Jason Gunthorpe
2021-09-21 23:56 ` Tian, Kevin
2021-09-22 0:55 ` Jason Gunthorpe
2021-09-22 1:07 ` Tian, Kevin
2021-09-22 12:31 ` Jason Gunthorpe
2021-09-22 3:22 ` Tian, Kevin
2021-09-22 12:50 ` Jason Gunthorpe
2021-09-22 14:09 ` Tian, Kevin
2021-09-21 19:56 ` Alex Williamson
2021-09-22 0:56 ` Tian, Kevin
2021-09-29 2:08 ` David Gibson
2021-09-29 19:05 ` Alex Williamson
2021-09-30 2:43 ` David Gibson
2021-10-20 12:39 ` Liu, Yi L
2021-09-19 6:38 ` [RFC 03/20] vfio: Add vfio_[un]register_device() Liu Yi L
2021-09-21 16:01 ` Jason Gunthorpe
2021-09-21 23:10 ` Tian, Kevin
2021-09-22 0:53 ` Jason Gunthorpe
2021-09-22 0:59 ` Tian, Kevin
2021-09-22 9:23 ` Tian, Kevin
2021-09-22 12:22 ` Jason Gunthorpe
2021-09-22 13:44 ` Tian, Kevin
2021-09-22 20:10 ` Alex Williamson
2021-09-22 22:34 ` Tian, Kevin
2021-09-22 22:45 ` Alex Williamson
2021-09-22 23:45 ` Tian, Kevin
2021-09-22 23:52 ` Jason Gunthorpe
2021-09-23 0:38 ` Tian, Kevin
2021-09-22 23:56 ` Jason Gunthorpe
2021-09-22 0:54 ` Tian, Kevin
2021-09-22 1:00 ` Jason Gunthorpe
2021-09-22 1:02 ` Tian, Kevin
2021-09-23 7:25 ` Eric Auger
2021-09-23 11:44 ` Jason Gunthorpe
2021-09-29 2:46 ` david
2021-09-29 12:22 ` Jason Gunthorpe
2021-09-30 2:48 ` david
2021-09-29 2:43 ` David Gibson
2021-09-29 3:40 ` Tian, Kevin
2021-09-29 5:30 ` Tian, Kevin
2021-09-29 7:08 ` Cornelia Huck
2021-09-29 12:15 ` Jason Gunthorpe
2021-09-19 6:38 ` [RFC 04/20] iommu: Add iommu_device_get_info interface Liu Yi L
2021-09-21 16:19 ` Jason Gunthorpe
2021-09-22 2:31 ` Lu Baolu
2021-09-22 5:07 ` Christoph Hellwig
2021-09-29 2:52 ` David Gibson
2021-09-29 9:25 ` Lu Baolu
2021-09-29 9:29 ` Lu Baolu
2021-09-19 6:38 ` [RFC 05/20] vfio/pci: Register device to /dev/vfio/devices Liu Yi L
2021-09-21 16:40 ` Jason Gunthorpe
2021-09-21 21:09 ` Alex Williamson
2021-09-21 21:58 ` Jason Gunthorpe
2021-09-22 1:24 ` Tian, Kevin
2021-09-22 1:19 ` Tian, Kevin
2021-09-22 21:17 ` Alex Williamson
2021-09-22 23:49 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 06/20] iommu: Add iommu_device_init[exit]_user_dma interfaces Liu Yi L
2021-09-21 17:09 ` Jason Gunthorpe
2021-09-22 1:47 ` Tian, Kevin
2021-09-22 12:39 ` Jason Gunthorpe
2021-09-22 13:56 ` Tian, Kevin
2021-09-27 9:42 ` Tian, Kevin
2021-09-27 11:34 ` Lu Baolu
2021-09-27 13:08 ` Tian, Kevin
2021-09-27 11:53 ` Jason Gunthorpe
2021-09-27 13:00 ` Tian, Kevin
2021-09-27 13:09 ` Jason Gunthorpe
2021-09-27 13:32 ` Tian, Kevin
2021-09-27 14:39 ` Jason Gunthorpe
2021-09-28 7:13 ` Tian, Kevin
2021-09-28 11:54 ` Jason Gunthorpe
2021-09-28 23:59 ` Tian, Kevin
2021-09-27 19:19 ` Alex Williamson
2021-09-28 7:43 ` Tian, Kevin
2021-09-28 16:26 ` Alex Williamson
2021-09-27 15:09 ` Jason Gunthorpe
2021-09-28 7:30 ` Tian, Kevin
2021-09-28 11:57 ` Jason Gunthorpe
2021-09-28 13:35 ` Lu Baolu
2021-09-28 14:07 ` Jason Gunthorpe
2021-09-29 0:38 ` Tian, Kevin
2021-09-29 12:59 ` Jason Gunthorpe
2021-10-15 1:29 ` Tian, Kevin
2021-10-15 11:09 ` Jason Gunthorpe
2021-10-18 1:52 ` Tian, Kevin
2021-09-29 2:22 ` Lu Baolu
2021-09-29 2:29 ` Tian, Kevin
2021-09-29 2:38 ` Lu Baolu
2021-09-29 4:55 ` David Gibson
2021-09-29 5:38 ` Tian, Kevin
2021-09-29 6:35 ` David Gibson
2021-09-29 7:31 ` Tian, Kevin
2021-09-30 3:05 ` David Gibson
2021-09-29 12:57 ` Jason Gunthorpe
2021-09-30 3:09 ` David Gibson
2021-09-30 22:28 ` Jason Gunthorpe
2021-10-01 3:54 ` David Gibson
2021-09-19 6:38 ` [RFC 07/20] iommu/iommufd: Add iommufd_[un]bind_device() Liu Yi L
2021-09-21 17:14 ` Jason Gunthorpe
2021-10-15 9:21 ` Liu, Yi L
2021-09-29 5:25 ` David Gibson
2021-09-29 12:24 ` Jason Gunthorpe
2021-09-30 3:10 ` David Gibson
2021-10-01 12:43 ` Jason Gunthorpe
2021-10-07 1:23 ` David Gibson
2021-10-07 11:35 ` Jason Gunthorpe
2021-10-11 3:24 ` David Gibson
2021-09-19 6:38 ` [RFC 08/20] vfio/pci: Add VFIO_DEVICE_BIND_IOMMUFD Liu Yi L
2021-09-21 17:29 ` Jason Gunthorpe
2021-09-22 21:01 ` Alex Williamson
2021-09-22 23:01 ` Jason Gunthorpe
2021-09-29 6:00 ` David Gibson
2021-09-29 6:41 ` Tian, Kevin
2021-09-29 12:28 ` Jason Gunthorpe
2021-09-29 22:34 ` Tian, Kevin
2021-09-30 3:12 ` David Gibson
2021-09-19 6:38 ` [RFC 09/20] iommu: Add page size and address width attributes Liu Yi L
2021-09-22 13:42 ` Eric Auger
2021-09-22 14:19 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 10/20] iommu/iommufd: Add IOMMU_DEVICE_GET_INFO Liu Yi L
2021-09-21 17:40 ` Jason Gunthorpe
2021-09-22 3:30 ` Tian, Kevin
2021-09-22 12:41 ` Jason Gunthorpe
2021-09-29 6:18 ` david
2021-09-22 21:24 ` Alex Williamson
2021-09-22 23:49 ` Jason Gunthorpe
2021-09-23 3:10 ` Tian, Kevin
2021-09-23 10:15 ` Jean-Philippe Brucker
2021-09-23 11:27 ` Jason Gunthorpe
2021-09-23 12:05 ` Tian, Kevin
2021-09-23 12:22 ` Jason Gunthorpe
2021-09-29 8:48 ` Tian, Kevin
2021-09-29 12:36 ` Jason Gunthorpe
2021-09-30 8:30 ` Tian, Kevin
2021-09-30 10:33 ` Jean-Philippe Brucker
2021-09-30 22:04 ` Jason Gunthorpe
2021-10-01 3:28 ` hch
2021-10-14 8:13 ` Tian, Kevin
2021-10-14 8:22 ` hch
2021-10-14 8:29 ` Tian, Kevin
2021-10-14 8:01 ` Tian, Kevin
2021-10-14 9:16 ` Jean-Philippe Brucker
2021-09-30 8:49 ` Tian, Kevin
2021-09-30 13:43 ` Lu Baolu
2021-10-01 3:24 ` hch
2021-09-30 22:08 ` Jason Gunthorpe
2021-09-23 11:36 ` Jason Gunthorpe
[not found] ` <BN9PR11MB5433409DF766AAEF1BB2CF258CA39@BN9PR11MB5433.namprd11.prod.outlook.com>
2021-09-23 3:38 ` Tian, Kevin
2021-09-23 11:42 ` Jason Gunthorpe
2021-09-30 9:35 ` Tian, Kevin
2021-09-30 22:23 ` Jason Gunthorpe
2021-10-01 3:30 ` hch
2021-10-14 9:11 ` Tian, Kevin
2021-10-14 15:42 ` Jason Gunthorpe
2021-10-15 1:01 ` Tian, Kevin
[not found] ` <BN9PR11MB543327BB6D58AEF91AD2C9D18CB99@BN9PR11MB5433.namprd11.prod.outlook.com>
2021-10-21 2:26 ` Tian, Kevin
2021-10-21 14:58 ` Jean-Philippe Brucker
2021-10-21 23:22 ` Jason Gunthorpe
2021-10-22 7:49 ` Jean-Philippe Brucker
2021-10-25 16:51 ` Jason Gunthorpe
2021-10-21 23:30 ` Jason Gunthorpe
2021-10-22 3:08 ` Tian, Kevin
2021-10-25 23:34 ` Jason Gunthorpe
2021-10-27 1:42 ` Tian, Kevin
2021-10-28 2:07 ` Tian, Kevin
2021-10-29 13:55 ` Jason Gunthorpe
2021-09-29 6:23 ` David Gibson
2021-09-19 6:38 ` [RFC 11/20] iommu/iommufd: Add IOMMU_IOASID_ALLOC/FREE Liu Yi L
2021-09-21 17:44 ` Jason Gunthorpe
2021-09-22 3:40 ` Tian, Kevin
2021-09-22 14:09 ` Jason Gunthorpe
2021-09-23 9:14 ` Tian, Kevin
2021-09-23 12:06 ` Jason Gunthorpe
2021-09-23 12:22 ` Tian, Kevin
2021-09-23 12:31 ` Jason Gunthorpe
2021-09-23 12:45 ` Tian, Kevin
2021-09-23 13:01 ` Jason Gunthorpe
2021-09-23 13:20 ` Tian, Kevin
2021-09-23 13:30 ` Jason Gunthorpe
2021-09-23 13:41 ` Tian, Kevin
2021-10-01 6:30 ` david
2021-10-01 6:26 ` david
2021-10-01 6:19 ` david
2021-10-01 12:25 ` Jason Gunthorpe
2021-10-02 4:21 ` david
2021-10-02 12:25 ` Jason Gunthorpe
2021-10-11 5:37 ` david
2021-10-11 17:17 ` Jason Gunthorpe
2021-10-14 4:33 ` david
2021-10-14 15:06 ` Jason Gunthorpe
2021-10-18 3:40 ` david
2021-10-01 6:15 ` david
2021-09-22 12:51 ` Liu, Yi L
2021-09-22 13:32 ` Jason Gunthorpe
2021-09-23 6:26 ` Liu, Yi L
2021-10-01 6:13 ` David Gibson
2021-10-01 12:22 ` Jason Gunthorpe
2021-10-11 6:02 ` David Gibson
2021-10-11 8:49 ` Jean-Philippe Brucker
2021-10-11 23:38 ` Jason Gunthorpe
2021-10-12 8:33 ` Jean-Philippe Brucker
2021-10-13 7:14 ` Tian, Kevin
2021-10-13 7:07 ` Tian, Kevin
2021-10-14 4:38 ` David Gibson
2021-10-11 18:49 ` Jason Gunthorpe
2021-10-14 4:53 ` David Gibson
2021-10-14 14:52 ` Jason Gunthorpe
2021-10-18 3:50 ` David Gibson [this message]
2021-10-18 17:42 ` Jason Gunthorpe
2021-09-22 13:45 ` Jean-Philippe Brucker
2021-09-29 10:47 ` Liu, Yi L
2021-10-01 6:11 ` David Gibson
2021-10-13 7:00 ` Tian, Kevin
2021-10-14 5:00 ` David Gibson
2021-10-14 6:53 ` Tian, Kevin
2021-10-25 5:05 ` David Gibson
2021-10-27 2:32 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 12/20] iommu/iommufd: Add IOMMU_CHECK_EXTENSION Liu Yi L
2021-09-21 17:47 ` Jason Gunthorpe
2021-09-22 3:41 ` Tian, Kevin
2021-09-22 12:55 ` Jason Gunthorpe
2021-09-22 14:13 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 13/20] iommu: Extend iommu_at[de]tach_device() for multiple devices group Liu Yi L
2021-10-14 5:24 ` David Gibson
2021-10-14 7:06 ` Tian, Kevin
2021-10-18 3:57 ` David Gibson
2021-10-18 16:32 ` Jason Gunthorpe
2021-10-25 5:14 ` David Gibson
2021-10-25 12:14 ` Jason Gunthorpe
2021-10-25 13:16 ` David Gibson
2021-10-25 23:36 ` Jason Gunthorpe
2021-10-26 9:23 ` David Gibson
2021-09-19 6:38 ` [RFC 14/20] iommu/iommufd: Add iommufd_device_[de]attach_ioasid() Liu Yi L
2021-09-21 18:02 ` Jason Gunthorpe
2021-09-22 3:53 ` Tian, Kevin
2021-09-22 12:57 ` Jason Gunthorpe
2021-09-22 14:16 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 15/20] vfio/pci: Add VFIO_DEVICE_[DE]ATTACH_IOASID Liu Yi L
2021-09-21 18:04 ` Jason Gunthorpe
2021-09-22 3:56 ` Tian, Kevin
2021-09-22 12:58 ` Jason Gunthorpe
2021-09-22 14:17 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 16/20] vfio/type1: Export symbols for dma [un]map code sharing Liu Yi L
2021-09-21 18:14 ` Jason Gunthorpe
2021-09-22 3:57 ` Tian, Kevin
2021-09-19 6:38 ` [RFC 17/20] iommu/iommufd: Report iova range to userspace Liu Yi L
2021-09-22 14:49 ` Jean-Philippe Brucker
2021-09-29 10:44 ` Liu, Yi L
2021-09-29 12:07 ` Jean-Philippe Brucker
2021-09-29 12:31 ` Jason Gunthorpe
2021-09-19 6:38 ` [RFC 18/20] iommu/iommufd: Add IOMMU_[UN]MAP_DMA on IOASID Liu Yi L
2021-09-19 6:38 ` [RFC 19/20] iommu/vt-d: Implement device_info iommu_ops callback Liu Yi L
2021-09-19 6:38 ` [RFC 20/20] Doc: Add documentation for /dev/iommu Liu Yi L
2021-10-29 0:15 ` David Gibson
2021-10-29 12:44 ` Jason Gunthorpe
2021-09-19 6:45 ` [RFC 00/20] Introduce /dev/iommu for userspace I/O address space management Liu, Yi L
2021-09-21 13:45 ` Jason Gunthorpe
2021-09-22 3:25 ` Liu, Yi L
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YWzvHsm7YMYS2sP3@yekko \
--to=david@gibson.dropbear.id.au \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@linux.intel.com \
--cc=corbet@lwn.net \
--cc=dave.jiang@intel.com \
--cc=dwmw2@infradead.org \
--cc=eric.auger@redhat.com \
--cc=hao.wu@intel.com \
--cc=hch@lst.de \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jasowang@redhat.com \
--cc=jean-philippe@linaro.org \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=jun.j.tian@intel.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lkml@metux.net \
--cc=lushenming@huawei.com \
--cc=nicolinc@nvidia.com \
--cc=parav@mellanox.com \
--cc=pbonzini@redhat.com \
--cc=robin.murphy@arm.com \
--cc=yi.l.liu@intel.com \
--cc=yi.l.liu@linux.intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).