iommu.lists.linux-foundation.org archive mirror
 help / color / mirror / Atom feed
From: David Gibson <david@gibson.dropbear.id.au>
To: Jason Gunthorpe <jgg@nvidia.com>
Cc: Jean-Philippe Brucker <jean-philippe@linaro.org>,
	"Tian, Kevin" <kevin.tian@intel.com>,
	"Alex Williamson \(alex.williamson@redhat.com\)"
	<alex.williamson@redhat.com>, "Raj, Ashok" <ashok.raj@intel.com>,
	"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
	Jonathan Corbet <corbet@lwn.net>,
	Robin Murphy <robin.murphy@arm.com>,
	LKML <linux-kernel@vger.kernel.org>,
	Kirti Wankhede <kwankhede@nvidia.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>,
	"Jiang, Dave" <dave.jiang@intel.com>,
	David Woodhouse <dwmw2@infradead.org>,
	Jason Wang <jasowang@redhat.com>
Subject: Re: [RFC] /dev/ioasid uAPI proposal
Date: Wed, 2 Jun 2021 16:32:27 +1000	[thread overview]
Message-ID: <YLcl+zaK6Y0gB54a@yekko> (raw)
In-Reply-To: <20210528173538.GA3816344@nvidia.com>


[-- Attachment #1.1: Type: text/plain, Size: 4577 bytes --]

On Fri, May 28, 2021 at 02:35:38PM -0300, Jason Gunthorpe wrote:
> On Thu, May 27, 2021 at 07:58:12AM +0000, Tian, Kevin wrote:
[snip]
> > With above design /dev/ioasid uAPI is all about I/O address spaces. 
> > It doesn't include any device routing information, which is only 
> > indirectly registered to the ioasid driver through VFIO uAPI. For
> > example, I/O page fault is always reported to userspace per IOASID,
> > although it's physically reported per device (RID+PASID). 
> 
> I agree with Jean-Philippe - at the very least erasing this
> information needs a major rational - but I don't really see why it
> must be erased? The HW reports the originating device, is it just a
> matter of labeling the devices attached to the /dev/ioasid FD so it
> can be reported to userspace?

HW reports the originating device as far as it knows.  In many cases
where you have multiple devices in an IOMMU group, it's because
although they're treated as separate devices at the kernel level, they
have the same RID at the HW level.  Which means a RID for something in
the right group is the closest you can count on supplying.

[snip]
> > However this way significantly 
> > violates the philosophy in this /dev/ioasid proposal. It is not one IOASID 
> > one address space any more. Device routing information (indirectly 
> > marking hidden I/O spaces) has to be carried in iotlb invalidation and 
> > page faulting uAPI to help connect vIOMMU with the underlying 
> > pIOMMU. This is one design choice to be confirmed with ARM guys.
> 
> I'm confused by this rational.
> 
> For a vIOMMU that has IO page tables in the guest the basic
> choices are:
>  - Do we have a hypervisor trap to bind the page table or not? (RID
>    and PASID may differ here)
>  - Do we have a hypervisor trap to invaliate the page tables or not?
> 
> If the first is a hypervisor trap then I agree it makes sense to create a
> child IOASID that points to each guest page table and manage it
> directly. This should not require walking guest page tables as it is
> really just informing the HW where the page table lives. HW will walk
> them.
> 
> If there are no hypervisor traps (does this exist?) then there is no
> way to involve the hypervisor here and the child IOASID should simply
> be a pointer to the guest's data structure that describes binding. In
> this case that IOASID should claim all PASIDs when bound to a
> RID. 

And in that case I think we should call that object something other
than an IOASID, since it represents multiple address spaces.

> Invalidation should be passed up the to the IOMMU driver in terms of
> the guest tables information and either the HW or software has to walk
> to guest tables to make sense of it.
> 
> Events from the IOMMU to userspace should be tagged with the attached
> device label and the PASID/substream ID. This means there is no issue
> to have a a 'all PASID' IOASID.
> 
> > Notes:
> > -   It might be confusing as IOASID is also used in the kernel (drivers/
> >     iommu/ioasid.c) to represent PCI PASID or ARM substream ID. We need
> >     find a better name later to differentiate.
> 
> +1 on Jean-Philippe's remarks
> 
> > -   PPC has not be considered yet as we haven't got time to fully understand
> >     its semantics. According to previous discussion there is some generality 
> >     between PPC window-based scheme and VFIO type1 semantics. Let's 
> >     first make consensus on this proposal and then further discuss how to 
> >     extend it to cover PPC's requirement.
> 
> From what I understood PPC is not so bad, Nesting IOASID's did its
> preload feature and it needed a way to specify/query the IOVA range a
> IOASID will cover.
> 
> > -   There is a protocol between vfio group and kvm. Needs to think about
> >     how it will be affected following this proposal.
> 
> Ugh, I always stop looking when I reach that boundary. Can anyone
> summarize what is going on there?
> 
> Most likely passing the /dev/ioasid into KVM's FD (or vicevera) is the
> right answer. Eg if ARM needs to get the VMID from KVM and set it to
> ioasid then a KVM "ioctl set_arm_vmid(/dev/ioasid)" call is
> reasonable. Certainly better than the symbol get sutff we have right
> now.
> 
> I will read through the detail below in another email
> 
> Jason
> 

-- 
David Gibson			| I'll have my music baroque, and my code
david AT gibson.dropbear.id.au	| minimalist, thank you.  NOT _the_ _other_
				| _way_ _around_!
http://www.ozlabs.org/~dgibson

[-- Attachment #1.2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

[-- Attachment #2: Type: text/plain, Size: 156 bytes --]

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu

  parent reply	other threads:[~2021-06-02  7:29 UTC|newest]

Thread overview: 260+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-05-27  7:58 [RFC] /dev/ioasid uAPI proposal Tian, Kevin
2021-05-28  2:24 ` Jason Wang
2021-05-28 20:25   ` Jason Gunthorpe
2021-05-31  8:41   ` Liu Yi L
2021-06-01  2:36     ` Jason Wang
2021-06-01  3:31       ` Liu Yi L
2021-06-01  5:08         ` Jason Wang
2021-06-01  5:23           ` Lu Baolu
2021-06-01  5:29             ` Jason Wang
2021-06-01  5:42               ` Tian, Kevin
2021-06-01  6:07                 ` Jason Wang
2021-06-01  6:16                   ` Tian, Kevin
2021-06-01  8:47                     ` Jason Wang
2021-06-01 17:31                       ` Jason Gunthorpe
2021-06-02  8:54                         ` Jason Wang
2021-06-02 17:21                           ` Jason Gunthorpe
2021-06-07 13:30                             ` Enrico Weigelt, metux IT consult
2021-06-07 18:01                               ` Jason Gunthorpe
2021-06-08 10:45                                 ` Enrico Weigelt, metux IT consult
2021-06-10  2:16                                   ` Jason Wang
2021-06-08  1:10                             ` Jason Wang
2021-06-08 13:20                               ` Jason Gunthorpe
2021-06-10  2:00                                 ` Jason Wang
2021-06-10  4:03                                   ` Jason Wang
2021-06-10 11:47                                   ` Jason Gunthorpe
2021-06-11  5:43                                     ` Jason Wang
2021-06-01 17:29                   ` Jason Gunthorpe
2021-06-02  8:58                     ` Jason Wang
2021-06-01  4:27       ` Shenming Lu
2021-06-01  5:10         ` Jason Wang
2021-05-28 16:23 ` Jean-Philippe Brucker
2021-05-28 20:16   ` Jason Gunthorpe
2021-06-01  7:50   ` Tian, Kevin
2021-05-28 17:35 ` Jason Gunthorpe
2021-06-01  8:10   ` Tian, Kevin
2021-06-01 17:42     ` Jason Gunthorpe
2021-06-02  1:33       ` Tian, Kevin
2021-06-02 16:09         ` Jason Gunthorpe
2021-06-03  1:29           ` Tian, Kevin
2021-06-03  5:09             ` David Gibson
2021-06-03  6:49               ` Tian, Kevin
2021-06-03 11:47                 ` Jason Gunthorpe
2021-06-04  2:15                   ` Tian, Kevin
2021-06-08  0:49                 ` David Gibson
2021-06-09  2:52                   ` Tian, Kevin
2021-06-02  6:32   ` David Gibson [this message]
2021-06-02 16:16     ` Jason Gunthorpe
2021-06-03  2:11       ` Tian, Kevin
2021-06-03  5:13       ` David Gibson
2021-06-03 11:52         ` Jason Gunthorpe
2021-06-08  0:53           ` David Gibson
2021-06-08 19:04             ` Jason Gunthorpe
2021-06-17  2:42               ` David Gibson
2021-05-28 19:58 ` Jason Gunthorpe
2021-06-01  8:38   ` Tian, Kevin
2021-06-01 17:56     ` Jason Gunthorpe
2021-06-02  2:00       ` Tian, Kevin
2021-06-02  6:57       ` David Gibson
2021-06-02 16:37         ` Jason Gunthorpe
2021-06-03  5:23           ` David Gibson
2021-06-03 12:28             ` Jason Gunthorpe
2021-06-08  6:04               ` David Gibson
2021-06-08 19:23                 ` Jason Gunthorpe
2021-06-02  6:48   ` David Gibson
2021-06-02 16:58     ` Jason Gunthorpe
2021-06-03  2:49       ` Tian, Kevin
2021-06-03  5:48         ` David Gibson
2021-06-03  5:45       ` David Gibson
2021-06-03 12:11         ` Jason Gunthorpe
2021-06-04  6:08           ` Tian, Kevin
2021-06-04 12:33             ` Jason Gunthorpe
2021-06-04 23:20               ` Tian, Kevin
2021-06-08  6:13           ` David Gibson
2021-06-04 10:24         ` Jean-Philippe Brucker
2021-06-04 12:05           ` Jason Gunthorpe
2021-06-04 17:27             ` Jacob Pan
2021-06-04 17:40               ` Jason Gunthorpe
2021-06-08  6:31           ` David Gibson
2021-06-10 16:37             ` Jean-Philippe Brucker
2021-06-17  3:00               ` David Gibson
2021-06-18 17:03                 ` Jean-Philippe Brucker
2021-06-18 18:30                   ` Jason Gunthorpe
2021-06-23  8:19                     ` Tian, Kevin
2021-06-23  7:57                   ` Tian, Kevin
2021-06-24  3:49                   ` David Gibson
2021-05-28 20:03 ` Jason Gunthorpe
2021-06-01  7:01   ` Tian, Kevin
2021-06-01 20:28     ` Jason Gunthorpe
2021-06-02  1:25       ` Tian, Kevin
2021-06-02 23:27         ` Jason Gunthorpe
2021-06-04  8:17         ` Jean-Philippe Brucker
2021-06-04  8:43           ` Tian, Kevin
2021-06-02  8:52       ` Jason Wang
2021-06-02 16:07         ` Jason Gunthorpe
2021-06-01 22:22     ` Alex Williamson
2021-06-02  2:20       ` Tian, Kevin
2021-06-02 16:01         ` Jason Gunthorpe
2021-06-02 17:11           ` Alex Williamson
2021-06-02 17:35             ` Jason Gunthorpe
2021-06-02 18:01               ` Alex Williamson
2021-06-02 18:09                 ` Jason Gunthorpe
2021-06-02 19:00                   ` Alex Williamson
2021-06-02 19:54                     ` Jason Gunthorpe
2021-06-02 20:37                       ` Alex Williamson
2021-06-02 22:45                         ` Jason Gunthorpe
2021-06-03  2:50                           ` Alex Williamson
2021-06-03  3:22                             ` Tian, Kevin
2021-06-03  4:14                               ` Alex Williamson
2021-06-03  5:18                                 ` Tian, Kevin
2021-06-03 12:40                               ` Jason Gunthorpe
2021-06-03 20:41                                 ` Alex Williamson
2021-06-04  9:19                                   ` Tian, Kevin
2021-06-04 15:37                                     ` Alex Williamson
2021-06-04 12:13                                   ` Jason Gunthorpe
2021-06-04 21:45                                     ` Alex Williamson
2021-06-04  7:33                                 ` Tian, Kevin
2021-06-03 12:34                             ` Jason Gunthorpe
2021-06-03 20:01                               ` Alex Williamson
2021-06-03 20:10                                 ` Jason Gunthorpe
2021-06-03 21:44                                   ` Alex Williamson
2021-06-04  8:38                                     ` Tian, Kevin
2021-06-04 12:28                                       ` Jason Gunthorpe
2021-06-04 15:26                                         ` Alex Williamson
2021-06-04 15:40                                           ` Paolo Bonzini
2021-06-04 15:50                                             ` Jason Gunthorpe
2021-06-04 15:57                                               ` Paolo Bonzini
2021-06-04 16:03                                                 ` Jason Gunthorpe
2021-06-04 16:10                                                   ` Paolo Bonzini
2021-06-04 17:22                                                     ` Jason Gunthorpe
2021-06-04 21:29                                                       ` Alex Williamson
2021-06-04 23:01                                                         ` Jason Gunthorpe
2021-06-07 15:41                                                           ` Alex Williamson
2021-06-07 18:18                                                             ` Jason Gunthorpe
2021-06-07 18:59                                                               ` Alex Williamson
2021-06-07 19:08                                                                 ` Jason Gunthorpe
2021-06-07 19:41                                                                   ` Alex Williamson
2021-06-07 23:03                                                                     ` Jason Gunthorpe
2021-06-08  0:30                                                                       ` Alex Williamson
2021-06-08  1:20                                                                     ` Jason Wang
2021-06-30  6:53                                                                       ` Christoph Hellwig
2021-06-30  6:49                                                                   ` Christoph Hellwig
2021-06-07  3:25                                                         ` Tian, Kevin
2021-06-07  6:51                                                           ` Paolo Bonzini
2021-06-07 18:01                                                             ` Jason Gunthorpe
2021-06-30  6:56                                                           ` Christoph Hellwig
2021-06-05  6:22                                                       ` Paolo Bonzini
2021-06-07  3:50                                                         ` Tian, Kevin
2021-06-07 17:59                                                         ` Jason Gunthorpe
2021-06-08  7:56                                                           ` Paolo Bonzini
2021-06-08 13:15                                                             ` Jason Gunthorpe
2021-06-08 13:44                                                               ` Paolo Bonzini
2021-06-08 18:47                                                                 ` Alex Williamson
2021-06-08 19:00                                                                   ` Jason Gunthorpe
2021-06-09  8:51                                                                     ` Enrico Weigelt, metux IT consult
2021-06-09  9:11                                                                       ` Paolo Bonzini
2021-06-09 11:54                                                                         ` Jason Gunthorpe
2021-06-09 14:31                                                                           ` Alex Williamson
2021-06-09 14:45                                                                             ` Jason Gunthorpe
2021-06-09 15:20                                                                               ` Paolo Bonzini
2021-10-27  6:18                                                                                 ` Tian, Kevin
2021-10-27 10:32                                                                                   ` Paolo Bonzini
2021-10-28  1:50                                                                                     ` Tian, Kevin
2021-06-09  2:49                                                                   ` Tian, Kevin
2021-06-09 11:57                                                                     ` Jason Gunthorpe
2021-06-09 12:46                                                                       ` Paolo Bonzini
2021-06-09 12:47                                                                         ` Jason Gunthorpe
2021-06-09 13:24                                                                           ` Paolo Bonzini
2021-06-09 14:32                                                                             ` Jason Gunthorpe
2021-06-30  7:01                                                                           ` Christoph Hellwig
2021-06-09 18:09                                                                     ` Alex Williamson
2021-06-03  2:52                         ` Jason Wang
2021-06-03 13:09                           ` Jason Gunthorpe
2021-06-04  1:11                             ` Jason Wang
2021-06-04 11:58                               ` Jason Gunthorpe
2021-06-07  3:18                                 ` Jason Wang
2021-06-07 14:14                                   ` Jason Gunthorpe
2021-06-08  1:00                                     ` Jason Wang
2021-06-08  8:54                                       ` Enrico Weigelt, metux IT consult
2021-06-08 12:52                                         ` Jason Gunthorpe
2021-06-30  7:07                                     ` Christoph Hellwig
2021-06-30  7:05                                 ` Christoph Hellwig
2021-06-08  2:37       ` David Gibson
2021-06-08 13:17         ` Jason Gunthorpe
2021-06-17  3:47           ` David Gibson
2021-06-23  7:59             ` Tian, Kevin
2021-06-24  3:53               ` David Gibson
2021-05-28 23:36 ` Jason Gunthorpe
2021-05-31 11:31   ` Liu Yi L
2021-05-31 18:09     ` Jason Gunthorpe
2021-06-01  3:08       ` Lu Baolu
2021-06-01 17:24         ` Jason Gunthorpe
2021-06-01  1:25     ` Lu Baolu
2021-06-01 11:09   ` Lu Baolu
2021-06-01 17:26     ` Jason Gunthorpe
2021-06-02  4:01       ` Lu Baolu
2021-06-02 23:23         ` Jason Gunthorpe
2021-06-03  5:49           ` Lu Baolu
2021-06-03  5:54     ` David Gibson
2021-06-03  6:50       ` Lu Baolu
2021-06-03 12:56         ` Jason Gunthorpe
2021-06-02  7:22   ` David Gibson
2021-06-03  6:39   ` Tian, Kevin
2021-06-03 13:05     ` Jason Gunthorpe
2021-06-04  6:37       ` Tian, Kevin
2021-06-04 12:09         ` Jason Gunthorpe
2021-06-04 23:10           ` Tian, Kevin
2021-06-07 17:54             ` Jason Gunthorpe
2021-06-15  8:59       ` Tian, Kevin
2021-06-15 15:06         ` Jason Gunthorpe
2021-06-15 22:59           ` Tian, Kevin
2021-06-15 23:02             ` Jason Gunthorpe
2021-06-15 23:09               ` Tian, Kevin
2021-06-15 23:40                 ` Jason Gunthorpe
2021-06-15 23:56                   ` Tian, Kevin
2021-06-15 23:59                     ` Jason Gunthorpe
2021-06-16  0:02                       ` Tian, Kevin
2021-05-31 17:37 ` Parav Pandit
2021-05-31 18:12   ` Jason Gunthorpe
2021-06-01 12:04     ` Parav Pandit
2021-06-01 17:36       ` Jason Gunthorpe
2021-06-02  8:38   ` Enrico Weigelt, metux IT consult
2021-06-02 12:41     ` Parav Pandit
2021-06-01  4:31 ` Shenming Lu
2021-06-01  5:10   ` Lu Baolu
2021-06-01  7:15     ` Shenming Lu
2021-06-01 12:30       ` Lu Baolu
2021-06-01 13:10         ` Shenming Lu
2021-06-01 17:33         ` Jason Gunthorpe
2021-06-02  4:50           ` Shenming Lu
2021-06-03 18:19             ` Jacob Pan
2021-06-04  1:30               ` Jason Wang
2021-06-04 16:22                 ` Jacob Pan
2021-06-04 16:22                   ` Jason Gunthorpe
2021-06-04 18:05                     ` Jacob Pan
2021-06-04  2:03               ` Shenming Lu
2021-06-07 12:19                 ` Liu, Yi L
2021-06-08  1:09                   ` Shenming Lu
2021-06-01 17:30 ` Parav Pandit
2021-06-03 20:58   ` Jacob Pan
2021-06-08  6:30     ` Parav Pandit
2021-06-02  6:15 ` David Gibson
2021-06-02 17:19   ` Jason Gunthorpe
2021-06-03  3:02     ` Tian, Kevin
2021-06-03  6:26     ` David Gibson
2021-06-03 12:46       ` Jason Gunthorpe
2021-06-04  6:27         ` Tian, Kevin
2021-06-03  7:17   ` Tian, Kevin
2021-06-03 12:49     ` Jason Gunthorpe
2021-06-08  5:49     ` David Gibson
2021-06-03  8:12   ` Tian, Kevin
2021-06-17  4:07     ` David Gibson
2021-06-23  8:00       ` Tian, Kevin
2021-06-24  3:55         ` David Gibson
2021-06-02  8:56 ` Enrico Weigelt, metux IT consult
2021-06-02 17:24   ` Jason Gunthorpe
2021-06-04 10:44     ` Enrico Weigelt, metux IT consult
2021-06-04 12:30       ` Jason Gunthorpe
2021-06-08  1:15         ` David Gibson
2021-06-08 10:43         ` Enrico Weigelt, metux IT consult
2021-06-08 13:11           ` Jason Gunthorpe

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=YLcl+zaK6Y0gB54a@yekko \
    --to=david@gibson.dropbear.id.au \
    --cc=alex.williamson@redhat.com \
    --cc=ashok.raj@intel.com \
    --cc=corbet@lwn.net \
    --cc=dave.jiang@intel.com \
    --cc=dwmw2@infradead.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=jasowang@redhat.com \
    --cc=jean-philippe@linaro.org \
    --cc=jgg@nvidia.com \
    --cc=kevin.tian@intel.com \
    --cc=kvm@vger.kernel.org \
    --cc=kwankhede@nvidia.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=robin.murphy@arm.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).