From: Eric Auger <eric.auger@redhat.com>
To: "Tian, Kevin" <kevin.tian@intel.com>,
Jason Gunthorpe <jgg@nvidia.com>,
"Alex Williamson (alex.williamson@redhat.com)"
<alex.williamson@redhat.com>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
David Gibson <david@gibson.dropbear.id.au>,
Jason Wang <jasowang@redhat.com>,
"parav@mellanox.com" <parav@mellanox.com>,
"Enrico Weigelt, metux IT consult" <lkml@metux.net>,
Paolo Bonzini <pbonzini@redhat.com>,
Shenming Lu <lushenming@huawei.com>,
Joerg Roedel <joro@8bytes.org>
Cc: Jonathan Corbet <corbet@lwn.net>,
"Raj, Ashok" <ashok.raj@intel.com>,
"Liu, Yi L" <yi.l.liu@intel.com>, "Wu, Hao" <hao.wu@intel.com>,
"Jiang, Dave" <dave.jiang@intel.com>,
Jacob Pan <jacob.jun.pan@linux.intel.com>,
Kirti Wankhede <kwankhede@nvidia.com>,
Robin Murphy <robin.murphy@arm.com>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
David Woodhouse <dwmw2@infradead.org>,
LKML <linux-kernel@vger.kernel.org>,
Lu Baolu <baolu.lu@linux.intel.com>
Subject: Re: [RFC v2] /dev/iommu uAPI proposal
Date: Tue, 10 Aug 2021 09:17:10 +0200 [thread overview]
Message-ID: <cec41751-c300-40f2-a8d6-f4916fb4a34e@redhat.com> (raw)
In-Reply-To: <BN9PR11MB5433453DED3546F5011C3BDD8CF29@BN9PR11MB5433.namprd11.prod.outlook.com>
Hi Kevin,
On 8/5/21 2:36 AM, Tian, Kevin wrote:
>> From: Eric Auger <eric.auger@redhat.com>
>> Sent: Wednesday, August 4, 2021 11:59 PM
>>
> [...]
>>> 1.2. Attach Device to I/O address space
>>> +++++++++++++++++++++++++++++++++++++++
>>>
>>> Device attach/bind is initiated through passthrough framework uAPI.
>>>
>>> Device attaching is allowed only after a device is successfully bound to
>>> the IOMMU fd. User should provide a device cookie when binding the
>>> device through VFIO uAPI. This cookie is used when the user queries
>>> device capability/format, issues per-device iotlb invalidation and
>>> receives per-device I/O page fault data via IOMMU fd.
>>>
>>> Successful binding puts the device into a security context which isolates
>>> its DMA from the rest system. VFIO should not allow user to access the
>> s/from the rest system/from the rest of the system
>>> device before binding is completed. Similarly, VFIO should prevent the
>>> user from unbinding the device before user access is withdrawn.
>> With Intel scalable IOV, I understand you could assign an RID/PASID to
>> one VM and another one to another VM (which is not the case for ARM). Is
>> it a targetted use case?How would it be handled? Is it related to the
>> sub-groups evoked hereafter?
> Not related to sub-group. Each mdev is bound to the IOMMU fd respectively
> with the defPASID which represents the mdev.
But how does it work in term of security. The device (RID) is bound to
an IOMMU fd. But then each SID/PASID may be working for a different VM.
How do you detect this is safe as each SID can work safely for a
different VM versus the ARM case where it is not possible.
1.3 says
"
1) A successful binding call for the first device in the group creates
the security context for the entire group, by:
"
What does it mean for above scalable IOV use case?
>
>> Actually all devices bound to an IOMMU fd should have the same parent
>> I/O address space or root address space, am I correct? If so, maybe add
>> this comment explicitly?
> in most cases yes but it's not mandatory. multiple roots are allowed
> (e.g. with vIOMMU but no nesting).
OK, right, this corresponds to example 4.2 for example. I misinterpreted
the notion of security context. The security context does not match the
IOMMU fd but is something implicit created on 1st device binding.
>
> [...]
>>> The device in the /dev/iommu context always refers to a physical one
>>> (pdev) which is identifiable via RID. Physically each pdev can support
>>> one default I/O address space (routed via RID) and optionally multiple
>>> non-default I/O address spaces (via RID+PASID).
>>>
>>> The device in VFIO context is a logic concept, being either a physical
>>> device (pdev) or mediated device (mdev or subdev). Each vfio device
>>> is represented by RID+cookie in IOMMU fd. User is allowed to create
>>> one default I/O address space (routed by vRID from user p.o.v) per
>>> each vfio_device.
>> The concept of default address space is not fully clear for me. I
>> currently understand this is a
>> root address space (not nesting). Is that coorect.This may need
>> clarification.
> w/o PASID there is only one address space (either GPA or GIOVA)
> per device. This one is called default. whether it's root is orthogonal
> (e.g. GIOVA could be also nested) to the device view of this space.
>
> w/ PASID additional address spaces can be targeted by the device.
> those are called non-default.
>
> I could also rename default to RID address space and non-default to
> RID+PASID address space if doing so makes it clearer.
Yes I think it is worth having a kind of glossary and defining root as,
default as as you clearly defined child/parent.
>
>>> VFIO decides the routing information for this default
>>> space based on device type:
>>>
>>> 1) pdev, routed via RID;
>>>
>>> 2) mdev/subdev with IOMMU-enforced DMA isolation, routed via
>>> the parent's RID plus the PASID marking this mdev;
>>>
>>> 3) a purely sw-mediated device (sw mdev), no routing required i.e. no
>>> need to install the I/O page table in the IOMMU. sw mdev just uses
>>> the metadata to assist its internal DMA isolation logic on top of
>>> the parent's IOMMU page table;
>> Maybe you should introduce this concept of SW mediated device earlier
>> because it seems to special case the way the attach behaves. I am
>> especially refering to
>>
>> "Successful attaching activates an I/O address space in the IOMMU, if the
>> device is not purely software mediated"
> makes sense.
>
>>> In addition, VFIO may allow user to create additional I/O address spaces
>>> on a vfio_device based on the hardware capability. In such case the user
>>> has its own view of the virtual routing information (vPASID) when marking
>>> these non-default address spaces.
>> I do not catch what does mean "marking these non default address space".
> as explained above, those non-default address spaces are identified/routed
> via PASID.
>
>>> 1.3. Group isolation
>>> ++++++++++++++++++++
> [...]
>>> 1) A successful binding call for the first device in the group creates
>>> the security context for the entire group, by:
>>>
>>> * Verifying group viability in a similar way as VFIO does;
>>>
>>> * Calling IOMMU-API to move the group into a block-dma state,
>>> which makes all devices in the group attached to an block-dma
>>> domain with an empty I/O page table;
>> this block-dma state/domain would deserve to be better defined (I know
>> you already evoked it in 1.1 with the dma mapping protocol though)
>> activates an empty I/O page table in the IOMMU (if the device is not
>> purely SW mediated)?
> sure. some explanations are scattered in following paragraph, but I
> can consider to further clarify it.
>
>> How does that relate to the default address space? Is it the same?
> different. this block-dma domain doesn't hold any valid mapping. The
> default address space is represented by a normal unmanaged domain.
> the ioasid attaching operation will detach the device from the block-dma
> domain and then attach it to the target ioasid.
OK
Thanks
Eric
>
>>> 2. uAPI Proposal
>>> ----------------------
> [...]
>>> /*
>>> * Allocate an IOASID.
>>> *
>>> * IOASID is the FD-local software handle representing an I/O address
>>> * space. Each IOASID is associated with a single I/O page table. User
>>> * must call this ioctl to get an IOASID for every I/O address space that is
>>> * intended to be tracked by the kernel.
>>> *
>>> * User needs to specify the attributes of the IOASID and associated
>>> * I/O page table format information according to one or multiple devices
>>> * which will be attached to this IOASID right after. The I/O page table
>>> * is activated in the IOMMU when it's attached by a device. Incompatible
>> .. if not SW mediated
>>> * format between device and IOASID will lead to attaching failure.
>>> *
>>> * The root IOASID should always have a kernel-managed I/O page
>>> * table for safety. Locked page accounting is also conducted on the root.
>> The definition of root IOASID is not easily found in this spec. Maybe
>> this would deserve some clarification.
> make sense.
>
> and thanks for other typo-related comments.
>
> Thanks
> Kevin
next prev parent reply other threads:[~2021-08-10 7:17 UTC|newest]
Thread overview: 60+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-07-09 7:48 [RFC v2] /dev/iommu uAPI proposal Tian, Kevin
2021-07-09 21:50 ` Alex Williamson
2021-07-12 1:22 ` Tian, Kevin
2021-07-12 18:41 ` Alex Williamson
2021-07-12 23:41 ` Tian, Kevin
2021-07-12 23:56 ` Tian, Kevin
2021-07-13 12:55 ` Jason Gunthorpe
2021-07-13 16:26 ` Alex Williamson
2021-07-13 16:32 ` Jason Gunthorpe
2021-07-13 22:48 ` Tian, Kevin
2021-07-13 23:02 ` Jason Gunthorpe
2021-07-13 23:20 ` Tian, Kevin
2021-07-13 23:22 ` Jason Gunthorpe
2021-07-13 23:24 ` Tian, Kevin
2021-07-15 3:20 ` Shenming Lu
2021-07-15 3:55 ` Tian, Kevin
2021-07-15 6:29 ` Shenming Lu
2021-07-15 6:49 ` Tian, Kevin
2021-07-15 8:14 ` Shenming Lu
2021-07-15 12:48 ` Jason Gunthorpe
2021-07-15 13:57 ` Raj, Ashok
2021-07-15 15:23 ` Jason Gunthorpe
2021-07-15 16:21 ` Raj, Ashok
2021-07-15 17:18 ` Jason Gunthorpe
2021-07-15 17:48 ` Raj, Ashok
2021-07-15 17:53 ` Jason Gunthorpe
2021-07-15 18:05 ` Raj, Ashok
2021-07-15 18:13 ` Jason Gunthorpe
2021-07-16 1:20 ` Tian, Kevin
2021-07-16 12:20 ` Shenming Lu
2021-07-21 2:13 ` Tian, Kevin
2021-07-22 16:30 ` Jason Gunthorpe
2021-07-16 18:30 ` Jason Gunthorpe
2021-07-21 2:11 ` Tian, Kevin
2021-07-26 4:50 ` David Gibson
2021-07-28 4:04 ` Tian, Kevin
2021-08-03 1:50 ` David Gibson
2021-08-03 3:19 ` Tian, Kevin
2021-08-06 4:45 ` David Gibson
2021-08-06 12:32 ` Jason Gunthorpe
2021-08-10 6:10 ` David Gibson
2021-08-09 8:34 ` Tian, Kevin
2021-08-10 4:47 ` David Gibson
2021-08-10 6:04 ` Tian, Kevin
2021-07-30 14:51 ` Jason Gunthorpe
2021-08-02 2:49 ` Tian, Kevin
2021-08-04 14:04 ` Jason Gunthorpe
2021-08-04 22:59 ` Tian, Kevin
2021-08-05 11:27 ` Jason Gunthorpe
2021-08-05 22:44 ` Tian, Kevin
2021-08-06 4:47 ` David Gibson
2021-08-03 1:58 ` David Gibson
2021-08-04 14:07 ` Jason Gunthorpe
2021-08-06 4:24 ` David Gibson
2021-07-26 8:14 ` Jean-Philippe Brucker
2021-07-28 4:05 ` Tian, Kevin
2021-08-04 15:59 ` Eric Auger
2021-08-05 0:36 ` Tian, Kevin
2021-08-10 7:17 ` Eric Auger [this message]
2021-08-10 9:00 ` Tian, Kevin
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=cec41751-c300-40f2-a8d6-f4916fb4a34e@redhat.com \
--to=eric.auger@redhat.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=baolu.lu@linux.intel.com \
--cc=corbet@lwn.net \
--cc=dave.jiang@intel.com \
--cc=david@gibson.dropbear.id.au \
--cc=dwmw2@infradead.org \
--cc=hao.wu@intel.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jasowang@redhat.com \
--cc=jean-philippe@linaro.org \
--cc=jgg@nvidia.com \
--cc=joro@8bytes.org \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=kwankhede@nvidia.com \
--cc=linux-kernel@vger.kernel.org \
--cc=lkml@metux.net \
--cc=lushenming@huawei.com \
--cc=parav@mellanox.com \
--cc=pbonzini@redhat.com \
--cc=robin.murphy@arm.com \
--cc=yi.l.liu@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).