From: "Liu, Yi L" <yi.l.liu@intel.com>
To: Auger Eric <eric.auger@redhat.com>,
Jean-Philippe Brucker <jean-philippe@linaro.org>,
"jacob.jun.pan@linux.intel.com" <jacob.jun.pan@linux.intel.com>
Cc: "alex.williamson@redhat.com" <alex.williamson@redhat.com>,
"Tian, Kevin" <kevin.tian@intel.com>,
"joro@8bytes.org" <joro@8bytes.org>,
"Raj, Ashok" <ashok.raj@intel.com>,
"Tian, Jun J" <jun.j.tian@intel.com>,
"Sun, Yi Y" <yi.y.sun@intel.com>,
"peterx@redhat.com" <peterx@redhat.com>,
"iommu@lists.linux-foundation.org"
<iommu@lists.linux-foundation.org>,
"kvm@vger.kernel.org" <kvm@vger.kernel.org>,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>,
"Wu, Hao" <hao.wu@intel.com>
Subject: RE: [PATCH v1 5/8] vfio/type1: Report 1st-level/stage-1 format to userspace
Date: Fri, 10 Apr 2020 03:48:52 +0000 [thread overview]
Message-ID: <A2975661238FB949B60364EF0F2C25743A229ABC@SHSMSX104.ccr.corp.intel.com> (raw)
In-Reply-To: <194923cd-666c-2244-bfc6-95a1c61d4d99@redhat.com>
Hi Eric,
> From: Auger Eric <eric.auger@redhat.com>
> Sent: Friday, April 10, 2020 11:28 AM
> To: Liu, Yi L <yi.l.liu@intel.com>; Jean-Philippe Brucker <jean-
> Subject: Re: [PATCH v1 5/8] vfio/type1: Report 1st-level/stage-1 format to
> userspace
>
> Hi Yi,
>
> On 4/9/20 2:47 PM, Liu, Yi L wrote:
> > Hi Jean,
> >
> >> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
> >> Sent: Thursday, April 9, 2020 4:15 PM
> >> Subject: Re: [PATCH v1 5/8] vfio/type1: Report 1st-level/stage-1
> >> format to userspace
> >>
> >> On Wed, Apr 08, 2020 at 12:27:58PM +0200, Auger Eric wrote:
> >>> Hi Yi,
> >>>
> >>> On 4/7/20 11:43 AM, Liu, Yi L wrote:
> >>>> Hi Jean,
> >>>>
> >>>>> From: Jean-Philippe Brucker <jean-philippe@linaro.org>
> >>>>> Sent: Friday, April 3, 2020 4:23 PM
> >>>>> To: Auger Eric <eric.auger@redhat.com> userspace
> >>>>>
> >>>>> On Wed, Apr 01, 2020 at 03:01:12PM +0200, Auger Eric wrote:
> >>>>>>>>> header = vfio_info_cap_add(caps, sizeof(*nesting_cap),
> >>>>>>>>>
> >> VFIO_IOMMU_TYPE1_INFO_CAP_NESTING, 1);
> >>>>> @@ -2254,6 +2309,7
> >>>>>>>>> @@ static int vfio_iommu_info_add_nesting_cap(struct
> >>>>>>>> vfio_iommu *iommu,
> >>>>>>>>> /* nesting iommu type supports PASID requests
> (alloc/free)
> >> */
> >>>>>>>>> nesting_cap->nesting_capabilities |=
> >> VFIO_IOMMU_PASID_REQS;
> >>>>>>>> What is the meaning for ARM?
> >>>>>>>
> >>>>>>> I think it's just a software capability exposed to userspace, on
> >>>>>>> userspace side, it has a choice to use it or not. :-) The reason
> >>>>>>> define it and report it in cap nesting is that I'd like to make
> >>>>>>> the pasid alloc/free be available just for IOMMU with type
> >>>>>>> VFIO_IOMMU_TYPE1_NESTING. Please feel free tell me if it is not
> >>>>>>> good for ARM. We can find a proper way to report the availability.
> >>>>>>
> >>>>>> Well it is more a question for jean-Philippe. Do we have a system
> >>>>>> wide PASID allocation on ARM?
> >>>>>
> >>>>> We don't, the PASID spaces are per-VM on Arm, so this function
> >>>>> should consult
> >> the
> >>>>> IOMMU driver before setting flags. As you said on patch 3, nested
> >>>>> doesn't necessarily imply PASID support. The SMMUv2 does not
> >>>>> support PASID but does support nesting stages 1 and 2 for the IOVA space.
> >>>>> SMMUv3 support of PASID depends on HW capabilities. So I think
> >>>>> this needs to
> >> be
> >>>>> finer grained:
> >>>>>
> >>>>> Does the container support:
> >>>>> * VFIO_IOMMU_PASID_REQUEST?
> >>>>> -> Yes for VT-d 3
> >>>>> -> No for Arm SMMU
> >>>>> * VFIO_IOMMU_{,UN}BIND_GUEST_PGTBL?
> >>>>> -> Yes for VT-d 3
> >>>>> -> Sometimes for SMMUv2
> >>>>> -> No for SMMUv3 (if we go with BIND_PASID_TABLE, which is simpler
> due to
> >>>>> PASID tables being in GPA space.)
> >>>>> * VFIO_IOMMU_BIND_PASID_TABLE?
> >>>>> -> No for VT-d
> >>>>> -> Sometimes for SMMUv3
> >>>>>
> >>>>> Any bind support implies VFIO_IOMMU_CACHE_INVALIDATE support.
> >>>>
> >>>> good summary. do you expect to see any
> >>>>
> >>>>>
> >>>>>>>>> + nesting_cap->stage1_formats = formats;
> >>>>>>>> as spotted by Kevin, since a single format is supported, rename
> >>>>>>>
> >>>>>>> ok, I was believing it may be possible on ARM or so. :-) will
> >>>>>>> rename it.
> >>>>>
> >>>>> Yes I don't think an u32 is going to cut it for Arm :( We need to
> >>>>> describe all sorts
> >> of
> >>>>> capabilities for page and PASID tables (granules, GPA size,
> >>>>> ASID/PASID size, HW access/dirty, etc etc.) Just saying "Arm
> >>>>> stage-1 format" wouldn't mean much. I guess we could have a secondary
> vendor capability for these?
> >>>>
> >>>> Actually, I'm wondering if we can define some formats to stands for
> >>>> a set of capabilities. e.g. VTD_STAGE1_FORMAT_V1 which may
> >>>> indicates the 1st level page table related caps (aw, a/d, SRE, EA
> >>>> and etc.). And vIOMMU can parse the capabilities.
> >>>
> >>> But eventually do we really need all those capability getters? I
> >>> mean can't we simply rely on the actual call to
> >>> VFIO_IOMMU_BIND_GUEST_PGTBL() to detect any mismatch? Definitively
> >>> the error handling may be heavier on userspace but can't we manage.
> >>
> >> I think we need to present these capabilities at boot time, long
> >> before the guest triggers a bind(). For example if the host SMMU
> >> doesn't support 16-bit ASID, we need to communicate that to the guest
> >> using vSMMU ID registers or PROBE properties. Otherwise a bind() will
> >> succeed, but if the guest uses 16-bit ASIDs in its CD, DMA will
> >> result in C_BAD_CD events which we'll inject into the guest, for no
> >> apparent reason from their perspective.
> >>
> >> In addition some VMMs may have fallbacks if shared page tables are
> >> not available. They could fall back to a MAP/UNMAP interface, or
> >> simply not present a vIOMMU to the guest.
> >>
> >
> > Based on the comments, I think it would be a need to report iommu caps
> > in detail. So I guess iommu uapi needs to provide something alike vfio
> > cap chain in iommu uapi. Please feel free let me know your thoughts.
> > :-)
>
> Yes to me it sounds sensible.
> >
> > In vfio, we can define a cap as below:
> >
> > struct vfio_iommu_type1_info_cap_nesting {
> > struct vfio_info_cap_header header;
> > __u64 iommu_model;
> > #define VFIO_IOMMU_PASID_REQS (1 << 0)
> I still think the name shall be changed
yes, I'll rename it per your suggestion.:-)
> > #define VFIO_IOMMU_BIND_GPASID (1 << 1)
> > #define VFIO_IOMMU_CACHE_INV (1 << 2)
> this operation seems mandated as soon as we have a nested paging based
> implementation?
oh, yes, should be. will remove it and comment in the code.
Regards,
Yi Liu
> > __u32 nesting_capabilities;
> > __u32 pasid_bits;
> > #define VFIO_IOMMU_VENDOR_SUB_CAP (1 << 3)
> > __u32 flags;
> > __u32 data_size;
> > __u8 data[]; /*iommu info caps defined by iommu uapi */
> > };
> >
> > VFIO needs new iommu APIs to ask iommu driver whether
> > PASID/bind_gpasid/ cache_inv/bind_gpasid_table is available or not and
> > also the pasid bits. After that VFIO will ask iommu driver about the
> > iommu_cap_info and fill in the @data[] field.
> >
> > iommu uapi:
> > struct iommu_info_cap_header {
> > __u16 id; /* Identifies capability */
> > __u16 version; /* Version specific to the capability ID */
> > __u32 next; /* Offset of next capability */
> > };
> >
> > #define IOMMU_INFO_CAP_INTEL_VTD 1
> > struct iommu_info_cap_intel_vtd {
> > struct iommu_info_cap_header header;
> > __u32 vaddr_width; /* VA addr_width*/
> > __u32 ipaddr_width; /* IPA addr_width, input of SL page table */
> > /* same definition with @flags instruct iommu_gpasid_bind_data_vtd */
> > __u64 flags;
> > };
> >
> > #define IOMMU_INFO_CAP_ARM_SMMUv3 2
> > struct iommu_info_cap_arm_smmuv3 {
> > struct iommu_info_cap_header header;
> > ...
> > };
>
> Thanks
>
> Eric
> >
> > Regards,
> > Yi Liu
> >
next prev parent reply other threads:[~2020-04-10 3:58 UTC|newest]
Thread overview: 110+ messages / expand[flat|nested] mbox.gz Atom feed top
2020-03-22 12:31 [PATCH v1 0/8] vfio: expose virtual Shared Virtual Addressing to VMs Liu, Yi L
2020-03-22 12:31 ` [PATCH v1 1/8] vfio: Add VFIO_IOMMU_PASID_REQUEST(alloc/free) Liu, Yi L
2020-03-22 16:21 ` kbuild test robot
2020-03-30 8:32 ` Tian, Kevin
2020-03-30 14:36 ` Liu, Yi L
2020-03-31 5:40 ` Tian, Kevin
2020-03-31 13:22 ` Liu, Yi L
2020-04-01 5:43 ` Tian, Kevin
2020-04-01 5:48 ` Liu, Yi L
2020-03-31 7:53 ` Christoph Hellwig
2020-03-31 8:17 ` Liu, Yi L
2020-03-31 8:32 ` Liu, Yi L
2020-03-31 8:36 ` Liu, Yi L
2020-03-31 9:15 ` Christoph Hellwig
2020-04-02 13:52 ` Jean-Philippe Brucker
2020-04-03 11:56 ` Liu, Yi L
2020-04-03 12:39 ` Jean-Philippe Brucker
2020-04-03 12:44 ` Liu, Yi L
2020-04-02 17:50 ` Alex Williamson
2020-04-03 5:58 ` Tian, Kevin
2020-04-03 15:14 ` Alex Williamson
2020-04-07 4:42 ` Tian, Kevin
2020-04-07 15:14 ` Alex Williamson
2020-04-03 13:12 ` Liu, Yi L
2020-04-03 17:50 ` Alex Williamson
2020-04-07 4:52 ` Tian, Kevin
2020-04-08 0:52 ` Liu, Yi L
2020-03-22 12:31 ` [PATCH v1 2/8] vfio/type1: Add vfio_iommu_type1 parameter for quota tuning Liu, Yi L
2020-03-22 17:20 ` kbuild test robot
2020-03-30 8:40 ` Tian, Kevin
2020-03-30 8:52 ` Liu, Yi L
2020-03-30 9:19 ` Tian, Kevin
2020-03-30 9:26 ` Liu, Yi L
2020-03-30 11:44 ` Tian, Kevin
2020-04-02 17:58 ` Alex Williamson
2020-04-03 8:15 ` Liu, Yi L
2020-03-22 12:32 ` [PATCH v1 3/8] vfio/type1: Report PASID alloc/free support to userspace Liu, Yi L
2020-03-30 9:43 ` Tian, Kevin
2020-04-01 7:46 ` Liu, Yi L
2020-04-01 9:41 ` Auger Eric
2020-04-01 13:13 ` Liu, Yi L
2020-04-02 18:01 ` Alex Williamson
2020-04-03 8:17 ` Liu, Yi L
2020-04-03 17:28 ` Alex Williamson
2020-04-04 11:36 ` Liu, Yi L
2020-03-22 12:32 ` [PATCH v1 4/8] vfio: Check nesting iommu uAPI version Liu, Yi L
2020-03-22 18:30 ` kbuild test robot
2020-03-22 12:32 ` [PATCH v1 5/8] vfio/type1: Report 1st-level/stage-1 format to userspace Liu, Yi L
2020-03-22 16:44 ` kbuild test robot
2020-03-30 11:48 ` Tian, Kevin
2020-04-01 7:38 ` Liu, Yi L
2020-04-01 7:56 ` Tian, Kevin
2020-04-01 8:06 ` Liu, Yi L
2020-04-01 8:08 ` Tian, Kevin
2020-04-01 8:09 ` Liu, Yi L
2020-04-01 8:51 ` Auger Eric
2020-04-01 12:51 ` Liu, Yi L
2020-04-01 13:01 ` Auger Eric
2020-04-03 8:23 ` Jean-Philippe Brucker
2020-04-07 9:43 ` Liu, Yi L
2020-04-08 1:02 ` Liu, Yi L
2020-04-08 10:27 ` Auger Eric
2020-04-09 8:14 ` Jean-Philippe Brucker
2020-04-09 9:01 ` Auger Eric
2020-04-09 12:47 ` Liu, Yi L
2020-04-10 3:28 ` Auger Eric
2020-04-10 3:48 ` Liu, Yi L [this message]
2020-04-10 12:30 ` Liu, Yi L
2020-04-02 19:20 ` Alex Williamson
2020-04-03 11:59 ` Liu, Yi L
2020-03-22 12:32 ` [PATCH v1 6/8] vfio/type1: Bind guest page tables to host Liu, Yi L
2020-03-22 18:10 ` kbuild test robot
2020-03-30 12:46 ` Tian, Kevin
2020-04-01 9:13 ` Liu, Yi L
2020-04-02 2:12 ` Tian, Kevin
2020-04-02 8:05 ` Liu, Yi L
2020-04-03 8:34 ` Jean-Philippe Brucker
2020-04-07 10:33 ` Liu, Yi L
2020-04-09 8:28 ` Jean-Philippe Brucker
2020-04-09 9:15 ` Liu, Yi L
2020-04-09 9:38 ` Jean-Philippe Brucker
2020-04-02 19:57 ` Alex Williamson
2020-04-03 13:30 ` Liu, Yi L
2020-04-03 18:11 ` Alex Williamson
2020-04-04 10:28 ` Liu, Yi L
2020-04-11 5:52 ` Liu, Yi L
2020-03-22 12:32 ` [PATCH v1 7/8] vfio/type1: Add VFIO_IOMMU_CACHE_INVALIDATE Liu, Yi L
2020-03-30 12:58 ` Tian, Kevin
2020-04-01 7:49 ` Liu, Yi L
2020-03-31 7:56 ` Christoph Hellwig
2020-03-31 10:48 ` Liu, Yi L
2020-04-02 20:24 ` Alex Williamson
2020-04-03 6:39 ` Tian, Kevin
2020-04-03 15:31 ` Jacob Pan
2020-04-03 15:34 ` Alex Williamson
2020-04-08 2:28 ` Liu, Yi L
2020-04-16 10:40 ` Liu, Yi L
2020-04-16 12:09 ` Tian, Kevin
2020-04-16 12:42 ` Auger Eric
2020-04-16 13:28 ` Tian, Kevin
2020-04-16 15:12 ` Auger Eric
2020-04-16 14:40 ` Alex Williamson
2020-04-16 14:48 ` Alex Williamson
2020-04-17 6:03 ` Liu, Yi L
2020-03-22 12:32 ` [PATCH v1 8/8] vfio/type1: Add vSVA support for IOMMU-backed mdevs Liu, Yi L
2020-03-30 13:18 ` Tian, Kevin
2020-04-01 7:51 ` Liu, Yi L
2020-04-02 20:33 ` Alex Williamson
2020-04-03 13:39 ` Liu, Yi L
2020-03-26 12:56 ` [PATCH v1 0/8] vfio: expose virtual Shared Virtual Addressing to VMs Liu, Yi L
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=A2975661238FB949B60364EF0F2C25743A229ABC@SHSMSX104.ccr.corp.intel.com \
--to=yi.l.liu@intel.com \
--cc=alex.williamson@redhat.com \
--cc=ashok.raj@intel.com \
--cc=eric.auger@redhat.com \
--cc=hao.wu@intel.com \
--cc=iommu@lists.linux-foundation.org \
--cc=jacob.jun.pan@linux.intel.com \
--cc=jean-philippe@linaro.org \
--cc=joro@8bytes.org \
--cc=jun.j.tian@intel.com \
--cc=kevin.tian@intel.com \
--cc=kvm@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=peterx@redhat.com \
--cc=yi.y.sun@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).