RE: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

["Tian, Kevin" <kevin.tian@intel.com>]

> From: Jason Wang
> Sent: Monday, September 14, 2020 4:57 PM
> 
> On 2020/9/14 下午4:01, Tian, Kevin wrote:
> >> From: Jason Wang <jasowang@redhat.com>
> >> Sent: Monday, September 14, 2020 12:20 PM
> >>
> >> On 2020/9/10 下午6:45, Liu Yi L wrote:
> >>> Shared Virtual Addressing (SVA), a.k.a, Shared Virtual Memory (SVM) on
> >>> Intel platforms allows address space sharing between device DMA and
> >>> applications. SVA can reduce programming complexity and enhance
> >> security.
> >>> This VFIO series is intended to expose SVA usage to VMs. i.e. Sharing
> >>> guest application address space with passthru devices. This is called
> >>> vSVA in this series. The whole vSVA enabling requires
> QEMU/VFIO/IOMMU
> >>> changes. For IOMMU and QEMU changes, they are in separate series
> (listed
> >>> in the "Related series").
> >>>
> >>> The high-level architecture for SVA virtualization is as below, the key
> >>> design of vSVA support is to utilize the dual-stage IOMMU translation (
> >>> also known as IOMMU nesting translation) capability in host IOMMU.
> >>>
> >>>
> >>>       .-------------.  .---------------------------.
> >>>       |   vIOMMU    |  | Guest process CR3, FL only|
> >>>       |             |  '---------------------------'
> >>>       .----------------/
> >>>       | PASID Entry |--- PASID cache flush -
> >>>       '-------------'                       |
> >>>       |             |                       V
> >>>       |             |                CR3 in GPA
> >>>       '-------------'
> >>> Guest
> >>> ------| Shadow |--------------------------|--------
> >>>         v        v                          v
> >>> Host
> >>>       .-------------.  .----------------------.
> >>>       |   pIOMMU    |  | Bind FL for GVA-GPA  |
> >>>       |             |  '----------------------'
> >>>       .----------------/  |
> >>>       | PASID Entry |     V (Nested xlate)
> >>>       '----------------\.------------------------------.
> >>>       |             ||SL for GPA-HPA, default domain|
> >>>       |             |   '------------------------------'
> >>>       '-------------'
> >>> Where:
> >>>    - FL = First level/stage one page tables
> >>>    - SL = Second level/stage two page tables
> >>>
> >>> Patch Overview:
> >>>    1. reports IOMMU nesting info to userspace ( patch 0001, 0002, 0003,
> >> 0015 , 0016)
> >>>    2. vfio support for PASID allocation and free for VMs (patch 0004, 0005,
> >> 0007)
> >>>    3. a fix to a revisit in intel iommu driver (patch 0006)
> >>>    4. vfio support for binding guest page table to host (patch 0008, 0009,
> >> 0010)
> >>>    5. vfio support for IOMMU cache invalidation from VMs (patch 0011)
> >>>    6. vfio support for vSVA usage on IOMMU-backed mdevs (patch 0012)
> >>>    7. expose PASID capability to VM (patch 0013)
> >>>    8. add doc for VFIO dual stage control (patch 0014)
> >>
> >> If it's possible, I would suggest a generic uAPI instead of a VFIO
> >> specific one.
> >>
> >> Jason suggest something like /dev/sva. There will be a lot of other
> >> subsystems that could benefit from this (e.g vDPA).
> >>
> > Just be curious. When does vDPA subsystem plan to support vSVA and
> > when could one expect a SVA-capable vDPA device in market?
> >
> > Thanks
> > Kevin
> 
> 
> vSVA is in the plan but there's no ETA. I think we might start the work
> after control vq support.  It will probably start from SVA first and
> then vSVA (since it might require platform support).
> 
> For the device part, it really depends on the chipset and other device
> vendors. We plan to do the prototype in virtio by introducing PASID
> support in the spec.
> 

Thanks for the info. Then here is my thought.

First, I don't think /dev/sva is the right interface. Once we start 
considering such generic uAPI, it better behaves as the one interface
for all kinds of DMA requirements on device/subdevice passthrough.
Nested page table thru vSVA is one way. Manual map/unmap is
another way. It doesn't make sense to have one through generic
uAPI and the other through subsystem specific uAPI. In the end
the interface might become /dev/iommu, for delegating certain
IOMMU operations to userspace. 

In addition, delegated IOMMU operations have different scopes.
PASID allocation is per process/VM. pgtbl-bind/unbind, map/unmap 
and cache invalidation are per iommu domain. page request/
response are per device/subdevice. This requires the uAPI to also
understand and manage the association between domain/group/
device/subdevice (such as group attach/detach), instead of doing 
it separately in VFIO or vDPA as today. 

Based on above, I feel a more reasonable way is to first make a 
/dev/iommu uAPI supporting DMA map/unmap usages and then 
introduce vSVA to it. Doing this order is because DMA map/unmap 
is widely used thus can better help verify the core logic with 
many existing devices. For vSVA, vDPA support has not be started
while VFIO support is close to be accepted. It doesn't make much 
sense by blocking the VFIO part until vDPA is ready for wide 
verification and /dev/iommu is mature enough. Yes, the newly-
added uAPIs will be finally deprecated when /dev/iommu starts 
to support vSVA. But using /dev/iommu will anyway deprecate 
some existing VFIO IOMMU uAPIs at that time...

Thanks
Kevin
_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu