Re: [PATCH v7 00/16] vfio: expose virtual Shared Virtual Addressing to VMs

[Jason Wang <jasowang@redhat.com>]

On 2020/9/17 上午7:09, Jacob Pan (Jun) wrote:
> Hi Jason,
> On Wed, 16 Sep 2020 15:38:41 -0300, Jason Gunthorpe <jgg@nvidia.com>
> wrote:
>
>> On Wed, Sep 16, 2020 at 11:21:10AM -0700, Jacob Pan (Jun) wrote:
>>> Hi Jason,
>>> On Wed, 16 Sep 2020 14:01:13 -0300, Jason Gunthorpe <jgg@nvidia.com>
>>> wrote:
>>>    
>>>> On Wed, Sep 16, 2020 at 09:33:43AM -0700, Raj, Ashok wrote:
>>>>> On Wed, Sep 16, 2020 at 12:07:54PM -0300, Jason Gunthorpe
>>>>> wrote:
>>>>>> On Tue, Sep 15, 2020 at 05:22:26PM -0700, Jacob Pan (Jun)
>>>>>> wrote:
>>>>>>>> If user space wants to bind page tables, create the PASID
>>>>>>>> with /dev/sva, use ioctls there to setup the page table
>>>>>>>> the way it wants, then pass the now configured PASID to a
>>>>>>>> driver that can use it.
>>>>>>> Are we talking about bare metal SVA?
>>>>>> What a weird term.
>>>>> Glad you noticed it at v7 :-)
>>>>>
>>>>> Any suggestions on something less weird than
>>>>> Shared Virtual Addressing? There is a reason why we moved from
>>>>> SVM to SVA.
>>>> SVA is fine, what is "bare metal" supposed to mean?
>>>>    
>>> What I meant here is sharing virtual address between DMA and host
>>> process. This requires devices perform DMA request with PASID and
>>> use IOMMU first level/stage 1 page tables.
>>> This can be further divided into 1) user SVA 2) supervisor SVA
>>> (sharing init_mm)
>>>
>>> My point is that /dev/sva is not useful here since the driver can
>>> perform PASID allocation while doing SVA bind.
>> No, you are thinking too small.
>>
>> Look at VDPA, it has a SVA uAPI. Some HW might use PASID for the SVA.
>>
> Could you point to me the SVA UAPI? I couldn't find it in the mainline.
> Seems VDPA uses VHOST interface?


It's the vhost_iotlb_msg defined in uapi/linux/vhost_types.h.


>
>> When VDPA is used by DPDK it makes sense that the PASID will be SVA
>> and 1:1 with the mm_struct.
>>
> I still don't see why bare metal DPDK needs to get a handle of the
> PASID.


My understanding is that it may:

- have a unified uAPI with vSVA: alloc, bind, unbind, free
- leave the binding policy to userspace instead of the using a implied 
one in the kenrel


> Perhaps the SVA patch would explain. Or are you talking about
> vDPA DPDK process that is used to support virtio-net-pmd in the guest?
>
>> When VDPA is used by qemu it makes sense that the PASID will be an
>> arbitary IOVA map constructed to be 1:1 with the guest vCPU physical
>> map. /dev/sva allows a single uAPI to do this kind of setup, and qemu
>> can support it while supporting a range of SVA kernel drivers. VDPA
>> and vfio-mdev are obvious initial targets.
>>
>> *BOTH* are needed.
>>
>> In general any uAPI for PASID should have the option to use either the
>> mm_struct SVA PASID *OR* a PASID from /dev/sva. It costs virtually
>> nothing to implement this in the driver as PASID is just a number, and
>> gives so much more flexability.
>>
> Not really nothing in terms of PASID life cycles. For example, if user
> uses uacce interface to open an accelerator, it gets an FD_acc. Then it
> opens /dev/sva to allocate PASID then get another FD_pasid. Then we
> pass FD_pasid to the driver to bind page tables, perhaps multiple
> drivers. Now we have to worry about If FD_pasid gets closed before
> FD_acc(s) closed and all these race conditions.


I'm not sure I understand this. But this demonstrates the flexibility of 
an unified uAPI. E.g it allows vDPA and VFIO device to use the same 
PAISD which can be shared with a process in the guest.

For the race condition, it could be probably solved with refcnt.

Thanks


>
> If we do not expose FD_pasid to the user, the teardown is much simpler
> and streamlined. Following each FD_acc close, PASID unbind is performed.
>
>>> Yi can correct me but this set is is about VFIO-PCI, VFIO-mdev will
>>> be introduced later.
>> Last patch is:
>>
>>    vfio/type1: Add vSVA support for IOMMU-backed mdevs
>>
>> So pretty hard to see how this is not about vfio-mdev, at least a
>> little..
>>
>> Jason
>
> Thanks,
>
> Jacob
>

_______________________________________________
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu