Re: Support SVM without PASID

[Jean-Philippe Brucker <jean-philippe.brucker@arm.com>]

Hi Bob,

On 07/08/17 13:18, Bob Liu wrote:
> On 2017/8/7 18:31, Jean-Philippe Brucker wrote:
>> On 05/08/17 06:14, valmiki wrote:
>> [...]
>>> Hi Jean, Thanks a lot, now i understood the flow. From vfio kernel
>>> documentation we fill vaddr and iova in struct vfio_iommu_type1_dma_map
>>> and pass them to VFIO. But if we use dynamic allocation in application
>>> (say malloc), do we need to use dma API to get iova and then call
>>> VFIO_IOMMU_MAP ioctl ?
>>> If application needs multiple such dynamic allocations, then it need to
>>> allocate large chunk and program it via VFIO_IOMMU_MAP ioctl and then
>>> manage rest allocations requirements from this buffer ?
>>
>> Yes, without SVM, the application allocates large buffers, allocates IOVAs
>> itself, and maps them with VFIO_IOMMU_MAP. Userspace doesn't rely on the
>> DMA API at all, it manages IOVAs as it wants. Sizes passed to
>> VFIO_IOMMU_MAP have to be multiples of the MMU or IOMMU page granularity
>> (that is at least 4kB), and both iova and vaddr have to be aligned on that
>> granularity as well. So malloc isn't really suitable in this case, you'll
>> need mmap. The application can then implement a small allocator to manage
>> the DMA pool created with VFIO_IOMMU_MAP.
>>
>> With SVM the application binds its address space to the device, and then
>> uses malloc for all DMA buffers, no need for VFIO_IOMMU_MAP.
>>
> 
> Hi Jean,
> 
> I think there is another way to support SVM without PASID.
> 
> Suppose there is a device in the same SOC-chip, the device access memory through SMMU(using internal bus instead of PCIe)
> Once page fault, the device send an event with (vaddr, substreamID) to SMMU, then SMMU triggers an event interrupt.
> 
> In the event interrupt handler, we can implement the same logic as PRI interrupt in your patch.
> What do you think about that?
What you're describing is the SMMU stall model for platform devices. From
the driver perspective it's the same as PRI and PASID (SubstreamID=PASID).

When a stall-capable device accesses unmapped memory, the SMMU parks the
transaction and sends an event marked "stall" on the event queue, with a
stall tag (STAG, roughly equivalent to PRG Index). The OS handles the
fault and sends a CMD_RESUME command with the status and the STAG. Then
the SMMU completes the access or terminates it.

In a prototype I have, the stall implementation reuses most of the
PASID/PRI code. The main difficulty is defining SSID and stall capability
in firmware, as there are no standard capability probing for platform
devices. Stall-capable devices must be able to wait an indefinite amount
of time that their DMA transactions returns, therefore the stall model
cannot work with PCI, only some integrated devices.

Thanks,
Jean