Re: Xen virtual IOMMU high level design doc V2

[Lan Tianyu <tianyu.lan@intel.com>]

On 2016年10月21日 04:36, Andrew Cooper wrote:
>>> >>
>>>> >>>                     u64 iova;
>>>> >>>                 /* Out parameters. */
>>>> >>>                     u64 translated_addr;
>>>> >>>                     u64 addr_mask; /* Translation page size */
>>>> >>>                     IOMMUAccessFlags permisson;
>>> >>
>>> >> How is this translation intended to be used?  How do you plan to avoid
>>> >> race conditions where qemu requests a translation, receives one, the
>>> >> guest invalidated the mapping, and then qemu tries to use its translated
>>> >> address?
>>> >>
>>> >> There are only two ways I can see of doing this race-free.  One is to
>>> >> implement a "memcpy with translation" hypercall, and the other is to
>>> >> require the use of ATS in the vIOMMU, where the guest OS is required to
>>> >> wait for a positive response from the vIOMMU before it can safely reuse
>>> >> the mapping.
>>> >>
>>> >> The former behaves like real hardware in that an intermediate entity
>>> >> performs the translation without interacting with the DMA source.  The
>>> >> latter explicitly exposing the fact that caching is going on at the
>>> >> endpoint to the OS.
>> >
>> > The former one seems to move DMA operation into hypervisor but Qemu
>> > vIOMMU framework just passes IOVA to dummy xen-vIOMMU without input
>> > data and access length. I will dig more to figure out solution.
> Yes - that does in principle actually move the DMA out of Qemu.

Hi Adnrew:

The first solution "Move the DMA out of Qemu": Qemu vIOMMU framework
just give a chance of doing DMA translation to dummy xen-vIOMMU device
model and DMA access operation is in the vIOMMU core code. It's hard to
move this out. There are a lot of places to call translation callback
and some these are not for DMA access(E,G Map guest memory in Qemu).

The second solution "Use ATS to sync invalidation operation.": This
requires to enable ATS for all virtual PCI devices. This is not easy to do.

The following is my proposal:
When IOMMU driver invalidates IOTLB, it also will wait until the
invalidation completion. We may use this to drain in-fly DMA operation.

Guest triggers invalidation operation and trip into vIOMMU in
hypervisor to flush cache data. After this, it should go to Qemu to
drain in-fly DMA translation.

To do that, dummy vIOMMU in Qemu registers the same MMIO region as
vIOMMU's and emulation part of invalidation operation returns
X86EMUL_UNHANDLEABLE after flush cache. MMIO emulation part is supposed
to send event to Qemu and dummy vIOMMU get a chance to starts a thread
to drain in-fly DMA and return emulation done.

Guest polls IVT(invalidate IOTLB) bit in the IOTLB invalidate register
until it's cleared. Dummy vIOMMU notifies vIOMMU drain operation
completed via hypercall, vIOMMU clears IVT bit and guest finish
invalidation operation.

-- 
Best regards
Tianyu Lan

_______________________________________________
Xen-devel mailing list
Xen-devel@lists.xen.org
https://lists.xen.org/xen-devel