Re: IOAT DMA w/IOMMU

[Logan Gunthorpe <logang@deltatee.com>]

On 10/08/18 10:31 AM, Dave Jiang wrote:
> 
> 
> On 08/10/2018 09:24 AM, Logan Gunthorpe wrote:
>>
>>
>> On 10/08/18 10:02 AM, Kit Chow wrote:
>>> Turns out there is no dma_map_resource routine on x86. get_dma_ops 
>>> returns intel_dma_ops which has map_resource pointing to NULL.
>>
>> Oh, yup. I wasn't aware of that. From a cursory view, it looks like it
>> shouldn't be too hard to implement though.
>>
>>> Will poke around some in the intel_map_page code but can you actually 
>>> get a valid struct page for a pci bar address (dma_map_single calls 
>>> virt_to_page)?  If not, does a map_resource routine that can properly 
>>> map a pci bar address need to be implemented?
>>
>> Yes, you can not get a struct page for a PCI bar address unless it's
>> mapped with ZONE_DEVICE like in my p2p work. So that would explain why
>> dma_map_single() didn't work.
>>
>> This all implies that ntb_transport doesn't work with DMA and the IOMMU
>> turned on. I'm not sure I've ever tried that configuration myself but it
>> is a bit surprising.
> 
> Hmm....that's surprising because it seems to work on Skylake platform
> when I tested it yesterday with Intel NTB. Kit is using a Haswell
> platform at the moment I think. Although I'm curious if it works with
> the PLX NTB he's using on Skylake.

Does that mean on Skylake the IOAT can bypass the IOMMU? Because it
looks like the ntb_transport code doesn't map the physical address of
the NTB MW into the IOMMU when doing DMA...

Logan