linux-pci.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Kit Chow <kchow@gigaio.com>
To: Logan Gunthorpe <logang@deltatee.com>,
	Dave Jiang <dave.jiang@intel.com>,
	Eric Pilmore <epilmore@gigaio.com>,
	Bjorn Helgaas <helgaas@kernel.org>
Cc: "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>,
	David Woodhouse <dwmw2@infradead.org>,
	Alex Williamson <alex.williamson@redhat.com>,
	"iommu@lists.linux-foundation.org"
	<iommu@lists.linux-foundation.org>
Subject: Re: IOAT DMA w/IOMMU
Date: Mon, 13 Aug 2018 07:23:22 -0700	[thread overview]
Message-ID: <a2391f6c-d73b-c140-edad-8cd6584dd437@gigaio.com> (raw)
In-Reply-To: <0a9fa9c3-dbe4-6451-9c1a-06b662702f61@deltatee.com>



On 08/10/2018 07:10 PM, Logan Gunthorpe wrote:
>
> On 10/08/18 06:53 PM, Kit Chow wrote:
>> I was able to finally succeed in doing the dma transfers over ioat only
>> when prot has DMA_PTE_WRITE set by setting the direction to either
>> DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. Any ideas if the prot settings
>> need to be changed? Are there any bad side effects if I used
>> DMA_BIDIRECTIONAL?
> Good to hear it. Without digging into the direction much all I can say
> is that it can sometimes be very confusing what the direction is. Adding
> another PCI device just adds to the confusion.
Yep, confusing :).

======================= =============================================
DMA_NONE		no direction (used for debugging)
DMA_TO_DEVICE		data is going from the memory to the device
DMA_FROM_DEVICE		data is coming from the device to the memory
DMA_BIDIRECTIONAL	direction isn't known
======================= =============================================

> I believe, the direction should be from the IOAT's point of view. So if
> the IOAT is writing to the BAR you'd set DMA_FROM_DEVICE (ie. data is
> coming from the IOAT) and if it's reading you'd set DMA_TO_DEVICE (ie.
> data is going to the IOAT).
It would certainly seem like DMA_TO_DEVICE would be the proper choice; 
IOAT is the plumbing to move host data (memory) to the bar address (device).

Will go with what works and set DMA_FROM_DEVICE.

In ntb_async_tx_submit, does the direction used for the dma_map routines 
for the src and dest addresses need to be consistent?

And does the direction setting for the dmaengine_unmap_data have to be 
consistent with the direction used in dma_map_*?

BTW, dmaengine_unmap routine only calls dma_unmap_page. Should it keep 
track of the dma_map routine used and call the corresponding dma_unmap 
routine?  In the case of the intel iommu, it doesn't matter.

Thanks
Kit

>
> Using DMA_BIDIRECTIONAL just forgoes any hardware security / protection
> that the buffer would have in terms of direction. Generally it's good
> practice to use the strictest direction you can.
>
>> Given that using the pci bar address as is without getting an iommu
>> address results in the same "PTE Write access" error, I wonder if there
>> is some internal 'prot' associated with the non-translated pci bar
>> address that just needs to be tweaked to include DMA_PTE_WRITE???
> No, I don't think so. The 'prot' will be a property of the IOMMU. Not
> having an entry is probably just the same (from the perspective of the
> error you see) as only having an entry for reading.
>
> Logan

  reply	other threads:[~2018-08-13 14:23 UTC|newest]

Thread overview: 48+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2018-08-09 18:14 IOAT DMA w/IOMMU Eric Pilmore
2018-08-09 18:43 ` Bjorn Helgaas
2018-08-09 18:51   ` Eric Pilmore
2018-08-09 19:35     ` Logan Gunthorpe
2018-08-09 19:47       ` Kit Chow
2018-08-09 20:11         ` Logan Gunthorpe
2018-08-09 20:57           ` Kit Chow
2018-08-09 21:11             ` Logan Gunthorpe
2018-08-09 21:47               ` Kit Chow
2018-08-09 22:40                 ` Jiang, Dave
2018-08-09 22:48                   ` Kit Chow
2018-08-09 22:50                     ` Logan Gunthorpe
2018-08-09 23:00                       ` Kit Chow
2018-08-10 16:02                         ` Kit Chow
2018-08-10 16:23                           ` Kit Chow
2018-08-10 16:24                             ` Logan Gunthorpe
2018-08-10 16:24                           ` Logan Gunthorpe
2018-08-10 16:31                             ` Dave Jiang
2018-08-10 16:33                               ` Logan Gunthorpe
2018-08-10 17:01                                 ` Dave Jiang
2018-08-10 17:15                                   ` Logan Gunthorpe
2018-08-10 17:46                                     ` Dave Jiang
2018-08-11  0:53                                       ` Kit Chow
2018-08-11  2:10                                         ` Logan Gunthorpe
2018-08-13 14:23                                           ` Kit Chow [this message]
2018-08-13 14:59                                             ` Robin Murphy
2018-08-13 15:21                                               ` Kit Chow
2018-08-13 23:30                                                 ` Kit Chow
2018-08-13 23:39                                                   ` Logan Gunthorpe
2018-08-13 23:48                                                     ` Kit Chow
2018-08-13 23:50                                                       ` Logan Gunthorpe
2018-08-14 13:47                                                         ` Kit Chow
2018-08-14 14:03                                                         ` Robin Murphy
2018-08-13 23:36                                                 ` Kit Chow
2018-08-09 21:31       ` Eric Pilmore
2018-08-09 21:36         ` Logan Gunthorpe
2018-08-16 17:16           ` Kit Chow
2018-08-16 17:21             ` Logan Gunthorpe
2018-08-16 18:53               ` Kit Chow
2018-08-16 18:56                 ` Logan Gunthorpe
2018-08-21 23:18                   ` Eric Pilmore
2018-08-21 23:20                     ` Logan Gunthorpe
2018-08-21 23:28                       ` Eric Pilmore
2018-08-21 23:35                         ` Logan Gunthorpe
2018-08-21 23:45                           ` Eric Pilmore
2018-08-21 23:53                             ` Logan Gunthorpe
2018-08-21 23:59                               ` Eric Pilmore
2018-08-21 23:30                       ` Eric Pilmore

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=a2391f6c-d73b-c140-edad-8cd6584dd437@gigaio.com \
    --to=kchow@gigaio.com \
    --cc=alex.williamson@redhat.com \
    --cc=dave.jiang@intel.com \
    --cc=dwmw2@infradead.org \
    --cc=epilmore@gigaio.com \
    --cc=helgaas@kernel.org \
    --cc=iommu@lists.linux-foundation.org \
    --cc=linux-pci@vger.kernel.org \
    --cc=logang@deltatee.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).