From: Logan Gunthorpe <logang@deltatee.com> To: Kit Chow <kchow@gigaio.com>, Dave Jiang <dave.jiang@intel.com>, Eric Pilmore <epilmore@gigaio.com>, Bjorn Helgaas <helgaas@kernel.org> Cc: "linux-pci@vger.kernel.org" <linux-pci@vger.kernel.org>, David Woodhouse <dwmw2@infradead.org>, Alex Williamson <alex.williamson@redhat.com>, "iommu@lists.linux-foundation.org" <iommu@lists.linux-foundation.org> Subject: Re: IOAT DMA w/IOMMU Date: Fri, 10 Aug 2018 20:10:20 -0600 [thread overview] Message-ID: <0a9fa9c3-dbe4-6451-9c1a-06b662702f61@deltatee.com> (raw) In-Reply-To: <9d135f61-20c2-e966-4d7d-3e154384ea08@gigaio.com> On 10/08/18 06:53 PM, Kit Chow wrote: > I was able to finally succeed in doing the dma transfers over ioat only > when prot has DMA_PTE_WRITE set by setting the direction to either > DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. Any ideas if the prot settings > need to be changed? Are there any bad side effects if I used > DMA_BIDIRECTIONAL? Good to hear it. Without digging into the direction much all I can say is that it can sometimes be very confusing what the direction is. Adding another PCI device just adds to the confusion. I believe, the direction should be from the IOAT's point of view. So if the IOAT is writing to the BAR you'd set DMA_FROM_DEVICE (ie. data is coming from the IOAT) and if it's reading you'd set DMA_TO_DEVICE (ie. data is going to the IOAT). Using DMA_BIDIRECTIONAL just forgoes any hardware security / protection that the buffer would have in terms of direction. Generally it's good practice to use the strictest direction you can. > Given that using the pci bar address as is without getting an iommu > address results in the same "PTE Write access" error, I wonder if there > is some internal 'prot' associated with the non-translated pci bar > address that just needs to be tweaked to include DMA_PTE_WRITE??? No, I don't think so. The 'prot' will be a property of the IOMMU. Not having an entry is probably just the same (from the perspective of the error you see) as only having an entry for reading. Logan
WARNING: multiple messages have this Message-ID (diff)
From: Logan Gunthorpe <logang-OTvnGxWRz7hWk0Htik3J/w@public.gmane.org> To: Kit Chow <kchow-Op3I1peydIbQT0dZR+AlfA@public.gmane.org>, Dave Jiang <dave.jiang-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>, Eric Pilmore <epilmore-Op3I1peydIbQT0dZR+AlfA@public.gmane.org>, Bjorn Helgaas <helgaas-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> Cc: "linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org" <linux-pci-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, Alex Williamson <alex.williamson-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>, David Woodhouse <dwmw2-wEGCiKHe2LqWVfeAwA7xHQ@public.gmane.org>, "iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org" <iommu-cunTk1MwBs9QetFLy7KEm3xJsTq8ys+cHZ5vskTnxNA@public.gmane.org> Subject: Re: IOAT DMA w/IOMMU Date: Fri, 10 Aug 2018 20:10:20 -0600 [thread overview] Message-ID: <0a9fa9c3-dbe4-6451-9c1a-06b662702f61@deltatee.com> (raw) In-Reply-To: <9d135f61-20c2-e966-4d7d-3e154384ea08-Op3I1peydIbQT0dZR+AlfA@public.gmane.org> On 10/08/18 06:53 PM, Kit Chow wrote: > I was able to finally succeed in doing the dma transfers over ioat only > when prot has DMA_PTE_WRITE set by setting the direction to either > DMA_FROM_DEVICE or DMA_BIDIRECTIONAL. Any ideas if the prot settings > need to be changed? Are there any bad side effects if I used > DMA_BIDIRECTIONAL? Good to hear it. Without digging into the direction much all I can say is that it can sometimes be very confusing what the direction is. Adding another PCI device just adds to the confusion. I believe, the direction should be from the IOAT's point of view. So if the IOAT is writing to the BAR you'd set DMA_FROM_DEVICE (ie. data is coming from the IOAT) and if it's reading you'd set DMA_TO_DEVICE (ie. data is going to the IOAT). Using DMA_BIDIRECTIONAL just forgoes any hardware security / protection that the buffer would have in terms of direction. Generally it's good practice to use the strictest direction you can. > Given that using the pci bar address as is without getting an iommu > address results in the same "PTE Write access" error, I wonder if there > is some internal 'prot' associated with the non-translated pci bar > address that just needs to be tweaked to include DMA_PTE_WRITE??? No, I don't think so. The 'prot' will be a property of the IOMMU. Not having an entry is probably just the same (from the perspective of the error you see) as only having an entry for reading. Logan
next prev parent reply other threads:[~2018-08-11 2:10 UTC|newest] Thread overview: 95+ messages / expand[flat|nested] mbox.gz Atom feed top 2018-08-09 18:14 IOAT DMA w/IOMMU Eric Pilmore 2018-08-09 18:43 ` Bjorn Helgaas 2018-08-09 18:43 ` Bjorn Helgaas 2018-08-09 18:51 ` Eric Pilmore 2018-08-09 18:51 ` Eric Pilmore 2018-08-09 19:35 ` Logan Gunthorpe 2018-08-09 19:35 ` Logan Gunthorpe 2018-08-09 19:47 ` Kit Chow 2018-08-09 19:47 ` Kit Chow 2018-08-09 20:11 ` Logan Gunthorpe 2018-08-09 20:11 ` Logan Gunthorpe 2018-08-09 20:57 ` Kit Chow 2018-08-09 20:57 ` Kit Chow 2018-08-09 21:11 ` Logan Gunthorpe 2018-08-09 21:11 ` Logan Gunthorpe 2018-08-09 21:47 ` Kit Chow 2018-08-09 21:47 ` Kit Chow 2018-08-09 22:40 ` Jiang, Dave 2018-08-09 22:40 ` Jiang, Dave 2018-08-09 22:48 ` Kit Chow 2018-08-09 22:48 ` Kit Chow 2018-08-09 22:50 ` Logan Gunthorpe 2018-08-09 22:50 ` Logan Gunthorpe 2018-08-09 23:00 ` Kit Chow 2018-08-09 23:00 ` Kit Chow 2018-08-10 16:02 ` Kit Chow 2018-08-10 16:02 ` Kit Chow 2018-08-10 16:23 ` Kit Chow 2018-08-10 16:23 ` Kit Chow 2018-08-10 16:24 ` Logan Gunthorpe 2018-08-10 16:24 ` Logan Gunthorpe 2018-08-10 16:24 ` Logan Gunthorpe 2018-08-10 16:24 ` Logan Gunthorpe 2018-08-10 16:31 ` Dave Jiang 2018-08-10 16:31 ` Dave Jiang 2018-08-10 16:33 ` Logan Gunthorpe 2018-08-10 16:33 ` Logan Gunthorpe 2018-08-10 17:01 ` Dave Jiang 2018-08-10 17:01 ` Dave Jiang 2018-08-10 17:15 ` Logan Gunthorpe 2018-08-10 17:15 ` Logan Gunthorpe 2018-08-10 17:46 ` Dave Jiang 2018-08-10 17:46 ` Dave Jiang 2018-08-11 0:53 ` Kit Chow 2018-08-11 0:53 ` Kit Chow 2018-08-11 2:10 ` Logan Gunthorpe [this message] 2018-08-11 2:10 ` Logan Gunthorpe 2018-08-13 14:23 ` Kit Chow 2018-08-13 14:23 ` Kit Chow 2018-08-13 14:59 ` Robin Murphy 2018-08-13 14:59 ` Robin Murphy 2018-08-13 15:21 ` Kit Chow 2018-08-13 15:21 ` Kit Chow 2018-08-13 23:30 ` Kit Chow 2018-08-13 23:30 ` Kit Chow 2018-08-13 23:39 ` Logan Gunthorpe 2018-08-13 23:39 ` Logan Gunthorpe 2018-08-13 23:48 ` Kit Chow 2018-08-13 23:48 ` Kit Chow 2018-08-13 23:50 ` Logan Gunthorpe 2018-08-13 23:50 ` Logan Gunthorpe 2018-08-14 13:47 ` Kit Chow 2018-08-14 13:47 ` Kit Chow 2018-08-14 14:03 ` Robin Murphy 2018-08-14 14:03 ` Robin Murphy 2018-08-13 23:36 ` Kit Chow 2018-08-13 23:36 ` Kit Chow 2018-08-09 21:31 ` Eric Pilmore 2018-08-09 21:31 ` Eric Pilmore 2018-08-09 21:36 ` Logan Gunthorpe 2018-08-09 21:36 ` Logan Gunthorpe 2018-08-16 17:16 ` Kit Chow 2018-08-16 17:16 ` Kit Chow 2018-08-16 17:21 ` Logan Gunthorpe 2018-08-16 17:21 ` Logan Gunthorpe 2018-08-16 18:53 ` Kit Chow 2018-08-16 18:53 ` Kit Chow 2018-08-16 18:56 ` Logan Gunthorpe 2018-08-16 18:56 ` Logan Gunthorpe 2018-08-21 23:18 ` Eric Pilmore 2018-08-21 23:18 ` Eric Pilmore 2018-08-21 23:20 ` Logan Gunthorpe 2018-08-21 23:20 ` Logan Gunthorpe 2018-08-21 23:28 ` Eric Pilmore 2018-08-21 23:28 ` Eric Pilmore 2018-08-21 23:35 ` Logan Gunthorpe 2018-08-21 23:35 ` Logan Gunthorpe 2018-08-21 23:45 ` Eric Pilmore 2018-08-21 23:45 ` Eric Pilmore 2018-08-21 23:53 ` Logan Gunthorpe 2018-08-21 23:53 ` Logan Gunthorpe 2018-08-21 23:59 ` Eric Pilmore 2018-08-21 23:59 ` Eric Pilmore 2018-08-21 23:30 ` Eric Pilmore 2018-08-21 23:30 ` Eric Pilmore
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=0a9fa9c3-dbe4-6451-9c1a-06b662702f61@deltatee.com \ --to=logang@deltatee.com \ --cc=alex.williamson@redhat.com \ --cc=dave.jiang@intel.com \ --cc=dwmw2@infradead.org \ --cc=epilmore@gigaio.com \ --cc=helgaas@kernel.org \ --cc=iommu@lists.linux-foundation.org \ --cc=kchow@gigaio.com \ --cc=linux-pci@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.