From: "Christian König" <christian.koenig@amd.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Oded Gabbay" <oded.gabbay@gmail.com>,
"Christian König" <ckoenig.leichtzumerken@gmail.com>,
"Gal Pressman" <galpress@amazon.com>,
sleybo@amazon.com, linux-rdma <linux-rdma@vger.kernel.org>,
"Oded Gabbay" <ogabbay@kernel.org>,
"Christoph Hellwig" <hch@lst.de>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
dri-devel <dri-devel@lists.freedesktop.org>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"Doug Ledford" <dledford@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Greg KH" <gregkh@linuxfoundation.org>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>
Subject: Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Tue, 22 Jun 2021 17:48:10 +0200 [thread overview]
Message-ID: <09df4a03-d99c-3949-05b2-8b49c71a109e@amd.com> (raw)
In-Reply-To: <20210622154027.GS1096940@ziepe.ca>
Am 22.06.21 um 17:40 schrieb Jason Gunthorpe:
> On Tue, Jun 22, 2021 at 05:29:01PM +0200, Christian König wrote:
>> [SNIP]
>> No absolutely not. NVidia GPUs work exactly the same way.
>>
>> And you have tons of similar cases in embedded and SoC systems where
>> intermediate memory between devices isn't directly addressable with the CPU.
> None of that is PCI P2P.
>
> It is all some specialty direct transfer.
>
> You can't reasonably call dma_map_resource() on non CPU mapped memory
> for instance, what address would you pass?
>
> Do not confuse "I am doing transfers between two HW blocks" with PCI
> Peer to Peer DMA transfers - the latter is a very narrow subcase.
>
>> No, just using the dma_map_resource() interface.
> Ik, but yes that does "work". Logan's series is better.
No it isn't. It makes devices depend on allocating struct pages for
their BARs which is not necessary nor desired.
How do you prevent direct I/O on those pages for example?
Allocating a struct pages has their use case, for example for exposing
VRAM as memory for HMM. But that is something very specific and should
not limit PCIe P2P DMA in general.
>> [SNIP]
>> Well that is certainly not true. I'm just not sure if that works with all
>> IOMMU drivers thought.
> Huh? All the iommu interfaces except for the dma_map_resource() are
> struct page based. dma_map_resource() is slow ad limited in what it
> can do.
Yeah, but that is exactly the functionality we need. And as far as I can
see that is also what Oded wants here.
Mapping stuff into userspace and then doing direct DMA to it is only a
very limited use case and we need to be more flexible here.
Christian.
>
> Jason
WARNING: multiple messages have this Message-ID (diff)
From: "Christian König" <christian.koenig@amd.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: linux-rdma <linux-rdma@vger.kernel.org>,
"Christian König" <ckoenig.leichtzumerken@gmail.com>,
sleybo@amazon.com, "Gal Pressman" <galpress@amazon.com>,
dri-devel <dri-devel@lists.freedesktop.org>,
"Christoph Hellwig" <hch@lst.de>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"Doug Ledford" <dledford@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Greg KH" <gregkh@linuxfoundation.org>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"Oded Gabbay" <ogabbay@kernel.org>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>
Subject: Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Tue, 22 Jun 2021 17:48:10 +0200 [thread overview]
Message-ID: <09df4a03-d99c-3949-05b2-8b49c71a109e@amd.com> (raw)
In-Reply-To: <20210622154027.GS1096940@ziepe.ca>
Am 22.06.21 um 17:40 schrieb Jason Gunthorpe:
> On Tue, Jun 22, 2021 at 05:29:01PM +0200, Christian König wrote:
>> [SNIP]
>> No absolutely not. NVidia GPUs work exactly the same way.
>>
>> And you have tons of similar cases in embedded and SoC systems where
>> intermediate memory between devices isn't directly addressable with the CPU.
> None of that is PCI P2P.
>
> It is all some specialty direct transfer.
>
> You can't reasonably call dma_map_resource() on non CPU mapped memory
> for instance, what address would you pass?
>
> Do not confuse "I am doing transfers between two HW blocks" with PCI
> Peer to Peer DMA transfers - the latter is a very narrow subcase.
>
>> No, just using the dma_map_resource() interface.
> Ik, but yes that does "work". Logan's series is better.
No it isn't. It makes devices depend on allocating struct pages for
their BARs which is not necessary nor desired.
How do you prevent direct I/O on those pages for example?
Allocating a struct pages has their use case, for example for exposing
VRAM as memory for HMM. But that is something very specific and should
not limit PCIe P2P DMA in general.
>> [SNIP]
>> Well that is certainly not true. I'm just not sure if that works with all
>> IOMMU drivers thought.
> Huh? All the iommu interfaces except for the dma_map_resource() are
> struct page based. dma_map_resource() is slow ad limited in what it
> can do.
Yeah, but that is exactly the functionality we need. And as far as I can
see that is also what Oded wants here.
Mapping stuff into userspace and then doing direct DMA to it is only a
very limited use case and we need to be more flexible here.
Christian.
>
> Jason
WARNING: multiple messages have this Message-ID (diff)
From: "Christian König" <christian.koenig@amd.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Oded Gabbay" <oded.gabbay@gmail.com>,
linux-rdma <linux-rdma@vger.kernel.org>,
"Christian König" <ckoenig.leichtzumerken@gmail.com>,
sleybo@amazon.com, "Gal Pressman" <galpress@amazon.com>,
dri-devel <dri-devel@lists.freedesktop.org>,
"Christoph Hellwig" <hch@lst.de>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"Doug Ledford" <dledford@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Greg KH" <gregkh@linuxfoundation.org>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"Oded Gabbay" <ogabbay@kernel.org>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>
Subject: Re: [Linaro-mm-sig] [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Tue, 22 Jun 2021 17:48:10 +0200 [thread overview]
Message-ID: <09df4a03-d99c-3949-05b2-8b49c71a109e@amd.com> (raw)
In-Reply-To: <20210622154027.GS1096940@ziepe.ca>
Am 22.06.21 um 17:40 schrieb Jason Gunthorpe:
> On Tue, Jun 22, 2021 at 05:29:01PM +0200, Christian König wrote:
>> [SNIP]
>> No absolutely not. NVidia GPUs work exactly the same way.
>>
>> And you have tons of similar cases in embedded and SoC systems where
>> intermediate memory between devices isn't directly addressable with the CPU.
> None of that is PCI P2P.
>
> It is all some specialty direct transfer.
>
> You can't reasonably call dma_map_resource() on non CPU mapped memory
> for instance, what address would you pass?
>
> Do not confuse "I am doing transfers between two HW blocks" with PCI
> Peer to Peer DMA transfers - the latter is a very narrow subcase.
>
>> No, just using the dma_map_resource() interface.
> Ik, but yes that does "work". Logan's series is better.
No it isn't. It makes devices depend on allocating struct pages for
their BARs which is not necessary nor desired.
How do you prevent direct I/O on those pages for example?
Allocating a struct pages has their use case, for example for exposing
VRAM as memory for HMM. But that is something very specific and should
not limit PCIe P2P DMA in general.
>> [SNIP]
>> Well that is certainly not true. I'm just not sure if that works with all
>> IOMMU drivers thought.
> Huh? All the iommu interfaces except for the dma_map_resource() are
> struct page based. dma_map_resource() is slow ad limited in what it
> can do.
Yeah, but that is exactly the functionality we need. And as far as I can
see that is also what Oded wants here.
Mapping stuff into userspace and then doing direct DMA to it is only a
very limited use case and we need to be more flexible here.
Christian.
>
> Jason
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-06-22 15:48 UTC|newest]
Thread overview: 143+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-18 12:36 [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF Oded Gabbay
2021-06-18 12:36 ` Oded Gabbay
2021-06-18 12:36 ` [PATCH v3 2/2] habanalabs: add support for dma-buf exporter Oded Gabbay
2021-06-18 12:36 ` Oded Gabbay
2021-06-21 12:28 ` [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF Daniel Vetter
2021-06-21 12:28 ` Daniel Vetter
2021-06-21 12:28 ` Daniel Vetter
2021-06-21 13:02 ` Greg KH
2021-06-21 13:02 ` Greg KH
2021-06-21 13:02 ` Greg KH
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 16:26 ` Oded Gabbay
2021-06-21 16:26 ` Oded Gabbay
2021-06-21 16:26 ` Oded Gabbay
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 18:27 ` Daniel Vetter
2021-06-21 18:27 ` Daniel Vetter
2021-06-21 18:27 ` Daniel Vetter
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-22 6:37 ` [Linaro-mm-sig] " Christian König
2021-06-22 6:37 ` Christian König
2021-06-22 6:37 ` Christian König
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:24 ` Christian König
2021-06-22 15:24 ` Christian König
2021-06-22 15:24 ` Christian König
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Christian König
2021-06-22 15:31 ` Christian König
2021-06-22 15:31 ` Christian König
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:49 ` Christian König
2021-06-22 15:49 ` Christian König
2021-06-22 15:49 ` Christian König
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 12:23 ` Christian König
2021-06-22 12:23 ` Christian König
2021-06-22 12:23 ` Christian König
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:29 ` Christian König
2021-06-22 15:29 ` Christian König
2021-06-22 15:29 ` Christian König
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:48 ` Christian König [this message]
2021-06-22 15:48 ` Christian König
2021-06-22 15:48 ` Christian König
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-23 8:57 ` Christian König
2021-06-23 8:57 ` Christian König
2021-06-23 8:57 ` Christian König
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:39 ` Oded Gabbay
2021-06-23 19:39 ` Oded Gabbay
2021-06-23 19:39 ` Oded Gabbay
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 5:40 ` Christoph Hellwig
2021-06-24 5:40 ` Christoph Hellwig
2021-06-24 5:34 ` Christoph Hellwig
2021-06-24 5:34 ` Christoph Hellwig
2021-06-24 8:07 ` Christian König
2021-06-24 8:07 ` Christian König
2021-06-24 8:07 ` Christian König
2021-06-24 8:12 ` Christoph Hellwig
2021-06-24 8:12 ` Christoph Hellwig
2021-06-24 9:52 ` Christian König
2021-06-24 9:52 ` Christian König
2021-06-24 9:52 ` Christian König
2021-06-24 13:22 ` Christoph Hellwig
2021-06-24 13:22 ` Christoph Hellwig
2021-06-22 16:50 ` Felix Kuehling
2021-06-22 16:50 ` Felix Kuehling
2021-06-22 16:50 ` Felix Kuehling
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:49 ` Jason Gunthorpe
2021-06-21 14:49 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=09df4a03-d99c-3949-05b2-8b49c71a109e@amd.com \
--to=christian.koenig@amd.com \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=ckoenig.leichtzumerken@gmail.com \
--cc=dledford@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=galpress@amazon.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=jgg@ziepe.ca \
--cc=leonro@nvidia.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=oded.gabbay@gmail.com \
--cc=ogabbay@kernel.org \
--cc=sleybo@amazon.com \
--cc=ttayar@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.