From: Oded Gabbay <oded.gabbay@gmail.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Greg KH" <gregkh@linuxfoundation.org>,
"Daniel Vetter" <daniel.vetter@ffwll.ch>,
"Oded Gabbay" <ogabbay@kernel.org>,
linux-rdma <linux-rdma@vger.kernel.org>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>,
"Doug Ledford" <dledford@redhat.com>,
"airlied@gmail.com" <airlied@gmail.com>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"Gal Pressman" <galpress@amazon.com>,
sleybo@amazon.com, dri-devel <dri-devel@lists.freedesktop.org>,
"Tomer Tayar" <ttayar@habana.ai>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"Christoph Hellwig" <hch@lst.de>
Subject: Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Mon, 21 Jun 2021 19:26:14 +0300 [thread overview]
Message-ID: <CAFCwf10KvCh0zfHEHqYR-Na6KJh4j+9i-6+==QaMdHHpLH1yEA@mail.gmail.com> (raw)
In-Reply-To: <20210621141217.GE1096940@ziepe.ca>
On Mon, Jun 21, 2021 at 5:12 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Mon, Jun 21, 2021 at 03:02:10PM +0200, Greg KH wrote:
> > On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote:
>
> > > Also I'm wondering which is the other driver that we share buffers
> > > with. The gaudi stuff doesn't have real struct pages as backing
> > > storage, it only fills out the dma_addr_t. That tends to blow up with
> > > other drivers, and the only place where this is guaranteed to work is
> > > if you have a dynamic importer which sets the allow_peer2peer flag.
> > > Adding maintainers from other subsystems who might want to chime in
> > > here. So even aside of the big question as-is this is broken.
> >
> > From what I can tell this driver is sending the buffers to other
> > instances of the same hardware,
>
> A dmabuf is consumed by something else in the kernel calling
> dma_buf_map_attachment() on the FD.
>
> What is the other side of this? I don't see any
> dma_buf_map_attachment() calls in drivers/misc, or added in this patch
> set.
This patch-set is only to enable the support for the exporter side.
The "other side" is any generic RDMA networking device that will want
to perform p2p communication over PCIe with our GAUDI accelerator.
An example is indeed the mlnx5 card which has already integrated
support for being an "importer".
This is *not* used for communication with another GAUDI device. If I
want to communicate with another GAUDI device, our userspace
communications library will use our internal network links, without
any need for dma-buf.
Oded
>
> AFAIK the only viable in-tree other side is in mlx5 (look in
> umem_dmabuf.c)
>
> Though as we already talked habana has their own networking (out of
> tree, presumably) so I suspect this is really to support some out of
> tree stuff??
>
> Jason
WARNING: multiple messages have this Message-ID (diff)
From: Oded Gabbay <oded.gabbay@gmail.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Gal Pressman" <galpress@amazon.com>,
sleybo@amazon.com, linux-rdma <linux-rdma@vger.kernel.org>,
"Greg KH" <gregkh@linuxfoundation.org>,
"Oded Gabbay" <ogabbay@kernel.org>,
"Christoph Hellwig" <hch@lst.de>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
dri-devel <dri-devel@lists.freedesktop.org>,
"Christian König" <christian.koenig@amd.com>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"Doug Ledford" <dledford@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Daniel Vetter" <daniel.vetter@ffwll.ch>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>
Subject: Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Mon, 21 Jun 2021 19:26:14 +0300 [thread overview]
Message-ID: <CAFCwf10KvCh0zfHEHqYR-Na6KJh4j+9i-6+==QaMdHHpLH1yEA@mail.gmail.com> (raw)
In-Reply-To: <20210621141217.GE1096940@ziepe.ca>
On Mon, Jun 21, 2021 at 5:12 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Mon, Jun 21, 2021 at 03:02:10PM +0200, Greg KH wrote:
> > On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote:
>
> > > Also I'm wondering which is the other driver that we share buffers
> > > with. The gaudi stuff doesn't have real struct pages as backing
> > > storage, it only fills out the dma_addr_t. That tends to blow up with
> > > other drivers, and the only place where this is guaranteed to work is
> > > if you have a dynamic importer which sets the allow_peer2peer flag.
> > > Adding maintainers from other subsystems who might want to chime in
> > > here. So even aside of the big question as-is this is broken.
> >
> > From what I can tell this driver is sending the buffers to other
> > instances of the same hardware,
>
> A dmabuf is consumed by something else in the kernel calling
> dma_buf_map_attachment() on the FD.
>
> What is the other side of this? I don't see any
> dma_buf_map_attachment() calls in drivers/misc, or added in this patch
> set.
This patch-set is only to enable the support for the exporter side.
The "other side" is any generic RDMA networking device that will want
to perform p2p communication over PCIe with our GAUDI accelerator.
An example is indeed the mlnx5 card which has already integrated
support for being an "importer".
This is *not* used for communication with another GAUDI device. If I
want to communicate with another GAUDI device, our userspace
communications library will use our internal network links, without
any need for dma-buf.
Oded
>
> AFAIK the only viable in-tree other side is in mlx5 (look in
> umem_dmabuf.c)
>
> Though as we already talked habana has their own networking (out of
> tree, presumably) so I suspect this is really to support some out of
> tree stuff??
>
> Jason
WARNING: multiple messages have this Message-ID (diff)
From: Oded Gabbay <oded.gabbay@gmail.com>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Gal Pressman" <galpress@amazon.com>,
sleybo@amazon.com, linux-rdma <linux-rdma@vger.kernel.org>,
"Greg KH" <gregkh@linuxfoundation.org>,
"Oded Gabbay" <ogabbay@kernel.org>,
"Christoph Hellwig" <hch@lst.de>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
dri-devel <dri-devel@lists.freedesktop.org>,
"Christian König" <christian.koenig@amd.com>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"Doug Ledford" <dledford@redhat.com>,
"Tomer Tayar" <ttayar@habana.ai>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Daniel Vetter" <daniel.vetter@ffwll.ch>,
"Alex Deucher" <alexander.deucher@amd.com>,
"airlied@gmail.com" <airlied@gmail.com>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Leon Romanovsky" <leonro@nvidia.com>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>
Subject: Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Mon, 21 Jun 2021 19:26:14 +0300 [thread overview]
Message-ID: <CAFCwf10KvCh0zfHEHqYR-Na6KJh4j+9i-6+==QaMdHHpLH1yEA@mail.gmail.com> (raw)
In-Reply-To: <20210621141217.GE1096940@ziepe.ca>
On Mon, Jun 21, 2021 at 5:12 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
>
> On Mon, Jun 21, 2021 at 03:02:10PM +0200, Greg KH wrote:
> > On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote:
>
> > > Also I'm wondering which is the other driver that we share buffers
> > > with. The gaudi stuff doesn't have real struct pages as backing
> > > storage, it only fills out the dma_addr_t. That tends to blow up with
> > > other drivers, and the only place where this is guaranteed to work is
> > > if you have a dynamic importer which sets the allow_peer2peer flag.
> > > Adding maintainers from other subsystems who might want to chime in
> > > here. So even aside of the big question as-is this is broken.
> >
> > From what I can tell this driver is sending the buffers to other
> > instances of the same hardware,
>
> A dmabuf is consumed by something else in the kernel calling
> dma_buf_map_attachment() on the FD.
>
> What is the other side of this? I don't see any
> dma_buf_map_attachment() calls in drivers/misc, or added in this patch
> set.
This patch-set is only to enable the support for the exporter side.
The "other side" is any generic RDMA networking device that will want
to perform p2p communication over PCIe with our GAUDI accelerator.
An example is indeed the mlnx5 card which has already integrated
support for being an "importer".
This is *not* used for communication with another GAUDI device. If I
want to communicate with another GAUDI device, our userspace
communications library will use our internal network links, without
any need for dma-buf.
Oded
>
> AFAIK the only viable in-tree other side is in mlx5 (look in
> umem_dmabuf.c)
>
> Though as we already talked habana has their own networking (out of
> tree, presumably) so I suspect this is really to support some out of
> tree stuff??
>
> Jason
_______________________________________________
amd-gfx mailing list
amd-gfx@lists.freedesktop.org
https://lists.freedesktop.org/mailman/listinfo/amd-gfx
next prev parent reply other threads:[~2021-06-21 16:55 UTC|newest]
Thread overview: 143+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-18 12:36 [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF Oded Gabbay
2021-06-18 12:36 ` Oded Gabbay
2021-06-18 12:36 ` [PATCH v3 2/2] habanalabs: add support for dma-buf exporter Oded Gabbay
2021-06-18 12:36 ` Oded Gabbay
2021-06-21 12:28 ` [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF Daniel Vetter
2021-06-21 12:28 ` Daniel Vetter
2021-06-21 12:28 ` Daniel Vetter
2021-06-21 13:02 ` Greg KH
2021-06-21 13:02 ` Greg KH
2021-06-21 13:02 ` Greg KH
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 16:26 ` Oded Gabbay [this message]
2021-06-21 16:26 ` Oded Gabbay
2021-06-21 16:26 ` Oded Gabbay
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 18:27 ` Daniel Vetter
2021-06-21 18:27 ` Daniel Vetter
2021-06-21 18:27 ` Daniel Vetter
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-22 6:37 ` [Linaro-mm-sig] " Christian König
2021-06-22 6:37 ` Christian König
2021-06-22 6:37 ` Christian König
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:24 ` Christian König
2021-06-22 15:24 ` Christian König
2021-06-22 15:24 ` Christian König
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Christian König
2021-06-22 15:31 ` Christian König
2021-06-22 15:31 ` Christian König
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:49 ` Christian König
2021-06-22 15:49 ` Christian König
2021-06-22 15:49 ` Christian König
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 12:23 ` Christian König
2021-06-22 12:23 ` Christian König
2021-06-22 12:23 ` Christian König
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:29 ` Christian König
2021-06-22 15:29 ` Christian König
2021-06-22 15:29 ` Christian König
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:48 ` Christian König
2021-06-22 15:48 ` Christian König
2021-06-22 15:48 ` Christian König
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-23 8:57 ` Christian König
2021-06-23 8:57 ` Christian König
2021-06-23 8:57 ` Christian König
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:39 ` Oded Gabbay
2021-06-23 19:39 ` Oded Gabbay
2021-06-23 19:39 ` Oded Gabbay
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 5:40 ` Christoph Hellwig
2021-06-24 5:40 ` Christoph Hellwig
2021-06-24 5:34 ` Christoph Hellwig
2021-06-24 5:34 ` Christoph Hellwig
2021-06-24 8:07 ` Christian König
2021-06-24 8:07 ` Christian König
2021-06-24 8:07 ` Christian König
2021-06-24 8:12 ` Christoph Hellwig
2021-06-24 8:12 ` Christoph Hellwig
2021-06-24 9:52 ` Christian König
2021-06-24 9:52 ` Christian König
2021-06-24 9:52 ` Christian König
2021-06-24 13:22 ` Christoph Hellwig
2021-06-24 13:22 ` Christoph Hellwig
2021-06-22 16:50 ` Felix Kuehling
2021-06-22 16:50 ` Felix Kuehling
2021-06-22 16:50 ` Felix Kuehling
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:49 ` Jason Gunthorpe
2021-06-21 14:49 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAFCwf10KvCh0zfHEHqYR-Na6KJh4j+9i-6+==QaMdHHpLH1yEA@mail.gmail.com' \
--to=oded.gabbay@gmail.com \
--cc=airlied@gmail.com \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
--cc=daniel.vetter@ffwll.ch \
--cc=dledford@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=galpress@amazon.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=jgg@ziepe.ca \
--cc=leonro@nvidia.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=ogabbay@kernel.org \
--cc=sleybo@amazon.com \
--cc=sumit.semwal@linaro.org \
--cc=ttayar@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.