From: Daniel Vetter <daniel.vetter@ffwll.ch>
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: "Oded Gabbay" <oded.gabbay@gmail.com>,
"Greg KH" <gregkh@linuxfoundation.org>,
"Oded Gabbay" <ogabbay@kernel.org>,
linux-rdma <linux-rdma@vger.kernel.org>,
"open list:DMA BUFFER SHARING FRAMEWORK"
<linux-media@vger.kernel.org>,
"Doug Ledford" <dledford@redhat.com>,
"airlied@gmail.com" <airlied@gmail.com>,
"Linux Kernel Mailing List" <linux-kernel@vger.kernel.org>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
"Christian König" <christian.koenig@amd.com>,
"Gal Pressman" <galpress@amazon.com>,
sleybo@amazon.com, dri-devel <dri-devel@lists.freedesktop.org>,
"Tomer Tayar" <ttayar@habana.ai>,
"moderated list:DMA BUFFER SHARING FRAMEWORK"
<linaro-mm-sig@lists.linaro.org>,
"amd-gfx list" <amd-gfx@lists.freedesktop.org>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Leon Romanovsky" <leonro@nvidia.com>,
"Christoph Hellwig" <hch@lst.de>
Subject: Re: [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF
Date: Mon, 21 Jun 2021 20:27:32 +0200 [thread overview]
Message-ID: <CAKMK7uEO1_B59DtM7N2g7kkH7pYtLM_WAkn+0f3FU3ps=XEjZQ@mail.gmail.com> (raw)
In-Reply-To: <20210621175511.GI1096940@ziepe.ca>
On Mon, Jun 21, 2021 at 7:55 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> On Mon, Jun 21, 2021 at 07:26:14PM +0300, Oded Gabbay wrote:
> > On Mon, Jun 21, 2021 at 5:12 PM Jason Gunthorpe <jgg@ziepe.ca> wrote:
> > >
> > > On Mon, Jun 21, 2021 at 03:02:10PM +0200, Greg KH wrote:
> > > > On Mon, Jun 21, 2021 at 02:28:48PM +0200, Daniel Vetter wrote:
> > >
> > > > > Also I'm wondering which is the other driver that we share buffers
> > > > > with. The gaudi stuff doesn't have real struct pages as backing
> > > > > storage, it only fills out the dma_addr_t. That tends to blow up with
> > > > > other drivers, and the only place where this is guaranteed to work is
> > > > > if you have a dynamic importer which sets the allow_peer2peer flag.
> > > > > Adding maintainers from other subsystems who might want to chime in
> > > > > here. So even aside of the big question as-is this is broken.
> > > >
> > > > From what I can tell this driver is sending the buffers to other
> > > > instances of the same hardware,
> > >
> > > A dmabuf is consumed by something else in the kernel calling
> > > dma_buf_map_attachment() on the FD.
> > >
> > > What is the other side of this? I don't see any
> > > dma_buf_map_attachment() calls in drivers/misc, or added in this patch
> > > set.
> >
> > This patch-set is only to enable the support for the exporter side.
> > The "other side" is any generic RDMA networking device that will want
> > to perform p2p communication over PCIe with our GAUDI accelerator.
> > An example is indeed the mlnx5 card which has already integrated
> > support for being an "importer".
>
> It raises the question of how you are testing this if you aren't using
> it with the only intree driver: mlx5.
For p2p dma-buf there's also amdgpu as a possible in-tree candiate
driver, that's why I added amdgpu folks. Otoh I'm not aware of AI+GPU
combos being much in use, at least with upstream gpu drivers (nvidia
blob is a different story ofc, but I don't care what they do in their
own world).
-Daniel
--
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
next prev parent reply other threads:[~2021-06-21 18:27 UTC|newest]
Thread overview: 50+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-18 12:36 [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF Oded Gabbay
2021-06-18 12:36 ` [PATCH v3 2/2] habanalabs: add support for dma-buf exporter Oded Gabbay
2021-06-21 12:28 ` [PATCH v3 1/2] habanalabs: define uAPI to export FD for DMA-BUF Daniel Vetter
2021-06-21 13:02 ` Greg KH
2021-06-21 14:12 ` Jason Gunthorpe
2021-06-21 16:26 ` Oded Gabbay
2021-06-21 17:55 ` Jason Gunthorpe
2021-06-21 18:27 ` Daniel Vetter [this message]
2021-06-21 19:24 ` Oded Gabbay
2021-06-21 23:29 ` Jason Gunthorpe
2021-06-22 6:37 ` [Linaro-mm-sig] " Christian König
2021-06-22 8:42 ` Oded Gabbay
2021-06-22 12:01 ` Jason Gunthorpe
2021-06-22 12:04 ` Oded Gabbay
2021-06-22 12:15 ` Jason Gunthorpe
2021-06-22 13:12 ` Oded Gabbay
2021-06-22 15:11 ` Jason Gunthorpe
2021-06-22 15:24 ` Christian König
2021-06-22 15:28 ` Jason Gunthorpe
2021-06-22 15:31 ` Oded Gabbay
2021-06-22 15:31 ` Christian König
2021-06-22 15:40 ` Oded Gabbay
2021-06-22 15:49 ` Christian König
2021-06-22 15:24 ` Oded Gabbay
2021-06-22 15:34 ` Jason Gunthorpe
2021-06-22 12:23 ` Christian König
2021-06-22 15:23 ` Jason Gunthorpe
2021-06-22 15:29 ` Christian König
2021-06-22 15:40 ` Jason Gunthorpe
2021-06-22 15:48 ` Christian König
2021-06-22 16:05 ` Jason Gunthorpe
2021-06-23 8:57 ` Christian König
2021-06-23 9:14 ` Oded Gabbay
2021-06-23 18:24 ` Jason Gunthorpe
2021-06-23 18:43 ` Oded Gabbay
2021-06-23 18:50 ` Jason Gunthorpe
2021-06-23 19:00 ` Oded Gabbay
2021-06-23 19:34 ` Jason Gunthorpe
2021-06-23 19:39 ` Oded Gabbay
2021-06-24 0:45 ` Jason Gunthorpe
2021-06-24 5:40 ` Christoph Hellwig
2021-06-24 5:34 ` Christoph Hellwig
2021-06-24 8:07 ` Christian König
2021-06-24 8:12 ` Christoph Hellwig
2021-06-24 9:52 ` Christian König
2021-06-24 13:22 ` Christoph Hellwig
2021-06-22 16:50 ` Felix Kuehling
2021-06-21 14:20 ` Daniel Vetter
2021-06-21 14:49 ` Jason Gunthorpe
2021-06-21 14:17 ` Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to='CAKMK7uEO1_B59DtM7N2g7kkH7pYtLM_WAkn+0f3FU3ps=XEjZQ@mail.gmail.com' \
--to=daniel.vetter@ffwll.ch \
--cc=airlied@gmail.com \
--cc=alexander.deucher@amd.com \
--cc=amd-gfx@lists.freedesktop.org \
--cc=christian.koenig@amd.com \
--cc=dledford@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=galpress@amazon.com \
--cc=gregkh@linuxfoundation.org \
--cc=hch@lst.de \
--cc=jgg@ziepe.ca \
--cc=leonro@nvidia.com \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=oded.gabbay@gmail.com \
--cc=ogabbay@kernel.org \
--cc=sleybo@amazon.com \
--cc=sumit.semwal@linaro.org \
--cc=ttayar@habana.ai \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).