linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [RFC] Make use of non-dynamic dmabuf in RDMA
@ 2021-08-18  7:43 Gal Pressman
  2021-08-18  8:00 ` Christian König
  2021-08-18  9:34 ` Daniel Vetter
  0 siblings, 2 replies; 33+ messages in thread
From: Gal Pressman @ 2021-08-18  7:43 UTC (permalink / raw)
  To: Sumit Semwal, Christian König, Doug Ledford, Jason Gunthorpe
  Cc: linux-media, dri-devel, linux-kernel, linux-rdma, Oded Gabbay,
	Tomer Tayar, Yossi Leybovich, Alexander Matushevsky,
	Leon Romanovsky, Jianxin Xiong, Gal Pressman

Hey all,

Currently, the RDMA subsystem can only work with dynamic dmabuf
attachments, which requires the RDMA device to support on-demand-paging
(ODP) which is not common on most devices (only supported by mlx5).

While the dynamic requirement makes sense for certain GPUs, some devices
(such as habanalabs) have device memory that is always "pinned" and do
not need/use the move_notify operation.

The motivation of this RFC is to use habanalabs as the dmabuf exporter,
and EFA as the importer to allow for peer2peer access through libibverbs.

This draft patch changes the dmabuf driver to differentiate between
static/dynamic attachments by looking at the move_notify op instead of
the importer_ops struct, and allowing the peer2peer flag to be enabled
in case of a static exporter.

Thanks

Signed-off-by: Gal Pressman <galpress@amazon.com>
---
 drivers/dma-buf/dma-buf.c             | 5 +++--
 drivers/infiniband/core/umem_dmabuf.c | 2 +-
 include/linux/dma-buf.h               | 2 +-
 3 files changed, 5 insertions(+), 4 deletions(-)

diff --git a/drivers/dma-buf/dma-buf.c b/drivers/dma-buf/dma-buf.c
index 511fe0d217a0..e3faad8f492c 100644
--- a/drivers/dma-buf/dma-buf.c
+++ b/drivers/dma-buf/dma-buf.c
@@ -727,7 +727,8 @@ dma_buf_dynamic_attach(struct dma_buf *dmabuf, struct device *dev,
 	if (WARN_ON(!dmabuf || !dev))
 		return ERR_PTR(-EINVAL);
 
-	if (WARN_ON(importer_ops && !importer_ops->move_notify))
+	if (WARN_ON(importer_ops && !importer_ops->move_notify &&
+		    dma_buf_is_dynamic(attach->dmabuf)))
 		return ERR_PTR(-EINVAL);
 
 	attach = kzalloc(sizeof(*attach), GFP_KERNEL);
@@ -1048,7 +1049,7 @@ void dma_buf_move_notify(struct dma_buf *dmabuf)
 	dma_resv_assert_held(dmabuf->resv);
 
 	list_for_each_entry(attach, &dmabuf->attachments, node)
-		if (attach->importer_ops)
+		if (attach->importer_ops && attach->importer_ops->move_notify)
 			attach->importer_ops->move_notify(attach);
 }
 EXPORT_SYMBOL_GPL(dma_buf_move_notify);
diff --git a/drivers/infiniband/core/umem_dmabuf.c b/drivers/infiniband/core/umem_dmabuf.c
index c6e875619fac..c502ae828bd3 100644
--- a/drivers/infiniband/core/umem_dmabuf.c
+++ b/drivers/infiniband/core/umem_dmabuf.c
@@ -118,7 +118,7 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
 	if (check_add_overflow(offset, (unsigned long)size, &end))
 		return ret;
 
-	if (unlikely(!ops || !ops->move_notify))
+	if (unlikely(!ops))
 		return ret;
 
 	dmabuf = dma_buf_get(fd);
diff --git a/include/linux/dma-buf.h b/include/linux/dma-buf.h
index efdc56b9d95f..4b2e99012cb1 100644
--- a/include/linux/dma-buf.h
+++ b/include/linux/dma-buf.h
@@ -473,7 +473,7 @@ static inline bool dma_buf_is_dynamic(struct dma_buf *dmabuf)
 static inline bool
 dma_buf_attachment_is_dynamic(struct dma_buf_attachment *attach)
 {
-	return !!attach->importer_ops;
+	return !!attach->importer_ops->move_notify;
 }
 
 struct dma_buf_attachment *dma_buf_attach(struct dma_buf *dmabuf,
-- 
2.32.0


^ permalink raw reply related	[flat|nested] 33+ messages in thread

end of thread, other threads:[~2021-09-02  6:57 UTC | newest]

Thread overview: 33+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-08-18  7:43 [RFC] Make use of non-dynamic dmabuf in RDMA Gal Pressman
2021-08-18  8:00 ` Christian König
2021-08-18  8:37   ` Gal Pressman
2021-08-18  9:34 ` Daniel Vetter
2021-08-19 23:06   ` Jason Gunthorpe
2021-08-20  7:25     ` Daniel Vetter
2021-08-20 12:33       ` Jason Gunthorpe
2021-08-20 12:58         ` Gal Pressman
2021-08-20 14:32           ` Jason Gunthorpe
2021-08-21  9:16             ` Gal Pressman
2021-08-23 10:43               ` Christian König
2021-08-24  9:06                 ` Gal Pressman
2021-08-24  9:32                   ` Christian König
2021-08-24 17:27                     ` John Hubbard
2021-08-24 17:32                       ` Jason Gunthorpe
2021-08-24 17:35                         ` John Hubbard
2021-08-24 19:15                           ` Dave Airlie
2021-08-24 19:30                             ` Jason Gunthorpe
2021-08-24 19:43                             ` Alex Deucher
2021-08-24 20:00                               ` Xiong, Jianxin
2021-08-25  6:17                         ` Christian König
2021-08-25  6:47                           ` John Hubbard
2021-08-25 12:18                           ` Jason Gunthorpe
2021-08-25 12:27                             ` Christian König
2021-08-25 12:38                               ` Jason Gunthorpe
2021-08-25 13:51                                 ` Christian König
2021-08-25 14:47                                   ` Jason Gunthorpe
2021-08-25 15:14                                     ` Christian König
2021-08-25 15:49                                       ` Jason Gunthorpe
2021-08-25 16:02                                       ` Oded Gabbay
2021-09-01 11:20                         ` Gal Pressman
2021-09-01 11:24                           ` Christian König
2021-09-02  6:56                             ` Gal Pressman

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).