From: Shunsuke Mie <mie@igel.co.jp>
To: Zhu Yanjun <zyjzyj2000@gmail.com>
Cc: "Shunsuke Mie" <mie@igel.co.jp>,
"Christian König" <christian.koenig@amd.com>,
"Alex Deucher" <alexander.deucher@amd.com>,
"Daniel Vetter" <daniel.vetter@ffwll.ch>,
"Doug Ledford" <dledford@redhat.com>,
"Jason Gunthorpe" <jgg@ziepe.ca>,
"Jianxin Xiong" <jianxin.xiong@intel.com>,
"Leon Romanovsky" <leon@kernel.org>,
"Maor Gottlieb" <maorg@nvidia.com>,
"Sean Hefty" <sean.hefty@intel.com>,
"Sumit Semwal" <sumit.semwal@linaro.org>,
dri-devel@lists.freedesktop.org, linaro-mm-sig@lists.linaro.org,
linux-media@vger.kernel.org, linux-kernel@vger.kernel.org,
linux-rdma@vger.kernel.org, dhobsong@igel.co.jp, taki@igel.co.jp,
etom@igel.co.jp
Subject: [RFC PATCH v2 1/2] RDMA/umem: Change for rdma devices has not dma device
Date: Wed, 29 Sep 2021 13:19:04 +0900 [thread overview]
Message-ID: <20210929041905.126454-2-mie@igel.co.jp> (raw)
In-Reply-To: <20210929041905.126454-1-mie@igel.co.jp>
Current implementation requires a dma device for RDMA driver to use
dma-buf memory space as RDMA buffer. However, software RDMA drivers has
not dma device and copy RDMA data using CPU instead of hardware.
This patch changes to be hold a dma-buf on struct ib_umem_dmabuf. This
allows the software RDMA driver to map dma-buf memory for CPU memory
accessing.
Signed-off-by: Shunsuke Mie <mie@igel.co.jp>
---
drivers/infiniband/core/umem_dmabuf.c | 20 ++++++++++++++++----
include/rdma/ib_umem.h | 1 +
2 files changed, 17 insertions(+), 4 deletions(-)
diff --git a/drivers/infiniband/core/umem_dmabuf.c b/drivers/infiniband/core/umem_dmabuf.c
index e824baf4640d..ebbb0a259fd4 100644
--- a/drivers/infiniband/core/umem_dmabuf.c
+++ b/drivers/infiniband/core/umem_dmabuf.c
@@ -117,9 +117,6 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
if (check_add_overflow(offset, (unsigned long)size, &end))
return ret;
- if (unlikely(!ops || !ops->move_notify))
- return ret;
-
dmabuf = dma_buf_get(fd);
if (IS_ERR(dmabuf))
return ERR_CAST(dmabuf);
@@ -133,6 +130,8 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
goto out_release_dmabuf;
}
+ umem_dmabuf->dmabuf = dmabuf;
+
umem = &umem_dmabuf->umem;
umem->ibdev = device;
umem->length = size;
@@ -143,6 +142,13 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
if (!ib_umem_num_pages(umem))
goto out_free_umem;
+ /* Software RDMA drivers has not dma device. Just get dmabuf from fd */
+ if (!device->dma_device)
+ goto done;
+
+ if (unlikely(!ops || !ops->move_notify))
+ goto out_free_umem;
+
umem_dmabuf->attach = dma_buf_dynamic_attach(
dmabuf,
device->dma_device,
@@ -152,6 +158,7 @@ struct ib_umem_dmabuf *ib_umem_dmabuf_get(struct ib_device *device,
ret = ERR_CAST(umem_dmabuf->attach);
goto out_free_umem;
}
+done:
return umem_dmabuf;
out_free_umem:
@@ -165,13 +172,18 @@ EXPORT_SYMBOL(ib_umem_dmabuf_get);
void ib_umem_dmabuf_release(struct ib_umem_dmabuf *umem_dmabuf)
{
- struct dma_buf *dmabuf = umem_dmabuf->attach->dmabuf;
+ struct dma_buf *dmabuf = umem_dmabuf->dmabuf;
+
+ if (!umem_dmabuf->attach)
+ goto free_dmabuf;
dma_resv_lock(dmabuf->resv, NULL);
ib_umem_dmabuf_unmap_pages(umem_dmabuf);
dma_resv_unlock(dmabuf->resv);
dma_buf_detach(dmabuf, umem_dmabuf->attach);
+
+free_dmabuf:
dma_buf_put(dmabuf);
kfree(umem_dmabuf);
}
diff --git a/include/rdma/ib_umem.h b/include/rdma/ib_umem.h
index 5ae9dff74dac..11c0cf7e0dd8 100644
--- a/include/rdma/ib_umem.h
+++ b/include/rdma/ib_umem.h
@@ -32,6 +32,7 @@ struct ib_umem {
struct ib_umem_dmabuf {
struct ib_umem umem;
struct dma_buf_attachment *attach;
+ struct dma_buf *dmabuf;
struct sg_table *sgt;
struct scatterlist *first_sg;
struct scatterlist *last_sg;
--
2.17.1
next prev parent reply other threads:[~2021-09-29 4:19 UTC|newest]
Thread overview: 10+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-09-29 4:19 [RFC PATCH v2 0/2] RDMA/rxe: Add dma-buf support Shunsuke Mie
2021-09-29 4:19 ` Shunsuke Mie [this message]
2021-09-29 4:19 ` [RFC PATCH v2 2/2] " Shunsuke Mie
2021-09-30 14:41 ` Daniel Vetter
2021-10-01 3:56 ` Shunsuke Mie
2021-10-05 10:20 ` Shunsuke Mie
2021-10-10 15:01 ` Zhu Yanjun
2021-10-13 11:42 ` Daniel Vetter
2021-10-13 23:10 ` Shunsuke Mie
2021-10-10 15:31 ` [RFC PATCH v2 0/2] " Zhu Yanjun
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210929041905.126454-2-mie@igel.co.jp \
--to=mie@igel.co.jp \
--cc=alexander.deucher@amd.com \
--cc=christian.koenig@amd.com \
--cc=daniel.vetter@ffwll.ch \
--cc=dhobsong@igel.co.jp \
--cc=dledford@redhat.com \
--cc=dri-devel@lists.freedesktop.org \
--cc=etom@igel.co.jp \
--cc=jgg@ziepe.ca \
--cc=jianxin.xiong@intel.com \
--cc=leon@kernel.org \
--cc=linaro-mm-sig@lists.linaro.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-media@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=maorg@nvidia.com \
--cc=sean.hefty@intel.com \
--cc=sumit.semwal@linaro.org \
--cc=taki@igel.co.jp \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).