* [RFC PATCH v2 0/1] Providers/rxe: Add dma-buf support @ 2021-09-30 6:20 Shunsuke Mie 2021-09-30 6:20 ` [RFC PATCH v2 1/1] " Shunsuke Mie 0 siblings, 1 reply; 8+ messages in thread From: Shunsuke Mie @ 2021-09-30 6:20 UTC (permalink / raw) To: Zhu Yanjun Cc: Shunsuke Mie, Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, linux-kernel, linux-rdma, dhobsong, taki, etom This is an user space counter-part of the kernel patch set to add dma-buf support to the RXE driver. Pull request at GitHub: https://github.com/linux-rdma/rdma-core/pull/1055 This is the secound version of the patch. Change log: v2: * Fix code formats that were indicated by Azure CI v1: https://www.spinics.net/lists/linux-rdma/msg105380.html * Initial patch set * Implement a callback function for reg_dmabuf_mr Shunsuke Mie (1): Providers/rxe: Add dma-buf support providers/rxe/rxe.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) -- 2.17.1 ^ permalink raw reply [flat|nested] 8+ messages in thread
* [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 6:20 [RFC PATCH v2 0/1] Providers/rxe: Add dma-buf support Shunsuke Mie @ 2021-09-30 6:20 ` Shunsuke Mie 2021-09-30 6:37 ` Zhu Yanjun 0 siblings, 1 reply; 8+ messages in thread From: Shunsuke Mie @ 2021-09-30 6:20 UTC (permalink / raw) To: Zhu Yanjun Cc: Shunsuke Mie, Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, linux-kernel, linux-rdma, dhobsong, taki, etom Implement a new provider method for dma-buf base memory registration. Signed-off-by: Shunsuke Mie <mie@igel.co.jp> --- providers/rxe/rxe.c | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c index 3c3ea8bb..84e00e60 100644 --- a/providers/rxe/rxe.c +++ b/providers/rxe/rxe.c @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, return &vmr->ibv_mr; } +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, + size_t length, uint64_t iova, int fd, + int access) +{ + struct verbs_mr *vmr; + int ret; + + vmr = malloc(sizeof(*vmr)); + if (!vmr) + return NULL; + + ret = ibv_cmd_reg_dmabuf_mr(pd, offset, length, iova, fd, access, vmr); + if (ret) { + free(vmr); + return NULL; + } + + return &vmr->ibv_mr; +} + static int rxe_dereg_mr(struct verbs_mr *vmr) { int ret; @@ -1706,6 +1726,7 @@ static const struct verbs_context_ops rxe_ctx_ops = { .alloc_pd = rxe_alloc_pd, .dealloc_pd = rxe_dealloc_pd, .reg_mr = rxe_reg_mr, + .reg_dmabuf_mr = rxe_reg_dmabuf_mr, .dereg_mr = rxe_dereg_mr, .alloc_mw = rxe_alloc_mw, .dealloc_mw = rxe_dealloc_mw, -- 2.17.1 ^ permalink raw reply related [flat|nested] 8+ messages in thread
* Re: [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 6:20 ` [RFC PATCH v2 1/1] " Shunsuke Mie @ 2021-09-30 6:37 ` Zhu Yanjun 2021-09-30 6:58 ` Shunsuke Mie 0 siblings, 1 reply; 8+ messages in thread From: Zhu Yanjun @ 2021-09-30 6:37 UTC (permalink / raw) To: Shunsuke Mie Cc: Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, LKML, RDMA mailing list, dhobsong, taki, etom On Thu, Sep 30, 2021 at 2:20 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > Implement a new provider method for dma-buf base memory registration. > > Signed-off-by: Shunsuke Mie <mie@igel.co.jp> > --- > providers/rxe/rxe.c | 21 +++++++++++++++++++++ > 1 file changed, 21 insertions(+) > > diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c > index 3c3ea8bb..84e00e60 100644 > --- a/providers/rxe/rxe.c > +++ b/providers/rxe/rxe.c > @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, > return &vmr->ibv_mr; > } > > +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, > + size_t length, uint64_t iova, int fd, > + int access) > +{ > + struct verbs_mr *vmr; > + int ret; > + > + vmr = malloc(sizeof(*vmr)); > + if (!vmr) > + return NULL; > + Do we need to set vmr to zero like the following? memset(vmr, 0, sizeof(*vmr)); Zhu Yanjun > + ret = ibv_cmd_reg_dmabuf_mr(pd, offset, length, iova, fd, access, vmr); > + if (ret) { > + free(vmr); > + return NULL; > + } > + > + return &vmr->ibv_mr; > +} > + > static int rxe_dereg_mr(struct verbs_mr *vmr) > { > int ret; > @@ -1706,6 +1726,7 @@ static const struct verbs_context_ops rxe_ctx_ops = { > .alloc_pd = rxe_alloc_pd, > .dealloc_pd = rxe_dealloc_pd, > .reg_mr = rxe_reg_mr, > + .reg_dmabuf_mr = rxe_reg_dmabuf_mr, > .dereg_mr = rxe_dereg_mr, > .alloc_mw = rxe_alloc_mw, > .dealloc_mw = rxe_dealloc_mw, > -- > 2.17.1 > ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 6:37 ` Zhu Yanjun @ 2021-09-30 6:58 ` Shunsuke Mie 2021-09-30 7:23 ` Zhu Yanjun 0 siblings, 1 reply; 8+ messages in thread From: Shunsuke Mie @ 2021-09-30 6:58 UTC (permalink / raw) To: Zhu Yanjun Cc: Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, LKML, RDMA mailing list, Damian Hobson-Garcia, Takanari Hayama, Tomohito Esaki 2021年9月30日(木) 15:37 Zhu Yanjun <zyjzyj2000@gmail.com>: > > On Thu, Sep 30, 2021 at 2:20 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > Implement a new provider method for dma-buf base memory registration. > > > > Signed-off-by: Shunsuke Mie <mie@igel.co.jp> > > --- > > providers/rxe/rxe.c | 21 +++++++++++++++++++++ > > 1 file changed, 21 insertions(+) > > > > diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c > > index 3c3ea8bb..84e00e60 100644 > > --- a/providers/rxe/rxe.c > > +++ b/providers/rxe/rxe.c > > @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, > > return &vmr->ibv_mr; > > } > > > > +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, > > + size_t length, uint64_t iova, int fd, > > + int access) > > +{ > > + struct verbs_mr *vmr; > > + int ret; > > + > > + vmr = malloc(sizeof(*vmr)); > > + if (!vmr) > > + return NULL; > > + > > Do we need to set vmr to zero like the following? > > memset(vmr, 0, sizeof(*vmr)); > > Zhu Yanjun Thank you for your quick response. I think it is better to clear the vmr. Actually the mlx5 driver allocates the vmr using calloc(). In addition, rxe_reg_mr() (not rxe_reg_dmabuf_mr()) is used the malloc and not clear the vmr. I think It has to be fixed too. Should I make another patch to fix this problem? Thanks a lot. Shunsuke ~ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 6:58 ` Shunsuke Mie @ 2021-09-30 7:23 ` Zhu Yanjun 2021-09-30 11:06 ` Shunsuke Mie 0 siblings, 1 reply; 8+ messages in thread From: Zhu Yanjun @ 2021-09-30 7:23 UTC (permalink / raw) To: Shunsuke Mie Cc: Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, LKML, RDMA mailing list, Damian Hobson-Garcia, Takanari Hayama, Tomohito Esaki On Thu, Sep 30, 2021 at 2:58 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > 2021年9月30日(木) 15:37 Zhu Yanjun <zyjzyj2000@gmail.com>: > > > > On Thu, Sep 30, 2021 at 2:20 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > > > Implement a new provider method for dma-buf base memory registration. > > > > > > Signed-off-by: Shunsuke Mie <mie@igel.co.jp> > > > --- > > > providers/rxe/rxe.c | 21 +++++++++++++++++++++ > > > 1 file changed, 21 insertions(+) > > > > > > diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c > > > index 3c3ea8bb..84e00e60 100644 > > > --- a/providers/rxe/rxe.c > > > +++ b/providers/rxe/rxe.c > > > @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, > > > return &vmr->ibv_mr; > > > } > > > > > > +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, > > > + size_t length, uint64_t iova, int fd, > > > + int access) > > > +{ > > > + struct verbs_mr *vmr; > > > + int ret; > > > + > > > + vmr = malloc(sizeof(*vmr)); > > > + if (!vmr) > > > + return NULL; > > > + > > > > Do we need to set vmr to zero like the following? > > > > memset(vmr, 0, sizeof(*vmr)); > > > > Zhu Yanjun > Thank you for your quick response. > > I think it is better to clear the vmr. Actually the mlx5 driver allocates > the vmr using calloc(). > > In addition, rxe_reg_mr() (not rxe_reg_dmabuf_mr()) is used the malloc > and not clear the vmr. I think It has to be fixed too. Should I make > another patch to fix this problem? Yes. Please. Zhu Yanjun > > Thanks a lot. > Shunsuke > > ~ ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 7:23 ` Zhu Yanjun @ 2021-09-30 11:06 ` Shunsuke Mie 2021-09-30 11:17 ` Zhu Yanjun 0 siblings, 1 reply; 8+ messages in thread From: Shunsuke Mie @ 2021-09-30 11:06 UTC (permalink / raw) To: Zhu Yanjun Cc: Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, LKML, RDMA mailing list, Damian Hobson-Garcia, Takanari Hayama, Tomohito Esaki 2021年9月30日(木) 16:23 Zhu Yanjun <zyjzyj2000@gmail.com>: > > On Thu, Sep 30, 2021 at 2:58 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > 2021年9月30日(木) 15:37 Zhu Yanjun <zyjzyj2000@gmail.com>: > > > > > > On Thu, Sep 30, 2021 at 2:20 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > > > > > Implement a new provider method for dma-buf base memory registration. > > > > > > > > Signed-off-by: Shunsuke Mie <mie@igel.co.jp> > > > > --- > > > > providers/rxe/rxe.c | 21 +++++++++++++++++++++ > > > > 1 file changed, 21 insertions(+) > > > > > > > > diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c > > > > index 3c3ea8bb..84e00e60 100644 > > > > --- a/providers/rxe/rxe.c > > > > +++ b/providers/rxe/rxe.c > > > > @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, > > > > return &vmr->ibv_mr; > > > > } > > > > > > > > +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, > > > > + size_t length, uint64_t iova, int fd, > > > > + int access) > > > > +{ > > > > + struct verbs_mr *vmr; > > > > + int ret; > > > > + > > > > + vmr = malloc(sizeof(*vmr)); > > > > + if (!vmr) > > > > + return NULL; > > > > + > > > > > > Do we need to set vmr to zero like the following? > > > > > > memset(vmr, 0, sizeof(*vmr)); > > > > > > Zhu Yanjun > > Thank you for your quick response. > > > > I think it is better to clear the vmr. Actually the mlx5 driver allocates > > the vmr using calloc(). > > > > In addition, rxe_reg_mr() (not rxe_reg_dmabuf_mr()) is used the malloc > > and not clear the vmr. I think It has to be fixed too. Should I make > > another patch to fix this problem? > > Yes. Please. > > Zhu Yanjun > > > > > Thanks a lot. > > Shunsuke > > > > ~ I looked into the vmr more, but there was no need to clear it. Moreover, some implementations also use malloc without memory clear. Thanks, Shunsuke ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 11:06 ` Shunsuke Mie @ 2021-09-30 11:17 ` Zhu Yanjun 2021-09-30 11:24 ` Shunsuke Mie 0 siblings, 1 reply; 8+ messages in thread From: Zhu Yanjun @ 2021-09-30 11:17 UTC (permalink / raw) To: Shunsuke Mie Cc: Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, LKML, RDMA mailing list, Damian Hobson-Garcia, Takanari Hayama, Tomohito Esaki On Thu, Sep 30, 2021 at 7:06 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > 2021年9月30日(木) 16:23 Zhu Yanjun <zyjzyj2000@gmail.com>: > > > > On Thu, Sep 30, 2021 at 2:58 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > > > 2021年9月30日(木) 15:37 Zhu Yanjun <zyjzyj2000@gmail.com>: > > > > > > > > On Thu, Sep 30, 2021 at 2:20 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > > > > > > > Implement a new provider method for dma-buf base memory registration. > > > > > > > > > > Signed-off-by: Shunsuke Mie <mie@igel.co.jp> > > > > > --- > > > > > providers/rxe/rxe.c | 21 +++++++++++++++++++++ > > > > > 1 file changed, 21 insertions(+) > > > > > > > > > > diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c > > > > > index 3c3ea8bb..84e00e60 100644 > > > > > --- a/providers/rxe/rxe.c > > > > > +++ b/providers/rxe/rxe.c > > > > > @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, > > > > > return &vmr->ibv_mr; > > > > > } > > > > > > > > > > +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, > > > > > + size_t length, uint64_t iova, int fd, > > > > > + int access) > > > > > +{ > > > > > + struct verbs_mr *vmr; > > > > > + int ret; > > > > > + > > > > > + vmr = malloc(sizeof(*vmr)); > > > > > + if (!vmr) > > > > > + return NULL; > > > > > + > > > > > > > > Do we need to set vmr to zero like the following? > > > > > > > > memset(vmr, 0, sizeof(*vmr)); > > > > > > > > Zhu Yanjun > > > Thank you for your quick response. > > > > > > I think it is better to clear the vmr. Actually the mlx5 driver allocates > > > the vmr using calloc(). > > > > > > In addition, rxe_reg_mr() (not rxe_reg_dmabuf_mr()) is used the malloc > > > and not clear the vmr. I think It has to be fixed too. Should I make > > > another patch to fix this problem? > > > > Yes. Please. > > > > Zhu Yanjun > > > > > > > > Thanks a lot. > > > Shunsuke > > > > > > ~ > > I looked into the vmr more, but there was no need to clear it. Moreover, > some implementations also use malloc without memory clear. > I confronted a lot of problems with memory not initialization. And a latest bug is https://www.spinics.net/lists/linux-rdma/msg105001.html So it is a good habit to clear a newly allocated memory. Zhu Yanjun > Thanks, > Shunsuke ^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: [RFC PATCH v2 1/1] Providers/rxe: Add dma-buf support 2021-09-30 11:17 ` Zhu Yanjun @ 2021-09-30 11:24 ` Shunsuke Mie 0 siblings, 0 replies; 8+ messages in thread From: Shunsuke Mie @ 2021-09-30 11:24 UTC (permalink / raw) To: Zhu Yanjun Cc: Christian König, Alex Deucher, Daniel Vetter, Doug Ledford, Jason Gunthorpe, Jianxin Xiong, Leon Romanovsky, Maor Gottlieb, Sean Hefty, Sumit Semwal, dri-devel, linaro-mm-sig, linux-media, LKML, RDMA mailing list, Damian Hobson-Garcia, Takanari Hayama, Tomohito Esaki 2021年9月30日(木) 20:18 Zhu Yanjun <zyjzyj2000@gmail.com>: > > On Thu, Sep 30, 2021 at 7:06 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > 2021年9月30日(木) 16:23 Zhu Yanjun <zyjzyj2000@gmail.com>: > > > > > > On Thu, Sep 30, 2021 at 2:58 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > > > > > 2021年9月30日(木) 15:37 Zhu Yanjun <zyjzyj2000@gmail.com>: > > > > > > > > > > On Thu, Sep 30, 2021 at 2:20 PM Shunsuke Mie <mie@igel.co.jp> wrote: > > > > > > > > > > > > Implement a new provider method for dma-buf base memory registration. > > > > > > > > > > > > Signed-off-by: Shunsuke Mie <mie@igel.co.jp> > > > > > > --- > > > > > > providers/rxe/rxe.c | 21 +++++++++++++++++++++ > > > > > > 1 file changed, 21 insertions(+) > > > > > > > > > > > > diff --git a/providers/rxe/rxe.c b/providers/rxe/rxe.c > > > > > > index 3c3ea8bb..84e00e60 100644 > > > > > > --- a/providers/rxe/rxe.c > > > > > > +++ b/providers/rxe/rxe.c > > > > > > @@ -239,6 +239,26 @@ static struct ibv_mr *rxe_reg_mr(struct ibv_pd *pd, void *addr, size_t length, > > > > > > return &vmr->ibv_mr; > > > > > > } > > > > > > > > > > > > +static struct ibv_mr *rxe_reg_dmabuf_mr(struct ibv_pd *pd, uint64_t offset, > > > > > > + size_t length, uint64_t iova, int fd, > > > > > > + int access) > > > > > > +{ > > > > > > + struct verbs_mr *vmr; > > > > > > + int ret; > > > > > > + > > > > > > + vmr = malloc(sizeof(*vmr)); > > > > > > + if (!vmr) > > > > > > + return NULL; > > > > > > + > > > > > > > > > > Do we need to set vmr to zero like the following? > > > > > > > > > > memset(vmr, 0, sizeof(*vmr)); > > > > > > > > > > Zhu Yanjun > > > > Thank you for your quick response. > > > > > > > > I think it is better to clear the vmr. Actually the mlx5 driver allocates > > > > the vmr using calloc(). > > > > > > > > In addition, rxe_reg_mr() (not rxe_reg_dmabuf_mr()) is used the malloc > > > > and not clear the vmr. I think It has to be fixed too. Should I make > > > > another patch to fix this problem? > > > > > > Yes. Please. > > > > > > Zhu Yanjun > > > > > > > > > > > Thanks a lot. > > > > Shunsuke > > > > > > > > ~ > > > > I looked into the vmr more, but there was no need to clear it. Moreover, > > some implementations also use malloc without memory clear. > > > > I confronted a lot of problems with memory not initialization. > And a latest bug is https://www.spinics.net/lists/linux-rdma/msg105001.html > > So it is a good habit to clear a newly allocated memory. > > Zhu Yanjun > > > Thanks, > > Shunsuke Ok, I understand that thought. I'll send a patch. Shunsuke. ^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2021-09-30 11:25 UTC | newest] Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2021-09-30 6:20 [RFC PATCH v2 0/1] Providers/rxe: Add dma-buf support Shunsuke Mie 2021-09-30 6:20 ` [RFC PATCH v2 1/1] " Shunsuke Mie 2021-09-30 6:37 ` Zhu Yanjun 2021-09-30 6:58 ` Shunsuke Mie 2021-09-30 7:23 ` Zhu Yanjun 2021-09-30 11:06 ` Shunsuke Mie 2021-09-30 11:17 ` Zhu Yanjun 2021-09-30 11:24 ` Shunsuke Mie
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.