linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH rdma-core 00/13] verbs: Introduce import verbs for device, PD, MR
@ 2020-06-17  7:45 Yishai Hadas
  2020-06-17  7:45 ` [PATCH rdma-core 01/13] Update kernel headers Yishai Hadas
                   ` (12 more replies)
  0 siblings, 13 replies; 30+ messages in thread
From: Yishai Hadas @ 2020-06-17  7:45 UTC (permalink / raw)
  To: linux-rdma; +Cc: jgg, yishaih, maorg

This series Introduces import verbs for device, PD, MR which enables processes
to share their ibv_context and then share PD(s) and MR(s) that are associated
with.

This functionality enables utilizing and optimizing some application flows, few
examples below.

Any solution which is a single business logic based on multi-process design
needs this. Example include NGINX, with TCP load balancing, sharing the RSS
indirection table with RQ per process. HPC frameworks with multi-rank (process)
solution on single hosts. UCX can share IB resources using the shared PD and
can help dispatch data to multiple processes/MR's in single RDMA operation.
Also, there are use cases when a primary processes registered a large shared
memory range, and each worker process spawned will create a private QP on the
shared PD, and use the shared MR to save the registration time per-process.

As part of this series was added also some pyverbs stuff to support and
demonstrate some usage of the APIs.

The verbs APIs were introduced in the mailing list by the below RFC [1], the
matching kernel series was sent to rdma-next,

PR: https://github.com/linux-rdma/rdma-core/pull/776
[1] https://patchwork.kernel.org/patch/11540665/

Yishai

Edward Srouji (3):
  pyverbs: Support verbs import APIs
  Documentation: Add usage example for verbs import
  tests: Add a shared PD Pyverbs test

Yishai Hadas (10):
  Update kernel headers
  verbs: Close async_fd only when it was previously created
  verbs: Introduce ibv_import_device() verb
  verbs: Handle async FD on an imported device
  mlx5: Refactor mlx5_alloc_context()
  mlx5: Implement the import device functionality
  verbs: Introduce ibv_import/unimport_pd() verbs
  mlx5: Implement the import/unimport PD verbs
  verbs: Introduce ibv_import/unimport_mr() verbs
  mlx5: Implement the import/unimport MR verbs

 Documentation/pyverbs.md                   |  40 ++++
 debian/libibverbs1.symbols                 |   6 +
 kernel-headers/rdma/ib_user_ioctl_cmds.h   |  15 ++
 kernel-headers/rdma/mlx5-abi.h             |   9 +-
 kernel-headers/rdma/mlx5_user_ioctl_cmds.h |  14 ++
 kernel-headers/rdma/rdma_netlink.h         |   8 +
 kernel-headers/rdma/rdma_user_cm.h         |  11 +-
 kernel-headers/rdma/rdma_user_ioctl_cmds.h |   2 +-
 libibverbs/CMakeLists.txt                  |   2 +-
 libibverbs/cmd_cq.c                        |   9 +-
 libibverbs/cmd_device.c                    |  32 ++-
 libibverbs/cmd_mr.c                        |  35 ++++
 libibverbs/cmd_qp.c                        |   4 +
 libibverbs/cmd_srq.c                       |   4 +
 libibverbs/cmd_wq.c                        |   4 +
 libibverbs/device.c                        |  73 ++++++-
 libibverbs/driver.h                        |  14 ++
 libibverbs/dummy_ops.c                     |  30 +++
 libibverbs/ibverbs.h                       |   1 +
 libibverbs/libibverbs.map.in               |  11 +
 libibverbs/man/CMakeLists.txt              |   5 +
 libibverbs/man/ibv_import_device.3.md      |  48 +++++
 libibverbs/man/ibv_import_mr.3.md          |  63 ++++++
 libibverbs/man/ibv_import_pd.3.md          |  57 ++++++
 libibverbs/verbs.c                         |  30 +++
 libibverbs/verbs.h                         |  26 +++
 providers/mlx5/mlx5.c                      | 317 ++++++++++++++++++-----------
 providers/mlx5/mlx5.h                      |   6 +
 providers/mlx5/verbs.c                     |  78 ++++++-
 pyverbs/device.pyx                         |  12 +-
 pyverbs/libibverbs.pxd                     |   5 +
 pyverbs/mr.pxd                             |   1 +
 pyverbs/mr.pyx                             |  60 +++++-
 pyverbs/pd.pxd                             |   1 +
 pyverbs/pd.pyx                             |  37 +++-
 tests/CMakeLists.txt                       |   1 +
 tests/base.py                              |  11 +-
 tests/test_shared_pd.py                    |  95 +++++++++
 38 files changed, 1022 insertions(+), 155 deletions(-)
 create mode 100644 libibverbs/man/ibv_import_device.3.md
 create mode 100644 libibverbs/man/ibv_import_mr.3.md
 create mode 100644 libibverbs/man/ibv_import_pd.3.md
 create mode 100644 tests/test_shared_pd.py

-- 
1.8.3.1


^ permalink raw reply	[flat|nested] 30+ messages in thread

end of thread, other threads:[~2020-07-02 17:43 UTC | newest]

Thread overview: 30+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-06-17  7:45 [PATCH rdma-core 00/13] verbs: Introduce import verbs for device, PD, MR Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 01/13] Update kernel headers Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 02/13] verbs: Close async_fd only when it was previously created Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 03/13] verbs: Introduce ibv_import_device() verb Yishai Hadas
2020-06-19 12:29   ` Jason Gunthorpe
2020-06-21  7:01     ` Yishai Hadas
2020-06-22 12:52       ` Jason Gunthorpe
2020-06-23 13:06         ` Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 04/13] verbs: Handle async FD on an imported device Yishai Hadas
2020-06-19 12:33   ` Jason Gunthorpe
2020-06-21  9:08     ` Yishai Hadas
2020-06-23 17:34       ` Jason Gunthorpe
2020-06-24  7:22         ` Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 05/13] mlx5: Refactor mlx5_alloc_context() Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 06/13] mlx5: Implement the import device functionality Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 07/13] verbs: Introduce ibv_import/unimport_pd() verbs Yishai Hadas
2020-06-19 12:48   ` Jason Gunthorpe
2020-06-21  8:30     ` Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 08/13] mlx5: Implement the import/unimport PD verbs Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 09/13] verbs: Introduce ibv_import/unimport_mr() verbs Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 10/13] mlx5: Implement the import/unimport MR verbs Yishai Hadas
2020-06-19 12:50   ` Jason Gunthorpe
2020-06-21  8:44     ` Yishai Hadas
2020-06-23 17:33       ` Jason Gunthorpe
2020-06-24  7:20         ` Yishai Hadas
2020-07-01 12:28         ` Yishai Hadas
2020-07-02 17:42           ` Jason Gunthorpe
2020-06-17  7:45 ` [PATCH rdma-core 11/13] pyverbs: Support verbs import APIs Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 12/13] Documentation: Add usage example for verbs import Yishai Hadas
2020-06-17  7:45 ` [PATCH rdma-core 13/13] tests: Add a shared PD Pyverbs test Yishai Hadas

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).