Re: [PATCH V1 rdma-core 0/8] verbs: Query GID table API

[Yishai Hadas <yishaih@nvidia.com>]

On 9/23/2020 1:26 PM, Yishai Hadas wrote:
> When an application is not using RDMA CM and if it is using multiple RDMA
> devices with one or more RoCE ports, finding the right GID table entry is a
> long process.
>
> For example, with two RoCE dual-port devices in a system, when IP failover is
> used between two RoCE ports, searching a suitable GID entry for a given source
> IP, matching netdevice of given RoCEv1/v2 type requires iterating over all 4
> ports * 256 entry GID table.
>
> Even though the best first match GID table for given criteria is used, when the
> matching entry is on the 4th port, it requires reading 3 ports * 256 entries *
> 3 files (GID, netdev, type) = 2304 files.  The GID table needs to be referred
> on every QP creation during IP failover on other netdevice of an RDMA device.
>
> We introduce this series of patches, which introduces an API to query the
> complete GID tables of an RDMA device, that returns all valid GID table
> entries.
>
> This is done through single ioctl, eliminating 2304 read, 2304 open and 2304
> close system calls to just a total of 2 calls (one for each device).
>
> While at it, we also introduce an API to query an individual GID entry over
> ioctl interface, which provides all GID attributes information.
>
> The APIs are based on the below RFC [1], the matching kernel part was sent to
> rdma-next.
>
> PR was sent as well [2].
>
> [1] https://www.spinics.net/lists/linux-rdma/msg91825.html
> [2] https://github.com/linux-rdma/rdma-core/pull/828
>
> Changelog:
> v1:
> * Move some GIDs stuff to cmd_device.c with better function's names.
> * Cleanup in few places.
>
> Avihai Horon (7):
>    verbs: Change the name of enum ibv_gid_type
>    verbs: Introduce a new query GID entry API
>    verbs: Implement ibv_query_gid and ibv_query_gid_type over ioctl
>    verbs: Optimize ibv_query_gid and ibv_query_gid_type
>    verbs: Introduce a new query GID table API
>    pyverbs: Add query_gid_table and query_gid_ex methods
>    tests: Add tests for ibv_query_gid_table and ibv_query_gid_ex
>
> Yishai Hadas (1):
>    Update kernel headers
>
>   debian/libibverbs1.symbols                |   3 +
>   kernel-headers/rdma/ib_user_ioctl_cmds.h  |  16 ++
>   kernel-headers/rdma/ib_user_ioctl_verbs.h |  14 ++
>   kernel-headers/rdma/ib_user_verbs.h       |  11 ++
>   kernel-headers/rdma/rdma_user_rxe.h       |   6 +-
>   libibverbs/CMakeLists.txt                 |   2 +-
>   libibverbs/cmd_device.c                   | 314 ++++++++++++++++++++++++++++++
>   libibverbs/driver.h                       |  19 +-
>   libibverbs/examples/devinfo.c             |  14 +-
>   libibverbs/libibverbs.map.in              |   6 +
>   libibverbs/man/CMakeLists.txt             |   2 +
>   libibverbs/man/ibv_query_gid_ex.3.md      |  93 +++++++++
>   libibverbs/man/ibv_query_gid_table.3.md   |  73 +++++++
>   libibverbs/verbs.c                        | 105 ++++------
>   libibverbs/verbs.h                        |  45 +++++
>   providers/mlx5/verbs.c                    |   2 +-
>   pyverbs/device.pxd                        |   3 +
>   pyverbs/device.pyx                        | 108 +++++++++-
>   pyverbs/libibverbs.pxd                    |  15 +-
>   pyverbs/libibverbs_enums.pxd              |  11 +-
>   tests/base.py                             |   3 +-
>   tests/test_device.py                      |  32 +++
>   22 files changed, 804 insertions(+), 93 deletions(-)
>   create mode 100644 libibverbs/man/ibv_query_gid_ex.3.md
>   create mode 100644 libibverbs/man/ibv_query_gid_table.3.md
>

The PR was merged.

Thanks,
Yishai