Re: [RFC PATCH v3 0/7] RDMA/rxe: Add RDMA FLUSH operation

["lizhijian@fujitsu.com" <lizhijian@fujitsu.com>]

kindly ping


On 15/03/2022 18:18, Li Zhijian wrote:
> Hey folks,
>
> These patches are going to implement a *NEW* RDMA opcode "RDMA FLUSH".
> In IB SPEC 1.5[1], 2 new opcodes, ATOMIC WRITE and RDMA FLUSH were
> added in the MEMORY PLACEMENT EXTENSIONS section.
>
> This patchset makes SoftRoCE support new RDMA FLUSH on RC and RD service.
>
> You can verify the patchset by building and running the rdma_flush example[2].
> server:
> $ ./rdma_flush_server -s [server_address] -p [port_number]
> client:
> $ ./rdma_flush_client -s [server_address] -p [port_number]
>
> - We introduce new packet format for FLUSH request.
> - We introduce FLUSH placement type attributes to HCA
> - We introduce FLUSH access flags that users are able to register with
>
> [1]: https://www.infinibandta.org/wp-content/uploads/2021/08/IBTA-Overview-of-IBTA-Volume-1-Release-1.5-and-MPE-2021-08-17-Secure.pptx
> [2]: https://github.com/zhijianli88/rdma-core/tree/rdma-flush
>
> CC: Xiao Yang <yangx.jy@fujitsu.com>
> CC: y-goto@fujitsu.com
> CC: Jason Gunthorpe <jgg@ziepe.ca>
> CC: Zhu Yanjun <zyjzyj2000@gmail.com
> CC: Leon Romanovsky <leon@kernel.org>
> CC: Bob Pearson <rpearsonhpe@gmail.com>
> CC: Mark Bloch <mbloch@nvidia.com>
> CC: Wenpeng Liang <liangwenpeng@huawei.com>
> CC: Aharon Landau <aharonl@nvidia.com>
> CC: Tom Talpey <tom@talpey.com>
> CC: "Gromadzki, Tomasz" <tomasz.gromadzki@intel.com>
> CC: Dan Williams <dan.j.williams@intel.com>
> CC: linux-rdma@vger.kernel.org
> CC: linux-kernel@vger.kernel.org
>
> Can also access the kernel source in:
> https://github.com/zhijianli88/linux/tree/rdma-flush
> Changes log
> V3:
> - Just rebase and commit log and comment updates
> - delete patch-1: "RDMA: mr: Introduce is_pmem", which will be combined into "Allow registering persistent flag for pmem MR only"
> - delete patch-7
>
> V2:
> RDMA: mr: Introduce is_pmem
>     check 1st byte to avoid crossing page boundary
>     new scheme to check is_pmem # Dan
>
> RDMA: Allow registering MR with flush access flags
>     combine with [03/10] RDMA/rxe: Allow registering FLUSH flags for supported device only to this patch # Jason
>     split RDMA_FLUSH to 2 capabilities
>
> RDMA/rxe: Allow registering persistent flag for pmem MR only
>     update commit message, get rid of confusing ib_check_flush_access_flags() # Tom
>
> RDMA/rxe: Implement RC RDMA FLUSH service in requester side
>     extend flush to include length field. # Tom and Tomasz
>
> RDMA/rxe: Implement flush execution in responder side
>     adjust start for WHOLE MR level # Tom
>     don't support DMA mr for flush # Tom
>     check flush return value
>
> RDMA/rxe: Enable RDMA FLUSH capability for rxe device
>     adjust patch's order. move it here from [04/10]
>
> Li Zhijian (7):
>    RDMA: Allow registering MR with flush access flags
>    RDMA/rxe: Allow registering persistent flag for pmem MR only
>    RDMA/rxe: Implement RC RDMA FLUSH service in requester side
>    RDMA/rxe: Implement flush execution in responder side
>    RDMA/rxe: Implement flush completion
>    RDMA/rxe: Enable RDMA FLUSH capability for rxe device
>    RDMA/rxe: Add RD FLUSH service support
>
>   drivers/infiniband/core/uverbs_cmd.c    |  17 +++
>   drivers/infiniband/sw/rxe/rxe_comp.c    |   4 +-
>   drivers/infiniband/sw/rxe/rxe_hdr.h     |  48 +++++++++
>   drivers/infiniband/sw/rxe/rxe_loc.h     |   2 +
>   drivers/infiniband/sw/rxe/rxe_mr.c      |  36 ++++++-
>   drivers/infiniband/sw/rxe/rxe_opcode.c  |  35 ++++++
>   drivers/infiniband/sw/rxe/rxe_opcode.h  |   3 +
>   drivers/infiniband/sw/rxe/rxe_param.h   |   4 +-
>   drivers/infiniband/sw/rxe/rxe_req.c     |  15 ++-
>   drivers/infiniband/sw/rxe/rxe_resp.c    | 135 ++++++++++++++++++++++--
>   include/rdma/ib_pack.h                  |   3 +
>   include/rdma/ib_verbs.h                 |  29 ++++-
>   include/uapi/rdma/ib_user_ioctl_verbs.h |   2 +
>   include/uapi/rdma/ib_user_verbs.h       |  19 ++++
>   include/uapi/rdma/rdma_user_rxe.h       |   7 ++
>   15 files changed, 346 insertions(+), 13 deletions(-)
>