From: Li Zhijian <lizhijian@fujitsu.com>
To: Bob Pearson <rpearsonhpe@gmail.com>,
Leon Romanovsky <leon@kernel.org>, Jason Gunthorpe <jgg@ziepe.ca>,
linux-rdma@vger.kernel.org
Cc: Zhu Yanjun <zyjzyj2000@gmail.com>,
yangx.jy@fujitsu.com, y-goto@fujitsu.com, mbloch@nvidia.com,
tom@talpey.com, tomasz.gromadzki@intel.com,
dan.j.williams@intel.com, linux-kernel@vger.kernel.org,
Li Zhijian <lizhijian@fujitsu.com>,
Wenpeng Liang <liangwenpeng@huawei.com>
Subject: [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation
Date: Tue, 6 Dec 2022 21:01:51 +0800 [thread overview]
Message-ID: <20221206130201.30986-1-lizhijian@fujitsu.com> (raw)
Changes in V7:
- rebase to jgg-for-next(Atomic Write applied)
- patch-09: enable FLUSH QP attribute for supported device only, and remove RVB # Jason
- patch-04: pr_debug -> rxe_dbg_mr # Jason
Changes in V6:
- rebase to v6.1-rc1
- add Yanjun's reviewed-by expect "Allow registering persistent flag for pmem MR only"
- minimize pmem checking side effect # Jason
- return EOPNOSUPP if HCA doesn't support flush operation
These patches are going to implement a *NEW* RDMA opcode "RDMA FLUSH".
In IB SPEC 1.5[1], 2 new opcodes, ATOMIC WRITE and RDMA FLUSH were
added in the MEMORY PLACEMENT EXTENSIONS section.
This patchset makes SoftRoCE support new RDMA FLUSH on RC service.
pyverbs tests and blktests(nvme over RDMA and srp) are tested.
You can verify the patchset by building and running the rdma_flush example[2].
server:
$ ./rdma_flush_server -s [server_address] -p [port_number]
client:
$ ./rdma_flush_client -s [server_address] -p [port_number]
Corresponding pyverbs and tests(tests.test_qpex.QpExTestCase.test_qp_ex_rc_rdma_flush)
are also added to rdma-core
[1]: https://www.infinibandta.org/wp-content/uploads/2021/08/IBTA-Overview-of-IBTA-Volume-1-Release-1.5-and-MPE-2021-08-17-Secure.pptx
[2]: https://github.com/zhijianli88/rdma-core/tree/rdma-flush-v5
CC: Xiao Yang <yangx.jy@fujitsu.com>
CC: "Gotou, Yasunori" <y-goto@fujitsu.com>
CC: Jason Gunthorpe <jgg@ziepe.ca>
CC: Zhu Yanjun <zyjzyj2000@gmail.com>
CC: Leon Romanovsky <leon@kernel.org>
CC: Bob Pearson <rpearsonhpe@gmail.com>
CC: Mark Bloch <mbloch@nvidia.com>
CC: Wenpeng Liang <liangwenpeng@huawei.com>
CC: Tom Talpey <tom@talpey.com>
CC: "Gromadzki, Tomasz" <tomasz.gromadzki@intel.com>
CC: Dan Williams <dan.j.williams@intel.com>
CC: linux-rdma@vger.kernel.org
CC: linux-kernel@vger.kernel.org
Can also access the kernel source in:
https://github.com/zhijianli88/linux/tree/rdma-flush-v7
Changes log
V6: https://lore.kernel.org/lkml/20221116081951.32750-10-lizhijian@fujitsu.com/T/
V5: https://lore.kernel.org/lkml/20220927055337.22630-12-lizhijian@fujitsu.com/t/
V4:
- rework responder process
- rebase to v5.19+
- remove [7/7]: RDMA/rxe: Add RD FLUSH service support since RD is not really supported
V3:
- Just rebase and commit log and comment updates
- delete patch-1: "RDMA: mr: Introduce is_pmem", which will be combined into "Allow registering persistent flag for pmem MR only"
- delete patch-7
V2:
RDMA: mr: Introduce is_pmem
check 1st byte to avoid crossing page boundary
new scheme to check is_pmem # Dan
RDMA: Allow registering MR with flush access flags
combine with [03/10] RDMA/rxe: Allow registering FLUSH flags for supported device only to this patch # Jason
split RDMA_FLUSH to 2 capabilities
RDMA/rxe: Allow registering persistent flag for pmem MR only
update commit message, get rid of confusing ib_check_flush_access_flags() # Tom
RDMA/rxe: Implement RC RDMA FLUSH service in requester side
extend flush to include length field. # Tom and Tomasz
RDMA/rxe: Implement flush execution in responder side
adjust start for WHOLE MR level # Tom
don't support DMA mr for flush # Tom
check flush return value
RDMA/rxe: Enable RDMA FLUSH capability for rxe device
adjust patch's order. move it here from [04/10]
Li Zhijian (10):
RDMA: Extend RDMA user ABI to support flush
RDMA: Extend RDMA kernel verbs ABI to support flush
RDMA/rxe: Extend rxe user ABI to support flush
RDMA/rxe: Allow registering persistent flag for pmem MR only
RDMA/rxe: Extend rxe packet format to support flush
RDMA/rxe: Implement RC RDMA FLUSH service in requester side
RDMA/rxe: Implement flush execution in responder side
RDMA/rxe: Implement flush completion
RDMA/cm: Make QP FLUSHABLE for supported device
RDMA/rxe: Enable RDMA FLUSH capability for rxe device
drivers/infiniband/core/cm.c | 13 +-
drivers/infiniband/sw/rxe/rxe_comp.c | 4 +-
drivers/infiniband/sw/rxe/rxe_hdr.h | 47 +++++++
drivers/infiniband/sw/rxe/rxe_loc.h | 1 +
drivers/infiniband/sw/rxe/rxe_mr.c | 58 ++++++++-
drivers/infiniband/sw/rxe/rxe_opcode.c | 17 +++
drivers/infiniband/sw/rxe/rxe_opcode.h | 14 ++-
drivers/infiniband/sw/rxe/rxe_param.h | 2 +
drivers/infiniband/sw/rxe/rxe_req.c | 15 ++-
drivers/infiniband/sw/rxe/rxe_resp.c | 160 +++++++++++++++++++++---
drivers/infiniband/sw/rxe/rxe_verbs.h | 6 +
include/rdma/ib_pack.h | 3 +
include/rdma/ib_verbs.h | 18 ++-
include/uapi/rdma/ib_user_ioctl_verbs.h | 2 +
include/uapi/rdma/ib_user_verbs.h | 17 +++
include/uapi/rdma/rdma_user_rxe.h | 7 ++
16 files changed, 352 insertions(+), 32 deletions(-)
--
2.31.1
next reply other threads:[~2022-12-06 13:03 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-06 13:01 Li Zhijian [this message]
2022-12-06 13:01 ` [for-next PATCH 01/10] RDMA: Extend RDMA user ABI to support flush Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 02/10] RDMA: Extend RDMA kernel verbs " Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 03/10] RDMA/rxe: Extend rxe user " Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 04/10] RDMA/rxe: Allow registering persistent flag for pmem MR only Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 05/10] RDMA/rxe: Extend rxe packet format to support flush Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 06/10] RDMA/rxe: Implement RC RDMA FLUSH service in requester side Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 07/10] RDMA/rxe: Implement flush execution in responder side Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 08/10] RDMA/rxe: Implement flush completion Li Zhijian
2022-12-06 13:02 ` [for-next PATCH 09/10] RDMA/cm: Make QP FLUSHABLE for supported device Li Zhijian
2022-12-06 13:02 ` [for-next PATCH 10/10] RDMA/rxe: Enable RDMA FLUSH capability for rxe device Li Zhijian
2022-12-09 23:43 ` [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221206130201.30986-1-lizhijian@fujitsu.com \
--to=lizhijian@fujitsu.com \
--cc=dan.j.williams@intel.com \
--cc=jgg@ziepe.ca \
--cc=leon@kernel.org \
--cc=liangwenpeng@huawei.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=rpearsonhpe@gmail.com \
--cc=tom@talpey.com \
--cc=tomasz.gromadzki@intel.com \
--cc=y-goto@fujitsu.com \
--cc=yangx.jy@fujitsu.com \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).