From: Li Zhijian <lizhijian@fujitsu.com>
To: Bob Pearson <rpearsonhpe@gmail.com>,
Leon Romanovsky <leon@kernel.org>, Jason Gunthorpe <jgg@ziepe.ca>,
linux-rdma@vger.kernel.org
Cc: Zhu Yanjun <zyjzyj2000@gmail.com>,
yangx.jy@fujitsu.com, y-goto@fujitsu.com, mbloch@nvidia.com,
tom@talpey.com, tomasz.gromadzki@intel.com,
dan.j.williams@intel.com, linux-kernel@vger.kernel.org,
Li Zhijian <lizhijian@fujitsu.com>
Subject: [for-next PATCH 04/10] RDMA/rxe: Allow registering persistent flag for pmem MR only
Date: Tue, 6 Dec 2022 21:01:55 +0800 [thread overview]
Message-ID: <20221206130201.30986-5-lizhijian@fujitsu.com> (raw)
In-Reply-To: <20221206130201.30986-1-lizhijian@fujitsu.com>
Memory region could support at most 2 flush access flags:
IB_ACCESS_FLUSH_PERSISTENT and IB_ACCESS_FLUSH_GLOBAL
But we only allow user to register persistent flush flags to the pmem MR
where it has the ability of persisting data across power cycles.
So registering a persistent access flag to a non-pmem MR will be rejected.
CC: Dan Williams <dan.j.williams@intel.com>
Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
---
V7: pr_debug -> rxe_dbg_mr
V6: Minimize pmem checking side effect # Jason
V5: make sure the whole MR is pmem # Bob
V4: set is_pmem more simple
V2: new scheme check is_pmem # Dan
update commit message, get rid of confusing ib_check_flush_access_flags() # Tom
---
drivers/infiniband/sw/rxe/rxe_mr.c | 22 ++++++++++++++++++++--
1 file changed, 20 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_mr.c b/drivers/infiniband/sw/rxe/rxe_mr.c
index b7c9ff1ddf0e..81a438e5010a 100644
--- a/drivers/infiniband/sw/rxe/rxe_mr.c
+++ b/drivers/infiniband/sw/rxe/rxe_mr.c
@@ -111,6 +111,15 @@ void rxe_mr_init_dma(int access, struct rxe_mr *mr)
mr->ibmr.type = IB_MR_TYPE_DMA;
}
+static bool is_pmem_page(struct page *pg)
+{
+ unsigned long paddr = page_to_phys(pg);
+
+ return REGION_INTERSECTS ==
+ region_intersects(paddr, PAGE_SIZE, IORESOURCE_MEM,
+ IORES_DESC_PERSISTENT_MEMORY);
+}
+
int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
int access, struct rxe_mr *mr)
{
@@ -146,16 +155,25 @@ int rxe_mr_init_user(struct rxe_dev *rxe, u64 start, u64 length, u64 iova,
num_buf = 0;
map = mr->map;
if (length > 0) {
- buf = map[0]->buf;
+ bool persistent_access = access & IB_ACCESS_FLUSH_PERSISTENT;
+ buf = map[0]->buf;
for_each_sgtable_page (&umem->sgt_append.sgt, &sg_iter, 0) {
+ struct page *pg = sg_page_iter_page(&sg_iter);
+
+ if (persistent_access && !is_pmem_page(pg)) {
+ rxe_dbg_mr(mr, "Unable to register persistent access to non-pmem device\n");
+ err = -EINVAL;
+ goto err_release_umem;
+ }
+
if (num_buf >= RXE_BUF_PER_MAP) {
map++;
buf = map[0]->buf;
num_buf = 0;
}
- vaddr = page_address(sg_page_iter_page(&sg_iter));
+ vaddr = page_address(pg);
if (!vaddr) {
rxe_dbg_mr(mr, "Unable to get virtual address\n");
err = -ENOMEM;
--
2.31.1
next prev parent reply other threads:[~2022-12-06 13:03 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-12-06 13:01 [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 01/10] RDMA: Extend RDMA user ABI to support flush Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 02/10] RDMA: Extend RDMA kernel verbs " Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 03/10] RDMA/rxe: Extend rxe user " Li Zhijian
2022-12-06 13:01 ` Li Zhijian [this message]
2022-12-06 13:01 ` [for-next PATCH 05/10] RDMA/rxe: Extend rxe packet format " Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 06/10] RDMA/rxe: Implement RC RDMA FLUSH service in requester side Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 07/10] RDMA/rxe: Implement flush execution in responder side Li Zhijian
2022-12-06 13:01 ` [for-next PATCH 08/10] RDMA/rxe: Implement flush completion Li Zhijian
2022-12-06 13:02 ` [for-next PATCH 09/10] RDMA/cm: Make QP FLUSHABLE for supported device Li Zhijian
2022-12-06 13:02 ` [for-next PATCH 10/10] RDMA/rxe: Enable RDMA FLUSH capability for rxe device Li Zhijian
2022-12-09 23:43 ` [for-next PATCH 00/10] RDMA/rxe: Add RDMA FLUSH operation Jason Gunthorpe
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221206130201.30986-5-lizhijian@fujitsu.com \
--to=lizhijian@fujitsu.com \
--cc=dan.j.williams@intel.com \
--cc=jgg@ziepe.ca \
--cc=leon@kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mbloch@nvidia.com \
--cc=rpearsonhpe@gmail.com \
--cc=tom@talpey.com \
--cc=tomasz.gromadzki@intel.com \
--cc=y-goto@fujitsu.com \
--cc=yangx.jy@fujitsu.com \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).