From: ira.weiny@intel.com
To: Jason Gunthorpe <jgg@ziepe.ca>
Cc: Ira Weiny <ira.weiny@intel.com>,
Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>,
Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
Doug Ledford <dledford@redhat.com>,
Faisal Latif <faisal.latif@intel.com>,
Shiraz Saleem <shiraz.saleem@intel.com>,
Bernard Metzler <bmt@zurich.ibm.com>,
Kamal Heib <kheib@redhat.com>,
linux-rdma@vger.kernel.org, linux-kernel@vger.kernel.org
Subject: [PATCH V2] RDMA/siw: Convert siw_tx_hdt() to kmap_local_page()
Date: Tue, 22 Jun 2021 13:34:32 -0700 [thread overview]
Message-ID: <20210622203432.2715659-1-ira.weiny@intel.com> (raw)
In-Reply-To: <20210622061422.2633501-5-ira.weiny@intel.com>
From: Ira Weiny <ira.weiny@intel.com>
kmap() is being deprecated and will break uses of device dax after PKS
protection is introduced.[1]
The use of kmap() in siw_tx_hdt() is all thread local therefore
kmap_local_page() is a sufficient replacement and will work with pgmap
protected pages when those are implemented.
siw_tx_hdt() tracks pages used in a page_array. It uses that array to
unmap pages which were mapped on function exit. Not all entries in the
array are mapped and this is tracked in kmap_mask.
kunmap_local() takes a mapped address rather than a page. Alter
siw_unmap_pages() to take the iov array to reuse the iov_base address of
each mapping. Use PAGE_MASK to get the proper address for
kunmap_local().
kmap_local_page() mappings are tracked in a stack and must be unmapped
in the opposite order they were mapped in. Because segments are mapped
into the page array in increasing index order, modify siw_unmap_pages()
to unmap pages in decreasing order.
Use kmap_local_page() instead of kmap() to map pages in the page_array.
[1] https://lore.kernel.org/lkml/20201009195033.3208459-59-ira.weiny@intel.com/
Signed-off-by: Ira Weiny <ira.weiny@intel.com>
---
Changes for V2:
From Bernard
Reuse iov[].iov_base rather than declaring another array of
pointers and preserve the use of kmap_mask to know which iov's
were kmapped.
---
drivers/infiniband/sw/siw/siw_qp_tx.c | 32 +++++++++++++++++----------
1 file changed, 20 insertions(+), 12 deletions(-)
diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c b/drivers/infiniband/sw/siw/siw_qp_tx.c
index db68a10d12cd..fd3b9e6a67d7 100644
--- a/drivers/infiniband/sw/siw/siw_qp_tx.c
+++ b/drivers/infiniband/sw/siw/siw_qp_tx.c
@@ -396,13 +396,20 @@ static int siw_0copy_tx(struct socket *s, struct page **page,
#define MAX_TRAILER (MPA_CRC_SIZE + 4)
-static void siw_unmap_pages(struct page **pp, unsigned long kmap_mask)
+static void siw_unmap_pages(struct kvec *iov, unsigned long kmap_mask, int len)
{
- while (kmap_mask) {
- if (kmap_mask & BIT(0))
- kunmap(*pp);
- pp++;
- kmap_mask >>= 1;
+ int i;
+
+ /*
+ * Work backwards through the array to honor the kmap_local_page()
+ * ordering requirements.
+ */
+ for (i = (len-1); i >= 0; i--) {
+ if (kmap_mask & BIT(i)) {
+ unsigned long addr = (unsigned long)iov[i].iov_base;
+
+ kunmap_local((void *)(addr & PAGE_MASK));
+ }
}
}
@@ -498,7 +505,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
p = siw_get_upage(mem->umem,
sge->laddr + sge_off);
if (unlikely(!p)) {
- siw_unmap_pages(page_array, kmap_mask);
+ siw_unmap_pages(iov, kmap_mask, MAX_ARRAY);
wqe->processed -= c_tx->bytes_unsent;
rv = -EFAULT;
goto done_crc;
@@ -506,11 +513,12 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
page_array[seg] = p;
if (!c_tx->use_sendpage) {
- iov[seg].iov_base = kmap(p) + fp_off;
- iov[seg].iov_len = plen;
+ void *kaddr = kmap_local_page(page_array[seg]);
/* Remember for later kunmap() */
kmap_mask |= BIT(seg);
+ iov[seg].iov_base = kaddr + fp_off;
+ iov[seg].iov_len = plen;
if (do_crc)
crypto_shash_update(
@@ -518,7 +526,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
iov[seg].iov_base,
plen);
} else if (do_crc) {
- kaddr = kmap_local_page(p);
+ kaddr = kmap_local_page(page_array[seg]);
crypto_shash_update(c_tx->mpa_crc_hd,
kaddr + fp_off,
plen);
@@ -542,7 +550,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
if (++seg > (int)MAX_ARRAY) {
siw_dbg_qp(tx_qp(c_tx), "to many fragments\n");
- siw_unmap_pages(page_array, kmap_mask);
+ siw_unmap_pages(iov, kmap_mask, MAX_ARRAY);
wqe->processed -= c_tx->bytes_unsent;
rv = -EMSGSIZE;
goto done_crc;
@@ -593,7 +601,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, struct socket *s)
} else {
rv = kernel_sendmsg(s, &msg, iov, seg + 1,
hdr_len + data_len + trl_len);
- siw_unmap_pages(page_array, kmap_mask);
+ siw_unmap_pages(iov, kmap_mask, MAX_ARRAY);
}
if (rv < (int)hdr_len) {
/* Not even complete hdr pushed or negative rv */
--
2.28.0.rc0.12.gb6a658bd00c9
next prev parent reply other threads:[~2021-06-22 20:35 UTC|newest]
Thread overview: 22+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-06-22 6:14 [PATCH 0/4] Remove use of kmap() ira.weiny
2021-06-22 6:14 ` [PATCH 1/4] RDMA/hfi1: " ira.weiny
2021-06-24 18:13 ` Jason Gunthorpe
2021-06-22 6:14 ` [PATCH 2/4] RDMA/i40iw: " ira.weiny
2021-06-22 12:14 ` Jason Gunthorpe
2021-06-22 16:56 ` [PATCH V2] RDMA/irdma: " ira.weiny
2021-06-24 18:13 ` Jason Gunthorpe
2021-06-22 6:14 ` [PATCH 3/4] RDMA/siw: Remove kmap() ira.weiny
2021-07-15 18:00 ` Jason Gunthorpe
2021-06-22 6:14 ` [PATCH 4/4] RDMA/siw: Convert siw_tx_hdt() to kmap_local_page() ira.weiny
2021-06-22 20:34 ` ira.weiny [this message]
2021-06-23 22:15 ` [PATCH V3] " ira.weiny
2021-06-24 17:48 ` [PATCH V4] " ira.weiny
2021-07-15 18:00 ` Jason Gunthorpe
2021-06-29 14:11 ` Bernard Metzler
2021-06-29 22:13 ` Ira Weiny
2021-06-24 15:45 ` [PATCH V3] " Bernard Metzler
2021-06-24 17:33 ` Ira Weiny
2021-06-23 14:36 ` [PATCH V2] " Bernard Metzler
2021-06-23 15:34 ` Ira Weiny
2021-06-22 16:42 ` [PATCH 4/4] " Bernard Metzler
2021-06-22 20:39 ` Ira Weiny
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20210622203432.2715659-1-ira.weiny@intel.com \
--to=ira.weiny@intel.com \
--cc=bmt@zurich.ibm.com \
--cc=dennis.dalessandro@cornelisnetworks.com \
--cc=dledford@redhat.com \
--cc=faisal.latif@intel.com \
--cc=jgg@ziepe.ca \
--cc=kheib@redhat.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=mike.marciniszyn@cornelisnetworks.com \
--cc=shiraz.saleem@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).