From mboxrd@z Thu Jan 1 00:00:00 1970 From: "Bernard Metzler" Date: Sat, 10 Oct 2020 11:36:49 +0000 Subject: Re: [PATCH RFC PKS/PMEM 10/58] drivers/rdma: Utilize new kmap_thread() Message-Id: List-Id: References: <20201009195033.3208459-11-ira.weiny@intel.com>, <20201009195033.3208459-1-ira.weiny@intel.com> In-Reply-To: <20201009195033.3208459-11-ira.weiny@intel.com> MIME-Version: 1.0 Content-Type: text/plain; charset="us-ascii" Content-Transfer-Encoding: 7bit To: ira.weiny@intel.com Cc: linux-aio@kvack.org, linux-efi@vger.kernel.org, kvm@vger.kernel.org, linux-doc@vger.kernel.org, Peter Zijlstra , linux-mmc@vger.kernel.org, Dave Hansen , dri-devel@lists.freedesktop.org, linux-mm@kvack.org, target-devel@vger.kernel.org, linux-mtd@lists.infradead.org, linux-kselftest@vger.kernel.org, samba-technical@lists.samba.org, Thomas Gleixner , devel@driverdev.osuosl.org, linux-cifs@vger.kernel.org, linux-nilfs@vger.kernel.org, linux-scsi@vger.kernel.org, linux-nvdimm@lists.01.org, linux-rdma@vger.kernel.org, x86@kernel.org, ceph-devel@vger.kernel.org, io-uring@vger.kernel.org, cluster-devel@redhat.com, Jason Gunthorpe , Doug Ledford , Ingo Molnar , intel-wired-lan@lists.osuosl.org, xen-devel@lists.xenproject.org, linux-ext4@vger.kernel.org, Fenghua Yu , linux-afs@lists.infradead.org, Faisal Latif , linux-um@lists.infradead.org, intel-gfx@lists.freedesktop.org, ecryptfs@vger.kernel.org, linux-erofs@lists.ozlabs.org, linux-nfs@vger.kernel.org, reiserfs-devel@vger.kernel.org, linux-block@vger.kernel.org, linux-bcache@vger.kernel.org, Borislav Petkov , Andy Lutomirski , drbd-dev@tron.linbit.com, amd-gfx@lists.freed.esktop.org, Dan Williams , Shiraz Saleem , bpf@vger.kernel.org, linux-cachefs@redhat.com, Mike Marciniszyn , linux-ntfs-dev@lists.sourceforge.net, netdev@vger.kernel.org, Dennis Dalessandro , kexec@lists.infradead.org, linux-kernel@vger.kernel.org, linux-f2fs-devel@lists.sourceforge.net, linux-fsdevel@vger.kernel.org, Andrew Morton , linuxppc-dev@lists.ozlabs.org, linux-btrfs@vger.kernel.org -----ira.weiny@intel.com wrote: ----- >To: "Andrew Morton" , "Thomas Gleixner" >, "Ingo Molnar" , "Borislav >Petkov" , "Andy Lutomirski" , "Peter >Zijlstra" >From: ira.weiny@intel.com >Date: 10/09/2020 09:52PM >Cc: "Ira Weiny" , "Mike Marciniszyn" >, "Dennis Dalessandro" >, "Doug Ledford" , >"Jason Gunthorpe" , "Faisal Latif" >, "Shiraz Saleem" , >"Bernard Metzler" , x86@kernel.org, "Dave Hansen" >, "Dan Williams" >, "Fenghua Yu" , >linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org, >linux-nvdimm@lists.01.org, linux-fsdevel@vger.kernel.org, >linux-mm@kvack.org, linux-kselftest@vger.kernel.org, >linuxppc-dev@lists.ozlabs.org, kvm@vger.kernel.org, >netdev@vger.kernel.org, bpf@vger.kernel.org, >kexec@lists.infradead.org, linux-bcache@vger.kernel.org, >linux-mtd@lists.infradead.org, devel@driverdev.osuosl.org, >linux-efi@vger.kernel.org, linux-mmc@vger.kernel.org, >linux-scsi@vger.kernel.org, target-devel@vger.kernel.org, >linux-nfs@vger.kernel.org, ceph-devel@vger.kernel.org, >linux-ext4@vger.kernel.org, linux-aio@kvack.org, >io-uring@vger.kernel.org, linux-erofs@lists.ozlabs.org, >linux-um@lists.infradead.org, linux-ntfs-dev@lists.sourceforge.net, >reiserfs-devel@vger.kernel.org, >linux-f2fs-devel@lists.sourceforge.net, linux-nilfs@vger.kernel.org, >cluster-devel@redhat.com, ecryptfs@vger.kernel.org, >linux-cifs@vger.kernel.org, linux-btrfs@vger.kernel.org, >linux-afs@lists.infradead.org, linux-rdma@vger.kernel.org, >amd-gfx@lists.freedesktop.org, dri-devel@lists.freedesktop.org, >intel-gfx@lists.freedesktop.org, drbd-dev@tron.linbit.com, >linux-block@vger.kernel.org, xen-devel@lists.xenproject.org, >linux-cachefs@redhat.com, samba-technical@lists.samba.org, >intel-wired-lan@lists.osuosl.org >Subject: [EXTERNAL] [PATCH RFC PKS/PMEM 10/58] drivers/rdma: Utilize >new kmap_thread() > >From: Ira Weiny > >The kmap() calls in these drivers are localized to a single thread. >To >avoid the over head of global PKRS updates use the new kmap_thread() >call. > >Cc: Mike Marciniszyn >Cc: Dennis Dalessandro >Cc: Doug Ledford >Cc: Jason Gunthorpe >Cc: Faisal Latif >Cc: Shiraz Saleem >Cc: Bernard Metzler >Signed-off-by: Ira Weiny >--- > drivers/infiniband/hw/hfi1/sdma.c | 4 ++-- > drivers/infiniband/hw/i40iw/i40iw_cm.c | 10 +++++----- > drivers/infiniband/sw/siw/siw_qp_tx.c | 14 +++++++------- > 3 files changed, 14 insertions(+), 14 deletions(-) > >diff --git a/drivers/infiniband/hw/hfi1/sdma.c >b/drivers/infiniband/hw/hfi1/sdma.c >index 04575c9afd61..09d206e3229a 100644 >--- a/drivers/infiniband/hw/hfi1/sdma.c >+++ b/drivers/infiniband/hw/hfi1/sdma.c >@@ -3130,7 +3130,7 @@ int ext_coal_sdma_tx_descs(struct hfi1_devdata >*dd, struct sdma_txreq *tx, > } > > if (type == SDMA_MAP_PAGE) { >- kvaddr = kmap(page); >+ kvaddr = kmap_thread(page); > kvaddr += offset; > } else if (WARN_ON(!kvaddr)) { > __sdma_txclean(dd, tx); >@@ -3140,7 +3140,7 @@ int ext_coal_sdma_tx_descs(struct hfi1_devdata >*dd, struct sdma_txreq *tx, > memcpy(tx->coalesce_buf + tx->coalesce_idx, kvaddr, len); > tx->coalesce_idx += len; > if (type == SDMA_MAP_PAGE) >- kunmap(page); >+ kunmap_thread(page); > > /* If there is more data, return */ > if (tx->tlen - tx->coalesce_idx) >diff --git a/drivers/infiniband/hw/i40iw/i40iw_cm.c >b/drivers/infiniband/hw/i40iw/i40iw_cm.c >index a3b95805c154..122d7a5642a1 100644 >--- a/drivers/infiniband/hw/i40iw/i40iw_cm.c >+++ b/drivers/infiniband/hw/i40iw/i40iw_cm.c >@@ -3721,7 +3721,7 @@ int i40iw_accept(struct iw_cm_id *cm_id, struct >iw_cm_conn_param *conn_param) > ibmr->device = iwpd->ibpd.device; > iwqp->lsmm_mr = ibmr; > if (iwqp->page) >- iwqp->sc_qp.qp_uk.sq_base = kmap(iwqp->page); >+ iwqp->sc_qp.qp_uk.sq_base = kmap_thread(iwqp->page); > dev->iw_priv_qp_ops->qp_send_lsmm(&iwqp->sc_qp, > iwqp->ietf_mem.va, > (accept.size + conn_param->private_data_len), >@@ -3729,12 +3729,12 @@ int i40iw_accept(struct iw_cm_id *cm_id, >struct iw_cm_conn_param *conn_param) > > } else { > if (iwqp->page) >- iwqp->sc_qp.qp_uk.sq_base = kmap(iwqp->page); >+ iwqp->sc_qp.qp_uk.sq_base = kmap_thread(iwqp->page); > dev->iw_priv_qp_ops->qp_send_lsmm(&iwqp->sc_qp, NULL, 0, 0); > } > > if (iwqp->page) >- kunmap(iwqp->page); >+ kunmap_thread(iwqp->page); > > iwqp->cm_id = cm_id; > cm_node->cm_id = cm_id; >@@ -4102,10 +4102,10 @@ static void i40iw_cm_event_connected(struct >i40iw_cm_event *event) > i40iw_cm_init_tsa_conn(iwqp, cm_node); > read0 = (cm_node->send_rdma0_op == SEND_RDMA_READ_ZERO); > if (iwqp->page) >- iwqp->sc_qp.qp_uk.sq_base = kmap(iwqp->page); >+ iwqp->sc_qp.qp_uk.sq_base = kmap_thread(iwqp->page); > dev->iw_priv_qp_ops->qp_send_rtt(&iwqp->sc_qp, read0); > if (iwqp->page) >- kunmap(iwqp->page); >+ kunmap_thread(iwqp->page); > > memset(&attr, 0, sizeof(attr)); > attr.qp_state = IB_QPS_RTS; >diff --git a/drivers/infiniband/sw/siw/siw_qp_tx.c >b/drivers/infiniband/sw/siw/siw_qp_tx.c >index d19d8325588b..4ed37c328d02 100644 >--- a/drivers/infiniband/sw/siw/siw_qp_tx.c >+++ b/drivers/infiniband/sw/siw/siw_qp_tx.c >@@ -76,7 +76,7 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, >void *paddr) > if (unlikely(!p)) > return -EFAULT; > >- buffer = kmap(p); >+ buffer = kmap_thread(p); > > if (likely(PAGE_SIZE - off >= bytes)) { > memcpy(paddr, buffer + off, bytes); >@@ -84,7 +84,7 @@ static int siw_try_1seg(struct siw_iwarp_tx *c_tx, >void *paddr) > unsigned long part = bytes - (PAGE_SIZE - off); > > memcpy(paddr, buffer + off, part); >- kunmap(p); >+ kunmap_thread(p); > > if (!mem->is_pbl) > p = siw_get_upage(mem->umem, >@@ -96,10 +96,10 @@ static int siw_try_1seg(struct siw_iwarp_tx >*c_tx, void *paddr) > if (unlikely(!p)) > return -EFAULT; > >- buffer = kmap(p); >+ buffer = kmap_thread(p); > memcpy(paddr + part, buffer, bytes - part); > } >- kunmap(p); >+ kunmap_thread(p); > } > } > return (int)bytes; >@@ -505,7 +505,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, >struct socket *s) > page_array[seg] = p; > > if (!c_tx->use_sendpage) { >- iov[seg].iov_base = kmap(p) + fp_off; >+ iov[seg].iov_base = kmap_thread(p) + fp_off; This misses a corresponding kunmap_thread() in siw_unmap_pages() (pls change line 403 in siw_qp_tx.c as well) Thanks, Bernard. > iov[seg].iov_len = plen; > > /* Remember for later kunmap() */ >@@ -518,9 +518,9 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx, >struct socket *s) > plen); > } else if (do_crc) { > crypto_shash_update(c_tx->mpa_crc_hd, >- kmap(p) + fp_off, >+ kmap_thread(p) + fp_off, > plen); >- kunmap(p); >+ kunmap_thread(p); > } > } else { > u64 va = sge->laddr + sge_off; >-- >2.28.0.rc0.12.gb6a658bd00c9 > >