Re: [PATCH V3] RDMA/siw: Convert siw_tx_hdt() to kmap_local_page()

From: Ira Weiny <ira.weiny@intel.com>
To: Bernard Metzler <BMT@zurich.ibm.com>
Cc: Jason Gunthorpe <jgg@ziepe.ca>,
	Mike Marciniszyn <mike.marciniszyn@cornelisnetworks.com>,
	Dennis Dalessandro <dennis.dalessandro@cornelisnetworks.com>,
	Doug Ledford <dledford@redhat.com>,
	Faisal Latif <faisal.latif@intel.com>,
	Shiraz Saleem <shiraz.saleem@intel.com>,
	Kamal Heib <kheib@redhat.com>,
	linux-rdma <linux-rdma@vger.kernel.org>,
	linux-kernel <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH V3] RDMA/siw: Convert siw_tx_hdt() to kmap_local_page()
Date: Thu, 24 Jun 2021 10:33:16 -0700	[thread overview]
Message-ID: <20210624173316.GA2799309@iweiny-DESK2.sc.intel.com> (raw)
In-Reply-To: <OF739F2480.B35209F8-ON002586FE.00569A1A-002586FE.00569A24@ch.ibm.com>

On Thu, Jun 24, 2021 at 03:45:55PM +0000, Bernard Metzler wrote:
> 
> >@@ -593,7 +601,7 @@ static int siw_tx_hdt(struct siw_iwarp_tx *c_tx,
> >struct socket *s)
> > 	} else {
> > 		rv = kernel_sendmsg(s, &msg, iov, seg + 1,
> > 				    hdr_len + data_len + trl_len);
> >-		siw_unmap_pages(page_array, kmap_mask);
> >+		siw_unmap_pages(iov, kmap_mask, seg+1);
> 
> seg+1 is one to many, since the last segment references the iWarp
> trailer (CRC). There are 2 reason for this multi-segment processing
> in the transmit path. (1) efficiency and (2) MTU based packet framing.
> The iov contains the complete iWarp frame with header, (potentially
> multiple) data fragments, and the CRC. It gets pushed to TCP in one
> go, praying for iWarp framing stays intact (which most time works).
> So the code can collect data form multiple SGE's of a WRITE or
> SEND and tries putting those into one frame, if MTU allows, and
> adds header and trailer. 
>
> The last segment (seg + 1) references the CRC, which is never kmap'ed.

siw_unmap_pages() take a length and seg is the index...

But ok so a further optimization...

Fair enough.

> 
> I'll try the code next days, but it looks good otherwise!

I believe this will work though.

Ira

> Thanks very much!
> > 	}
> > 	if (rv < (int)hdr_len) {
> > 		/* Not even complete hdr pushed or negative rv */
> >-- 
> >2.28.0.rc0.12.gb6a658bd00c9
> >
> >