linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH] svcrdma: fix bounce buffers for non-zero page offsets
@ 2020-10-02 14:48 Dan Aloni
  2020-10-02 14:54 ` Chuck Lever
  0 siblings, 1 reply; 6+ messages in thread
From: Dan Aloni @ 2020-10-02 14:48 UTC (permalink / raw)
  To: Chuck Lever; +Cc: linux-rdma, linux-nfs

This was discovered using O_DIRECT and small unaligned file offsets
at the client side.

Fixes: e248aa7be86 ("svcrdma: Remove max_sge check at connect time")
Signed-off-by: Dan Aloni <dan@kernelim.com>
---
 net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index 7b94d971feb3..c991eb1fd4e3 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,
 		while (remaining) {
 			len = min_t(u32, PAGE_SIZE - pageoff, remaining);
 
-			memcpy(dst, page_address(*ppages), len);
+			memcpy(dst, page_address(*ppages) + pageoff, len);
 			remaining -= len;
 			dst += len;
 			pageoff = 0;
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH] svcrdma: fix bounce buffers for non-zero page offsets
  2020-10-02 14:48 [PATCH] svcrdma: fix bounce buffers for non-zero page offsets Dan Aloni
@ 2020-10-02 14:54 ` Chuck Lever
  2020-10-02 15:18   ` Dan Aloni
  0 siblings, 1 reply; 6+ messages in thread
From: Chuck Lever @ 2020-10-02 14:54 UTC (permalink / raw)
  To: Dan Aloni; +Cc: linux-rdma, Linux NFS Mailing List

Hi Dan-

> On Oct 2, 2020, at 10:48 AM, Dan Aloni <dan@kernelim.com> wrote:
> 
> This was discovered using O_DIRECT and small unaligned file offsets
> at the client side.
> 
> Fixes: e248aa7be86 ("svcrdma: Remove max_sge check at connect time")
> Signed-off-by: Dan Aloni <dan@kernelim.com>
> ---
> net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 +-
> 1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> index 7b94d971feb3..c991eb1fd4e3 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> @@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,
> 		while (remaining) {
> 			len = min_t(u32, PAGE_SIZE - pageoff, remaining);
> 
> -			memcpy(dst, page_address(*ppages), len);
> +			memcpy(dst, page_address(*ppages) + pageoff, len);

I'm assuming the only relevant place that sets xdr->page_base
is nfsd_splice_actor() ?


> 			remaining -= len;
> 			dst += len;
> 			pageoff = 0;
> -- 
> 2.26.2
> 

--
Chuck Lever




^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] svcrdma: fix bounce buffers for non-zero page offsets
  2020-10-02 14:54 ` Chuck Lever
@ 2020-10-02 15:18   ` Dan Aloni
  2020-10-02 15:23     ` Chuck Lever
  0 siblings, 1 reply; 6+ messages in thread
From: Dan Aloni @ 2020-10-02 15:18 UTC (permalink / raw)
  To: Chuck Lever; +Cc: linux-rdma, Linux NFS Mailing List

On Fri, Oct 02, 2020 at 10:54:28AM -0400, Chuck Lever wrote:
> Hi Dan-
> 
> > On Oct 2, 2020, at 10:48 AM, Dan Aloni <dan@kernelim.com> wrote:
> > 
> > This was discovered using O_DIRECT and small unaligned file offsets
> > at the client side.
> > 
> > Fixes: e248aa7be86 ("svcrdma: Remove max_sge check at connect time")
> > Signed-off-by: Dan Aloni <dan@kernelim.com>
> > ---
> > net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 +-
> > 1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> > index 7b94d971feb3..c991eb1fd4e3 100644
> > --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> > +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> > @@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,
> > 		while (remaining) {
> > 			len = min_t(u32, PAGE_SIZE - pageoff, remaining);
> > 
> > -			memcpy(dst, page_address(*ppages), len);
> > +			memcpy(dst, page_address(*ppages) + pageoff, len);
> 
> I'm assuming the only relevant place that sets xdr->page_base
> is nfsd_splice_actor() ?

Yes, and traces at the server indeed indicate that splicing happened.
This works for both tmpfs and ext4 as host FSes.

-- 
Dan Aloni

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] svcrdma: fix bounce buffers for non-zero page offsets
  2020-10-02 15:18   ` Dan Aloni
@ 2020-10-02 15:23     ` Chuck Lever
  2020-10-02 19:33       ` [PATCH v2] svcrdma: fix bounce buffers for unaligned offsets and multiple pages Dan Aloni
  0 siblings, 1 reply; 6+ messages in thread
From: Chuck Lever @ 2020-10-02 15:23 UTC (permalink / raw)
  To: Dan Aloni; +Cc: linux-rdma, Linux NFS Mailing List



> On Oct 2, 2020, at 11:18 AM, Dan Aloni <dan@kernelim.com> wrote:
> 
> On Fri, Oct 02, 2020 at 10:54:28AM -0400, Chuck Lever wrote:
>> Hi Dan-
>> 
>>> On Oct 2, 2020, at 10:48 AM, Dan Aloni <dan@kernelim.com> wrote:
>>> 
>>> This was discovered using O_DIRECT and small unaligned file offsets
>>> at the client side.
>>> 
>>> Fixes: e248aa7be86 ("svcrdma: Remove max_sge check at connect time")
>>> Signed-off-by: Dan Aloni <dan@kernelim.com>
>>> ---
>>> net/sunrpc/xprtrdma/svc_rdma_sendto.c | 2 +-
>>> 1 file changed, 1 insertion(+), 1 deletion(-)
>>> 
>>> diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
>>> index 7b94d971feb3..c991eb1fd4e3 100644
>>> --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
>>> +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
>>> @@ -638,7 +638,7 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,
>>> 		while (remaining) {
>>> 			len = min_t(u32, PAGE_SIZE - pageoff, remaining);
>>> 
>>> -			memcpy(dst, page_address(*ppages), len);
>>> +			memcpy(dst, page_address(*ppages) + pageoff, len);
>> 
>> I'm assuming the only relevant place that sets xdr->page_base
>> is nfsd_splice_actor() ?
> 
> Yes, and traces at the server indeed indicate that splicing happened.
> This works for both tmpfs and ext4 as host FSes.

Seems plausible, thanks. I'll throw it into my test environment.


--
Chuck Lever




^ permalink raw reply	[flat|nested] 6+ messages in thread

* [PATCH v2] svcrdma: fix bounce buffers for unaligned offsets and multiple pages
  2020-10-02 15:23     ` Chuck Lever
@ 2020-10-02 19:33       ` Dan Aloni
  2020-10-14 21:26         ` J. Bruce Fields
  0 siblings, 1 reply; 6+ messages in thread
From: Dan Aloni @ 2020-10-02 19:33 UTC (permalink / raw)
  To: Chuck Lever; +Cc: linux-rdma, linux-nfs

This was discovered using O_DIRECT at the client side, with small
unaligned file offsets or IOs that span multiple file pages.

Fixes: e248aa7be86 ("svcrdma: Remove max_sge check at connect time")
Signed-off-by: Dan Aloni <dan@kernelim.com>
---
 net/sunrpc/xprtrdma/svc_rdma_sendto.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

Extended testing found another issue with the loop.

diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index 7b94d971feb3..c3d588b149aa 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -638,10 +638,11 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,
 		while (remaining) {
 			len = min_t(u32, PAGE_SIZE - pageoff, remaining);
 
-			memcpy(dst, page_address(*ppages), len);
+			memcpy(dst, page_address(*ppages) + pageoff, len);
 			remaining -= len;
 			dst += len;
 			pageoff = 0;
+			ppages++;
 		}
 	}
 
-- 
2.26.2


^ permalink raw reply related	[flat|nested] 6+ messages in thread

* Re: [PATCH v2] svcrdma: fix bounce buffers for unaligned offsets and multiple pages
  2020-10-02 19:33       ` [PATCH v2] svcrdma: fix bounce buffers for unaligned offsets and multiple pages Dan Aloni
@ 2020-10-14 21:26         ` J. Bruce Fields
  0 siblings, 0 replies; 6+ messages in thread
From: J. Bruce Fields @ 2020-10-14 21:26 UTC (permalink / raw)
  To: Dan Aloni; +Cc: Chuck Lever, linux-rdma, linux-nfs

Thanks, applying for 5.10.--b.

On Fri, Oct 02, 2020 at 10:33:43PM +0300, Dan Aloni wrote:
> This was discovered using O_DIRECT at the client side, with small
> unaligned file offsets or IOs that span multiple file pages.
> 
> Fixes: e248aa7be86 ("svcrdma: Remove max_sge check at connect time")
> Signed-off-by: Dan Aloni <dan@kernelim.com>
> ---
>  net/sunrpc/xprtrdma/svc_rdma_sendto.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> Extended testing found another issue with the loop.
> 
> diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> index 7b94d971feb3..c3d588b149aa 100644
> --- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> +++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
> @@ -638,10 +638,11 @@ static int svc_rdma_pull_up_reply_msg(struct svcxprt_rdma *rdma,
>  		while (remaining) {
>  			len = min_t(u32, PAGE_SIZE - pageoff, remaining);
>  
> -			memcpy(dst, page_address(*ppages), len);
> +			memcpy(dst, page_address(*ppages) + pageoff, len);
>  			remaining -= len;
>  			dst += len;
>  			pageoff = 0;
> +			ppages++;
>  		}
>  	}
>  
> -- 
> 2.26.2

^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, other threads:[~2020-10-15  2:19 UTC | newest]

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-10-02 14:48 [PATCH] svcrdma: fix bounce buffers for non-zero page offsets Dan Aloni
2020-10-02 14:54 ` Chuck Lever
2020-10-02 15:18   ` Dan Aloni
2020-10-02 15:23     ` Chuck Lever
2020-10-02 19:33       ` [PATCH v2] svcrdma: fix bounce buffers for unaligned offsets and multiple pages Dan Aloni
2020-10-14 21:26         ` J. Bruce Fields

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).