All of lore.kernel.org
 help / color / mirror / Atom feed
From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
To: anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org
Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
Subject: [PATCH v2 5/5] xprtrdma: Shrink send SGEs array
Date: Mon, 23 Jan 2017 15:53:19 -0500	[thread overview]
Message-ID: <20170123205319.21699.50374.stgit@manet.1015granger.net> (raw)
In-Reply-To: <20170123205159.21699.47373.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>

We no longer need to accommodate an xdr_buf whose pages start at an
offset and cross extra page boundaries. If there are more partial or
whole pages to send than there are available SGEs, the marshaling
logic is now smart enough to use a Read chunk instead of failing.

Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
---
 net/sunrpc/xprtrdma/xprt_rdma.h |   11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index 3d7e9c9..852dd0a 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -305,16 +305,19 @@ struct rpcrdma_mr_seg {		/* chunk descriptors */
 	char		*mr_offset;	/* kva if no page, else offset */
 };
 
-/* Reserve enough Send SGEs to send a maximum size inline request:
+/* The Send SGE array is provisioned to send a maximum size
+ * inline request:
  * - RPC-over-RDMA header
  * - xdr_buf head iovec
- * - RPCRDMA_MAX_INLINE bytes, possibly unaligned, in pages
+ * - RPCRDMA_MAX_INLINE bytes, in pages
  * - xdr_buf tail iovec
+ *
+ * The actual number of array elements consumed by each RPC
+ * depends on the device's max_sge limit.
  */
 enum {
 	RPCRDMA_MIN_SEND_SGES = 3,
-	RPCRDMA_MAX_SEND_PAGES = PAGE_SIZE + RPCRDMA_MAX_INLINE - 1,
-	RPCRDMA_MAX_PAGE_SGES = (RPCRDMA_MAX_SEND_PAGES >> PAGE_SHIFT) + 1,
+	RPCRDMA_MAX_PAGE_SGES = RPCRDMA_MAX_INLINE >> PAGE_SHIFT,
 	RPCRDMA_MAX_SEND_SGES = 1 + 1 + RPCRDMA_MAX_PAGE_SGES + 1,
 };
 

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

WARNING: multiple messages have this Message-ID (diff)
From: Chuck Lever <chuck.lever@oracle.com>
To: anna.schumaker@netapp.com
Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org
Subject: [PATCH v2 5/5] xprtrdma: Shrink send SGEs array
Date: Mon, 23 Jan 2017 15:53:19 -0500	[thread overview]
Message-ID: <20170123205319.21699.50374.stgit@manet.1015granger.net> (raw)
In-Reply-To: <20170123205159.21699.47373.stgit@manet.1015granger.net>

We no longer need to accommodate an xdr_buf whose pages start at an
offset and cross extra page boundaries. If there are more partial or
whole pages to send than there are available SGEs, the marshaling
logic is now smart enough to use a Read chunk instead of failing.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 net/sunrpc/xprtrdma/xprt_rdma.h |   11 +++++++----
 1 file changed, 7 insertions(+), 4 deletions(-)

diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h
index 3d7e9c9..852dd0a 100644
--- a/net/sunrpc/xprtrdma/xprt_rdma.h
+++ b/net/sunrpc/xprtrdma/xprt_rdma.h
@@ -305,16 +305,19 @@ struct rpcrdma_mr_seg {		/* chunk descriptors */
 	char		*mr_offset;	/* kva if no page, else offset */
 };
 
-/* Reserve enough Send SGEs to send a maximum size inline request:
+/* The Send SGE array is provisioned to send a maximum size
+ * inline request:
  * - RPC-over-RDMA header
  * - xdr_buf head iovec
- * - RPCRDMA_MAX_INLINE bytes, possibly unaligned, in pages
+ * - RPCRDMA_MAX_INLINE bytes, in pages
  * - xdr_buf tail iovec
+ *
+ * The actual number of array elements consumed by each RPC
+ * depends on the device's max_sge limit.
  */
 enum {
 	RPCRDMA_MIN_SEND_SGES = 3,
-	RPCRDMA_MAX_SEND_PAGES = PAGE_SIZE + RPCRDMA_MAX_INLINE - 1,
-	RPCRDMA_MAX_PAGE_SGES = (RPCRDMA_MAX_SEND_PAGES >> PAGE_SHIFT) + 1,
+	RPCRDMA_MAX_PAGE_SGES = RPCRDMA_MAX_INLINE >> PAGE_SHIFT,
 	RPCRDMA_MAX_SEND_SGES = 1 + 1 + RPCRDMA_MAX_PAGE_SGES + 1,
 };
 


  parent reply	other threads:[~2017-01-23 20:53 UTC|newest]

Thread overview: 28+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2017-01-23 20:52 [PATCH v2 0/5] Fix "support large inline thresholds" Chuck Lever
2017-01-23 20:52 ` Chuck Lever
     [not found] ` <20170123205159.21699.47373.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-01-23 20:52   ` [PATCH v2 1/5] xprtrdma: Fix Read chunk padding Chuck Lever
2017-01-23 20:52     ` Chuck Lever
2017-01-23 20:52   ` [PATCH v2 2/5] xprtrdma: Per-connection pad optimization Chuck Lever
2017-01-23 20:52     ` Chuck Lever
     [not found]     ` <20170123205254.21699.44329.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org>
2017-01-24 19:12       ` Anna Schumaker
2017-01-24 19:12         ` Anna Schumaker
     [not found]         ` <19a00a6f-9502-161a-d37b-7371b07662ed-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org>
2017-01-24 19:16           ` Chuck Lever
2017-01-24 19:16             ` Chuck Lever
     [not found]             ` <575D5F73-2D4B-4FDA-B196-0D683B459F9D-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-01-24 19:23               ` Anna Schumaker
2017-01-24 19:23                 ` Anna Schumaker
2017-01-23 20:53   ` [PATCH v2 3/5] xprtrdma: Disable pad optimization by default Chuck Lever
2017-01-23 20:53     ` Chuck Lever
2017-01-23 20:53   ` [PATCH v2 4/5] xprtrdma: Reduce required number of send SGEs Chuck Lever
2017-01-23 20:53     ` Chuck Lever
2017-01-23 20:53   ` Chuck Lever [this message]
2017-01-23 20:53     ` [PATCH v2 5/5] xprtrdma: Shrink send SGEs array Chuck Lever
2017-01-24 21:35   ` [PATCH v2 0/5] Fix "support large inline thresholds" Anna Schumaker
2017-01-24 21:35     ` Anna Schumaker
     [not found]     ` <353a72a5-549d-535b-28b7-411ba81483a1-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org>
2017-01-24 21:46       ` Chuck Lever
2017-01-24 21:46         ` Chuck Lever
     [not found]         ` <82302622-9524-4020-869E-81F57629EC86-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-02-01 18:15           ` Chuck Lever
2017-02-01 18:15             ` Chuck Lever
     [not found]             ` <68619BE7-CBFD-41DE-8227-ABCD797E81F7-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-02-08 16:47               ` Chuck Lever
2017-02-08 16:47                 ` Chuck Lever
     [not found]                 ` <D12FEB95-6948-4E51-84EA-6D138C7F8B12-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
2017-02-08 18:28                   ` Anna Schumaker
2017-02-08 18:28                     ` Anna Schumaker

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20170123205319.21699.50374.stgit@manet.1015granger.net \
    --to=chuck.lever-qhclzuegtsvqt0dzr+alfa@public.gmane.org \
    --cc=anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org \
    --cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.