From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> To: anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Subject: [PATCH v2 5/5] xprtrdma: Shrink send SGEs array Date: Mon, 23 Jan 2017 15:53:19 -0500 [thread overview] Message-ID: <20170123205319.21699.50374.stgit@manet.1015granger.net> (raw) In-Reply-To: <20170123205159.21699.47373.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> We no longer need to accommodate an xdr_buf whose pages start at an offset and cross extra page boundaries. If there are more partial or whole pages to send than there are available SGEs, the marshaling logic is now smart enough to use a Read chunk instead of failing. Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> --- net/sunrpc/xprtrdma/xprt_rdma.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 3d7e9c9..852dd0a 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -305,16 +305,19 @@ struct rpcrdma_mr_seg { /* chunk descriptors */ char *mr_offset; /* kva if no page, else offset */ }; -/* Reserve enough Send SGEs to send a maximum size inline request: +/* The Send SGE array is provisioned to send a maximum size + * inline request: * - RPC-over-RDMA header * - xdr_buf head iovec - * - RPCRDMA_MAX_INLINE bytes, possibly unaligned, in pages + * - RPCRDMA_MAX_INLINE bytes, in pages * - xdr_buf tail iovec + * + * The actual number of array elements consumed by each RPC + * depends on the device's max_sge limit. */ enum { RPCRDMA_MIN_SEND_SGES = 3, - RPCRDMA_MAX_SEND_PAGES = PAGE_SIZE + RPCRDMA_MAX_INLINE - 1, - RPCRDMA_MAX_PAGE_SGES = (RPCRDMA_MAX_SEND_PAGES >> PAGE_SHIFT) + 1, + RPCRDMA_MAX_PAGE_SGES = RPCRDMA_MAX_INLINE >> PAGE_SHIFT, RPCRDMA_MAX_SEND_SGES = 1 + 1 + RPCRDMA_MAX_PAGE_SGES + 1, }; -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Chuck Lever <chuck.lever@oracle.com> To: anna.schumaker@netapp.com Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Subject: [PATCH v2 5/5] xprtrdma: Shrink send SGEs array Date: Mon, 23 Jan 2017 15:53:19 -0500 [thread overview] Message-ID: <20170123205319.21699.50374.stgit@manet.1015granger.net> (raw) In-Reply-To: <20170123205159.21699.47373.stgit@manet.1015granger.net> We no longer need to accommodate an xdr_buf whose pages start at an offset and cross extra page boundaries. If there are more partial or whole pages to send than there are available SGEs, the marshaling logic is now smart enough to use a Read chunk instead of failing. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> --- net/sunrpc/xprtrdma/xprt_rdma.h | 11 +++++++---- 1 file changed, 7 insertions(+), 4 deletions(-) diff --git a/net/sunrpc/xprtrdma/xprt_rdma.h b/net/sunrpc/xprtrdma/xprt_rdma.h index 3d7e9c9..852dd0a 100644 --- a/net/sunrpc/xprtrdma/xprt_rdma.h +++ b/net/sunrpc/xprtrdma/xprt_rdma.h @@ -305,16 +305,19 @@ struct rpcrdma_mr_seg { /* chunk descriptors */ char *mr_offset; /* kva if no page, else offset */ }; -/* Reserve enough Send SGEs to send a maximum size inline request: +/* The Send SGE array is provisioned to send a maximum size + * inline request: * - RPC-over-RDMA header * - xdr_buf head iovec - * - RPCRDMA_MAX_INLINE bytes, possibly unaligned, in pages + * - RPCRDMA_MAX_INLINE bytes, in pages * - xdr_buf tail iovec + * + * The actual number of array elements consumed by each RPC + * depends on the device's max_sge limit. */ enum { RPCRDMA_MIN_SEND_SGES = 3, - RPCRDMA_MAX_SEND_PAGES = PAGE_SIZE + RPCRDMA_MAX_INLINE - 1, - RPCRDMA_MAX_PAGE_SGES = (RPCRDMA_MAX_SEND_PAGES >> PAGE_SHIFT) + 1, + RPCRDMA_MAX_PAGE_SGES = RPCRDMA_MAX_INLINE >> PAGE_SHIFT, RPCRDMA_MAX_SEND_SGES = 1 + 1 + RPCRDMA_MAX_PAGE_SGES + 1, };
next prev parent reply other threads:[~2017-01-23 20:53 UTC|newest] Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-01-23 20:52 [PATCH v2 0/5] Fix "support large inline thresholds" Chuck Lever 2017-01-23 20:52 ` Chuck Lever [not found] ` <20170123205159.21699.47373.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> 2017-01-23 20:52 ` [PATCH v2 1/5] xprtrdma: Fix Read chunk padding Chuck Lever 2017-01-23 20:52 ` Chuck Lever 2017-01-23 20:52 ` [PATCH v2 2/5] xprtrdma: Per-connection pad optimization Chuck Lever 2017-01-23 20:52 ` Chuck Lever [not found] ` <20170123205254.21699.44329.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> 2017-01-24 19:12 ` Anna Schumaker 2017-01-24 19:12 ` Anna Schumaker [not found] ` <19a00a6f-9502-161a-d37b-7371b07662ed-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org> 2017-01-24 19:16 ` Chuck Lever 2017-01-24 19:16 ` Chuck Lever [not found] ` <575D5F73-2D4B-4FDA-B196-0D683B459F9D-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-01-24 19:23 ` Anna Schumaker 2017-01-24 19:23 ` Anna Schumaker 2017-01-23 20:53 ` [PATCH v2 3/5] xprtrdma: Disable pad optimization by default Chuck Lever 2017-01-23 20:53 ` Chuck Lever 2017-01-23 20:53 ` [PATCH v2 4/5] xprtrdma: Reduce required number of send SGEs Chuck Lever 2017-01-23 20:53 ` Chuck Lever 2017-01-23 20:53 ` Chuck Lever [this message] 2017-01-23 20:53 ` [PATCH v2 5/5] xprtrdma: Shrink send SGEs array Chuck Lever 2017-01-24 21:35 ` [PATCH v2 0/5] Fix "support large inline thresholds" Anna Schumaker 2017-01-24 21:35 ` Anna Schumaker [not found] ` <353a72a5-549d-535b-28b7-411ba81483a1-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org> 2017-01-24 21:46 ` Chuck Lever 2017-01-24 21:46 ` Chuck Lever [not found] ` <82302622-9524-4020-869E-81F57629EC86-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-02-01 18:15 ` Chuck Lever 2017-02-01 18:15 ` Chuck Lever [not found] ` <68619BE7-CBFD-41DE-8227-ABCD797E81F7-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-02-08 16:47 ` Chuck Lever 2017-02-08 16:47 ` Chuck Lever [not found] ` <D12FEB95-6948-4E51-84EA-6D138C7F8B12-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-02-08 18:28 ` Anna Schumaker 2017-02-08 18:28 ` Anna Schumaker
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170123205319.21699.50374.stgit@manet.1015granger.net \ --to=chuck.lever-qhclzuegtsvqt0dzr+alfa@public.gmane.org \ --cc=anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org \ --cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.