Linux-NFS Archive on lore.kernel.org
 help / color / Atom feed
From: Chuck Lever <chuck.lever@oracle.com>
To: anna.schumaker@netapp.com
Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org
Subject: [PATCH v1 02/23] xprtrdma: Check inline size before providing a Write chunk
Date: Mon, 11 Feb 2019 11:23:49 -0500
Message-ID: <20190211162349.2817.19297.stgit@manet.1015granger.net> (raw)
In-Reply-To: <20190211161920.2817.20881.stgit@manet.1015granger.net>

In very rare cases, an NFS READ operation might predict that the
non-payload part of the RPC Call is large. For instance, an
NFSv4 COMPOUND with a large GETATTR result, in combination with a
large Kerberos credential, could push the non-payload part to be
several kilobytes.

If the non-payload part is larger than the connection's inline
threshold, the client is required to provision a Reply chunk. The
current Linux client does not check for this case. There are two
obvious ways to handle it:

a. Provision a Write chunk for the payload and a Reply chunk for
   the non-payload part

b. Provision a Reply chunk for the whole RPC Reply

Some testing at a recent NFS bake-a-thon showed that servers can
mostly handle a. but there are some corner cases that do not work
yet. b. already works (it has to, to handle krb5i/p), but could be
somewhat less efficient. However, I expect this scenario to be very
rare -- no-one has reported a problem yet.

So I'm going to implement b. Sometime later I will provide some
patches to help make b. a little more efficient by more carefully
choosing the Reply chunk's segment sizes to ensure the payload is
optimally aligned.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 net/sunrpc/xprtrdma/rpc_rdma.c |   18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)

diff --git a/net/sunrpc/xprtrdma/rpc_rdma.c b/net/sunrpc/xprtrdma/rpc_rdma.c
index d18614e..7774aee 100644
--- a/net/sunrpc/xprtrdma/rpc_rdma.c
+++ b/net/sunrpc/xprtrdma/rpc_rdma.c
@@ -164,6 +164,21 @@ static bool rpcrdma_results_inline(struct rpcrdma_xprt *r_xprt,
 	return rqst->rq_rcv_buf.buflen <= ia->ri_max_inline_read;
 }
 
+/* The client is required to provide a Reply chunk if the maximum
+ * size of the non-payload part of the RPC Reply is larger than
+ * the inline threshold.
+ */
+static bool
+rpcrdma_nonpayload_inline(const struct rpcrdma_xprt *r_xprt,
+			  const struct rpc_rqst *rqst)
+{
+	const struct xdr_buf *buf = &rqst->rq_rcv_buf;
+	const struct rpcrdma_ia *ia = &r_xprt->rx_ia;
+
+	return buf->head[0].iov_len + buf->tail[0].iov_len <
+		ia->ri_max_inline_read;
+}
+
 /* Split @vec on page boundaries into SGEs. FMR registers pages, not
  * a byte range. Other modes coalesce these SGEs into a single MR
  * when they can.
@@ -762,7 +777,8 @@ static bool rpcrdma_results_inline(struct rpcrdma_xprt *r_xprt,
 	 */
 	if (rpcrdma_results_inline(r_xprt, rqst))
 		wtype = rpcrdma_noch;
-	else if (ddp_allowed && rqst->rq_rcv_buf.flags & XDRBUF_READ)
+	else if ((ddp_allowed && rqst->rq_rcv_buf.flags & XDRBUF_READ) &&
+		 rpcrdma_nonpayload_inline(r_xprt, rqst))
 		wtype = rpcrdma_writech;
 	else
 		wtype = rpcrdma_replych;


  parent reply index

Thread overview: 37+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2019-02-11 16:23 [PATCH v1 00/23] NFS client patches for v5.1 (complete) Chuck Lever
2019-02-11 16:23 ` [PATCH v1 01/23] xprtrdma: Fix sparse warnings Chuck Lever
2019-02-11 16:23 ` Chuck Lever [this message]
2019-02-11 16:23 ` [PATCH v1 03/23] xprtrdma: Reduce the doorbell rate (Receive) Chuck Lever
2019-02-11 16:24 ` [PATCH v1 04/23] SUNRPC: Display symbolic flag names in RPC trace events Chuck Lever
2019-02-11 16:24 ` [PATCH v1 05/23] SUNRPC: Add xdr_stream::rqst field Chuck Lever
2019-02-11 16:24 ` [PATCH v1 06/23] SUNRPC: Add XDR overflow trace event Chuck Lever
2019-02-11 16:24 ` [PATCH v1 07/23] SUNRPC: Add trace event that reports reply page vector alignment Chuck Lever
2019-02-11 16:24 ` [PATCH v1 08/23] NFS: Remove print_overflow_msg() Chuck Lever
2019-02-11 16:24 ` [PATCH v1 09/23] NFS: Add trace events to report non-zero NFS status codes Chuck Lever
2019-02-11 16:24 ` [PATCH v1 10/23] SUNRPC: Remove some dprintk() call sites from auth functions Chuck Lever
2019-02-11 16:24 ` [PATCH v1 11/23] SUNRPC: Remove rpc_xprt::tsh_size Chuck Lever
2019-02-11 16:24 ` [PATCH v1 12/23] SUNRPC: Add build option to disable support for insecure enctypes Chuck Lever
2019-02-11 16:24 ` [PATCH v1 13/23] SUNRPC: Use struct xdr_stream when constructing RPC Call header Chuck Lever
2019-02-11 16:24 ` [PATCH v1 14/23] SUNRPC: Clean up rpc_verify_header() Chuck Lever
2019-02-11 16:24 ` [PATCH v1 15/23] SUNRPC: Use struct xdr_stream when decoding RPC Reply header Chuck Lever
2019-02-11 16:25 ` [PATCH v1 16/23] SUNRPC: Introduce trace points in rpc_auth_gss.ko Chuck Lever
2019-02-11 16:25 ` [PATCH v1 17/23] SUNRPC: Remove xdr_buf_trim() Chuck Lever
2019-02-11 16:25 ` [PATCH v1 18/23] SUNRPC: Add SPDX IDs to some net/sunrpc/auth_gss/ files Chuck Lever
2019-02-11 16:25 ` [PATCH v1 19/23] SUNRPC: Introduce rpc_prepare_reply_pages() Chuck Lever
2019-02-11 16:25 ` [PATCH v1 20/23] NFS: Account for XDR pad of buf->pages Chuck Lever
2019-04-05 17:36   ` Olga Kornievskaia
2019-04-05 17:51     ` Chuck Lever
2019-04-05 19:17       ` Olga Kornievskaia
2019-04-05 19:23         ` Chuck Lever
2019-04-05 19:27           ` Olga Kornievskaia
2019-04-05 19:42             ` Chuck Lever
2019-04-08 14:36               ` Olga Kornievskaia
2019-04-08 14:43                 ` Chuck Lever
2019-04-08 15:21                   ` Olga Kornievskaia
2019-04-08 15:26                     ` Olga Kornievskaia
2019-04-08 15:50                       ` Olga Kornievskaia
2019-04-08 16:02                         ` Olga Kornievskaia
2019-04-08 16:29                         ` Chuck Lever
2019-02-11 16:25 ` [PATCH v1 21/23] SUNRPC: Make AUTH_SYS and AUTH_NULL set au_verfsize Chuck Lever
2019-02-11 16:25 ` [PATCH v1 22/23] SUNRPC: Add rpc_auth::au_ralign field Chuck Lever
2019-02-11 16:25 ` [PATCH v1 23/23] SUNRPC: Use au_rslack when computing reply buffer size Chuck Lever

Reply instructions:

You may reply publically to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20190211162349.2817.19297.stgit@manet.1015granger.net \
    --to=chuck.lever@oracle.com \
    --cc=anna.schumaker@netapp.com \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-NFS Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-nfs/0 linux-nfs/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-nfs linux-nfs/ https://lore.kernel.org/linux-nfs \
		linux-nfs@vger.kernel.org
	public-inbox-index linux-nfs

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kernel.vger.linux-nfs


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git