From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> To: anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org Cc: linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Subject: [PATCH v2 3/5] xprtrdma: Disable pad optimization by default Date: Mon, 23 Jan 2017 15:53:02 -0500 [thread overview] Message-ID: <20170123205302.21699.89801.stgit@manet.1015granger.net> (raw) In-Reply-To: <20170123205159.21699.47373.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> Commit d5440e27d3e5 ('xprtrdma: Enable pad optimization') made the Linux client omit XDR round-up padding in normal Read and Write chunks so that the client doesn't have to register and invalidate 3-byte memory regions that contain no real data. Unfortunately, my cheery 2014 assessment that this optimization "is supported now by both Linux and Solaris servers" was premature. We've found bugs in Solaris in this area since commit d5440e27d3e5 was merged (SYMLINK is the main culprit). So for maximum interoperability, I'm disabling this optimization again. If a CM private message is exchanged when connecting, the client recognizes that the server is Linux, and enables the optimization for that connection. Until now the Solaris server bugs did not impact common operations, and were thus largely unnoticed. Soon, less capable devices on Linux NFS/RDMA clients will make use of Read chunks more often, and these Solaris bugs will prevent interoperation in more cases. Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> --- net/sunrpc/xprtrdma/transport.c | 2 +- net/sunrpc/xprtrdma/verbs.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 534c178..6990581 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -67,7 +67,7 @@ static unsigned int xprt_rdma_max_inline_write = RPCRDMA_DEF_INLINE; static unsigned int xprt_rdma_inline_write_padding; static unsigned int xprt_rdma_memreg_strategy = RPCRDMA_FRMR; - int xprt_rdma_pad_optimize = 1; + int xprt_rdma_pad_optimize = 0; #if IS_ENABLED(CONFIG_SUNRPC_DEBUG) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 2a6a367..23f4da4 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -216,6 +216,7 @@ pmsg->cp_magic == rpcrdma_cmp_magic && pmsg->cp_version == RPCRDMA_CMP_VERSION) { r_xprt->rx_ia.ri_reminv_expected = true; + r_xprt->rx_ia.ri_implicit_roundup = true; rsize = rpcrdma_decode_buffer_size(pmsg->cp_send_size); wsize = rpcrdma_decode_buffer_size(pmsg->cp_recv_size); } -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Chuck Lever <chuck.lever@oracle.com> To: anna.schumaker@netapp.com Cc: linux-rdma@vger.kernel.org, linux-nfs@vger.kernel.org Subject: [PATCH v2 3/5] xprtrdma: Disable pad optimization by default Date: Mon, 23 Jan 2017 15:53:02 -0500 [thread overview] Message-ID: <20170123205302.21699.89801.stgit@manet.1015granger.net> (raw) In-Reply-To: <20170123205159.21699.47373.stgit@manet.1015granger.net> Commit d5440e27d3e5 ('xprtrdma: Enable pad optimization') made the Linux client omit XDR round-up padding in normal Read and Write chunks so that the client doesn't have to register and invalidate 3-byte memory regions that contain no real data. Unfortunately, my cheery 2014 assessment that this optimization "is supported now by both Linux and Solaris servers" was premature. We've found bugs in Solaris in this area since commit d5440e27d3e5 was merged (SYMLINK is the main culprit). So for maximum interoperability, I'm disabling this optimization again. If a CM private message is exchanged when connecting, the client recognizes that the server is Linux, and enables the optimization for that connection. Until now the Solaris server bugs did not impact common operations, and were thus largely unnoticed. Soon, less capable devices on Linux NFS/RDMA clients will make use of Read chunks more often, and these Solaris bugs will prevent interoperation in more cases. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> --- net/sunrpc/xprtrdma/transport.c | 2 +- net/sunrpc/xprtrdma/verbs.c | 1 + 2 files changed, 2 insertions(+), 1 deletion(-) diff --git a/net/sunrpc/xprtrdma/transport.c b/net/sunrpc/xprtrdma/transport.c index 534c178..6990581 100644 --- a/net/sunrpc/xprtrdma/transport.c +++ b/net/sunrpc/xprtrdma/transport.c @@ -67,7 +67,7 @@ static unsigned int xprt_rdma_max_inline_write = RPCRDMA_DEF_INLINE; static unsigned int xprt_rdma_inline_write_padding; static unsigned int xprt_rdma_memreg_strategy = RPCRDMA_FRMR; - int xprt_rdma_pad_optimize = 1; + int xprt_rdma_pad_optimize = 0; #if IS_ENABLED(CONFIG_SUNRPC_DEBUG) diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 2a6a367..23f4da4 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -216,6 +216,7 @@ pmsg->cp_magic == rpcrdma_cmp_magic && pmsg->cp_version == RPCRDMA_CMP_VERSION) { r_xprt->rx_ia.ri_reminv_expected = true; + r_xprt->rx_ia.ri_implicit_roundup = true; rsize = rpcrdma_decode_buffer_size(pmsg->cp_send_size); wsize = rpcrdma_decode_buffer_size(pmsg->cp_recv_size); }
next prev parent reply other threads:[~2017-01-23 20:53 UTC|newest] Thread overview: 28+ messages / expand[flat|nested] mbox.gz Atom feed top 2017-01-23 20:52 [PATCH v2 0/5] Fix "support large inline thresholds" Chuck Lever 2017-01-23 20:52 ` Chuck Lever [not found] ` <20170123205159.21699.47373.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> 2017-01-23 20:52 ` [PATCH v2 1/5] xprtrdma: Fix Read chunk padding Chuck Lever 2017-01-23 20:52 ` Chuck Lever 2017-01-23 20:52 ` [PATCH v2 2/5] xprtrdma: Per-connection pad optimization Chuck Lever 2017-01-23 20:52 ` Chuck Lever [not found] ` <20170123205254.21699.44329.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> 2017-01-24 19:12 ` Anna Schumaker 2017-01-24 19:12 ` Anna Schumaker [not found] ` <19a00a6f-9502-161a-d37b-7371b07662ed-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org> 2017-01-24 19:16 ` Chuck Lever 2017-01-24 19:16 ` Chuck Lever [not found] ` <575D5F73-2D4B-4FDA-B196-0D683B459F9D-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-01-24 19:23 ` Anna Schumaker 2017-01-24 19:23 ` Anna Schumaker 2017-01-23 20:53 ` Chuck Lever [this message] 2017-01-23 20:53 ` [PATCH v2 3/5] xprtrdma: Disable pad optimization by default Chuck Lever 2017-01-23 20:53 ` [PATCH v2 4/5] xprtrdma: Reduce required number of send SGEs Chuck Lever 2017-01-23 20:53 ` Chuck Lever 2017-01-23 20:53 ` [PATCH v2 5/5] xprtrdma: Shrink send SGEs array Chuck Lever 2017-01-23 20:53 ` Chuck Lever 2017-01-24 21:35 ` [PATCH v2 0/5] Fix "support large inline thresholds" Anna Schumaker 2017-01-24 21:35 ` Anna Schumaker [not found] ` <353a72a5-549d-535b-28b7-411ba81483a1-ZwjVKphTwtPQT0dZR+AlfA@public.gmane.org> 2017-01-24 21:46 ` Chuck Lever 2017-01-24 21:46 ` Chuck Lever [not found] ` <82302622-9524-4020-869E-81F57629EC86-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-02-01 18:15 ` Chuck Lever 2017-02-01 18:15 ` Chuck Lever [not found] ` <68619BE7-CBFD-41DE-8227-ABCD797E81F7-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-02-08 16:47 ` Chuck Lever 2017-02-08 16:47 ` Chuck Lever [not found] ` <D12FEB95-6948-4E51-84EA-6D138C7F8B12-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2017-02-08 18:28 ` Anna Schumaker 2017-02-08 18:28 ` Anna Schumaker
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20170123205302.21699.89801.stgit@manet.1015granger.net \ --to=chuck.lever-qhclzuegtsvqt0dzr+alfa@public.gmane.org \ --cc=anna.schumaker-HgOvQuBEEgTQT0dZR+AlfA@public.gmane.org \ --cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.