From: Chuck Lever <chuck.lever@oracle.com> To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org Subject: [PATCH 3/8] xprtrdma: Disable ALLPHYSICAL mode by default Date: Mon, 14 Apr 2014 18:22:49 -0400 [thread overview] Message-ID: <20140414222249.20646.47882.stgit@manet.1015granger.net> (raw) In-Reply-To: <20140414220041.20646.63991.stgit@manet.1015granger.net> ALLPHYSICAL is not a safe memory registration mode because it permits NFS servers to write anywhere in a client's memory. NFS server bugs could result in client memory being overwritten. This can be useful for embedded systems which do not support more surgical RDMA memory registration and protection methods. However, enterprise Linux distributions have expressed a desire to disable or remove it entirely. As a compromise (or as a step towards deprecation), add a CONFIG setting (by default N) to enable ALLPHYSICAL support. ALLPHYSICAL is now never a fallback choice when HCAs do not support the selected memory registration mode. Signed-off-by: Chuck Lever <chuck.lever@oracle.com> --- include/linux/sunrpc/xprtrdma.h | 3 --- net/sunrpc/Kconfig | 15 +++++++++++++++ net/sunrpc/xprtrdma/verbs.c | 22 +++------------------- 3 files changed, 18 insertions(+), 22 deletions(-) diff --git a/include/linux/sunrpc/xprtrdma.h b/include/linux/sunrpc/xprtrdma.h index c2f04e1..54e9b47 100644 --- a/include/linux/sunrpc/xprtrdma.h +++ b/include/linux/sunrpc/xprtrdma.h @@ -61,9 +61,6 @@ #define RPCRDMA_INLINE_PAD_THRESH (512)/* payload threshold to pad (bytes) */ -/* memory registration strategies */ -#define RPCRDMA_PERSISTENT_REGISTRATION (1) - enum rpcrdma_memreg { RPCRDMA_BOUNCEBUFFERS = 0, RPCRDMA_REGISTER, diff --git a/net/sunrpc/Kconfig b/net/sunrpc/Kconfig index 0754d0f..af56554 100644 --- a/net/sunrpc/Kconfig +++ b/net/sunrpc/Kconfig @@ -58,6 +58,21 @@ config SUNRPC_XPRT_RDMA_CLIENT If unsure, say N. +config SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL + bool "Enable ALLPHYSICAL memory registration mode" + depends on SUNRPC_XPRT_RDMA_CLIENT + default N + help + This option enables support for the ALLPHYSICAL memory + registration mode. + + This mode is very fast but not safe because it registers + and exposes all of local memory. This could allow an + NFS server bug to corrupt client memory. + + N is almost always the correct choice unless you are + sure what you are doing. + config SUNRPC_XPRT_RDMA_SERVER tristate "RPC over RDMA Server Support" depends on SUNRPC && INFINIBAND && INFINIBAND_ADDR_TRANS diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 1fe554f..6373232 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -506,19 +506,11 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr *addr, int memreg) break; case RPCRDMA_MTHCAFMR: if (!ia->ri_id->device->alloc_fmr) { -#if RPCRDMA_PERSISTENT_REGISTRATION - dprintk("RPC: %s: MTHCAFMR registration " - "specified but not supported by adapter, " - "using riskier RPCRDMA_ALLPHYSICAL\n", - __func__); - memreg = RPCRDMA_ALLPHYSICAL; -#else dprintk("RPC: %s: MTHCAFMR registration " "specified but not supported by adapter, " "using slower RPCRDMA_REGISTER\n", __func__); memreg = RPCRDMA_REGISTER; -#endif } break; case RPCRDMA_FRMR: @@ -526,19 +518,11 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr *addr, int memreg) if ((devattr.device_cap_flags & (IB_DEVICE_MEM_MGT_EXTENSIONS|IB_DEVICE_LOCAL_DMA_LKEY)) != (IB_DEVICE_MEM_MGT_EXTENSIONS|IB_DEVICE_LOCAL_DMA_LKEY)) { -#if RPCRDMA_PERSISTENT_REGISTRATION - dprintk("RPC: %s: FRMR registration " - "specified but not supported by adapter, " - "using riskier RPCRDMA_ALLPHYSICAL\n", - __func__); - memreg = RPCRDMA_ALLPHYSICAL; -#else dprintk("RPC: %s: FRMR registration " "specified but not supported by adapter, " "using slower RPCRDMA_REGISTER\n", __func__); memreg = RPCRDMA_REGISTER; -#endif } else { /* Mind the ia limit on FRMR page list depth */ ia->ri_max_frmr_depth = min_t(unsigned int, @@ -560,7 +544,7 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr *addr, int memreg) case RPCRDMA_REGISTER: case RPCRDMA_FRMR: break; -#if RPCRDMA_PERSISTENT_REGISTRATION +#ifdef CONFIG_SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL case RPCRDMA_ALLPHYSICAL: mem_priv = IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | @@ -1824,7 +1808,7 @@ rpcrdma_register_external(struct rpcrdma_mr_seg *seg, switch (ia->ri_memreg_strategy) { -#if RPCRDMA_PERSISTENT_REGISTRATION +#ifdef CONFIG_SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL case RPCRDMA_ALLPHYSICAL: rpcrdma_map_one(ia, seg, writing); seg->mr_rkey = ia->ri_bind_mem->rkey; @@ -1870,7 +1854,7 @@ rpcrdma_deregister_external(struct rpcrdma_mr_seg *seg, switch (ia->ri_memreg_strategy) { -#if RPCRDMA_PERSISTENT_REGISTRATION +#ifdef CONFIG_SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL case RPCRDMA_ALLPHYSICAL: BUG_ON(nsegs != 1); rpcrdma_unmap_one(ia, seg);
WARNING: multiple messages have this Message-ID (diff)
From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> To: linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org Subject: [PATCH 3/8] xprtrdma: Disable ALLPHYSICAL mode by default Date: Mon, 14 Apr 2014 18:22:49 -0400 [thread overview] Message-ID: <20140414222249.20646.47882.stgit@manet.1015granger.net> (raw) In-Reply-To: <20140414220041.20646.63991.stgit-FYjufvaPoItvLzlybtyyYzGyq/o6K9yX@public.gmane.org> ALLPHYSICAL is not a safe memory registration mode because it permits NFS servers to write anywhere in a client's memory. NFS server bugs could result in client memory being overwritten. This can be useful for embedded systems which do not support more surgical RDMA memory registration and protection methods. However, enterprise Linux distributions have expressed a desire to disable or remove it entirely. As a compromise (or as a step towards deprecation), add a CONFIG setting (by default N) to enable ALLPHYSICAL support. ALLPHYSICAL is now never a fallback choice when HCAs do not support the selected memory registration mode. Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> --- include/linux/sunrpc/xprtrdma.h | 3 --- net/sunrpc/Kconfig | 15 +++++++++++++++ net/sunrpc/xprtrdma/verbs.c | 22 +++------------------- 3 files changed, 18 insertions(+), 22 deletions(-) diff --git a/include/linux/sunrpc/xprtrdma.h b/include/linux/sunrpc/xprtrdma.h index c2f04e1..54e9b47 100644 --- a/include/linux/sunrpc/xprtrdma.h +++ b/include/linux/sunrpc/xprtrdma.h @@ -61,9 +61,6 @@ #define RPCRDMA_INLINE_PAD_THRESH (512)/* payload threshold to pad (bytes) */ -/* memory registration strategies */ -#define RPCRDMA_PERSISTENT_REGISTRATION (1) - enum rpcrdma_memreg { RPCRDMA_BOUNCEBUFFERS = 0, RPCRDMA_REGISTER, diff --git a/net/sunrpc/Kconfig b/net/sunrpc/Kconfig index 0754d0f..af56554 100644 --- a/net/sunrpc/Kconfig +++ b/net/sunrpc/Kconfig @@ -58,6 +58,21 @@ config SUNRPC_XPRT_RDMA_CLIENT If unsure, say N. +config SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL + bool "Enable ALLPHYSICAL memory registration mode" + depends on SUNRPC_XPRT_RDMA_CLIENT + default N + help + This option enables support for the ALLPHYSICAL memory + registration mode. + + This mode is very fast but not safe because it registers + and exposes all of local memory. This could allow an + NFS server bug to corrupt client memory. + + N is almost always the correct choice unless you are + sure what you are doing. + config SUNRPC_XPRT_RDMA_SERVER tristate "RPC over RDMA Server Support" depends on SUNRPC && INFINIBAND && INFINIBAND_ADDR_TRANS diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c index 1fe554f..6373232 100644 --- a/net/sunrpc/xprtrdma/verbs.c +++ b/net/sunrpc/xprtrdma/verbs.c @@ -506,19 +506,11 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr *addr, int memreg) break; case RPCRDMA_MTHCAFMR: if (!ia->ri_id->device->alloc_fmr) { -#if RPCRDMA_PERSISTENT_REGISTRATION - dprintk("RPC: %s: MTHCAFMR registration " - "specified but not supported by adapter, " - "using riskier RPCRDMA_ALLPHYSICAL\n", - __func__); - memreg = RPCRDMA_ALLPHYSICAL; -#else dprintk("RPC: %s: MTHCAFMR registration " "specified but not supported by adapter, " "using slower RPCRDMA_REGISTER\n", __func__); memreg = RPCRDMA_REGISTER; -#endif } break; case RPCRDMA_FRMR: @@ -526,19 +518,11 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr *addr, int memreg) if ((devattr.device_cap_flags & (IB_DEVICE_MEM_MGT_EXTENSIONS|IB_DEVICE_LOCAL_DMA_LKEY)) != (IB_DEVICE_MEM_MGT_EXTENSIONS|IB_DEVICE_LOCAL_DMA_LKEY)) { -#if RPCRDMA_PERSISTENT_REGISTRATION - dprintk("RPC: %s: FRMR registration " - "specified but not supported by adapter, " - "using riskier RPCRDMA_ALLPHYSICAL\n", - __func__); - memreg = RPCRDMA_ALLPHYSICAL; -#else dprintk("RPC: %s: FRMR registration " "specified but not supported by adapter, " "using slower RPCRDMA_REGISTER\n", __func__); memreg = RPCRDMA_REGISTER; -#endif } else { /* Mind the ia limit on FRMR page list depth */ ia->ri_max_frmr_depth = min_t(unsigned int, @@ -560,7 +544,7 @@ rpcrdma_ia_open(struct rpcrdma_xprt *xprt, struct sockaddr *addr, int memreg) case RPCRDMA_REGISTER: case RPCRDMA_FRMR: break; -#if RPCRDMA_PERSISTENT_REGISTRATION +#ifdef CONFIG_SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL case RPCRDMA_ALLPHYSICAL: mem_priv = IB_ACCESS_LOCAL_WRITE | IB_ACCESS_REMOTE_WRITE | @@ -1824,7 +1808,7 @@ rpcrdma_register_external(struct rpcrdma_mr_seg *seg, switch (ia->ri_memreg_strategy) { -#if RPCRDMA_PERSISTENT_REGISTRATION +#ifdef CONFIG_SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL case RPCRDMA_ALLPHYSICAL: rpcrdma_map_one(ia, seg, writing); seg->mr_rkey = ia->ri_bind_mem->rkey; @@ -1870,7 +1854,7 @@ rpcrdma_deregister_external(struct rpcrdma_mr_seg *seg, switch (ia->ri_memreg_strategy) { -#if RPCRDMA_PERSISTENT_REGISTRATION +#ifdef CONFIG_SUNRPC_XPRT_RDMA_CLIENT_ALLPHYSICAL case RPCRDMA_ALLPHYSICAL: BUG_ON(nsegs != 1); rpcrdma_unmap_one(ia, seg); -- To unsubscribe from this list: send the line "unsubscribe linux-nfs" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html
next prev parent reply other threads:[~2014-04-14 22:22 UTC|newest] Thread overview: 58+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-04-14 22:22 [PATCH 0/8] NFS/RDMA patches for review Chuck Lever 2014-04-14 22:22 ` Chuck Lever 2014-04-14 22:22 ` [PATCH 1/8] xprtrdma: RPC/RDMA must invoke xprt_wake_pending_tasks() in process context Chuck Lever 2014-04-14 22:22 ` Chuck Lever 2014-04-14 22:22 ` [PATCH 2/8] xprtrdma: Remove BOUNCEBUFFERS memory registration mode Chuck Lever 2014-04-14 22:22 ` Chuck Lever 2014-04-14 22:22 ` Chuck Lever [this message] 2014-04-14 22:22 ` [PATCH 3/8] xprtrdma: Disable ALLPHYSICAL mode by default Chuck Lever 2014-04-14 22:22 ` [PATCH 4/8] xprtrdma: Remove support for MEMWINDOWS registration mode Chuck Lever 2014-04-14 22:22 ` Chuck Lever 2014-04-14 22:23 ` [PATCH 5/8] xprtrdma: Simplify rpcrdma_deregister_external() synopsis Chuck Lever 2014-04-14 22:23 ` Chuck Lever 2014-04-14 22:23 ` [PATCH 6/8] xprtrdma: Make rpcrdma_ep_destroy() return void Chuck Lever 2014-04-14 22:23 ` Chuck Lever 2014-04-14 22:23 ` [PATCH 7/8] xprtrdma: Split the completion queue Chuck Lever 2014-04-14 22:23 ` Chuck Lever 2014-04-16 12:48 ` Sagi Grimberg 2014-04-16 12:48 ` Sagi Grimberg 2014-04-16 13:30 ` Steve Wise 2014-04-16 13:30 ` Steve Wise 2014-04-16 14:12 ` Sagi Grimberg 2014-04-16 14:12 ` Sagi Grimberg 2014-04-16 14:25 ` Steve Wise 2014-04-16 14:25 ` Steve Wise 2014-04-16 14:35 ` Sagi Grimberg 2014-04-16 14:35 ` Sagi Grimberg 2014-04-16 14:43 ` Steve Wise 2014-04-16 14:43 ` Steve Wise 2014-04-16 15:18 ` Sagi Grimberg 2014-04-16 15:18 ` Sagi Grimberg 2014-04-16 15:46 ` Steve Wise 2014-04-16 15:46 ` Steve Wise 2014-04-16 15:08 ` Chuck Lever 2014-04-16 15:08 ` Chuck Lever 2014-04-16 15:23 ` Sagi Grimberg 2014-04-16 15:23 ` Sagi Grimberg 2014-04-16 18:21 ` Chuck Lever 2014-04-16 18:21 ` Chuck Lever 2014-04-17 7:06 ` Sagi Grimberg 2014-04-17 7:06 ` Sagi Grimberg 2014-04-17 13:55 ` Chuck Lever 2014-04-17 13:55 ` Chuck Lever 2014-04-17 14:34 ` Steve Wise 2014-04-17 14:34 ` Steve Wise 2014-04-17 19:11 ` Sagi Grimberg 2014-04-17 19:11 ` Sagi Grimberg 2014-04-19 16:31 ` Chuck Lever 2014-04-19 16:31 ` Chuck Lever 2014-04-20 12:42 ` Sagi Grimberg 2014-04-20 12:42 ` Sagi Grimberg 2014-04-17 19:08 ` Sagi Grimberg 2014-04-17 19:08 ` Sagi Grimberg 2014-04-14 22:23 ` [PATCH 8/8] xprtrdma: Reduce the number of hardway buffer allocations Chuck Lever 2014-04-14 22:23 ` Chuck Lever 2014-04-15 20:15 ` [PATCH 0/8] NFS/RDMA patches for review Steve Wise 2014-04-15 20:15 ` Steve Wise 2014-04-15 20:20 ` Steve Wise 2014-04-15 20:20 ` Steve Wise
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=20140414222249.20646.47882.stgit@manet.1015granger.net \ --to=chuck.lever@oracle.com \ --cc=linux-nfs@vger.kernel.org \ --cc=linux-rdma@vger.kernel.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.