From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> To: Devesh Sharma <devesh.sharma-laKkSmNT4hbQT0dZR+AlfA@public.gmane.org> Cc: Linux NFS Mailing List <linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>, linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org, Trond Myklebust <trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org> Subject: Re: [PATCH V1] NFS-RDMA: fix qp pointer validation checks Date: Wed, 9 Apr 2014 16:26:34 -0400 [thread overview] Message-ID: <F1C70AD6-BDD4-4534-8DC4-61D2767581D9@oracle.com> (raw) In-Reply-To: <D7AB2150-5F25-4BA2-80D9-94890AD11F8F-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org> On Apr 9, 2014, at 4:22 PM, Trond Myklebust <trond.myklebust@primarydata.com> wrote: > Hi Devesh, > > This looks a lot better. I still have a couple of small suggestions, though. > > On Apr 9, 2014, at 14:40, Devesh Sharma <devesh.sharma-laKkSmNT4hbQT0dZR+AlfA@public.gmane.org> wrote: > >> If the rdma_create_qp fails to create qp due to device firmware being in invalid state >> xprtrdma still tries to destroy the non-existant qp and ends up in a NULL pointer reference >> crash. >> Adding proper checks for vaidating QP pointer avoids this to happen. >> >> Signed-off-by: Devesh Sharma <devesh.sharma-laKkSmNT4hbQT0dZR+AlfA@public.gmane.org> >> --- >> net/sunrpc/xprtrdma/verbs.c | 29 +++++++++++++++++++++++++---- >> 1 files changed, 25 insertions(+), 4 deletions(-) >> >> diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c >> index 9372656..902ac78 100644 >> --- a/net/sunrpc/xprtrdma/verbs.c >> +++ b/net/sunrpc/xprtrdma/verbs.c >> @@ -831,10 +831,12 @@ rpcrdma_ep_connect(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia) >> if (ep->rep_connected != 0) { >> struct rpcrdma_xprt *xprt; >> retry: >> - rc = rpcrdma_ep_disconnect(ep, ia); >> - if (rc && rc != -ENOTCONN) >> - dprintk("RPC: %s: rpcrdma_ep_disconnect" >> + if (ia->ri_id->qp) { >> + rc = rpcrdma_ep_disconnect(ep, ia); >> + if (rc && rc != -ENOTCONN) >> + dprintk("RPC: %s: rpcrdma_ep_disconnect" >> " status %i\n", __func__, rc); >> + } >> rpcrdma_clean_cq(ep->rep_cq); >> >> xprt = container_of(ia, struct rpcrdma_xprt, rx_ia); >> @@ -859,7 +861,9 @@ retry: >> goto out; >> } >> /* END TEMP */ >> - rdma_destroy_qp(ia->ri_id); >> + if (ia->ri_id->qp) { >> + rdma_destroy_qp(ia->ri_id); >> + } > > Nit: No need for braces here. > >> rdma_destroy_id(ia->ri_id); >> ia->ri_id = id; >> } >> @@ -1557,6 +1561,13 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, >> frmr_wr.wr.fast_reg.rkey = seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey; >> DECR_CQCOUNT(&r_xprt->rx_ep); I don’t think you can DECR_CQCOUNT, then exit without posting the send. That will screw up the completion counter and result in a transport hang, won’t it? >> >> + if (!ia->ri_is->qp) { >> + rc = -EINVAL; >> + while (i--) >> + rpcrdma_unmap_one(ia, --seg); >> + goto out; >> + } > > Instead of duplicating the rpcrdma_unmap_one() cleanup here, why not just do > > if (ia->ri_is->qp) > rc = ib_post_send(…) > else > rc = -EINVAL; > > BTW: can we not simply test for ia->ri_is->qp before we even call rpcrdma_map_one() and hence bail out before we have to do any cleanup? > >> + >> rc = ib_post_send(ia->ri_id->qp, post_wr, &bad_wr); >> >> if (rc) { >> @@ -1571,6 +1582,7 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, >> seg1->mr_len = len; >> } >> *nsegs = i; >> +out: >> return rc; >> } >> >> @@ -1592,6 +1604,9 @@ rpcrdma_deregister_frmr_external(struct rpcrdma_mr_seg *seg, >> invalidate_wr.ex.invalidate_rkey = seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey; >> DECR_CQCOUNT(&r_xprt->rx_ep); Ditto. >> >> + if (!ia->ri_id->qp) >> + return -EINVAL; >> + >> rc = ib_post_send(ia->ri_id->qp, &invalidate_wr, &bad_wr); >> if (rc) >> dprintk("RPC: %s: failed ib_post_send for invalidate," >> @@ -1923,6 +1938,9 @@ rpcrdma_ep_post(struct rpcrdma_ia *ia, >> send_wr.send_flags = IB_SEND_SIGNALED; >> } Ditto. >> >> + if (!ia->ri_id->qp) >> + return -EINVAL; >> + >> rc = ib_post_send(ia->ri_id->qp, &send_wr, &send_wr_fail); >> if (rc) >> dprintk("RPC: %s: ib_post_send returned %i\n", __func__, >> @@ -1951,6 +1969,9 @@ rpcrdma_ep_post_recv(struct rpcrdma_ia *ia, >> rep->rr_iov.addr, rep->rr_iov.length, DMA_BIDIRECTIONAL); >> >> DECR_CQCOUNT(ep); And here. >> + >> + if (!ia->ri_id->qp) >> + return -EINVAL; >> rc = ib_post_recv(ia->ri_id->qp, &recv_wr, &recv_wr_fail); >> >> if (rc) >> -- >> 1.7.1 >> > > _________________________________ > Trond Myklebust > Linux NFS client maintainer, PrimaryData > trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org > > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Chuck Lever chuck[dot]lever[at]oracle[dot]com -- To unsubscribe from this list: send the line "unsubscribe linux-rdma" in the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org More majordomo info at http://vger.kernel.org/majordomo-info.html
WARNING: multiple messages have this Message-ID (diff)
From: Chuck Lever <chuck.lever@oracle.com> To: Devesh Sharma <devesh.sharma@emulex.com> Cc: Linux NFS Mailing List <linux-nfs@vger.kernel.org>, linux-rdma@vger.kernel.org, Trond Myklebust <trond.myklebust@primarydata.com> Subject: Re: [PATCH V1] NFS-RDMA: fix qp pointer validation checks Date: Wed, 9 Apr 2014 16:26:34 -0400 [thread overview] Message-ID: <F1C70AD6-BDD4-4534-8DC4-61D2767581D9@oracle.com> (raw) In-Reply-To: <D7AB2150-5F25-4BA2-80D9-94890AD11F8F@primarydata.com> On Apr 9, 2014, at 4:22 PM, Trond Myklebust <trond.myklebust@primarydata.com> wrote: > Hi Devesh, > > This looks a lot better. I still have a couple of small suggestions, though. > > On Apr 9, 2014, at 14:40, Devesh Sharma <devesh.sharma@emulex.com> wrote: > >> If the rdma_create_qp fails to create qp due to device firmware being in invalid state >> xprtrdma still tries to destroy the non-existant qp and ends up in a NULL pointer reference >> crash. >> Adding proper checks for vaidating QP pointer avoids this to happen. >> >> Signed-off-by: Devesh Sharma <devesh.sharma@emulex.com> >> --- >> net/sunrpc/xprtrdma/verbs.c | 29 +++++++++++++++++++++++++---- >> 1 files changed, 25 insertions(+), 4 deletions(-) >> >> diff --git a/net/sunrpc/xprtrdma/verbs.c b/net/sunrpc/xprtrdma/verbs.c >> index 9372656..902ac78 100644 >> --- a/net/sunrpc/xprtrdma/verbs.c >> +++ b/net/sunrpc/xprtrdma/verbs.c >> @@ -831,10 +831,12 @@ rpcrdma_ep_connect(struct rpcrdma_ep *ep, struct rpcrdma_ia *ia) >> if (ep->rep_connected != 0) { >> struct rpcrdma_xprt *xprt; >> retry: >> - rc = rpcrdma_ep_disconnect(ep, ia); >> - if (rc && rc != -ENOTCONN) >> - dprintk("RPC: %s: rpcrdma_ep_disconnect" >> + if (ia->ri_id->qp) { >> + rc = rpcrdma_ep_disconnect(ep, ia); >> + if (rc && rc != -ENOTCONN) >> + dprintk("RPC: %s: rpcrdma_ep_disconnect" >> " status %i\n", __func__, rc); >> + } >> rpcrdma_clean_cq(ep->rep_cq); >> >> xprt = container_of(ia, struct rpcrdma_xprt, rx_ia); >> @@ -859,7 +861,9 @@ retry: >> goto out; >> } >> /* END TEMP */ >> - rdma_destroy_qp(ia->ri_id); >> + if (ia->ri_id->qp) { >> + rdma_destroy_qp(ia->ri_id); >> + } > > Nit: No need for braces here. > >> rdma_destroy_id(ia->ri_id); >> ia->ri_id = id; >> } >> @@ -1557,6 +1561,13 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, >> frmr_wr.wr.fast_reg.rkey = seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey; >> DECR_CQCOUNT(&r_xprt->rx_ep); I don’t think you can DECR_CQCOUNT, then exit without posting the send. That will screw up the completion counter and result in a transport hang, won’t it? >> >> + if (!ia->ri_is->qp) { >> + rc = -EINVAL; >> + while (i--) >> + rpcrdma_unmap_one(ia, --seg); >> + goto out; >> + } > > Instead of duplicating the rpcrdma_unmap_one() cleanup here, why not just do > > if (ia->ri_is->qp) > rc = ib_post_send(…) > else > rc = -EINVAL; > > BTW: can we not simply test for ia->ri_is->qp before we even call rpcrdma_map_one() and hence bail out before we have to do any cleanup? > >> + >> rc = ib_post_send(ia->ri_id->qp, post_wr, &bad_wr); >> >> if (rc) { >> @@ -1571,6 +1582,7 @@ rpcrdma_register_frmr_external(struct rpcrdma_mr_seg *seg, >> seg1->mr_len = len; >> } >> *nsegs = i; >> +out: >> return rc; >> } >> >> @@ -1592,6 +1604,9 @@ rpcrdma_deregister_frmr_external(struct rpcrdma_mr_seg *seg, >> invalidate_wr.ex.invalidate_rkey = seg1->mr_chunk.rl_mw->r.frmr.fr_mr->rkey; >> DECR_CQCOUNT(&r_xprt->rx_ep); Ditto. >> >> + if (!ia->ri_id->qp) >> + return -EINVAL; >> + >> rc = ib_post_send(ia->ri_id->qp, &invalidate_wr, &bad_wr); >> if (rc) >> dprintk("RPC: %s: failed ib_post_send for invalidate," >> @@ -1923,6 +1938,9 @@ rpcrdma_ep_post(struct rpcrdma_ia *ia, >> send_wr.send_flags = IB_SEND_SIGNALED; >> } Ditto. >> >> + if (!ia->ri_id->qp) >> + return -EINVAL; >> + >> rc = ib_post_send(ia->ri_id->qp, &send_wr, &send_wr_fail); >> if (rc) >> dprintk("RPC: %s: ib_post_send returned %i\n", __func__, >> @@ -1951,6 +1969,9 @@ rpcrdma_ep_post_recv(struct rpcrdma_ia *ia, >> rep->rr_iov.addr, rep->rr_iov.length, DMA_BIDIRECTIONAL); >> >> DECR_CQCOUNT(ep); And here. >> + >> + if (!ia->ri_id->qp) >> + return -EINVAL; >> rc = ib_post_recv(ia->ri_id->qp, &recv_wr, &recv_wr_fail); >> >> if (rc) >> -- >> 1.7.1 >> > > _________________________________ > Trond Myklebust > Linux NFS client maintainer, PrimaryData > trond.myklebust@primarydata.com > > -- > To unsubscribe from this list: send the line "unsubscribe linux-nfs" in > the body of a message to majordomo@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html -- Chuck Lever chuck[dot]lever[at]oracle[dot]com
next prev parent reply other threads:[~2014-04-09 20:26 UTC|newest] Thread overview: 54+ messages / expand[flat|nested] mbox.gz Atom feed top 2014-04-09 18:40 [PATCH V1] NFS-RDMA: fix qp pointer validation checks Devesh Sharma 2014-04-09 18:40 ` Devesh Sharma [not found] ` <014738b6-698e-4ea1-82f9-287378bfec19-3RiH6ntJJkOPfaB/Gd0HpljyZtpTMMwT@public.gmane.org> 2014-04-09 20:22 ` Trond Myklebust 2014-04-09 20:22 ` Trond Myklebust [not found] ` <D7AB2150-5F25-4BA2-80D9-94890AD11F8F-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org> 2014-04-09 20:26 ` Chuck Lever [this message] 2014-04-09 20:26 ` Chuck Lever [not found] ` <F1C70AD6-BDD4-4534-8DC4-61D2767581D9-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-09 23:56 ` Devesh Sharma 2014-04-09 23:56 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDEAA43-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-10 0:26 ` Chuck Lever 2014-04-10 0:26 ` Chuck Lever [not found] ` <E66D006A-0D04-4602-8BF5-6834CACD2E24-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-10 15:01 ` Steve Wise 2014-04-10 15:01 ` Steve Wise [not found] ` <5346B22D.3060706-7bPotxP6k4+P2YhJcF5u+vpXobYPEAuW@public.gmane.org> 2014-04-10 17:43 ` Chuck Lever 2014-04-10 17:43 ` Chuck Lever [not found] ` <D7836AB3-FCB6-40EF-9954-B58A05A87791-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-10 18:34 ` Steve Wise 2014-04-10 18:34 ` Steve Wise 2014-04-10 17:42 ` Devesh Sharma 2014-04-10 17:42 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDEB3B4-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-10 17:51 ` Chuck Lever 2014-04-10 17:51 ` Chuck Lever [not found] ` <BD7B05C0-4733-4DD1-83F3-B30B6B0EE48C-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-10 17:54 ` Devesh Sharma 2014-04-10 17:54 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDEB3DF-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-10 19:53 ` Chuck Lever 2014-04-10 19:53 ` Chuck Lever [not found] ` <56C87770-7940-4006-948C-FEF3C0EC4ACC-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-11 23:51 ` Devesh Sharma 2014-04-11 23:51 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDEBD66-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-13 4:01 ` Chuck Lever 2014-04-13 4:01 ` Chuck Lever [not found] ` <5710A71F-C4D5-408B-9B41-07F21B5853F0-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-14 20:53 ` Chuck Lever 2014-04-14 20:53 ` Chuck Lever [not found] ` <6837A427-B677-4CC7-A022-4FB9E52A3FC6-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-14 22:46 ` Devesh Sharma 2014-04-14 22:46 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDED915-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-15 0:39 ` Chuck Lever 2014-04-15 0:39 ` Chuck Lever [not found] ` <C689AB91-46F6-4E96-A673-0DE76FE54CC4-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-15 18:25 ` Devesh Sharma 2014-04-15 18:25 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDEE11F-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-23 23:30 ` Devesh Sharma 2014-04-23 23:30 ` Devesh Sharma [not found] ` <1bab6615-60c4-4865-a6a0-c53bb1c32341-3RiH6ntJJkP8BX6JNMqfyFjyZtpTMMwT@public.gmane.org> 2014-04-24 7:12 ` Sagi Grimberg 2014-04-24 7:12 ` Sagi Grimberg [not found] ` <5358B975.4020207-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2014-04-24 15:01 ` Chuck Lever 2014-04-24 15:01 ` Chuck Lever [not found] ` <B39C0B38-357F-4BDA-BDA7-048BD38853F7-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-24 15:48 ` Devesh Sharma 2014-04-24 15:48 ` Devesh Sharma [not found] ` <EE7902D3F51F404C82415C4803930ACD3FDF4F83-DWYeeINJQrxExQ8dmkPuX0M9+F4ksjoh@public.gmane.org> 2014-04-24 17:44 ` Chuck Lever 2014-04-24 17:44 ` Chuck Lever 2014-04-27 10:12 ` Sagi Grimberg 2014-04-27 10:12 ` Sagi Grimberg [not found] ` <535CD819.3050508@dev! .mellanox.co.il> [not found] ` <535CD819.3050508-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org> 2014-04-27 12:37 ` Chuck Lever 2014-04-27 12:37 ` Chuck Lever [not found] ` <4ACED3B0-CC8B-4F1F-8DB6-6C272AB17C99-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org> 2014-04-28 8:58 ` Sagi Grimberg 2014-04-28 8:58 ` Sagi Grimberg 2014-04-14 23:55 ` Devesh Sharma 2014-04-14 23:55 ` Devesh Sharma
Reply instructions: You may reply publicly to this message via plain-text email using any one of the following methods: * Save the following mbox file, import it into your mail client, and reply-to-all from there: mbox Avoid top-posting and favor interleaved quoting: https://en.wikipedia.org/wiki/Posting_style#Interleaved_style * Reply using the --to, --cc, and --in-reply-to switches of git-send-email(1): git send-email \ --in-reply-to=F1C70AD6-BDD4-4534-8DC4-61D2767581D9@oracle.com \ --to=chuck.lever-qhclzuegtsvqt0dzr+alfa@public.gmane.org \ --cc=devesh.sharma-laKkSmNT4hbQT0dZR+AlfA@public.gmane.org \ --cc=linux-nfs-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org \ --cc=trond.myklebust-7I+n7zu2hftEKMMhf/gKZA@public.gmane.org \ /path/to/YOUR_REPLY https://kernel.org/pub/software/scm/git/docs/git-send-email.html * If your mail client supports setting the In-Reply-To header via mailto: links, try the mailto: linkBe sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.