All of lore.kernel.org
 help / color / mirror / Atom feed
From: Chuck Lever <cel@kernel.org>
To: linux-nfs@vger.kernel.org, linux-rdma@vger.kernel.org
Subject: [PATCH v2 07/12] svcrdma: Fix retry loop in svc_rdma_send()
Date: Sun, 04 Feb 2024 18:17:15 -0500	[thread overview]
Message-ID: <170708863598.28128.10828477777645028041.stgit@bazille.1015granger.net> (raw)
In-Reply-To: <170708844422.28128.2979813721958631192.stgit@bazille.1015granger.net>

From: Chuck Lever <chuck.lever@oracle.com>

Don't call ib_post_send() at all if the transport is already
shutting down.

Signed-off-by: Chuck Lever <chuck.lever@oracle.com>
---
 net/sunrpc/xprtrdma/svc_rdma_sendto.c |   28 +++++++++++++++++-----------
 1 file changed, 17 insertions(+), 11 deletions(-)

diff --git a/net/sunrpc/xprtrdma/svc_rdma_sendto.c b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
index b6fc9299b472..0ee9185f5f3f 100644
--- a/net/sunrpc/xprtrdma/svc_rdma_sendto.c
+++ b/net/sunrpc/xprtrdma/svc_rdma_sendto.c
@@ -320,8 +320,9 @@ static void svc_rdma_wc_send(struct ib_cq *cq, struct ib_wc *wc)
  * that these values remain available after the ib_post_send() call.
  * In some error flow cases, svc_rdma_wc_send() releases @ctxt.
  *
- * Returns zero if the Send WR was posted successfully. Otherwise, a
- * negative errno is returned.
+ * Return values:
+ *   %0: @ctxt's WR chain was posted successfully
+ *   %-ENOTCONN: The connection was lost
  */
 int svc_rdma_send(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt)
 {
@@ -338,30 +339,35 @@ int svc_rdma_send(struct svcxprt_rdma *rdma, struct svc_rdma_send_ctxt *ctxt)
 				      DMA_TO_DEVICE);
 
 	/* If the SQ is full, wait until an SQ entry is available */
-	while (1) {
+	while (!test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags)) {
 		if ((atomic_dec_return(&rdma->sc_sq_avail) < 0)) {
 			svc_rdma_wake_send_waiters(rdma, 1);
+
+			/* When the transport is torn down, assume
+			 * ib_drain_sq() will trigger enough Send
+			 * completions to wake us. The XPT_CLOSE test
+			 * above should then cause the while loop to
+			 * exit.
+			 */
 			percpu_counter_inc(&svcrdma_stat_sq_starve);
 			trace_svcrdma_sq_full(rdma, &cid);
 			wait_event(rdma->sc_send_wait,
 				   atomic_read(&rdma->sc_sq_avail) > 0);
-			if (test_bit(XPT_CLOSE, &rdma->sc_xprt.xpt_flags))
-				return -ENOTCONN;
 			trace_svcrdma_sq_retry(rdma, &cid);
 			continue;
 		}
 
 		trace_svcrdma_post_send(ctxt);
 		ret = ib_post_send(rdma->sc_qp, wr, NULL);
-		if (ret)
+		if (ret) {
+			trace_svcrdma_sq_post_err(rdma, &cid, ret);
+			svc_xprt_deferred_close(&rdma->sc_xprt);
+			svc_rdma_wake_send_waiters(rdma, 1);
 			break;
+		}
 		return 0;
 	}
-
-	trace_svcrdma_sq_post_err(rdma, &cid, ret);
-	svc_xprt_deferred_close(&rdma->sc_xprt);
-	svc_rdma_wake_send_waiters(rdma, 1);
-	return ret;
+	return -ENOTCONN;
 }
 
 /**



  parent reply	other threads:[~2024-02-04 23:17 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2024-02-04 23:16 [PATCH v2 00/12] NFSD RDMA transport improvements Chuck Lever
2024-02-04 23:16 ` [PATCH v2 01/12] svcrdma: Reserve an extra WQE for ib_drain_rq() Chuck Lever
2024-02-04 23:16 ` [PATCH v2 02/12] svcrdma: Report CQ depths in debugging output Chuck Lever
2024-02-04 23:16 ` [PATCH v2 03/12] svcrdma: Update max_send_sges after QP is created Chuck Lever
2024-02-04 23:16 ` [PATCH v2 04/12] svcrdma: Increase the per-transport rw_ctx count Chuck Lever
2024-02-04 23:17 ` [PATCH v2 05/12] svcrdma: Fix SQ wake-ups Chuck Lever
2024-02-04 23:17 ` [PATCH v2 06/12] svcrdma: Prevent a UAF in svc_rdma_send() Chuck Lever
2024-02-04 23:17 ` Chuck Lever [this message]
2024-02-04 23:17 ` [PATCH v2 08/12] svcrdma: Post Send WR chain Chuck Lever
2024-02-04 23:17 ` [PATCH v2 09/12] svcrdma: Move write_info for Reply chunks into struct svc_rdma_send_ctxt Chuck Lever
2024-02-04 23:17 ` [PATCH v2 10/12] svcrdma: Post the Reply chunk and Send WR together Chuck Lever
2024-02-04 23:17 ` [PATCH v2 11/12] svcrdma: Post WRs for Write chunks in svc_rdma_sendto() Chuck Lever
2024-02-04 23:17 ` [PATCH v2 12/12] svcrdma: Add Write chunk WRs to the RPC's Send WR chain Chuck Lever

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=170708863598.28128.10828477777645028041.stgit@bazille.1015granger.net \
    --to=cel@kernel.org \
    --cc=linux-nfs@vger.kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.