From: Bob Pearson <rpearsonhpe@gmail.com>
To: jgg@nvidia.com, leon@kernel.org, zyjzyj2000@gmail.com,
jhack@hpe.com, linux-rdma@vger.kernel.org
Cc: Bob Pearson <rpearsonhpe@gmail.com>, Ian Ziemba <ian.ziemba@hpe.com>
Subject: [PATCH for-next v3 06/13] RDMA/rxe: Remove __rxe_do_task()
Date: Fri, 28 Oct 2022 22:10:03 -0500 [thread overview]
Message-ID: <20221029031009.64467-7-rpearsonhpe@gmail.com> (raw)
In-Reply-To: <20221029031009.64467-1-rpearsonhpe@gmail.com>
The subroutine __rxe_do_task is not thread safe. It is only
used in the rxe_qp_reset() and rxe_qp_do_cleanup() routines.
After changes in error handling in the tasklet functions the
queues can be drained by calling them once outside of the
tasklet code. This allows __rxe_do_task() to be removed.
Signed-off-by: Ian Ziemba <ian.ziemba@hpe.com>
Signed-off-by: Bob Pearson <rpearsonhpe@gmail.com>
---
drivers/infiniband/sw/rxe/rxe_qp.c | 60 ++++++++++++++--------------
drivers/infiniband/sw/rxe/rxe_task.c | 13 ------
drivers/infiniband/sw/rxe/rxe_task.h | 6 ---
3 files changed, 29 insertions(+), 50 deletions(-)
diff --git a/drivers/infiniband/sw/rxe/rxe_qp.c b/drivers/infiniband/sw/rxe/rxe_qp.c
index 3691eb97c576..50f6b8b8ad9d 100644
--- a/drivers/infiniband/sw/rxe/rxe_qp.c
+++ b/drivers/infiniband/sw/rxe/rxe_qp.c
@@ -477,28 +477,23 @@ static void rxe_qp_reset(struct rxe_qp *qp)
{
/* stop tasks from running */
rxe_disable_task(&qp->resp.task);
-
- /* stop request/comp */
- if (qp->sq.queue) {
- rxe_disable_task(&qp->comp.task);
- rxe_disable_task(&qp->req.task);
- }
+ rxe_disable_task(&qp->comp.task);
+ rxe_disable_task(&qp->req.task);
/* move qp to the reset state */
qp->req.state = QP_STATE_RESET;
qp->comp.state = QP_STATE_RESET;
qp->resp.state = QP_STATE_RESET;
- /* let state machines reset themselves drain work and packet queues
- * etc.
- */
- __rxe_do_task(&qp->resp.task);
+ /* drain work and packet queues */
+ rxe_responder(qp);
+ rxe_completer(qp);
+ rxe_requester(qp);
- if (qp->sq.queue) {
- __rxe_do_task(&qp->comp.task);
- __rxe_do_task(&qp->req.task);
+ if (qp->rq.queue)
+ rxe_queue_reset(qp->rq.queue);
+ if (qp->sq.queue)
rxe_queue_reset(qp->sq.queue);
- }
/* cleanup attributes */
atomic_set(&qp->ssn, 0);
@@ -521,11 +516,8 @@ static void rxe_qp_reset(struct rxe_qp *qp)
/* reenable tasks */
rxe_enable_task(&qp->resp.task);
-
- if (qp->sq.queue) {
- rxe_enable_task(&qp->comp.task);
- rxe_enable_task(&qp->req.task);
- }
+ rxe_enable_task(&qp->comp.task);
+ rxe_enable_task(&qp->req.task);
}
/* drain the send queue */
@@ -543,15 +535,25 @@ static void rxe_qp_drain(struct rxe_qp *qp)
/* move the qp to the error state */
void rxe_qp_error(struct rxe_qp *qp)
{
+ /* stop tasks from running */
+ rxe_disable_task(&qp->resp.task);
+ rxe_disable_task(&qp->comp.task);
+ rxe_disable_task(&qp->req.task);
+
qp->req.state = QP_STATE_ERROR;
qp->resp.state = QP_STATE_ERROR;
qp->comp.state = QP_STATE_ERROR;
qp->attr.qp_state = IB_QPS_ERR;
/* drain work and packet queues */
- rxe_sched_task(&qp->resp.task);
- rxe_sched_task(&qp->comp.task);
- rxe_sched_task(&qp->req.task);
+ rxe_responder(qp);
+ rxe_completer(qp);
+ rxe_requester(qp);
+
+ /* reenable tasks */
+ rxe_enable_task(&qp->resp.task);
+ rxe_enable_task(&qp->comp.task);
+ rxe_enable_task(&qp->req.task);
}
/* called by the modify qp verb */
@@ -770,24 +772,20 @@ static void rxe_qp_do_cleanup(struct work_struct *work)
qp->valid = 0;
qp->qp_timeout_jiffies = 0;
- rxe_cleanup_task(&qp->resp.task);
if (qp_type(qp) == IB_QPT_RC) {
del_timer_sync(&qp->retrans_timer);
del_timer_sync(&qp->rnr_nak_timer);
}
+ rxe_cleanup_task(&qp->resp.task);
rxe_cleanup_task(&qp->req.task);
rxe_cleanup_task(&qp->comp.task);
- /* flush out any receive wr's or pending requests */
- if (qp->req.task.func)
- __rxe_do_task(&qp->req.task);
-
- if (qp->sq.queue) {
- __rxe_do_task(&qp->comp.task);
- __rxe_do_task(&qp->req.task);
- }
+ /* drain any receive wr's or pending requests */
+ rxe_responder(qp);
+ rxe_completer(qp);
+ rxe_requester(qp);
if (qp->sq.queue)
rxe_queue_cleanup(qp->sq.queue);
diff --git a/drivers/infiniband/sw/rxe/rxe_task.c b/drivers/infiniband/sw/rxe/rxe_task.c
index 8dfbfa164eff..120693c9a795 100644
--- a/drivers/infiniband/sw/rxe/rxe_task.c
+++ b/drivers/infiniband/sw/rxe/rxe_task.c
@@ -6,19 +6,6 @@
#include "rxe.h"
-int __rxe_do_task(struct rxe_task *task)
-
-{
- int ret;
-
- while ((ret = task->func(task->arg)) == 0)
- ;
-
- task->ret = ret;
-
- return ret;
-}
-
/*
* this locking is due to a potential race where
* a second caller finds the task already running
diff --git a/drivers/infiniband/sw/rxe/rxe_task.h b/drivers/infiniband/sw/rxe/rxe_task.h
index 31963129ff7a..d594468fcf56 100644
--- a/drivers/infiniband/sw/rxe/rxe_task.h
+++ b/drivers/infiniband/sw/rxe/rxe_task.h
@@ -48,12 +48,6 @@ struct rxe_task {
int rxe_init_task(struct rxe_task *task, void *arg, int (*func)(void *),
enum rxe_task_type type);
-/*
- * raw call to func in loop without any checking
- * can call when tasklets are disabled
- */
-int __rxe_do_task(struct rxe_task *task);
-
void rxe_run_task(struct rxe_task *task);
void rxe_sched_task(struct rxe_task *task);
--
2.34.1
next prev parent reply other threads:[~2022-10-29 3:10 UTC|newest]
Thread overview: 23+ messages / expand[flat|nested] mbox.gz Atom feed top
2022-10-29 3:09 [PATCH for-next v3 00/13] Implement work queues for rdma_rxe Bob Pearson
2022-10-29 3:09 ` [PATCH for-next v3 01/13] RDMA/rxe: Make task interface pluggable Bob Pearson
2022-11-11 2:28 ` Yanjun Zhu
2022-10-29 3:09 ` [PATCH for-next v3 02/13] RDMA/rxe: Split rxe_drain_resp_pkts() Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 03/13] RDMA/rxe: Simplify reset state handling in rxe_resp.c Bob Pearson
2022-11-11 3:04 ` Yanjun Zhu
2022-10-29 3:10 ` [PATCH for-next v3 04/13] RDMA/rxe: Handle qp error " Bob Pearson
2022-10-29 3:10 ` [PATCH v3 05/13] RDMA/rxe: Cleanup comp tasks in rxe_qp.c Bob Pearson
2022-10-29 3:10 ` Bob Pearson [this message]
2022-10-29 3:10 ` [PATCH for-next v3 07/13] RDMA/rxe: Make tasks schedule each other Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 08/13] RDMA/rxe: Implement disable/enable_task() Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 09/13] RDMA/rxe: Replace TASK_STATE_START by TASK_STATE_IDLE Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 10/13] RDMA/rxe: Replace task->destroyed by task state INVALID Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 11/13] RDMA/rxe: Add workqueue support for tasks Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 12/13] RDMA/rxe: Make WORKQUEUE default for RC tasks Bob Pearson
2022-10-29 3:10 ` [PATCH for-next v3 13/13] RDMA/rxe: Remove tasklets from rxe_task.c Bob Pearson
2022-11-02 10:17 ` [PATCH for-next v3 00/13] Implement work queues for rdma_rxe Daisuke Matsuda (Fujitsu)
2022-11-02 11:20 ` Bob Pearson
2022-11-04 4:59 ` Daisuke Matsuda (Fujitsu)
2022-11-05 21:15 ` Bob Pearson
2022-11-07 8:21 ` Daisuke Matsuda (Fujitsu)
2022-11-07 16:06 ` Bob Pearson
2022-11-18 5:02 ` Daisuke Matsuda (Fujitsu)
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20221029031009.64467-7-rpearsonhpe@gmail.com \
--to=rpearsonhpe@gmail.com \
--cc=ian.ziemba@hpe.com \
--cc=jgg@nvidia.com \
--cc=jhack@hpe.com \
--cc=leon@kernel.org \
--cc=linux-rdma@vger.kernel.org \
--cc=zyjzyj2000@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).