[PATCH 1/3] RDMA/rxe: Remove useless parameters for update_state()

[PATCH 1/3] RDMA/rxe: Remove useless parameters for update_state()

[Li Zhijian]

wqe was not used by update_state() so far.

aaaf62e06623 ("RDMA/rxe: Remove useless argument for update_state()")
just did a partial fixes.

Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
---
 drivers/infiniband/sw/rxe/rxe_req.c | 5 ++---
 1 file changed, 2 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index 5f7348b11268..bf7493bab9b9 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -525,8 +525,7 @@ static void rollback_state(struct rxe_send_wqe *wqe,
 	qp->req.psn    = rollback_psn;
 }
 
-static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
-			 struct rxe_pkt_info *pkt)
+static void update_state(struct rxe_qp *qp, struct rxe_pkt_info *pkt)
 {
 	qp->req.opcode = pkt->opcode;
 
@@ -753,7 +752,7 @@ int rxe_requester(void *arg)
 		goto err;
 	}
 
-	update_state(qp, wqe, &pkt);
+	update_state(qp, &pkt);
 
 	goto next_wqe;
 
-- 
2.31.1

[PATCH 2/3] RDMA/rxe: Update wqe_index for each wqe error completion

[Li Zhijian]

Previously, if user space keep sending abnormal wqe, qeueu.prod will
keep increasing while queue.index doesn't. Once
queue.index==qeueu.prod in next round, req_next_wqe() will treat queue
is empty. In such case, no new completion would be generated.

Update wqe_index for each wqe completion so that
req_next_wqe() can get next wqe properly.

Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
---
 drivers/infiniband/sw/rxe/rxe_req.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index bf7493bab9b9..c369bebaf02e 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -760,6 +760,8 @@ int rxe_requester(void *arg)
 	if (ah)
 		rxe_put(ah);
 err:
+	/* update wqe_index for each wqe completion */
+	qp->req.wqe_index = queue_next_index(qp->sq.queue, qp->req.wqe_index);
 	wqe->state = wqe_state_error;
 	__rxe_do_task(&qp->comp.task);
 
-- 
2.31.1

[PATCH 3/3] RDMA/rxe: Generate error completion for error requester state

[Li Zhijian]

SoftRoCE always returns success when user space is posting a new wqe where
it usually just euqueues a wqe.

Once requester state becomes QP_STATE_ERROR, we should generate error
completion for all subsequent wqe. So user is able to poll the completion
event to check if the former wqe is handled correctly.

Here we check QP_STATE_ERROR after req_next_wqe() so that the completion
can associate with its wqe.

Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
---
 drivers/infiniband/sw/rxe/rxe_req.c | 10 +++++++++-
 1 file changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
index c369bebaf02e..9b35f6528d73 100644
--- a/drivers/infiniband/sw/rxe/rxe_req.c
+++ b/drivers/infiniband/sw/rxe/rxe_req.c
@@ -611,7 +611,7 @@ int rxe_requester(void *arg)
 	rxe_get(qp);
 
 next_wqe:
-	if (unlikely(!qp->valid || qp->req.state == QP_STATE_ERROR))
+	if (unlikely(!qp->valid))
 		goto exit;
 
 	if (unlikely(qp->req.state == QP_STATE_RESET)) {
@@ -633,6 +633,14 @@ int rxe_requester(void *arg)
 	if (unlikely(!wqe))
 		goto exit;
 
+	if (qp->req.state == QP_STATE_ERROR) {
+		/*
+		 * Generate an error completion so that user space is able to
+		 * poll this completion.
+		 */
+		goto err;
+	}
+
 	if (wqe->mask & WR_LOCAL_OP_MASK) {
 		ret = rxe_do_local_ops(qp, wqe);
 		if (unlikely(ret))
-- 
2.31.1

Re: [PATCH 1/3] RDMA/rxe: Remove useless parameters for update_state()

[lizhijian]

ping


On 12/04/2022 10:29, Li Zhijian wrote:
> wqe was not used by update_state() so far.
>
> aaaf62e06623 ("RDMA/rxe: Remove useless argument for update_state()")
> just did a partial fixes.
>
> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
> ---
>   drivers/infiniband/sw/rxe/rxe_req.c | 5 ++---
>   1 file changed, 2 insertions(+), 3 deletions(-)
>
> diff --git a/drivers/infiniband/sw/rxe/rxe_req.c b/drivers/infiniband/sw/rxe/rxe_req.c
> index 5f7348b11268..bf7493bab9b9 100644
> --- a/drivers/infiniband/sw/rxe/rxe_req.c
> +++ b/drivers/infiniband/sw/rxe/rxe_req.c
> @@ -525,8 +525,7 @@ static void rollback_state(struct rxe_send_wqe *wqe,
>   	qp->req.psn    = rollback_psn;
>   }
>   
> -static void update_state(struct rxe_qp *qp, struct rxe_send_wqe *wqe,
> -			 struct rxe_pkt_info *pkt)
> +static void update_state(struct rxe_qp *qp, struct rxe_pkt_info *pkt)
>   {
>   	qp->req.opcode = pkt->opcode;
>   
> @@ -753,7 +752,7 @@ int rxe_requester(void *arg)
>   		goto err;
>   	}
>   
> -	update_state(qp, wqe, &pkt);
> +	update_state(qp, &pkt);
>   
>   	goto next_wqe;
>   

Re: [PATCH 1/3] RDMA/rxe: Remove useless parameters for update_state()

[Jason Gunthorpe]

On Fri, Apr 22, 2022 at 01:07:46AM +0000, lizhijian@fujitsu.com wrote:
> ping

I'd like to see some of the people testing rxe look over these, at
least until we get rxe back to being stable

Jason

Re: [PATCH 1/3] RDMA/rxe: Remove useless parameters for update_state()

[Jason Gunthorpe]

On Tue, Apr 12, 2022 at 10:29:01AM +0800, Li Zhijian wrote:
> wqe was not used by update_state() so far.
> 
> aaaf62e06623 ("RDMA/rxe: Remove useless argument for update_state()")
> just did a partial fixes.
> 
> Signed-off-by: Li Zhijian <lizhijian@fujitsu.com>
> ---
>  drivers/infiniband/sw/rxe/rxe_req.c | 5 ++---
>  1 file changed, 2 insertions(+), 3 deletions(-)

I applied just this patch to for-next

Thanks,
Jason