linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* [PATCH rdma-core] irdma: Remove optimization algorithm for QP doorbell
@ 2021-09-20 21:37 Tatyana Nikolova
  2021-09-22  8:40 ` Leon Romanovsky
  0 siblings, 1 reply; 2+ messages in thread
From: Tatyana Nikolova @ 2021-09-20 21:37 UTC (permalink / raw)
  To: jgg, dledford, leon; +Cc: linux-rdma, Tatyana Nikolova

Remove optimization algorithm for QP doorbell, because
without an mfence the algorithm incorrectly skips ringing
the doorbell. This causes applicaitons like OpenMPI with
high number of connections to stall waiting for completion.

Enforcing the order of the write of the WQE valid bit
and the read of the SQ tail is required by the algorithm,
but furher investigation is necessary because this does not
appear sufficient for the algorithm to work. In the meantime,
remove the doorbell optimization and fix the MPI failures.

Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 providers/irdma/uk.c | 29 ++---------------------------
 1 file changed, 2 insertions(+), 27 deletions(-)

diff --git a/providers/irdma/uk.c b/providers/irdma/uk.c
index c7053c52..098b8682 100644
--- a/providers/irdma/uk.c
+++ b/providers/irdma/uk.c
@@ -113,35 +113,10 @@ void irdma_clr_wqes(struct irdma_qp_uk *qp, __u32 qp_wqe_idx)
  */
 void irdma_uk_qp_post_wr(struct irdma_qp_uk *qp)
 {
-	__u64 temp;
-	__u32 hw_sq_tail;
-	__u32 sw_sq_head;
-
-	/* valid bit is written and loads completed before reading shadow */
+	/* valid bit is written before ringing doorbell */
 	udma_to_device_barrier();
 
-	/* read the doorbell shadow area */
-	get_64bit_val(qp->shadow_area, 0, &temp);
-
-	hw_sq_tail = (__u32)FIELD_GET(IRDMA_QP_DBSA_HW_SQ_TAIL, temp);
-	sw_sq_head = IRDMA_RING_CURRENT_HEAD(qp->sq_ring);
-	if (sw_sq_head != qp->initial_ring.head) {
-		if (qp->push_dropped) {
-			db_wr32(qp->qp_id, qp->wqe_alloc_db);
-			qp->push_dropped = false;
-		} else if (sw_sq_head != hw_sq_tail) {
-			if (sw_sq_head > qp->initial_ring.head) {
-				if (hw_sq_tail >= qp->initial_ring.head &&
-				    hw_sq_tail < sw_sq_head)
-					db_wr32(qp->qp_id, qp->wqe_alloc_db);
-			} else {
-				if (hw_sq_tail >= qp->initial_ring.head ||
-				    hw_sq_tail < sw_sq_head)
-					db_wr32(qp->qp_id, qp->wqe_alloc_db);
-			}
-		}
-	}
-
+	db_wr32(qp->qp_id, qp->wqe_alloc_db);
 	qp->initial_ring.head = qp->sq_ring.head;
 }
 
-- 
2.27.0


^ permalink raw reply	[flat|nested] 2+ messages in thread

* Re: [PATCH rdma-core] irdma: Remove optimization algorithm for QP doorbell
  2021-09-20 21:37 [PATCH rdma-core] irdma: Remove optimization algorithm for QP doorbell Tatyana Nikolova
@ 2021-09-22  8:40 ` Leon Romanovsky
  0 siblings, 0 replies; 2+ messages in thread
From: Leon Romanovsky @ 2021-09-22  8:40 UTC (permalink / raw)
  To: Tatyana Nikolova; +Cc: jgg, dledford, linux-rdma

On Mon, Sep 20, 2021 at 04:37:23PM -0500, Tatyana Nikolova wrote:
> Remove optimization algorithm for QP doorbell, because
> without an mfence the algorithm incorrectly skips ringing
> the doorbell. This causes applicaitons like OpenMPI with
> high number of connections to stall waiting for completion.
> 
> Enforcing the order of the write of the WQE valid bit
> and the read of the SQ tail is required by the algorithm,
> but furher investigation is necessary because this does not
> appear sufficient for the algorithm to work. In the meantime,
> remove the doorbell optimization and fix the MPI failures.
> 
> Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
> ---
>  providers/irdma/uk.c | 29 ++---------------------------
>  1 file changed, 2 insertions(+), 27 deletions(-)
> 

Thanks, applied.

^ permalink raw reply	[flat|nested] 2+ messages in thread

end of thread, other threads:[~2021-09-22  8:40 UTC | newest]

Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-09-20 21:37 [PATCH rdma-core] irdma: Remove optimization algorithm for QP doorbell Tatyana Nikolova
2021-09-22  8:40 ` Leon Romanovsky

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).