linux-rdma.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
To: jgg@nvidia.com, dledford@redhat.com, leon@kernel.org
Cc: linux-rdma@vger.kernel.org,
	Tatyana Nikolova <tatyana.e.nikolova@intel.com>
Subject: [PATCH rdma-core] irdma: Remove optimization algorithm for QP doorbell
Date: Mon, 20 Sep 2021 16:37:23 -0500	[thread overview]
Message-ID: <20210920213723.1279-1-tatyana.e.nikolova@intel.com> (raw)

Remove optimization algorithm for QP doorbell, because
without an mfence the algorithm incorrectly skips ringing
the doorbell. This causes applicaitons like OpenMPI with
high number of connections to stall waiting for completion.

Enforcing the order of the write of the WQE valid bit
and the read of the SQ tail is required by the algorithm,
but furher investigation is necessary because this does not
appear sufficient for the algorithm to work. In the meantime,
remove the doorbell optimization and fix the MPI failures.

Signed-off-by: Tatyana Nikolova <tatyana.e.nikolova@intel.com>
---
 providers/irdma/uk.c | 29 ++---------------------------
 1 file changed, 2 insertions(+), 27 deletions(-)

diff --git a/providers/irdma/uk.c b/providers/irdma/uk.c
index c7053c52..098b8682 100644
--- a/providers/irdma/uk.c
+++ b/providers/irdma/uk.c
@@ -113,35 +113,10 @@ void irdma_clr_wqes(struct irdma_qp_uk *qp, __u32 qp_wqe_idx)
  */
 void irdma_uk_qp_post_wr(struct irdma_qp_uk *qp)
 {
-	__u64 temp;
-	__u32 hw_sq_tail;
-	__u32 sw_sq_head;
-
-	/* valid bit is written and loads completed before reading shadow */
+	/* valid bit is written before ringing doorbell */
 	udma_to_device_barrier();
 
-	/* read the doorbell shadow area */
-	get_64bit_val(qp->shadow_area, 0, &temp);
-
-	hw_sq_tail = (__u32)FIELD_GET(IRDMA_QP_DBSA_HW_SQ_TAIL, temp);
-	sw_sq_head = IRDMA_RING_CURRENT_HEAD(qp->sq_ring);
-	if (sw_sq_head != qp->initial_ring.head) {
-		if (qp->push_dropped) {
-			db_wr32(qp->qp_id, qp->wqe_alloc_db);
-			qp->push_dropped = false;
-		} else if (sw_sq_head != hw_sq_tail) {
-			if (sw_sq_head > qp->initial_ring.head) {
-				if (hw_sq_tail >= qp->initial_ring.head &&
-				    hw_sq_tail < sw_sq_head)
-					db_wr32(qp->qp_id, qp->wqe_alloc_db);
-			} else {
-				if (hw_sq_tail >= qp->initial_ring.head ||
-				    hw_sq_tail < sw_sq_head)
-					db_wr32(qp->qp_id, qp->wqe_alloc_db);
-			}
-		}
-	}
-
+	db_wr32(qp->qp_id, qp->wqe_alloc_db);
 	qp->initial_ring.head = qp->sq_ring.head;
 }
 
-- 
2.27.0


             reply	other threads:[~2021-09-20 21:44 UTC|newest]

Thread overview: 2+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2021-09-20 21:37 Tatyana Nikolova [this message]
2021-09-22  8:40 ` Leon Romanovsky

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20210920213723.1279-1-tatyana.e.nikolova@intel.com \
    --to=tatyana.e.nikolova@intel.com \
    --cc=dledford@redhat.com \
    --cc=jgg@nvidia.com \
    --cc=leon@kernel.org \
    --cc=linux-rdma@vger.kernel.org \
    --subject='Re: [PATCH rdma-core] irdma: Remove optimization algorithm for QP doorbell' \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).