* [PATCH for-rc 1/5] IB/hfi1: Drop stale TID RDMA packets
2019-08-15 19:20 [PATCH for-rc 0/5] Fixes TID packet ordering issues Mike Marciniszyn
@ 2019-08-15 19:20 ` Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 2/5] IB/hfi1: Unsafe PSN checking for TID RDMA READ Resp packet Mike Marciniszyn
` (4 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Mike Marciniszyn @ 2019-08-15 19:20 UTC (permalink / raw)
To: jgg, dledford; +Cc: linux-rdma
From: Kaike Wan <kaike.wan@intel.com>
In a congested fabric with adaptive routing enabled, traces show that
the sender could receive stale TID RDMA NAK packets that contain newer
KDETH PSNs and older Verbs PSNs. If not dropped, these packets could
cause the incorrect rewinding of the software flows and the incorrect
completion of TID RDMA WRITE requests, and eventually leading to memory
corruption and kernel crash.
The current code drops stale TID RDMA ACK/NAK packets solely based
on KDETH PSNs, which may lead to erroneous processing. This patch
fixes the issue by also checking the Verbs PSN. Addition checks are
added before rewinding the TID RDMA WRITE DATA packets.
Fixes: 9e93e967f7b4 ("IB/hfi1: Add a function to receive TID RDMA ACK packet")
Cc: <stable@vger.kernel.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
drivers/infiniband/hw/hfi1/tid_rdma.c | 13 +++++++++++--
1 file changed, 11 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c
index 996fc2982..9407014 100644
--- a/drivers/infiniband/hw/hfi1/tid_rdma.c
+++ b/drivers/infiniband/hw/hfi1/tid_rdma.c
@@ -4509,7 +4509,7 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
struct rvt_swqe *wqe;
struct tid_rdma_request *req;
struct tid_rdma_flow *flow;
- u32 aeth, psn, req_psn, ack_psn, resync_psn, ack_kpsn;
+ u32 aeth, psn, req_psn, ack_psn, flpsn, resync_psn, ack_kpsn;
unsigned long flags;
u16 fidx;
@@ -4538,6 +4538,9 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
ack_kpsn--;
}
+ if (unlikely(qp->s_acked == qp->s_tail))
+ goto ack_op_err;
+
wqe = rvt_get_swqe_ptr(qp, qp->s_acked);
if (wqe->wr.opcode != IB_WR_TID_RDMA_WRITE)
@@ -4550,7 +4553,8 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
trace_hfi1_tid_flow_rcv_tid_ack(qp, req->acked_tail, flow);
/* Drop stale ACK/NAK */
- if (cmp_psn(psn, full_flow_psn(flow, flow->flow_state.spsn)) < 0)
+ if (cmp_psn(psn, full_flow_psn(flow, flow->flow_state.spsn)) < 0 ||
+ cmp_psn(req_psn, flow->flow_state.resp_ib_psn) < 0)
goto ack_op_err;
while (cmp_psn(ack_kpsn,
@@ -4712,7 +4716,12 @@ void hfi1_rc_rcv_tid_rdma_ack(struct hfi1_packet *packet)
switch ((aeth >> IB_AETH_CREDIT_SHIFT) &
IB_AETH_CREDIT_MASK) {
case 0: /* PSN sequence error */
+ if (!req->flows)
+ break;
flow = &req->flows[req->acked_tail];
+ flpsn = full_flow_psn(flow, flow->flow_state.lpsn);
+ if (cmp_psn(psn, flpsn) > 0)
+ break;
trace_hfi1_tid_flow_rcv_tid_ack(qp, req->acked_tail,
flow);
req->r_ack_psn = mask_psn(be32_to_cpu(ohdr->bth[2]));
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH for-rc 2/5] IB/hfi1: Unsafe PSN checking for TID RDMA READ Resp packet
2019-08-15 19:20 [PATCH for-rc 0/5] Fixes TID packet ordering issues Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 1/5] IB/hfi1: Drop stale TID RDMA packets Mike Marciniszyn
@ 2019-08-15 19:20 ` Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 3/5] IB/hfi1: Add additional checks when handling TID RDMA READ RESP packet Mike Marciniszyn
` (3 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Mike Marciniszyn @ 2019-08-15 19:20 UTC (permalink / raw)
To: jgg, dledford; +Cc: linux-rdma
From: Kaike Wan <kaike.wan@intel.com>
When processing a TID RDMA READ RESP packet that causes KDETH EFLAGS
errors, the packet's IB PSN is checked against qp->s_last_psn and
qp->s_psn without the protection of qp->s_lock, which is not safe.
This patch fixes the issue by acquiring qp->s_lock first.
Fixes: 9905bf06e890 ("IB/hfi1: Add functions to receive TID RDMA READ response")
Cc: <stable@vger.kernel.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
drivers/infiniband/hw/hfi1/tid_rdma.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c
index 9407014..01c8b02 100644
--- a/drivers/infiniband/hw/hfi1/tid_rdma.c
+++ b/drivers/infiniband/hw/hfi1/tid_rdma.c
@@ -2687,12 +2687,12 @@ static bool handle_read_kdeth_eflags(struct hfi1_ctxtdata *rcd,
u32 fpsn;
lockdep_assert_held(&qp->r_lock);
+ spin_lock(&qp->s_lock);
/* If the psn is out of valid range, drop the packet */
if (cmp_psn(ibpsn, qp->s_last_psn) < 0 ||
cmp_psn(ibpsn, qp->s_psn) > 0)
- return ret;
+ goto s_unlock;
- spin_lock(&qp->s_lock);
/*
* Note that NAKs implicitly ACK outstanding SEND and RDMA write
* requests and implicitly NAK RDMA read and atomic requests issued
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH for-rc 3/5] IB/hfi1: Add additional checks when handling TID RDMA READ RESP packet
2019-08-15 19:20 [PATCH for-rc 0/5] Fixes TID packet ordering issues Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 1/5] IB/hfi1: Drop stale TID RDMA packets Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 2/5] IB/hfi1: Unsafe PSN checking for TID RDMA READ Resp packet Mike Marciniszyn
@ 2019-08-15 19:20 ` Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 4/5] IB/hfi1: Add additional checks when handling TID RDMA WRITE DATA packet Mike Marciniszyn
` (2 subsequent siblings)
5 siblings, 0 replies; 7+ messages in thread
From: Mike Marciniszyn @ 2019-08-15 19:20 UTC (permalink / raw)
To: jgg, dledford; +Cc: linux-rdma
From: Kaike Wan <kaike.wan@intel.com>
In a congested fabric with adaptive routing enabled, traces show that
packets could be delivered out of order, which could cause incorrect
processing of stale packets. For stale TID RDMA READ RESP packets that
cause KDETH EFLAGS errors, this patch adds additional checks before
processing the packets.
Fixes: 9905bf06e890 ("IB/hfi1: Add functions to receive TID RDMA READ response")
Cc: <stable@vger.kernel.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
drivers/infiniband/hw/hfi1/tid_rdma.c | 5 ++++-
1 file changed, 4 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c
index 01c8b02..23bb249 100644
--- a/drivers/infiniband/hw/hfi1/tid_rdma.c
+++ b/drivers/infiniband/hw/hfi1/tid_rdma.c
@@ -2740,9 +2740,12 @@ static bool handle_read_kdeth_eflags(struct hfi1_ctxtdata *rcd,
wqe = do_rc_completion(qp, wqe, ibp);
if (qp->s_acked == qp->s_tail)
- break;
+ goto s_unlock;
}
+ if (qp->s_acked == qp->s_tail)
+ goto s_unlock;
+
/* Handle the eflags for the request */
if (wqe->wr.opcode != IB_WR_TID_RDMA_READ)
goto s_unlock;
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH for-rc 4/5] IB/hfi1: Add additional checks when handling TID RDMA WRITE DATA packet
2019-08-15 19:20 [PATCH for-rc 0/5] Fixes TID packet ordering issues Mike Marciniszyn
` (2 preceding siblings ...)
2019-08-15 19:20 ` [PATCH for-rc 3/5] IB/hfi1: Add additional checks when handling TID RDMA READ RESP packet Mike Marciniszyn
@ 2019-08-15 19:20 ` Mike Marciniszyn
2019-08-15 19:20 ` [PATCH for-rc 5/5] IB/hfi1: Drop stale TID RDMA packets that cause TIDErr Mike Marciniszyn
2019-08-20 16:41 ` [PATCH for-rc 0/5] Fixes TID packet ordering issues Doug Ledford
5 siblings, 0 replies; 7+ messages in thread
From: Mike Marciniszyn @ 2019-08-15 19:20 UTC (permalink / raw)
To: jgg, dledford; +Cc: linux-rdma
From: Kaike Wan <kaike.wan@intel.com>
In a congested fabric with adaptive routing enabled, traces show that
packets could be delivered out of order, which could cause incorrect
processing of stale packets. For stale TID RDMA WRITE DATA packets that
cause KDETH EFLAGS errors, this patch adds additional checks before
processing the packets.
Fixes: d72fe7d5008b ("IB/hfi1: Add a function to receive TID RDMA WRITE DATA packet")
Cc: <stable@vger.kernel.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
drivers/infiniband/hw/hfi1/tid_rdma.c | 7 +++++++
1 file changed, 7 insertions(+)
diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c
index 23bb249..7bccb59 100644
--- a/drivers/infiniband/hw/hfi1/tid_rdma.c
+++ b/drivers/infiniband/hw/hfi1/tid_rdma.c
@@ -2945,8 +2945,15 @@ bool hfi1_handle_kdeth_eflags(struct hfi1_ctxtdata *rcd,
*/
spin_lock(&qp->s_lock);
qpriv = qp->priv;
+ if (qpriv->r_tid_tail == HFI1_QP_WQE_INVALID ||
+ qpriv->r_tid_tail == qpriv->r_tid_head)
+ goto unlock;
e = &qp->s_ack_queue[qpriv->r_tid_tail];
+ if (e->opcode != TID_OP(WRITE_REQ))
+ goto unlock;
req = ack_to_tid_req(e);
+ if (req->comp_seg == req->cur_seg)
+ goto unlock;
flow = &req->flows[req->clear_tail];
trace_hfi1_eflags_err_write(qp, rcv_type, rte, psn);
trace_hfi1_rsp_handle_kdeth_eflags(qp, psn);
^ permalink raw reply related [flat|nested] 7+ messages in thread* [PATCH for-rc 5/5] IB/hfi1: Drop stale TID RDMA packets that cause TIDErr
2019-08-15 19:20 [PATCH for-rc 0/5] Fixes TID packet ordering issues Mike Marciniszyn
` (3 preceding siblings ...)
2019-08-15 19:20 ` [PATCH for-rc 4/5] IB/hfi1: Add additional checks when handling TID RDMA WRITE DATA packet Mike Marciniszyn
@ 2019-08-15 19:20 ` Mike Marciniszyn
2019-08-20 16:41 ` [PATCH for-rc 0/5] Fixes TID packet ordering issues Doug Ledford
5 siblings, 0 replies; 7+ messages in thread
From: Mike Marciniszyn @ 2019-08-15 19:20 UTC (permalink / raw)
To: jgg, dledford; +Cc: linux-rdma
From: Kaike Wan <kaike.wan@intel.com>
In a congested fabric with adaptive routing enabled, traces show that
packets could be delivered out of order. A stale TID RDMA data packet
could lead to TidErr if the TID entries have been released by duplicate
data packets generated from retries, and subsequently erroneously force
the qp into error state in the current implementation.
Since the payload has already been dropped by hardware, the packet can
be simply dropped and it is no longer necessary to put the qp into
error state.
Fixes: 9905bf06e890 ("IB/hfi1: Add functions to receive TID RDMA READ response")
Cc: <stable@vger.kernel.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn@intel.com>
Signed-off-by: Kaike Wan <kaike.wan@intel.com>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro@intel.com>
---
drivers/infiniband/hw/hfi1/tid_rdma.c | 47 ++-------------------------------
1 file changed, 3 insertions(+), 44 deletions(-)
diff --git a/drivers/infiniband/hw/hfi1/tid_rdma.c b/drivers/infiniband/hw/hfi1/tid_rdma.c
index 7bccb59..6141f4e 100644
--- a/drivers/infiniband/hw/hfi1/tid_rdma.c
+++ b/drivers/infiniband/hw/hfi1/tid_rdma.c
@@ -2574,18 +2574,9 @@ void hfi1_kern_read_tid_flow_free(struct rvt_qp *qp)
hfi1_kern_clear_hw_flow(priv->rcd, qp);
}
-static bool tid_rdma_tid_err(struct hfi1_ctxtdata *rcd,
- struct hfi1_packet *packet, u8 rcv_type,
- u8 opcode)
+static bool tid_rdma_tid_err(struct hfi1_packet *packet, u8 rcv_type)
{
struct rvt_qp *qp = packet->qp;
- struct hfi1_qp_priv *qpriv = qp->priv;
- u32 ipsn;
- struct ib_other_headers *ohdr = packet->ohdr;
- struct rvt_ack_entry *e;
- struct tid_rdma_request *req;
- struct rvt_dev_info *rdi = ib_to_rvt(qp->ibqp.device);
- u32 i;
if (rcv_type >= RHF_RCV_TYPE_IB)
goto done;
@@ -2602,41 +2593,9 @@ static bool tid_rdma_tid_err(struct hfi1_ctxtdata *rcd,
if (rcv_type == RHF_RCV_TYPE_EAGER) {
hfi1_restart_rc(qp, qp->s_last_psn + 1, 1);
hfi1_schedule_send(qp);
- goto done_unlock;
}
- /*
- * For TID READ response, error out QP after freeing the tid
- * resources.
- */
- if (opcode == TID_OP(READ_RESP)) {
- ipsn = mask_psn(be32_to_cpu(ohdr->u.tid_rdma.r_rsp.verbs_psn));
- if (cmp_psn(ipsn, qp->s_last_psn) > 0 &&
- cmp_psn(ipsn, qp->s_psn) < 0) {
- hfi1_kern_read_tid_flow_free(qp);
- spin_unlock(&qp->s_lock);
- rvt_rc_error(qp, IB_WC_LOC_QP_OP_ERR);
- goto done;
- }
- goto done_unlock;
- }
-
- /*
- * Error out the qp for TID RDMA WRITE
- */
- hfi1_kern_clear_hw_flow(qpriv->rcd, qp);
- for (i = 0; i < rvt_max_atomic(rdi); i++) {
- e = &qp->s_ack_queue[i];
- if (e->opcode == TID_OP(WRITE_REQ)) {
- req = ack_to_tid_req(e);
- hfi1_kern_exp_rcv_clear_all(req);
- }
- }
- spin_unlock(&qp->s_lock);
- rvt_rc_error(qp, IB_WC_LOC_LEN_ERR);
- goto done;
-
-done_unlock:
+ /* Since no payload is delivered, just drop the packet */
spin_unlock(&qp->s_lock);
done:
return true;
@@ -2925,7 +2884,7 @@ bool hfi1_handle_kdeth_eflags(struct hfi1_ctxtdata *rcd,
if (lnh == HFI1_LRH_GRH)
goto r_unlock;
- if (tid_rdma_tid_err(rcd, packet, rcv_type, opcode))
+ if (tid_rdma_tid_err(packet, rcv_type))
goto r_unlock;
}
^ permalink raw reply related [flat|nested] 7+ messages in thread* Re: [PATCH for-rc 0/5] Fixes TID packet ordering issues
2019-08-15 19:20 [PATCH for-rc 0/5] Fixes TID packet ordering issues Mike Marciniszyn
` (4 preceding siblings ...)
2019-08-15 19:20 ` [PATCH for-rc 5/5] IB/hfi1: Drop stale TID RDMA packets that cause TIDErr Mike Marciniszyn
@ 2019-08-20 16:41 ` Doug Ledford
5 siblings, 0 replies; 7+ messages in thread
From: Doug Ledford @ 2019-08-20 16:41 UTC (permalink / raw)
To: Mike Marciniszyn, jgg; +Cc: linux-rdma
[-- Attachment #1: Type: text/plain, Size: 877 bytes --]
On Thu, 2019-08-15 at 15:20 -0400, Mike Marciniszyn wrote:
> This patch patch series fixes a issues caused
> by packet ordering when adaptive routing is in
> use.
>
> ---
> Kaike Wan (5):
> IB/hfi1: Drop stale TID RDMA packets
> IB/hfi1: Unsafe PSN checking for TID RDMA READ Resp packet
> IB/hfi1: Add additional checks when handling TID RDMA READ RESP
> packet
> IB/hfi1: Add additional checks when handling TID RDMA WRITE DATA
> packet
> IB/hfi1: Drop stale TID RDMA packets that cause TIDErr
>
>
> drivers/infiniband/hw/hfi1/tid_rdma.c | 76 ++++++++++++----------
> -----------
> 1 file changed, 27 insertions(+), 49 deletions(-)
>
Thanks, series applied to for-rc.
--
Doug Ledford <dledford@redhat.com>
GPG KeyID: B826A3330E572FDD
Fingerprint = AE6B 1BDA 122B 23B4 265B 1274 B826 A333 0E57 2FDD
[-- Attachment #2: This is a digitally signed message part --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 7+ messages in thread