[PATCH 00/10] IB/hfi1,rdmavt: Another set of fixes for 4.7

[PATCH 00/10] IB/hfi1,rdmavt: Another set of fixes for 4.7

[Dennis Dalessandro]

Doug, here are a few more fixes that we would like to get into 4.7 if possible.
These apply on top of the series to get the driver out of staging. I can always
respin if needed.

Included here is one patch which has a stable tag for 4.6 since it fixes a hard
lockup. The others are minor bug fixes.

Patches can also be viewed in my repo at:
https://github.com/ddalessa/kernel/tree/for-4.7

---

Dean Luick (2):
      IB/hfi1: Add a retry for the first-time QSFP access
      IB/hfi1: Correct 8051 link parameter settings

Jianxin Xiong (2):
      IB/hfi1,qib: Add ieth to the packet header definitions
      IB/hfi1: Add tracing support for send with invalidate opcode

Jubin John (2):
      IB/hfi1: Fix pio map initialization
      IB/hfi1: Fix credit return threshold adjustment

Mike Marciniszyn (3):
      IB/hfi1: Fix hard lockup due to not using save/restore spin lock
      IB/rdmavt: Max atomic value should be a u8
      IB/rdamvt: Fix rdmavt s_ack_queue sizing

Sebastian Sanchez (1):
      IB/hfi1: Update pkey table properly after link down or FM start


 drivers/infiniband/hw/hfi1/chip.c     |   16 ++++++++++++++--
 drivers/infiniband/hw/hfi1/chip.h     |    6 ++++++
 drivers/infiniband/hw/hfi1/mad.c      |    6 ++++++
 drivers/infiniband/hw/hfi1/pio.c      |   24 ++++++++++++++++++++----
 drivers/infiniband/hw/hfi1/platform.c |    6 ++++--
 drivers/infiniband/hw/hfi1/qp.c       |    5 +++--
 drivers/infiniband/hw/hfi1/qsfp.c     |   16 ++++++++++++++++
 drivers/infiniband/hw/hfi1/trace.c    |    7 +++++++
 drivers/infiniband/hw/hfi1/trace.h    |    2 ++
 drivers/infiniband/hw/hfi1/verbs.c    |    2 ++
 drivers/infiniband/hw/hfi1/verbs.h    |    1 +
 drivers/infiniband/hw/qib/qib_verbs.h |    1 +
 drivers/infiniband/sw/rdmavt/qp.c     |   22 ++++++++++++++++++++--
 include/rdma/rdma_vt.h                |   13 +++++++++++--
 include/rdma/rdmavt_qp.h              |    5 +----
 15 files changed, 114 insertions(+), 18 deletions(-)

-- 
-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 01/10] IB/hfi1, qib: Add ieth to the packet header definitions

[Dennis Dalessandro]

From: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

A new union member "ieth" (Invalidate Extended Transport Header) is
added to the packet header definition in preparation of supporting
the send with invalidate opcode.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/verbs.h    |    1 +
 drivers/infiniband/hw/qib/qib_verbs.h |    1 +
 2 files changed, 2 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/verbs.h b/drivers/infiniband/hw/hfi1/verbs.h
index 3ee2239..4883567 100644
--- a/drivers/infiniband/hw/hfi1/verbs.h
+++ b/drivers/infiniband/hw/hfi1/verbs.h
@@ -152,6 +152,7 @@ union ib_ehdrs {
 	} at;
 	__be32 imm_data;
 	__be32 aeth;
+	__be32 ieth;
 	struct ib_atomic_eth atomic_eth;
 }  __packed;
 
diff --git a/drivers/infiniband/hw/qib/qib_verbs.h b/drivers/infiniband/hw/qib/qib_verbs.h
index 6888f03..4f87815 100644
--- a/drivers/infiniband/hw/qib/qib_verbs.h
+++ b/drivers/infiniband/hw/qib/qib_verbs.h
@@ -159,6 +159,7 @@ struct qib_other_headers {
 		} at;
 		__be32 imm_data;
 		__be32 aeth;
+		__be32 ieth;
 		struct ib_atomic_eth atomic_eth;
 	} u;
 } __packed;

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 02/10] IB/hfi1: Add tracing support for send with invalidate opcode

[Dennis Dalessandro]

From: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

Enable trace generation for packets with the "Send Last with
Invalidate" and "Send Only with Invalidate" opcodes.

Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/trace.c |    7 +++++++
 drivers/infiniband/hw/hfi1/trace.h |    2 ++
 drivers/infiniband/hw/hfi1/verbs.c |    2 ++
 3 files changed, 11 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/trace.c b/drivers/infiniband/hw/hfi1/trace.c
index caddb2a..79b2952 100644
--- a/drivers/infiniband/hw/hfi1/trace.c
+++ b/drivers/infiniband/hw/hfi1/trace.c
@@ -66,6 +66,7 @@ u8 ibhdr_exhdr_len(struct hfi1_ib_header *hdr)
 #define RETH_PRN "reth vaddr 0x%.16llx rkey 0x%.8x dlen 0x%.8x"
 #define AETH_PRN "aeth syn 0x%.2x %s msn 0x%.8x"
 #define DETH_PRN "deth qkey 0x%.8x sqpn 0x%.6x"
+#define IETH_PRN "ieth rkey 0x%.8x"
 #define ATOMICACKETH_PRN "origdata %lld"
 #define ATOMICETH_PRN "vaddr 0x%llx rkey 0x%.8x sdata %lld cdata %lld"
 
@@ -166,6 +167,12 @@ const char *parse_everbs_hdrs(
 				 be32_to_cpu(eh->ud.deth[0]),
 				 be32_to_cpu(eh->ud.deth[1]) & RVT_QPN_MASK);
 		break;
+	/* ieth */
+	case OP(RC, SEND_LAST_WITH_INVALIDATE):
+	case OP(RC, SEND_ONLY_WITH_INVALIDATE):
+		trace_seq_printf(p, IETH_PRN,
+				 be32_to_cpu(eh->ieth));
+		break;
 	}
 	trace_seq_putc(p, 0);
 	return ret;
diff --git a/drivers/infiniband/hw/hfi1/trace.h b/drivers/infiniband/hw/hfi1/trace.h
index a6c1adf..28c1d08 100644
--- a/drivers/infiniband/hw/hfi1/trace.h
+++ b/drivers/infiniband/hw/hfi1/trace.h
@@ -392,6 +392,8 @@ __print_symbolic(opcode,                                   \
 	ib_opcode_name(RC_ATOMIC_ACKNOWLEDGE),             \
 	ib_opcode_name(RC_COMPARE_SWAP),                   \
 	ib_opcode_name(RC_FETCH_ADD),                      \
+	ib_opcode_name(RC_SEND_LAST_WITH_INVALIDATE),      \
+	ib_opcode_name(RC_SEND_ONLY_WITH_INVALIDATE),      \
 	ib_opcode_name(UC_SEND_FIRST),                     \
 	ib_opcode_name(UC_SEND_MIDDLE),                    \
 	ib_opcode_name(UC_SEND_LAST),                      \
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index ae92ac5..849c4b9 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -335,6 +335,8 @@ const u8 hdr_len_by_opcode[256] = {
 	[IB_OPCODE_RC_ATOMIC_ACKNOWLEDGE]             = 12 + 8 + 4,
 	[IB_OPCODE_RC_COMPARE_SWAP]                   = 12 + 8 + 28,
 	[IB_OPCODE_RC_FETCH_ADD]                      = 12 + 8 + 28,
+	[IB_OPCODE_RC_SEND_LAST_WITH_INVALIDATE]      = 12 + 8 + 4,
+	[IB_OPCODE_RC_SEND_ONLY_WITH_INVALIDATE]      = 12 + 8 + 4,
 	/* UC */
 	[IB_OPCODE_UC_SEND_FIRST]                     = 12 + 8,
 	[IB_OPCODE_UC_SEND_MIDDLE]                    = 12 + 8,

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 03/10] IB/hfi1: Fix hard lockup due to not using save/restore spin lock

[Dennis Dalessandro]

From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

Commit b9b06cb6feda
("IB/hfi1: Fix missing lock/unlock in verbs drain callback")
added a spin lock.

Unfortunately, the new lock code can be called from a base
level interrupt state, and an interrupt that can get stacked
will attempt to get the same lock.

Fix by using the flag save/restore spin lock variation.

Cc: stable-u79uwXL29TY76Z2rM5mHXA@public.gmane.org # 4.6+
Reviewed-by: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/qp.c |    5 +++--
 1 files changed, 3 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/qp.c b/drivers/infiniband/hw/hfi1/qp.c
index 14f889e..1a942ff 100644
--- a/drivers/infiniband/hw/hfi1/qp.c
+++ b/drivers/infiniband/hw/hfi1/qp.c
@@ -512,6 +512,7 @@ static void iowait_wakeup(struct iowait *wait, int reason)
 static void iowait_sdma_drained(struct iowait *wait)
 {
 	struct rvt_qp *qp = iowait_to_qp(wait);
+	unsigned long flags;
 
 	/*
 	 * This happens when the send engine notes
@@ -519,12 +520,12 @@ static void iowait_sdma_drained(struct iowait *wait)
 	 * do the flush work until that QP's
 	 * sdma work has finished.
 	 */
-	spin_lock(&qp->s_lock);
+	spin_lock_irqsave(&qp->s_lock, flags);
 	if (qp->s_flags & RVT_S_WAIT_DMA) {
 		qp->s_flags &= ~RVT_S_WAIT_DMA;
 		hfi1_schedule_send(qp);
 	}
-	spin_unlock(&qp->s_lock);
+	spin_unlock_irqrestore(&qp->s_lock, flags);
 }
 
 /**

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 04/10] IB/rdmavt: Max atomic value should be a u8

[Dennis Dalessandro]

From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

This matches the ib_qp_attr size and
avoids a extremely large value when the lower level
driver registers.

As part of the patch, the u8 ordinals are moved to the
end of the struct to reduce pahole noted excesses.

Reviewed-by: Mitko Haralanov <mitko.haralanov-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 include/rdma/rdma_vt.h |    4 ++--
 1 files changed, 2 insertions(+), 2 deletions(-)

diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index d57ceee..3f12da9 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -149,15 +149,15 @@ struct rvt_driver_params {
 	int qpn_res_end;
 	int nports;
 	int npkeys;
-	u8 qos_shift;
 	char cq_name[RVT_CQN_MAX];
 	int node;
-	int max_rdma_atomic;
 	int psn_mask;
 	int psn_shift;
 	int psn_modify_mask;
 	u32 core_cap_flags;
 	u32 max_mad_size;
+	u8 qos_shift;
+	u8 max_rdma_atomic;
 };
 
 /* Protection domain */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 05/10] IB/rdamvt: Fix rdmavt s_ack_queue sizing

[Dennis Dalessandro]

From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

rdmavt allows the driver to specify the size of the ack queue, but
only uses it for the modify QP limit testing for setting the atomic
limit value.

The driver dependent size is now used to size the s_ack_queue ring
dynamicially.

Since the driver knows its size, the driver will use its define
for any ring size dependent code.

Reviewed-by: Mitko Haralanov <mitko.haralanov-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/sw/rdmavt/qp.c |   22 ++++++++++++++++++++--
 include/rdma/rdma_vt.h            |    9 +++++++++
 include/rdma/rdmavt_qp.h          |    5 +----
 3 files changed, 30 insertions(+), 6 deletions(-)

diff --git a/drivers/infiniband/sw/rdmavt/qp.c b/drivers/infiniband/sw/rdmavt/qp.c
index dd03b23..5fa4d4d 100644
--- a/drivers/infiniband/sw/rdmavt/qp.c
+++ b/drivers/infiniband/sw/rdmavt/qp.c
@@ -397,6 +397,7 @@ static void free_qpn(struct rvt_qpn_table *qpt, u32 qpn)
 static void rvt_clear_mr_refs(struct rvt_qp *qp, int clr_sends)
 {
 	unsigned n;
+	struct rvt_dev_info *rdi = ib_to_rvt(qp->ibqp.device);
 
 	if (test_and_clear_bit(RVT_R_REWIND_SGE, &qp->r_aflags))
 		rvt_put_ss(&qp->s_rdma_read_sge);
@@ -431,7 +432,7 @@ static void rvt_clear_mr_refs(struct rvt_qp *qp, int clr_sends)
 	if (qp->ibqp.qp_type != IB_QPT_RC)
 		return;
 
-	for (n = 0; n < ARRAY_SIZE(qp->s_ack_queue); n++) {
+	for (n = 0; n < rvt_max_atomic(rdi); n++) {
 		struct rvt_ack_entry *e = &qp->s_ack_queue[n];
 
 		if (e->opcode == IB_OPCODE_RC_RDMA_READ_REQUEST &&
@@ -569,7 +570,12 @@ static void rvt_reset_qp(struct rvt_dev_info *rdi, struct rvt_qp *qp,
 	qp->s_ssn = 1;
 	qp->s_lsn = 0;
 	qp->s_mig_state = IB_MIG_MIGRATED;
-	memset(qp->s_ack_queue, 0, sizeof(qp->s_ack_queue));
+	if (qp->s_ack_queue)
+		memset(
+			qp->s_ack_queue,
+			0,
+			rvt_max_atomic(rdi) *
+				sizeof(*qp->s_ack_queue));
 	qp->r_head_ack_queue = 0;
 	qp->s_tail_ack_queue = 0;
 	qp->s_num_rd_atomic = 0;
@@ -677,6 +683,16 @@ struct ib_qp *rvt_create_qp(struct ib_pd *ibpd,
 			goto bail_swq;
 
 		RCU_INIT_POINTER(qp->next, NULL);
+		if (init_attr->qp_type == IB_QPT_RC) {
+			qp->s_ack_queue =
+				kzalloc_node(
+					sizeof(*qp->s_ack_queue) *
+					 rvt_max_atomic(rdi),
+					gfp,
+					rdi->dparms.node);
+			if (!qp->s_ack_queue)
+				goto bail_qp;
+		}
 
 		/*
 		 * Driver needs to set up it's private QP structure and do any
@@ -857,6 +873,7 @@ bail_driver_priv:
 	rdi->driver_f.qp_priv_free(rdi, qp);
 
 bail_qp:
+	kfree(qp->s_ack_queue);
 	kfree(qp);
 
 bail_swq:
@@ -1284,6 +1301,7 @@ int rvt_destroy_qp(struct ib_qp *ibqp)
 		vfree(qp->r_rq.wq);
 	vfree(qp->s_wq);
 	rdi->driver_f.qp_priv_free(rdi, qp);
+	kfree(qp->s_ack_queue);
 	kfree(qp);
 	return 0;
 }
diff --git a/include/rdma/rdma_vt.h b/include/rdma/rdma_vt.h
index 3f12da9..16274e2 100644
--- a/include/rdma/rdma_vt.h
+++ b/include/rdma/rdma_vt.h
@@ -426,6 +426,15 @@ static inline unsigned rvt_get_npkeys(struct rvt_dev_info *rdi)
 }
 
 /*
+ * Return the max atomic suitable for determining
+ * the size of the ack ring buffer in a QP.
+ */
+static inline unsigned int rvt_max_atomic(struct rvt_dev_info *rdi)
+{
+	return rdi->dparms.max_rdma_atomic + 1;
+}
+
+/*
  * Return the indexed PKEY from the port PKEY table.
  */
 static inline u16 rvt_get_pkey(struct rvt_dev_info *rdi,
diff --git a/include/rdma/rdmavt_qp.h b/include/rdma/rdmavt_qp.h
index 0e1ff2a..6d23b87 100644
--- a/include/rdma/rdmavt_qp.h
+++ b/include/rdma/rdmavt_qp.h
@@ -211,8 +211,6 @@ struct rvt_mmap_info {
 	unsigned size;
 };
 
-#define RVT_MAX_RDMA_ATOMIC	16
-
 /*
  * This structure holds the information that the send tasklet needs
  * to send a RDMA read response or atomic operation.
@@ -282,8 +280,7 @@ struct rvt_qp {
 	atomic_t refcount ____cacheline_aligned_in_smp;
 	wait_queue_head_t wait;
 
-	struct rvt_ack_entry s_ack_queue[RVT_MAX_RDMA_ATOMIC + 1]
-		____cacheline_aligned_in_smp;
+	struct rvt_ack_entry *s_ack_queue;
 	struct rvt_sge_state s_rdma_read_sge;
 
 	spinlock_t r_lock ____cacheline_aligned_in_smp;      /* used for APM */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 06/10] IB/hfi1: Update pkey table properly after link down or FM start

[Dennis Dalessandro]

From: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

When FM is disabled, and the HFI port on the switch is
changed from MgmtAllowed=YES to MgmtAllowed=NO and the
link is bounced, FULL_MGMT_P_KEY doesn't get cleared
from the pkey table. This also occurs when the QSFP
cable is moved from a switch port with MgmtAllowed=YES
to a MgmtAllowed=NO port. Clear pkey entry properly.

Also, when the driver is loaded and the switch port is
set to MgmtAllowed=NO, FULL_MGMT_P_KEY shouldn't be added
to pkey table after FM is started. Only set FULL_MGMT_P_KEY
in the pkey table if switch port is configured to
MgmtAllowed=YES.

Reviewed-by: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Sebastian Sanchez <sebastian.sanchez-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/chip.c |    9 +++++++++
 drivers/infiniband/hw/hfi1/mad.c  |    6 ++++++
 2 files changed, 15 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index d8cc329..3b876da 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -1037,6 +1037,7 @@ static void dc_shutdown(struct hfi1_devdata *);
 static void dc_start(struct hfi1_devdata *);
 static int qos_rmt_entries(struct hfi1_devdata *dd, unsigned int *mp,
 			   unsigned int *np);
+static void remove_full_mgmt_pkey(struct hfi1_pportdata *ppd);
 
 /*
  * Error interrupt table entry.  This is used as input to the interrupt
@@ -6961,6 +6962,8 @@ void handle_link_down(struct work_struct *work)
 	}
 
 	reset_neighbor_info(ppd);
+	if (ppd->mgmt_allowed)
+		remove_full_mgmt_pkey(ppd);
 
 	/* disable the port */
 	clear_rcvctrl(ppd->dd, RCV_CTRL_RCV_PORT_ENABLE_SMASK);
@@ -7069,6 +7072,12 @@ static void add_full_mgmt_pkey(struct hfi1_pportdata *ppd)
 	(void)hfi1_set_ib_cfg(ppd, HFI1_IB_CFG_PKEYS, 0);
 }
 
+static void remove_full_mgmt_pkey(struct hfi1_pportdata *ppd)
+{
+	ppd->pkeys[2] = 0;
+	(void)hfi1_set_ib_cfg(ppd, HFI1_IB_CFG_PKEYS, 0);
+}
+
 /*
  * Convert the given link width to the OPA link width bitmask.
  */
diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c
index 17882dc..2190295 100644
--- a/drivers/infiniband/hw/hfi1/mad.c
+++ b/drivers/infiniband/hw/hfi1/mad.c
@@ -1403,6 +1403,12 @@ static int set_pkeys(struct hfi1_devdata *dd, u8 port, u16 *pkeys)
 		if (key == okey)
 			continue;
 		/*
+		 * Don't update pkeys[2], if an HFI port without MgmtAllowed
+		 * by neighbor is a switch.
+		 */
+		if (i == 2 && !ppd->mgmt_allowed && ppd->neighbor_type == 1)
+			continue;
+		/*
 		 * The SM gives us the complete PKey table. We have
 		 * to ensure that we put the PKeys in the matching
 		 * slots.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 07/10] IB/hfi1: Add a retry for the first-time QSFP access

[Dennis Dalessandro]

From: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

Some QSFPs do not respond within the expected time, but later
appear fine.  Add a limited retry on the first access.

Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/qsfp.c |   16 ++++++++++++++++
 1 files changed, 16 insertions(+), 0 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/qsfp.c b/drivers/infiniband/hw/hfi1/qsfp.c
index 2441669..aa7ed23 100644
--- a/drivers/infiniband/hw/hfi1/qsfp.c
+++ b/drivers/infiniband/hw/hfi1/qsfp.c
@@ -362,6 +362,7 @@ int refresh_qsfp_cache(struct hfi1_pportdata *ppd, struct qsfp_data *cp)
 {
 	u32 target = ppd->dd->hfi1_id;
 	int ret;
+	int retry_count;
 	unsigned long flags;
 	u8 *cache = &cp->cache[0];
 
@@ -376,8 +377,23 @@ int refresh_qsfp_cache(struct hfi1_pportdata *ppd, struct qsfp_data *cp)
 		goto bail;
 	}
 
+	retry_count = 0;
+retry:
 	ret = qsfp_read(ppd, target, 0, cache, QSFP_PAGESIZE);
 	if (ret != QSFP_PAGESIZE) {
+		/*
+		 * This is the first QSFP access the driver makes.
+		 * Some QSFPs don't respond within the expected time,
+		 * but later appear fine.  Retry at 2s intervals for up
+		 * to 10s.
+		 */
+		if (ret < 0 && retry_count < 5) {
+			retry_count++;
+			dd_dev_info(ppd->dd, "%s: QSFP not responding, waiting and retrying %d\n",
+				    __func__, retry_count);
+			msleep(2000);
+			goto retry;
+		}
 		dd_dev_info(ppd->dd,
 			    "%s: Page 0 read failed, expected %d, got %d\n",
 			    __func__, QSFP_PAGESIZE, ret);

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 08/10] IB/hfi1: Correct 8051 link parameter settings

[Dennis Dalessandro]

From: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

Two 8051 link settings, external device config and tuning method,
were written in the wrong location and the previous settings were
not cleared.  For both, clear the old value and write the new
value.

Fixes: 8ebd4cf1852a ("staging/rdma/hfi1: Add active and optical cable support")
Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/chip.h     |    6 ++++++
 drivers/infiniband/hw/hfi1/platform.c |    6 ++++--
 2 files changed, 10 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/chip.h b/drivers/infiniband/hw/hfi1/chip.h
index 1948706..66a3279 100644
--- a/drivers/infiniband/hw/hfi1/chip.h
+++ b/drivers/infiniband/hw/hfi1/chip.h
@@ -398,6 +398,12 @@
 /* Lane ID for general configuration registers */
 #define GENERAL_CONFIG 4
 
+/* LINK_TUNING_PARAMETERS fields */
+#define TUNING_METHOD_SHIFT 24
+
+/* LINK_OPTIMIZATION_SETTINGS fields */
+#define ENABLE_EXT_DEV_CONFIG_SHIFT 24
+
 /* LOAD_DATA 8051 command shifts and fields */
 #define LOAD_DATA_FIELD_ID_SHIFT 40
 #define LOAD_DATA_FIELD_ID_MASK 0xfull
diff --git a/drivers/infiniband/hw/hfi1/platform.c b/drivers/infiniband/hw/hfi1/platform.c
index ae0e498..03df932 100644
--- a/drivers/infiniband/hw/hfi1/platform.c
+++ b/drivers/infiniband/hw/hfi1/platform.c
@@ -540,7 +540,8 @@ static void apply_tunings(
 	/* Enable external device config if channel is limiting active */
 	read_8051_config(ppd->dd, LINK_OPTIMIZATION_SETTINGS,
 			 GENERAL_CONFIG, &config_data);
-	config_data |= limiting_active;
+	config_data &= ~(0xff << ENABLE_EXT_DEV_CONFIG_SHIFT);
+	config_data |= ((u32)limiting_active << ENABLE_EXT_DEV_CONFIG_SHIFT);
 	ret = load_8051_config(ppd->dd, LINK_OPTIMIZATION_SETTINGS,
 			       GENERAL_CONFIG, config_data);
 	if (ret != HCMD_SUCCESS)
@@ -553,7 +554,8 @@ static void apply_tunings(
 	/* Pass tuning method to 8051 */
 	read_8051_config(ppd->dd, LINK_TUNING_PARAMETERS, GENERAL_CONFIG,
 			 &config_data);
-	config_data |= tuning_method;
+	config_data &= ~(0xff << TUNING_METHOD_SHIFT);
+	config_data |= ((u32)tuning_method << TUNING_METHOD_SHIFT);
 	ret = load_8051_config(ppd->dd, LINK_TUNING_PARAMETERS, GENERAL_CONFIG,
 			       config_data);
 	if (ret != HCMD_SUCCESS)

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 09/10] IB/hfi1: Fix pio map initialization

[Dennis Dalessandro]

From: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

The pio map initialization function is off by 1 causing the last
kernel send context that is allocated to not get mapped into the
pio map which leads to the last kernel send context not being used
by any of the qps.

The send context reserved for VL15 is taken care of by setting the
scontext variable that is used as the index into the kernel send
context array to 1 and does not need to be accounted for in the
kernel send context counting loop as it is currently done.

Fix the kernel send context counting loop to account for all the
allocated send contexts and map all of them to the different VLs.

Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/pio.c |    3 +--
 1 files changed, 1 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
index c67b9ad..d5edb1a 100644
--- a/drivers/infiniband/hw/hfi1/pio.c
+++ b/drivers/infiniband/hw/hfi1/pio.c
@@ -1835,8 +1835,7 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
 	struct pio_vl_map *oldmap, *newmap;
 
 	if (!vl_scontexts) {
-		/* send context 0 reserved for VL15 */
-		for (i = 1; i < dd->num_send_contexts; i++)
+		for (i = 0; i < dd->num_send_contexts; i++)
 			if (dd->send_contexts[i].type == SC_KERNEL)
 				num_kernel_send_contexts++;
 		/* truncate divide */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment

[Dennis Dalessandro]

From: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

The credit return threshold adjustment on mtu change algorithm does not
take into account all the kernel send contexts that are assigned per VL.
Use the pio send context map to adjust the credit return thresholds for
all the allocated and assigned kernel send contexts based on the MTU
adjustment per VL.

The pio send context map can be changed dynamically based on the actual
number of operational vls which is set by the fabric manager. When this
happens update the credit return threshold values for all the remapped
kernel send contexts.

Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/hw/hfi1/chip.c |    7 +++++--
 drivers/infiniband/hw/hfi1/pio.c  |   21 +++++++++++++++++++--
 2 files changed, 24 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
index 3b876da..be4fd3e 100644
--- a/drivers/infiniband/hw/hfi1/chip.c
+++ b/drivers/infiniband/hw/hfi1/chip.c
@@ -9777,7 +9777,7 @@ static void set_send_length(struct hfi1_pportdata *ppd)
 	u64 len1 = 0, len2 = (((dd->vld[15].mtu + max_hb) >> 2)
 			      & SEND_LEN_CHECK1_LEN_VL15_MASK) <<
 		SEND_LEN_CHECK1_LEN_VL15_SHIFT;
-	int i;
+	int i, j;
 	u32 thres;
 
 	for (i = 0; i < ppd->vls_supported; i++) {
@@ -9801,7 +9801,10 @@ static void set_send_length(struct hfi1_pportdata *ppd)
 			    sc_mtu_to_threshold(dd->vld[i].sc,
 						dd->vld[i].mtu,
 						dd->rcd[0]->rcvhdrqentsize));
-		sc_set_cr_threshold(dd->vld[i].sc, thres);
+		for (j = 0; j < INIT_SC_PER_VL; j++)
+			sc_set_cr_threshold(
+					pio_select_send_context_vl(dd, j, i),
+					    thres);
 	}
 	thres = min(sc_percent_to_threshold(dd->vld[15].sc, 50),
 		    sc_mtu_to_threshold(dd->vld[15].sc,
diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
index d5edb1a..d5a9a5c 100644
--- a/drivers/infiniband/hw/hfi1/pio.c
+++ b/drivers/infiniband/hw/hfi1/pio.c
@@ -1832,6 +1832,7 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
 	int scontext = 1;
 	int num_kernel_send_contexts = 0;
 	u8 lvl_scontexts[OPA_MAX_VLS];
+	u32 thres;
 	struct pio_vl_map *oldmap, *newmap;
 
 	if (!vl_scontexts) {
@@ -1872,11 +1873,27 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
 			if (!newmap->map[i])
 				goto bail;
 			newmap->map[i]->mask = (1 << ilog2(sz)) - 1;
-			/* assign send contexts */
+			/*
+			 * assign send contexts and
+			 * adjust credit return threshold
+			 */
 			for (j = 0; j < sz; j++) {
-				if (dd->kernel_send_context[scontext])
+				if (dd->kernel_send_context[scontext]) {
 					newmap->map[i]->ksc[j] =
 					dd->kernel_send_context[scontext];
+					thres = min(sc_percent_to_threshold(
+						    dd->kernel_send_context
+						    [scontext], 50),
+						    sc_mtu_to_threshold(
+						    dd->kernel_send_context
+						    [scontext], dd->vld[i].mtu,
+						    dd->rcd[0]->rcvhdrqentsize)
+						    );
+					sc_set_cr_threshold(
+						dd->
+						kernel_send_context[scontext],
+						thres);
+				}
 				if (++scontext >= first_scontext +
 						  vl_scontexts[i])
 					/* wrap back to first send context */

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 4143 bytes --]

On Tue, May 24, 2016 at 12:51:12PM -0700, Dennis Dalessandro wrote:
> From: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> 
> The credit return threshold adjustment on mtu change algorithm does not
> take into account all the kernel send contexts that are assigned per VL.
> Use the pio send context map to adjust the credit return thresholds for
> all the allocated and assigned kernel send contexts based on the MTU
> adjustment per VL.
> 
> The pio send context map can be changed dynamically based on the actual
> number of operational vls which is set by the fabric manager. When this
> happens update the credit return threshold values for all the remapped
> kernel send contexts.
> 
> Reviewed-by: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Reviewed-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Reviewed-by: Jianxin Xiong <jianxin.xiong-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Signed-off-by: Jubin John <jubin.john-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> ---
>  drivers/infiniband/hw/hfi1/chip.c |    7 +++++--
>  drivers/infiniband/hw/hfi1/pio.c  |   21 +++++++++++++++++++--
>  2 files changed, 24 insertions(+), 4 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/hfi1/chip.c b/drivers/infiniband/hw/hfi1/chip.c
> index 3b876da..be4fd3e 100644
> --- a/drivers/infiniband/hw/hfi1/chip.c
> +++ b/drivers/infiniband/hw/hfi1/chip.c
> @@ -9777,7 +9777,7 @@ static void set_send_length(struct hfi1_pportdata *ppd)
>  	u64 len1 = 0, len2 = (((dd->vld[15].mtu + max_hb) >> 2)
>  			      & SEND_LEN_CHECK1_LEN_VL15_MASK) <<
>  		SEND_LEN_CHECK1_LEN_VL15_SHIFT;
> -	int i;
> +	int i, j;
>  	u32 thres;
>  
>  	for (i = 0; i < ppd->vls_supported; i++) {
> @@ -9801,7 +9801,10 @@ static void set_send_length(struct hfi1_pportdata *ppd)
>  			    sc_mtu_to_threshold(dd->vld[i].sc,
>  						dd->vld[i].mtu,
>  						dd->rcd[0]->rcvhdrqentsize));
> -		sc_set_cr_threshold(dd->vld[i].sc, thres);
> +		for (j = 0; j < INIT_SC_PER_VL; j++)
> +			sc_set_cr_threshold(
> +					pio_select_send_context_vl(dd, j, i),
> +					    thres);
>  	}
>  	thres = min(sc_percent_to_threshold(dd->vld[15].sc, 50),
>  		    sc_mtu_to_threshold(dd->vld[15].sc,
> diff --git a/drivers/infiniband/hw/hfi1/pio.c b/drivers/infiniband/hw/hfi1/pio.c
> index d5edb1a..d5a9a5c 100644
> --- a/drivers/infiniband/hw/hfi1/pio.c
> +++ b/drivers/infiniband/hw/hfi1/pio.c
> @@ -1832,6 +1832,7 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
>  	int scontext = 1;
>  	int num_kernel_send_contexts = 0;
>  	u8 lvl_scontexts[OPA_MAX_VLS];
> +	u32 thres;
>  	struct pio_vl_map *oldmap, *newmap;
>  
>  	if (!vl_scontexts) {
> @@ -1872,11 +1873,27 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
>  			if (!newmap->map[i])
>  				goto bail;
>  			newmap->map[i]->mask = (1 << ilog2(sz)) - 1;
> -			/* assign send contexts */
> +			/*
> +			 * assign send contexts and
> +			 * adjust credit return threshold
> +			 */
>  			for (j = 0; j < sz; j++) {
> -				if (dd->kernel_send_context[scontext])
> +				if (dd->kernel_send_context[scontext]) {
>  					newmap->map[i]->ksc[j] =
>  					dd->kernel_send_context[scontext];
> +					thres = min(sc_percent_to_threshold(
> +						    dd->kernel_send_context
> +						    [scontext], 50),
> +						    sc_mtu_to_threshold(
> +						    dd->kernel_send_context
> +						    [scontext], dd->vld[i].mtu,
> +						    dd->rcd[0]->rcvhdrqentsize)
> +						    );
> +					sc_set_cr_threshold(
> +						dd->
> +						kernel_send_context[scontext],

You took checkpatch warnings output to the extreme.

> +						thres);
> +				}
>  				if (++scontext >= first_scontext +
>  						  vl_scontexts[i])
>  					/* wrap back to first send context */
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment

[Bart Van Assche]

On 05/25/2016 06:36 AM, Leon Romanovsky wrote:
> On Tue, May 24, 2016 at 12:51:12PM -0700, Dennis Dalessandro wrote:
>> @@ -1872,11 +1873,27 @@ int pio_map_init(struct hfi1_devdata *dd, u8 port, u8 num_vls, u8 *vl_scontexts)
>>  			if (!newmap->map[i])
>>  				goto bail;
>>  			newmap->map[i]->mask = (1 << ilog2(sz)) - 1;
>> -			/* assign send contexts */
>> +			/*
>> +			 * assign send contexts and
>> +			 * adjust credit return threshold
>> +			 */
>>  			for (j = 0; j < sz; j++) {
>> -				if (dd->kernel_send_context[scontext])
>> +				if (dd->kernel_send_context[scontext]) {
>>  					newmap->map[i]->ksc[j] =
>>  					dd->kernel_send_context[scontext];
>> +					thres = min(sc_percent_to_threshold(
>> +						    dd->kernel_send_context
>> +						    [scontext], 50),
>> +						    sc_mtu_to_threshold(
>> +						    dd->kernel_send_context
>> +						    [scontext], dd->vld[i].mtu,
>> +						    dd->rcd[0]->rcvhdrqentsize)
>> +						    );
>> +					sc_set_cr_threshold(
>> +						dd->
>> +						kernel_send_context[scontext],
>
> You took checkpatch warnings output to the extreme.

Hello Dennis and Leon,

Such formatting usually means that the indentation level is too deep. 
One way to address this is to move the body of the for loop into a new 
function. Choosing a shorter name for kernel_send_context and 
introducing temporary variables for the results of 
sc_percent_to_threshold(...) would also help.

Bart.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 05/10] IB/rdamvt: Fix rdmavt s_ack_queue sizing

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1329 bytes --]

On Tue, May 24, 2016 at 12:50:40PM -0700, Dennis Dalessandro wrote:
> From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> 
> rdmavt allows the driver to specify the size of the ack queue, but
> only uses it for the modify QP limit testing for setting the atomic
> limit value.
> 
> The driver dependent size is now used to size the s_ack_queue ring
> dynamicially.
> 
> Since the driver knows its size, the driver will use its define
> for any ring size dependent code.
> 
> Reviewed-by: Mitko Haralanov <mitko.haralanov-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> ---
>  drivers/infiniband/sw/rdmavt/qp.c |   22 ++++++++++++++++++++--
>  include/rdma/rdma_vt.h            |    9 +++++++++
>  include/rdma/rdmavt_qp.h          |    5 +----
>  3 files changed, 30 insertions(+), 6 deletions(-)
> 

...

>  /*
> + * Return the max atomic suitable for determining
> + * the size of the ack ring buffer in a QP.
> + */
> +static inline unsigned int rvt_max_atomic(struct rvt_dev_info *rdi)
> +{

In previous patch, you changed max_rdma_atomic to be u8 instead of int,
so why are you returning unsigned int and not u8?

> +	return rdi->dparms.max_rdma_atomic + 1;
> +}
> +

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH 10/10] IB/hfi1: Fix credit return threshold adjustment

[Dennis Dalessandro]

On Wed, May 25, 2016 at 08:27:37AM -0700, Bart Van Assche wrote:
>On 05/25/2016 06:36 AM, Leon Romanovsky wrote:
>>On Tue, May 24, 2016 at 12:51:12PM -0700, Dennis Dalessandro wrote:
>>>@@ -1872,11 +1873,27 @@ int pio_map_init(struct hfi1_devdata *dd, u8 
>>>port, u8 num_vls, u8 *vl_scontexts)
>>>  			if (!newmap->map[i])
>>>  				goto bail;
>>>  			newmap->map[i]->mask = (1 << ilog2(sz)) - 1;
>>>-			/* assign send contexts */
>>>+			/*
>>>+			 * assign send contexts and
>>>+			 * adjust credit return threshold
>>>+			 */
>>>  			for (j = 0; j < sz; j++) {
>>>-				if (dd->kernel_send_context[scontext])
>>>+				if (dd->kernel_send_context[scontext]) {
>>>  					newmap->map[i]->ksc[j] =
>>>  					dd->kernel_send_context[scontext];
>>>+					thres = min(sc_percent_to_threshold(
>>>+						    dd->kernel_send_context
>>>+						    [scontext], 50),
>>>+						    sc_mtu_to_threshold(
>>>+						    dd->kernel_send_context
>>>+						    [scontext], 
>>>dd->vld[i].mtu,
>>>+						    
>>>dd->rcd[0]->rcvhdrqentsize)
>>>+						    );
>>>+					sc_set_cr_threshold(
>>>+						dd->
>>>+					 kernel_send_context[scontext],
>>
>>You took checkpatch warnings output to the extreme.
>
>Hello Dennis and Leon,
>
>Such formatting usually means that the indentation level is too deep. 
>One way to address this is to move the body of the for loop into a new 
>function. Choosing a shorter name for kernel_send_context and 
>introducing temporary variables for the results of 
>sc_percent_to_threshold(...) would also help.

Yeah, we sometimes take the checkpatch thing to the extreme. Problem is 
while we sit in staging the hfi1 driver is a big target for folks to send 
checkpatch type fixes. We try to prevent that when we can.

We'll take a stab at prettying this up and reducing the indentation level.

-Denny

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 05/10] IB/rdamvt: Fix rdmavt s_ack_queue sizing

[Dennis Dalessandro]

On Wed, May 25, 2016 at 09:29:28PM +0300, Leon Romanovsky wrote:
>On Tue, May 24, 2016 at 12:50:40PM -0700, Dennis Dalessandro wrote:
>> From: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> 
>> rdmavt allows the driver to specify the size of the ack queue, but
>> only uses it for the modify QP limit testing for setting the atomic
>> limit value.
>> 
>> The driver dependent size is now used to size the s_ack_queue ring
>> dynamicially.
>> 
>> Since the driver knows its size, the driver will use its define
>> for any ring size dependent code.
>> 
>> Reviewed-by: Mitko Haralanov <mitko.haralanov-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> Signed-off-by: Mike Marciniszyn <mike.marciniszyn-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> ---
>>  drivers/infiniband/sw/rdmavt/qp.c |   22 ++++++++++++++++++++--
>>  include/rdma/rdma_vt.h            |    9 +++++++++
>>  include/rdma/rdmavt_qp.h          |    5 +----
>>  3 files changed, 30 insertions(+), 6 deletions(-)
>> 
>
>...
>
>>  /*
>> + * Return the max atomic suitable for determining
>> + * the size of the ack ring buffer in a QP.
>> + */
>> +static inline unsigned int rvt_max_atomic(struct rvt_dev_info *rdi)
>> +{
>
>In previous patch, you changed max_rdma_atomic to be u8 instead of int,
>so why are you returning unsigned int and not u8?
>
>> +	return rdi->dparms.max_rdma_atomic + 1;
>> +}
>> +

Because of the "+1", so we don't have a wrapping issue. 255 is a valid 
value.  Now we could bump it up to a u16 vs int if it matters that much.

-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 00/10] IB/hfi1,rdmavt: Another set of fixes for 4.7

[Doug Ledford]

[-- Attachment #1: Type: text/plain, Size: 2351 bytes --]

On 05/24/2016 03:50 PM, Dennis Dalessandro wrote:
> Doug, here are a few more fixes that we would like to get into 4.7 if possible.
> These apply on top of the series to get the driver out of staging. I can always
> respin if needed.
> 
> Included here is one patch which has a stable tag for 4.6 since it fixes a hard
> lockup. The others are minor bug fixes.
> 
> Patches can also be viewed in my repo at:
> https://github.com/ddalessa/kernel/tree/for-4.7
> 
> ---
> 
> Dean Luick (2):
>       IB/hfi1: Add a retry for the first-time QSFP access
>       IB/hfi1: Correct 8051 link parameter settings
> 
> Jianxin Xiong (2):
>       IB/hfi1,qib: Add ieth to the packet header definitions
>       IB/hfi1: Add tracing support for send with invalidate opcode
> 
> Jubin John (2):
>       IB/hfi1: Fix pio map initialization
>       IB/hfi1: Fix credit return threshold adjustment
> 
> Mike Marciniszyn (3):
>       IB/hfi1: Fix hard lockup due to not using save/restore spin lock
>       IB/rdmavt: Max atomic value should be a u8
>       IB/rdamvt: Fix rdmavt s_ack_queue sizing
> 
> Sebastian Sanchez (1):
>       IB/hfi1: Update pkey table properly after link down or FM start
> 
> 
>  drivers/infiniband/hw/hfi1/chip.c     |   16 ++++++++++++++--
>  drivers/infiniband/hw/hfi1/chip.h     |    6 ++++++
>  drivers/infiniband/hw/hfi1/mad.c      |    6 ++++++
>  drivers/infiniband/hw/hfi1/pio.c      |   24 ++++++++++++++++++++----
>  drivers/infiniband/hw/hfi1/platform.c |    6 ++++--
>  drivers/infiniband/hw/hfi1/qp.c       |    5 +++--
>  drivers/infiniband/hw/hfi1/qsfp.c     |   16 ++++++++++++++++
>  drivers/infiniband/hw/hfi1/trace.c    |    7 +++++++
>  drivers/infiniband/hw/hfi1/trace.h    |    2 ++
>  drivers/infiniband/hw/hfi1/verbs.c    |    2 ++
>  drivers/infiniband/hw/hfi1/verbs.h    |    1 +
>  drivers/infiniband/hw/qib/qib_verbs.h |    1 +
>  drivers/infiniband/sw/rdmavt/qp.c     |   22 ++++++++++++++++++++--
>  include/rdma/rdma_vt.h                |   13 +++++++++++--
>  include/rdma/rdmavt_qp.h              |    5 +----
>  15 files changed, 114 insertions(+), 18 deletions(-)
> 

I took everything but 7 and 10.  Please address the comments on those two.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
              GPG KeyID: 0E572FDD



[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [PATCH 07/10] IB/hfi1: Add a retry for the first-time QSFP access

[Dennis Dalessandro]

On Thu, May 26, 2016 at 12:17:16PM -0400, Doug Ledford wrote:
>On 05/24/2016 03:50 PM, Dennis Dalessandro wrote:
>> From: Dean Luick <dean.luick-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> 
>> Some QSFPs do not respond within the expected time, but later
>> appear fine.  Add a limited retry on the first access.
>
>10 seconds is an awful long delay period.  Admittedly I didn't look
>through the sources to see if the refresh is already happening in the
>context of a delayed work queue or similar, so maybe you can ignore
>this, but if you're going to delay for 10 seconds, it should probably be
>done from a workqueue and not via msleep(2000); goto retry;.

Yeah the 10 seconds may be a bit long. I think we can go ahead and drop this 
from the queue for 4.7 merge window since it is winding down. 

Using a workqueue may be the right way to go but there may be some subtle 
issues there I'd like to think through so rather than rush we can wait for 
rc or 4.8 even.

-Denny
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html