[PATCH V2 for-next 0/7] Add CM and a bugfix to hip08

[PATCH V2 for-next 0/7] Add CM and a bugfix to hip08

[Lijun Ou]

This patch series adds the CM(Connection Mangement) support
to hip08 RoCE driver. Changes done are primarily to add
support of APIs in IB device and some fixes over the
origin driver to support RDMA CM. This patch series
also adjust some codes style in order to fill the
sq wqe of ud type and fix a bug for tx_cqn field.

Change from V1:
- Add a new patch named assign the correct value for
  tx_cqn splited from the first patchset
- Fix the comments given by Leon Romanovsky
- Add the options for free sq lock when qp is illegal
  qp type
 
Lijun Ou (7):
  RDMA/hns: Assign the correct value for tx_cqn
  RDMA/hns: Create gsi qp in hip08
  RDMA/hns: Add gsi qp support for modifying qp in hip08
  RDMA/hns: Fill sq wqe context of ud type in hip08
  RDMA/hns: Assign zero for pkey_index of wc in hip08
  RDMA/hns: Update the verbs of polling for completion
  RDMA/hns: Set the guid for hip08 RoCE device

 drivers/infiniband/hw/hns/hns_roce_device.h |   1 +
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c  | 531 +++++++++++++++++++---------
 drivers/infiniband/hw/hns/hns_roce_hw_v2.h  |  86 ++++-
 drivers/infiniband/hw/hns/hns_roce_qp.c     |  18 +-
 4 files changed, 471 insertions(+), 165 deletions(-)

-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 1/7] RDMA/hns: Assign the correct value for tx_cqn

[Lijun Ou]

When modifying qp from init to init, it need to
assign the cqn of send cq for tx cqn field of
qp context. Otherwise, it will cause a mistake
when the send and recv cq sizes is the difference.
This patch mainly fixes it.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 2ca35e3..dd50336 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -2342,7 +2342,7 @@ static void modify_qp_init_to_init(struct ib_qp *ibqp,
 		       V2_QPC_BYTE_80_RX_CQN_S, 0);
 
 	roce_set_field(context->byte_252_err_txcqn, V2_QPC_BYTE_252_TX_CQN_M,
-		       V2_QPC_BYTE_252_TX_CQN_S, to_hr_cq(ibqp->recv_cq)->cqn);
+		       V2_QPC_BYTE_252_TX_CQN_S, to_hr_cq(ibqp->send_cq)->cqn);
 	roce_set_field(qpc_mask->byte_252_err_txcqn, V2_QPC_BYTE_252_TX_CQN_M,
 		       V2_QPC_BYTE_252_TX_CQN_S, 0);
 
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 2/7] RDMA/hns: Create gsi qp in hip08

[Lijun Ou]

The gsi qp and rc qp use the same qp context
structure and the created flow, only
differentiate them by qpn and qp type.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
 drivers/infiniband/hw/hns/hns_roce_qp.c | 18 ++++++++++++++++--
 1 file changed, 16 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c
index 351fa31..4414cea 100644
--- a/drivers/infiniband/hw/hns/hns_roce_qp.c
+++ b/drivers/infiniband/hw/hns/hns_roce_qp.c
@@ -455,6 +455,13 @@ static int hns_roce_set_kernel_sq_size(struct hns_roce_dev *hr_dev,
 		hr_qp->sge.sge_shift = 4;
 	}
 
+	/* ud sqwqe's sge use extend sge */
+	if (hr_dev->caps.max_sq_sg > 2 && hr_qp->ibqp.qp_type == IB_QPT_GSI) {
+		hr_qp->sge.sge_cnt = roundup_pow_of_two(hr_qp->sq.wqe_cnt *
+				     hr_qp->sq.max_gs);
+		hr_qp->sge.sge_shift = 4;
+	}
+
 	/* Get buf size, SQ and RQ are aligned to PAGE_SIZE */
 	page_size = 1 << (hr_dev->caps.mtt_buf_pg_sz + PAGE_SHIFT);
 	hr_qp->sq.offset = 0;
@@ -502,6 +509,8 @@ static int hns_roce_create_qp_common(struct hns_roce_dev *hr_dev,
 
 	hr_qp->state = IB_QPS_RESET;
 
+	hr_qp->ibqp.qp_type = init_attr->qp_type;
+
 	if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
 		hr_qp->sq_signal_bits = IB_SIGNAL_ALL_WR;
 	else
@@ -764,8 +773,13 @@ struct ib_qp *hns_roce_create_qp(struct ib_pd *pd,
 		hr_qp = &hr_sqp->hr_qp;
 		hr_qp->port = init_attr->port_num - 1;
 		hr_qp->phy_port = hr_dev->iboe.phy_port[hr_qp->port];
-		hr_qp->ibqp.qp_num = HNS_ROCE_MAX_PORTS +
-				     hr_dev->iboe.phy_port[hr_qp->port];
+
+		/* when hw version is v1, the sqpn is allocated */
+		if (hr_dev->caps.max_sq_sg <= 2)
+			hr_qp->ibqp.qp_num = HNS_ROCE_MAX_PORTS +
+					     hr_dev->iboe.phy_port[hr_qp->port];
+		else
+			hr_qp->ibqp.qp_num = 1;
 
 		ret = hns_roce_create_qp_common(hr_dev, pd, init_attr, udata,
 						hr_qp->ibqp.qp_num, hr_qp);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 3/7] RDMA/hns: Add gsi qp support for modifying qp in hip08

[Lijun Ou]

It needs to Assign the values for some fields in qp context
when qp type is gsi qp type in hip08.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V2:
- Remove the options of assigning the correct value for
  tx cqn field of qp context and generate a separate patch

V1:
- The initial submit
---
 drivers/infiniband/hw/hns/hns_roce_device.h |  1 +
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c  | 64 +++++++++++++++++++++--------
 2 files changed, 49 insertions(+), 16 deletions(-)

diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 4afa070..42c3b5a 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -485,6 +485,7 @@ struct hns_roce_qp {
 	u32			access_flags;
 	u32                     atomic_rd_en;
 	u32			pkey_index;
+	u32			qkey;
 	void			(*event)(struct hns_roce_qp *,
 					 enum hns_roce_event);
 	unsigned long		qpn;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index dd50336..e53cd7d 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1975,6 +1975,7 @@ static void set_access_flags(struct hns_roce_qp *hr_qp,
 
 static void modify_qp_reset_to_init(struct ib_qp *ibqp,
 				    const struct ib_qp_attr *attr,
+				    int attr_mask,
 				    struct hns_roce_v2_qp_context *context,
 				    struct hns_roce_v2_qp_context *qpc_mask)
 {
@@ -1991,9 +1992,18 @@ static void modify_qp_reset_to_init(struct ib_qp *ibqp,
 	roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_TST_M,
 		       V2_QPC_BYTE_4_TST_S, 0);
 
-	roce_set_field(context->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
-		       V2_QPC_BYTE_4_SGE_SHIFT_S, hr_qp->sq.max_gs > 2 ?
-		       ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+	if (ibqp->qp_type == IB_QPT_GSI)
+		roce_set_field(context->byte_4_sqpn_tst,
+			       V2_QPC_BYTE_4_SGE_SHIFT_M,
+			       V2_QPC_BYTE_4_SGE_SHIFT_S,
+			       ilog2((unsigned int)hr_qp->sge.sge_cnt));
+	else
+		roce_set_field(context->byte_4_sqpn_tst,
+			       V2_QPC_BYTE_4_SGE_SHIFT_M,
+			       V2_QPC_BYTE_4_SGE_SHIFT_S,
+			       hr_qp->sq.max_gs > 2 ?
+			       ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+
 	roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
 		       V2_QPC_BYTE_4_SGE_SHIFT_S, 0);
 
@@ -2058,6 +2068,12 @@ static void modify_qp_reset_to_init(struct ib_qp *ibqp,
 	roce_set_bit(qpc_mask->byte_28_at_fl, V2_QPC_BYTE_28_CNP_TX_FLAG_S, 0);
 	roce_set_bit(qpc_mask->byte_28_at_fl, V2_QPC_BYTE_28_CE_FLAG_S, 0);
 
+	if (attr_mask & IB_QP_QKEY) {
+		context->qkey_xrcd = attr->qkey;
+		qpc_mask->qkey_xrcd = 0;
+		hr_qp->qkey = attr->qkey;
+	}
+
 	roce_set_bit(context->byte_76_srqn_op_en, V2_QPC_BYTE_76_RQIE_S, 1);
 	roce_set_bit(qpc_mask->byte_76_srqn_op_en, V2_QPC_BYTE_76_RQIE_S, 0);
 
@@ -2279,9 +2295,17 @@ static void modify_qp_init_to_init(struct ib_qp *ibqp,
 	roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_TST_M,
 		       V2_QPC_BYTE_4_TST_S, 0);
 
-	roce_set_field(context->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
-		       V2_QPC_BYTE_4_SGE_SHIFT_S, hr_qp->sq.max_gs > 2 ?
-		       ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+	if (ibqp->qp_type == IB_QPT_GSI)
+		roce_set_field(context->byte_4_sqpn_tst,
+			       V2_QPC_BYTE_4_SGE_SHIFT_M,
+			       V2_QPC_BYTE_4_SGE_SHIFT_S,
+			       ilog2((unsigned int)hr_qp->sge.sge_cnt));
+	else
+		roce_set_field(context->byte_4_sqpn_tst,
+			       V2_QPC_BYTE_4_SGE_SHIFT_M,
+			       V2_QPC_BYTE_4_SGE_SHIFT_S, hr_qp->sq.max_gs > 2 ?
+			       ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+
 	roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
 		       V2_QPC_BYTE_4_SGE_SHIFT_S, 0);
 
@@ -2358,10 +2382,10 @@ static void modify_qp_init_to_init(struct ib_qp *ibqp,
 			       V2_QPC_BYTE_76_SRQN_M, V2_QPC_BYTE_76_SRQN_S, 0);
 	}
 
-	if (attr_mask & IB_QP_PKEY_INDEX)
-		context->qkey_xrcd = attr->pkey_index;
-	else
-		context->qkey_xrcd = hr_qp->pkey_index;
+	if (attr_mask & IB_QP_QKEY) {
+		context->qkey_xrcd = attr->qkey;
+		qpc_mask->qkey_xrcd = 0;
+	}
 
 	roce_set_field(context->byte_4_sqpn_tst, V2_QPC_BYTE_4_SQPN_M,
 		       V2_QPC_BYTE_4_SQPN_S, hr_qp->qpn);
@@ -2457,7 +2481,8 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp,
 	roce_set_field(context->byte_20_smac_sgid_idx,
 		       V2_QPC_BYTE_20_SGE_HOP_NUM_M,
 		       V2_QPC_BYTE_20_SGE_HOP_NUM_S,
-		       hr_qp->sq.max_gs > 2 ? hr_dev->caps.mtt_hop_num : 0);
+		       ((ibqp->qp_type == IB_QPT_GSI) || hr_qp->sq.max_gs > 2) ?
+		       hr_dev->caps.mtt_hop_num : 0);
 	roce_set_field(qpc_mask->byte_20_smac_sgid_idx,
 		       V2_QPC_BYTE_20_SGE_HOP_NUM_M,
 		       V2_QPC_BYTE_20_SGE_HOP_NUM_S, 0);
@@ -2617,8 +2642,13 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp,
 	roce_set_field(qpc_mask->byte_24_mtu_tc, V2_QPC_BYTE_24_TC_M,
 		       V2_QPC_BYTE_24_TC_S, 0);
 
-	roce_set_field(context->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
-		       V2_QPC_BYTE_24_MTU_S, attr->path_mtu);
+	if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_UD)
+		roce_set_field(context->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
+			       V2_QPC_BYTE_24_MTU_S, IB_MTU_4096);
+	else
+		roce_set_field(context->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
+			       V2_QPC_BYTE_24_MTU_S, attr->path_mtu);
+
 	roce_set_field(qpc_mask->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
 		       V2_QPC_BYTE_24_MTU_S, 0);
 
@@ -2725,13 +2755,14 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp,
 		       V2_QPC_BYTE_168_SQ_CUR_BLK_ADDR_S, 0);
 
 	page_size = 1 << (hr_dev->caps.mtt_buf_pg_sz + PAGE_SHIFT);
-	context->sq_cur_sge_blk_addr = hr_qp->sq.max_gs > 2 ?
+	context->sq_cur_sge_blk_addr =
+		       ((ibqp->qp_type == IB_QPT_GSI) || hr_qp->sq.max_gs > 2) ?
 				      ((u32)(mtts[hr_qp->sge.offset / page_size]
 				      >> PAGE_ADDR_SHIFT)) : 0;
 	roce_set_field(context->byte_184_irrl_idx,
 		       V2_QPC_BYTE_184_SQ_CUR_SGE_BLK_ADDR_M,
 		       V2_QPC_BYTE_184_SQ_CUR_SGE_BLK_ADDR_S,
-		       hr_qp->sq.max_gs > 2 ?
+		       ((ibqp->qp_type == IB_QPT_GSI) || hr_qp->sq.max_gs > 2) ?
 		       (mtts[hr_qp->sge.offset / page_size] >>
 		       (32 + PAGE_ADDR_SHIFT)) : 0);
 	qpc_mask->sq_cur_sge_blk_addr = 0;
@@ -2902,7 +2933,8 @@ static int hns_roce_v2_modify_qp(struct ib_qp *ibqp,
 	 */
 	memset(qpc_mask, 0xff, sizeof(*qpc_mask));
 	if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
-		modify_qp_reset_to_init(ibqp, attr, context, qpc_mask);
+		modify_qp_reset_to_init(ibqp, attr, attr_mask, context,
+					qpc_mask);
 	} else if (cur_state == IB_QPS_INIT && new_state == IB_QPS_INIT) {
 		modify_qp_init_to_init(ibqp, attr, attr_mask, context,
 				       qpc_mask);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type in hip08

[Lijun Ou]

This patch mainly configure the fileds of sq wqe of ud
type when posting wr of gsi qp type.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V1->V2:
- return errno code directly and delete unnecessary
  initialized value of ret from Leon Romanvosky's
  advice.
- Add necessary option for free sq lock when qp is
  illegal qp type.

V1:
- The initial submit
---
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 446 +++++++++++++++++++----------
 drivers/infiniband/hw/hns/hns_roce_hw_v2.h |  84 ++++++
 2 files changed, 385 insertions(+), 145 deletions(-)

diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index e53cd7d..cb3ac54 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -51,26 +51,99 @@ static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
 	dseg->len  = cpu_to_le32(sg->length);
 }
 
+static int set_rwqe_data_seg(struct ib_qp *ibqp, struct ib_send_wr *wr,
+			     struct hns_roce_v2_rc_send_wqe *rc_sq_wqe,
+			     void *wqe, unsigned int *sge_ind,
+			     struct ib_send_wr **bad_wr)
+{
+	struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+	struct hns_roce_v2_wqe_data_seg *dseg = wqe;
+	struct hns_roce_qp *qp = to_hr_qp(ibqp);
+	int i;
+
+	if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
+		if (rc_sq_wqe->msg_len > hr_dev->caps.max_sq_inline) {
+			*bad_wr = wr;
+			dev_err(hr_dev->dev, "inline len(1-%d)=%d, illegal",
+				rc_sq_wqe->msg_len, hr_dev->caps.max_sq_inline);
+			return -EINVAL;
+		}
+
+		for (i = 0; i < wr->num_sge; i++) {
+			memcpy(wqe, ((void *)wr->sg_list[i].addr),
+			       wr->sg_list[i].length);
+			wqe += wr->sg_list[i].length;
+		}
+
+		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_INLINE_S,
+			     1);
+	} else {
+		if (wr->num_sge <= 2) {
+			for (i = 0; i < wr->num_sge; i++) {
+				if (likely(wr->sg_list[i].length)) {
+					set_data_seg_v2(dseg, wr->sg_list + i);
+					dseg++;
+				}
+			}
+		} else {
+			roce_set_field(rc_sq_wqe->byte_20,
+				     V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
+				     V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+				     (*sge_ind) & (qp->sge.sge_cnt - 1));
+
+			for (i = 0; i < 2; i++) {
+				if (likely(wr->sg_list[i].length)) {
+					set_data_seg_v2(dseg, wr->sg_list + i);
+					dseg++;
+				}
+			}
+
+			dseg = get_send_extend_sge(qp,
+					    (*sge_ind) & (qp->sge.sge_cnt - 1));
+
+			for (i = 0; i < wr->num_sge - 2; i++) {
+				if (likely(wr->sg_list[i + 2].length)) {
+					set_data_seg_v2(dseg,
+							wr->sg_list + 2 + i);
+					dseg++;
+					(*sge_ind)++;
+				}
+			}
+		}
+
+		roce_set_field(rc_sq_wqe->byte_16,
+			       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
+			       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
+	}
+
+	return 0;
+}
+
 static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
 				 struct ib_send_wr **bad_wr)
 {
 	struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
+	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe;
 	struct hns_roce_v2_rc_send_wqe *rc_sq_wqe;
 	struct hns_roce_qp *qp = to_hr_qp(ibqp);
 	struct hns_roce_v2_wqe_data_seg *dseg;
 	struct device *dev = hr_dev->dev;
 	struct hns_roce_v2_db sq_db;
 	unsigned int sge_ind = 0;
-	unsigned int wqe_sz = 0;
 	unsigned int owner_bit;
 	unsigned long flags;
 	unsigned int ind;
 	void *wqe = NULL;
+	bool loopback;
 	int ret = 0;
+	u8 *smac;
 	int nreq;
 	int i;
 
-	if (unlikely(ibqp->qp_type != IB_QPT_RC)) {
+	if (unlikely(ibqp->qp_type != IB_QPT_RC &&
+		     ibqp->qp_type != IB_QPT_GSI &&
+		     ibqp->qp_type != IB_QPT_UD)) {
 		dev_err(dev, "Not supported QP(0x%x)type!\n", ibqp->qp_type);
 		*bad_wr = NULL;
 		return -EOPNOTSUPP;
@@ -107,172 +180,255 @@ static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
 								      wr->wr_id;
 
 		owner_bit = ~(qp->sq.head >> ilog2(qp->sq.wqe_cnt)) & 0x1;
-		rc_sq_wqe = wqe;
-		memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
-		for (i = 0; i < wr->num_sge; i++)
-			rc_sq_wqe->msg_len += wr->sg_list[i].length;
 
-		rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
+		/* Corresponding to the QP type, wqe process separately */
+		if (ibqp->qp_type == IB_QPT_GSI) {
+			ud_sq_wqe = wqe;
+			memset(ud_sq_wqe, 0, sizeof(*ud_sq_wqe));
+
+			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_0_M,
+				       V2_UD_SEND_WQE_DMAC_0_S, ah->av.mac[0]);
+			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_1_M,
+				       V2_UD_SEND_WQE_DMAC_1_S, ah->av.mac[1]);
+			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_2_M,
+				       V2_UD_SEND_WQE_DMAC_2_S, ah->av.mac[2]);
+			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_3_M,
+				       V2_UD_SEND_WQE_DMAC_3_S, ah->av.mac[3]);
+			roce_set_field(ud_sq_wqe->byte_48,
+				       V2_UD_SEND_WQE_BYTE_48_DMAC_4_M,
+				       V2_UD_SEND_WQE_BYTE_48_DMAC_4_S,
+				       ah->av.mac[4]);
+			roce_set_field(ud_sq_wqe->byte_48,
+				       V2_UD_SEND_WQE_BYTE_48_DMAC_5_M,
+				       V2_UD_SEND_WQE_BYTE_48_DMAC_5_S,
+				       ah->av.mac[5]);
+
+			/* MAC loopback */
+			smac = (u8 *)hr_dev->dev_addr[qp->port];
+			loopback = ether_addr_equal_unaligned(ah->av.mac,
+							      smac) ? 1 : 0;
+
+			roce_set_bit(ud_sq_wqe->byte_40,
+				     V2_UD_SEND_WQE_BYTE_40_LBI_S, loopback);
+
+			roce_set_field(ud_sq_wqe->byte_4,
+				       V2_UD_SEND_WQE_BYTE_4_OPCODE_M,
+				       V2_UD_SEND_WQE_BYTE_4_OPCODE_S,
+				       HNS_ROCE_V2_WQE_OP_SEND);
 
-		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_FENCE_S,
-			    (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
+			for (i = 0; i < wr->num_sge; i++)
+				ud_sq_wqe->msg_len += wr->sg_list[i].length;
 
-		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_SE_S,
-			    (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
+			ud_sq_wqe->immtdata = send_ieth(wr);
 
-		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_CQE_S,
-			    (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
+			/* Set sig attr */
+			roce_set_bit(ud_sq_wqe->byte_4,
+				   V2_UD_SEND_WQE_BYTE_4_CQE_S,
+				   (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
 
-		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_OWNER_S,
-			     owner_bit);
+			/* Set se attr */
+			roce_set_bit(ud_sq_wqe->byte_4,
+				  V2_UD_SEND_WQE_BYTE_4_SE_S,
+				  (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
 
-		switch (wr->opcode) {
-		case IB_WR_RDMA_READ:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_RDMA_READ);
-			rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
-			rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
-			break;
-		case IB_WR_RDMA_WRITE:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
-			rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
-			rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
-			break;
-		case IB_WR_RDMA_WRITE_WITH_IMM:
-			roce_set_field(rc_sq_wqe->byte_4,
+			roce_set_bit(ud_sq_wqe->byte_4,
+				     V2_UD_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
+
+			roce_set_field(ud_sq_wqe->byte_16,
+				       V2_UD_SEND_WQE_BYTE_16_PD_M,
+				       V2_UD_SEND_WQE_BYTE_16_PD_S,
+				       to_hr_pd(ibqp->pd)->pdn);
+
+			roce_set_field(ud_sq_wqe->byte_16,
+				       V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M,
+				       V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S,
+				       wr->num_sge);
+
+			roce_set_field(ud_sq_wqe->byte_20,
+				     V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
+				     V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+				     sge_ind & (qp->sge.sge_cnt - 1));
+
+			roce_set_field(ud_sq_wqe->byte_24,
+				       V2_UD_SEND_WQE_BYTE_24_UDPSPN_M,
+				       V2_UD_SEND_WQE_BYTE_24_UDPSPN_S, 0);
+			ud_sq_wqe->qkey =
+			     cpu_to_be32(ud_wr(wr)->remote_qkey & 0x80000000) ?
+			     qp->qkey : ud_wr(wr)->remote_qkey;
+			roce_set_field(ud_sq_wqe->byte_32,
+				       V2_UD_SEND_WQE_BYTE_32_DQPN_M,
+				       V2_UD_SEND_WQE_BYTE_32_DQPN_S,
+				       ud_wr(wr)->remote_qpn);
+
+			roce_set_field(ud_sq_wqe->byte_36,
+				       V2_UD_SEND_WQE_BYTE_36_VLAN_M,
+				       V2_UD_SEND_WQE_BYTE_36_VLAN_S,
+				       ah->av.vlan);
+			roce_set_field(ud_sq_wqe->byte_36,
+				       V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M,
+				       V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S,
+				       ah->av.hop_limit);
+			roce_set_field(ud_sq_wqe->byte_36,
+				       V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
+				       V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
+				       0);
+			roce_set_field(ud_sq_wqe->byte_36,
+				       V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
+				       V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
+				       0);
+			roce_set_field(ud_sq_wqe->byte_40,
+				       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
+				       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, 0);
+			roce_set_field(ud_sq_wqe->byte_40,
+				       V2_UD_SEND_WQE_BYTE_40_SL_M,
+				       V2_UD_SEND_WQE_BYTE_40_SL_S,
+				       ah->av.sl_tclass_flowlabel >>
+				       HNS_ROCE_SL_SHIFT);
+			roce_set_field(ud_sq_wqe->byte_40,
+				       V2_UD_SEND_WQE_BYTE_40_PORTN_M,
+				       V2_UD_SEND_WQE_BYTE_40_PORTN_S,
+				       qp->port);
+
+			roce_set_field(ud_sq_wqe->byte_48,
+				       V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M,
+				       V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S,
+				       hns_get_gid_index(hr_dev, qp->phy_port,
+							 ah->av.gid_index));
+
+			memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0],
+			       GID_LEN_V2);
+
+			dseg = get_send_extend_sge(qp,
+					    sge_ind & (qp->sge.sge_cnt - 1));
+			for (i = 0; i < wr->num_sge; i++) {
+				set_data_seg_v2(dseg + i, wr->sg_list + i);
+				sge_ind++;
+			}
+
+			ind++;
+		} else if (ibqp->qp_type == IB_QPT_RC) {
+			rc_sq_wqe = wqe;
+			memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
+			for (i = 0; i < wr->num_sge; i++)
+				rc_sq_wqe->msg_len += wr->sg_list[i].length;
+
+			rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
+
+			roce_set_bit(rc_sq_wqe->byte_4,
+				     V2_RC_SEND_WQE_BYTE_4_FENCE_S,
+				     (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
+
+			roce_set_bit(rc_sq_wqe->byte_4,
+				  V2_RC_SEND_WQE_BYTE_4_SE_S,
+				  (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
+
+			roce_set_bit(rc_sq_wqe->byte_4,
+				   V2_RC_SEND_WQE_BYTE_4_CQE_S,
+				   (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
+
+			roce_set_bit(rc_sq_wqe->byte_4,
+				     V2_RC_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
+
+			switch (wr->opcode) {
+			case IB_WR_RDMA_READ:
+				roce_set_field(rc_sq_wqe->byte_4,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					       HNS_ROCE_V2_WQE_OP_RDMA_READ);
+				rc_sq_wqe->rkey =
+					cpu_to_le32(rdma_wr(wr)->rkey);
+				rc_sq_wqe->va =
+					cpu_to_le64(rdma_wr(wr)->remote_addr);
+				break;
+			case IB_WR_RDMA_WRITE:
+				roce_set_field(rc_sq_wqe->byte_4,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					       HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
+				rc_sq_wqe->rkey =
+					cpu_to_le32(rdma_wr(wr)->rkey);
+				rc_sq_wqe->va =
+					cpu_to_le64(rdma_wr(wr)->remote_addr);
+				break;
+			case IB_WR_RDMA_WRITE_WITH_IMM:
+				roce_set_field(rc_sq_wqe->byte_4,
 				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
 				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
 				       HNS_ROCE_V2_WQE_OP_RDMA_WRITE_WITH_IMM);
-			rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
-			rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
-			break;
-		case IB_WR_SEND:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_SEND);
-			break;
-		case IB_WR_SEND_WITH_INV:
-			roce_set_field(rc_sq_wqe->byte_4,
+				rc_sq_wqe->rkey =
+					cpu_to_le32(rdma_wr(wr)->rkey);
+				rc_sq_wqe->va =
+					cpu_to_le64(rdma_wr(wr)->remote_addr);
+				break;
+			case IB_WR_SEND:
+				roce_set_field(rc_sq_wqe->byte_4,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					       HNS_ROCE_V2_WQE_OP_SEND);
+				break;
+			case IB_WR_SEND_WITH_INV:
+				roce_set_field(rc_sq_wqe->byte_4,
 				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
 				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
 				       HNS_ROCE_V2_WQE_OP_SEND_WITH_INV);
-			break;
-		case IB_WR_SEND_WITH_IMM:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
-			break;
-		case IB_WR_LOCAL_INV:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_LOCAL_INV);
-			break;
-		case IB_WR_ATOMIC_CMP_AND_SWP:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
-			break;
-		case IB_WR_ATOMIC_FETCH_AND_ADD:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
-			break;
-		case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
-			roce_set_field(rc_sq_wqe->byte_4,
+				break;
+			case IB_WR_SEND_WITH_IMM:
+				roce_set_field(rc_sq_wqe->byte_4,
+					      V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					      V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					      HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
+				break;
+			case IB_WR_LOCAL_INV:
+				roce_set_field(rc_sq_wqe->byte_4,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					       HNS_ROCE_V2_WQE_OP_LOCAL_INV);
+				break;
+			case IB_WR_ATOMIC_CMP_AND_SWP:
+				roce_set_field(rc_sq_wqe->byte_4,
+					  V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					  V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					  HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
+				break;
+			case IB_WR_ATOMIC_FETCH_AND_ADD:
+				roce_set_field(rc_sq_wqe->byte_4,
+					 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					 HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
+				break;
+			case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
+				roce_set_field(rc_sq_wqe->byte_4,
 				      V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
 				      V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
 				      HNS_ROCE_V2_WQE_OP_ATOM_MSK_CMP_AND_SWAP);
-			break;
-		case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
-			roce_set_field(rc_sq_wqe->byte_4,
+				break;
+			case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
+				roce_set_field(rc_sq_wqe->byte_4,
 				     V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
 				     V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
 				     HNS_ROCE_V2_WQE_OP_ATOM_MSK_FETCH_AND_ADD);
-			break;
-		default:
-			roce_set_field(rc_sq_wqe->byte_4,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
-				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
-				       HNS_ROCE_V2_WQE_OP_MASK);
-			break;
-		}
-
-		wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
-		dseg = wqe;
-		if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
-			if (rc_sq_wqe->msg_len >
-				hr_dev->caps.max_sq_inline) {
-				ret = -EINVAL;
-				*bad_wr = wr;
-				dev_err(dev, "inline len(1-%d)=%d, illegal",
-					rc_sq_wqe->msg_len,
-					hr_dev->caps.max_sq_inline);
-				goto out;
+				break;
+			default:
+				roce_set_field(rc_sq_wqe->byte_4,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+					       HNS_ROCE_V2_WQE_OP_MASK);
+				break;
 			}
 
-			for (i = 0; i < wr->num_sge; i++) {
-				memcpy(wqe, ((void *)wr->sg_list[i].addr),
-				       wr->sg_list[i].length);
-				wqe += wr->sg_list[i].length;
-				wqe_sz += wr->sg_list[i].length;
-			}
+			wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
+			dseg = wqe;
 
-			roce_set_bit(rc_sq_wqe->byte_4,
-				     V2_RC_SEND_WQE_BYTE_4_INLINE_S, 1);
+			ret = set_rwqe_data_seg(ibqp, wr, rc_sq_wqe, wqe,
+						&sge_ind, bad_wr);
+			if (ret)
+				goto out;
+			ind++;
 		} else {
-			if (wr->num_sge <= 2) {
-				for (i = 0; i < wr->num_sge; i++) {
-					if (likely(wr->sg_list[i].length)) {
-						set_data_seg_v2(dseg,
-							       wr->sg_list + i);
-						dseg++;
-					}
-				}
-			} else {
-				roce_set_field(rc_sq_wqe->byte_20,
-				V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
-				V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
-				sge_ind & (qp->sge.sge_cnt - 1));
-
-				for (i = 0; i < 2; i++) {
-					if (likely(wr->sg_list[i].length)) {
-						set_data_seg_v2(dseg,
-							       wr->sg_list + i);
-						dseg++;
-					}
-				}
-
-				dseg = get_send_extend_sge(qp,
-					sge_ind & (qp->sge.sge_cnt - 1));
-
-				for (i = 0; i < wr->num_sge - 2; i++) {
-					if (likely(wr->sg_list[i + 2].length)) {
-						set_data_seg_v2(dseg,
-							   wr->sg_list + 2 + i);
-						dseg++;
-						sge_ind++;
-					}
-				}
-			}
-
-			roce_set_field(rc_sq_wqe->byte_16,
-				       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
-				       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S,
-				       wr->num_sge);
-			wqe_sz += wr->num_sge *
-				  sizeof(struct hns_roce_v2_wqe_data_seg);
+			dev_err(dev, "Illegal qp_type(0x%x)\n", ibqp->qp_type);
+			spin_unlock_irqrestore(&qp->sq.lock, flags);
+			return -EOPNOTSUPP;
 		}
-		ind++;
 	}
 
 out:
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
index 463edab..c11b253 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
@@ -916,6 +916,90 @@ struct hns_roce_v2_cq_db {
 
 #define V2_CQ_DB_PARAMETER_NOTIFY_S 24
 
+struct hns_roce_v2_ud_send_wqe {
+	u32	byte_4;
+	u32	msg_len;
+	u32	immtdata;
+	u32	byte_16;
+	u32	byte_20;
+	u32	byte_24;
+	u32	qkey;
+	u32	byte_32;
+	u32	byte_36;
+	u32	byte_40;
+	u32	dmac;
+	u32	byte_48;
+	u8	dgid[GID_LEN_V2];
+
+};
+#define	V2_UD_SEND_WQE_BYTE_4_OPCODE_S 0
+#define V2_UD_SEND_WQE_BYTE_4_OPCODE_M GENMASK(4, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_4_OWNER_S 7
+
+#define	V2_UD_SEND_WQE_BYTE_4_CQE_S 8
+
+#define	V2_UD_SEND_WQE_BYTE_4_SE_S 11
+
+#define	V2_UD_SEND_WQE_BYTE_16_PD_S 0
+#define V2_UD_SEND_WQE_BYTE_16_PD_M GENMASK(23, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S 24
+#define V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M GENMASK(31, 24)
+
+#define	V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S 0
+#define V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M GENMASK(23, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_24_UDPSPN_S 16
+#define V2_UD_SEND_WQE_BYTE_24_UDPSPN_M GENMASK(31, 16)
+
+#define	V2_UD_SEND_WQE_BYTE_32_DQPN_S 0
+#define V2_UD_SEND_WQE_BYTE_32_DQPN_M GENMASK(23, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_36_VLAN_S 0
+#define V2_UD_SEND_WQE_BYTE_36_VLAN_M GENMASK(15, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S 16
+#define V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M GENMASK(23, 16)
+
+#define	V2_UD_SEND_WQE_BYTE_36_TCLASS_S 24
+#define V2_UD_SEND_WQE_BYTE_36_TCLASS_M GENMASK(31, 24)
+
+#define	V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S 0
+#define V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M GENMASK(19, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_40_SL_S 20
+#define V2_UD_SEND_WQE_BYTE_40_SL_M GENMASK(23, 20)
+
+#define	V2_UD_SEND_WQE_BYTE_40_PORTN_S 24
+#define V2_UD_SEND_WQE_BYTE_40_PORTN_M GENMASK(26, 24)
+
+#define	V2_UD_SEND_WQE_BYTE_40_LBI_S 31
+
+#define	V2_UD_SEND_WQE_DMAC_0_S 0
+#define V2_UD_SEND_WQE_DMAC_0_M GENMASK(7, 0)
+
+#define	V2_UD_SEND_WQE_DMAC_1_S 8
+#define V2_UD_SEND_WQE_DMAC_1_M GENMASK(15, 8)
+
+#define	V2_UD_SEND_WQE_DMAC_2_S 16
+#define V2_UD_SEND_WQE_DMAC_2_M GENMASK(23, 16)
+
+#define	V2_UD_SEND_WQE_DMAC_3_S 24
+#define V2_UD_SEND_WQE_DMAC_3_M GENMASK(31, 24)
+
+#define	V2_UD_SEND_WQE_BYTE_48_DMAC_4_S 0
+#define V2_UD_SEND_WQE_BYTE_48_DMAC_4_M GENMASK(7, 0)
+
+#define	V2_UD_SEND_WQE_BYTE_48_DMAC_5_S 8
+#define V2_UD_SEND_WQE_BYTE_48_DMAC_5_M GENMASK(15, 8)
+
+#define	V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S 16
+#define V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M GENMASK(23, 16)
+
+#define	V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_S 24
+#define V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_M GENMASK(31, 24)
+
 struct hns_roce_v2_rc_send_wqe {
 	u32		byte_4;
 	u32		msg_len;
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type in hip08

[kbuild test robot]

Hi Lijun,

I love your patch! Perhaps something to improve:

[auto build test WARNING on rdma/for-next]
[also build test WARNING on next-20180112]
[cannot apply to linus/master v4.15-rc7]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]

url:    https://github.com/0day-ci/linux/commits/Lijun-Ou/Add-CM-and-a-bugfix-to-hip08/20180112-215900
base:   https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
reproduce:
        # apt-get install sparse
        make ARCH=x86_64 allmodconfig
        make C=1 CF=-D__CHECK_ENDIAN__


sparse warnings: (new ones prefixed by >>)

   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:49:20: sparse: incorrect type in assignment (different base types) @@ expected restricted __be32 lkey @@ got restricted __be32 lkey @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:49:20: expected restricted __be32 lkey
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:49:20: got restricted __le32 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:50:20: sparse: incorrect type in assignment (different base types) @@ expected restricted __be64 addr @@ got restricted __be64 addr @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:50:20: expected restricted __be64 addr
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:50:20: got restricted __le64 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:51:20: sparse: incorrect type in assignment (different base types) @@ expected restricted __be32 len @@ got restricted __be32 len @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:51:20: expected restricted __be32 len
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:51:20: got restricted __le32 <noident>
>> drivers/infiniband/hw/hns/hns_roce_hw_v2.c:222:45: sparse: incorrect type in assignment (different base types) @@ expected unsigned int immtdata @@ got gned] immtdata @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:222:45: expected unsigned int immtdata
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:222:45: got restricted __be32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:263:25: sparse: cast from restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:282:25: sparse: restricted __le32 degrades to integer
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:315:53: sparse: incorrect type in assignment (different base types) @@ expected unsigned int inv_key_immtdata @@ got gned] inv_key_immtdata @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:315:53: expected unsigned int inv_key_immtdata
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:315:53: got restricted __be32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:338:49: sparse: incorrect type in assignment (different base types) @@ expected unsigned int rkey @@ got ed int rkey @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:338:49: expected unsigned int rkey
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:338:49: got restricted __le32 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:340:47: sparse: incorrect type in assignment (different base types) @@ expected unsigned long long va @@ got g long va @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:340:47: expected unsigned long long va
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:340:47: got restricted __le64 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:348:49: sparse: incorrect type in assignment (different base types) @@ expected unsigned int rkey @@ got ed int rkey @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:348:49: expected unsigned int rkey
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:348:49: got restricted __le32 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:350:47: sparse: incorrect type in assignment (different base types) @@ expected unsigned long long va @@ got g long va @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:350:47: expected unsigned long long va
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:350:47: got restricted __le64 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:358:49: sparse: incorrect type in assignment (different base types) @@ expected unsigned int rkey @@ got ed int rkey @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:358:49: expected unsigned int rkey
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:358:49: got restricted __le32 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:360:47: sparse: incorrect type in assignment (different base types) @@ expected unsigned long long va @@ got g long va @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:360:47: expected unsigned long long va
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:360:47: got restricted __le64 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:443:17: sparse: cast from restricted __le64
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:692:22: sparse: incorrect type in assignment (different base types) @@ expected unsigned short opcode @@ got short opcode @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:692:22: expected unsigned short opcode
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:692:22: got restricted __le16 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:693:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short flag @@ got short flag @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:693:20: expected unsigned short flag
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:693:20: got restricted __le16 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:696:28: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:696:28: left side has type unsigned short
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:696:28: right side has type restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:698:28: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:698:28: left side has type unsigned short
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:698:28: right side has type restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:832:26: sparse: cast to restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:832:26: sparse: cast from restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:833:29: sparse: cast to restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:833:29: sparse: cast from restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:870:38: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:870:38: left side has type unsigned short
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:870:38: right side has type restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:872:38: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:872:38: left side has type unsigned short
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:872:38: right side has type fouled restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:881:35: sparse: restricted __le32 degrades to integer
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:884:36: sparse: restricted __le32 degrades to integer
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:887:35: sparse: restricted __le32 degrades to integer
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:890:35: sparse: restricted __le32 degrades to integer
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:913:38: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:913:38: left side has type unsigned short
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:913:38: right side has type restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:915:38: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:915:38: left side has type unsigned short
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:915:38: right side has type fouled restricted __le16
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1202:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned long val @@ got restunsigned long val @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1202:22: expected unsigned long val
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1202:22: got restricted __le64 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1203:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned long val @@ got restunsigned long val @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1203:22: expected unsigned long val
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1203:22: got restricted __le64 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1208:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned int val @@ got restrunsigned int val @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1208:22: expected unsigned int val
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1208:22: got restricted __le32 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1209:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned int val @@ got restrunsigned int val @@
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1209:22: expected unsigned int val
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1209:22: got restricted __le32 <noident>
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: sparse: invalid assignment: &=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: right side has type unsigned long
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: sparse: invalid assignment: |=
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: left side has type restricted __le32
   drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: right side has type unsigned long

vim +222 drivers/infiniband/hw/hns/hns_roce_hw_v2.c

    45	
    46	static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
    47				    struct ib_sge *sg)
    48	{
  > 49		dseg->lkey = cpu_to_le32(sg->lkey);
    50		dseg->addr = cpu_to_le64(sg->addr);
    51		dseg->len  = cpu_to_le32(sg->length);
    52	}
    53	
    54	static int set_rwqe_data_seg(struct ib_qp *ibqp, struct ib_send_wr *wr,
    55				     struct hns_roce_v2_rc_send_wqe *rc_sq_wqe,
    56				     void *wqe, unsigned int *sge_ind,
    57				     struct ib_send_wr **bad_wr)
    58	{
    59		struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
    60		struct hns_roce_v2_wqe_data_seg *dseg = wqe;
    61		struct hns_roce_qp *qp = to_hr_qp(ibqp);
    62		int i;
    63	
    64		if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
    65			if (rc_sq_wqe->msg_len > hr_dev->caps.max_sq_inline) {
    66				*bad_wr = wr;
    67				dev_err(hr_dev->dev, "inline len(1-%d)=%d, illegal",
    68					rc_sq_wqe->msg_len, hr_dev->caps.max_sq_inline);
    69				return -EINVAL;
    70			}
    71	
    72			for (i = 0; i < wr->num_sge; i++) {
    73				memcpy(wqe, ((void *)wr->sg_list[i].addr),
    74				       wr->sg_list[i].length);
    75				wqe += wr->sg_list[i].length;
    76			}
    77	
    78			roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_INLINE_S,
    79				     1);
    80		} else {
    81			if (wr->num_sge <= 2) {
    82				for (i = 0; i < wr->num_sge; i++) {
    83					if (likely(wr->sg_list[i].length)) {
    84						set_data_seg_v2(dseg, wr->sg_list + i);
    85						dseg++;
    86					}
    87				}
    88			} else {
    89				roce_set_field(rc_sq_wqe->byte_20,
    90					     V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
    91					     V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
    92					     (*sge_ind) & (qp->sge.sge_cnt - 1));
    93	
    94				for (i = 0; i < 2; i++) {
    95					if (likely(wr->sg_list[i].length)) {
    96						set_data_seg_v2(dseg, wr->sg_list + i);
    97						dseg++;
    98					}
    99				}
   100	
   101				dseg = get_send_extend_sge(qp,
   102						    (*sge_ind) & (qp->sge.sge_cnt - 1));
   103	
   104				for (i = 0; i < wr->num_sge - 2; i++) {
   105					if (likely(wr->sg_list[i + 2].length)) {
   106						set_data_seg_v2(dseg,
   107								wr->sg_list + 2 + i);
   108						dseg++;
   109						(*sge_ind)++;
   110					}
   111				}
   112			}
   113	
   114			roce_set_field(rc_sq_wqe->byte_16,
   115				       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
   116				       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
   117		}
   118	
   119		return 0;
   120	}
   121	
   122	static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
   123					 struct ib_send_wr **bad_wr)
   124	{
   125		struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
   126		struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
   127		struct hns_roce_v2_ud_send_wqe *ud_sq_wqe;
   128		struct hns_roce_v2_rc_send_wqe *rc_sq_wqe;
   129		struct hns_roce_qp *qp = to_hr_qp(ibqp);
   130		struct hns_roce_v2_wqe_data_seg *dseg;
   131		struct device *dev = hr_dev->dev;
   132		struct hns_roce_v2_db sq_db;
   133		unsigned int sge_ind = 0;
   134		unsigned int owner_bit;
   135		unsigned long flags;
   136		unsigned int ind;
   137		void *wqe = NULL;
   138		bool loopback;
   139		int ret = 0;
   140		u8 *smac;
   141		int nreq;
   142		int i;
   143	
   144		if (unlikely(ibqp->qp_type != IB_QPT_RC &&
   145			     ibqp->qp_type != IB_QPT_GSI &&
   146			     ibqp->qp_type != IB_QPT_UD)) {
   147			dev_err(dev, "Not supported QP(0x%x)type!\n", ibqp->qp_type);
   148			*bad_wr = NULL;
   149			return -EOPNOTSUPP;
   150		}
   151	
   152		if (unlikely(qp->state == IB_QPS_RESET || qp->state == IB_QPS_INIT ||
   153			     qp->state == IB_QPS_RTR)) {
   154			dev_err(dev, "Post WQE fail, QP state %d err!\n", qp->state);
   155			*bad_wr = wr;
   156			return -EINVAL;
   157		}
   158	
   159		spin_lock_irqsave(&qp->sq.lock, flags);
   160		ind = qp->sq_next_wqe;
   161		sge_ind = qp->next_sge;
   162	
   163		for (nreq = 0; wr; ++nreq, wr = wr->next) {
   164			if (hns_roce_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) {
   165				ret = -ENOMEM;
   166				*bad_wr = wr;
   167				goto out;
   168			}
   169	
   170			if (unlikely(wr->num_sge > qp->sq.max_gs)) {
   171				dev_err(dev, "num_sge=%d > qp->sq.max_gs=%d\n",
   172					wr->num_sge, qp->sq.max_gs);
   173				ret = -EINVAL;
   174				*bad_wr = wr;
   175				goto out;
   176			}
   177	
   178			wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1));
   179			qp->sq.wrid[(qp->sq.head + nreq) & (qp->sq.wqe_cnt - 1)] =
   180									      wr->wr_id;
   181	
   182			owner_bit = ~(qp->sq.head >> ilog2(qp->sq.wqe_cnt)) & 0x1;
   183	
   184			/* Corresponding to the QP type, wqe process separately */
   185			if (ibqp->qp_type == IB_QPT_GSI) {
   186				ud_sq_wqe = wqe;
   187				memset(ud_sq_wqe, 0, sizeof(*ud_sq_wqe));
   188	
   189				roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_0_M,
   190					       V2_UD_SEND_WQE_DMAC_0_S, ah->av.mac[0]);
   191				roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_1_M,
   192					       V2_UD_SEND_WQE_DMAC_1_S, ah->av.mac[1]);
   193				roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_2_M,
   194					       V2_UD_SEND_WQE_DMAC_2_S, ah->av.mac[2]);
   195				roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_3_M,
   196					       V2_UD_SEND_WQE_DMAC_3_S, ah->av.mac[3]);
   197				roce_set_field(ud_sq_wqe->byte_48,
   198					       V2_UD_SEND_WQE_BYTE_48_DMAC_4_M,
   199					       V2_UD_SEND_WQE_BYTE_48_DMAC_4_S,
   200					       ah->av.mac[4]);
   201				roce_set_field(ud_sq_wqe->byte_48,
   202					       V2_UD_SEND_WQE_BYTE_48_DMAC_5_M,
   203					       V2_UD_SEND_WQE_BYTE_48_DMAC_5_S,
   204					       ah->av.mac[5]);
   205	
   206				/* MAC loopback */
   207				smac = (u8 *)hr_dev->dev_addr[qp->port];
   208				loopback = ether_addr_equal_unaligned(ah->av.mac,
   209								      smac) ? 1 : 0;
   210	
   211				roce_set_bit(ud_sq_wqe->byte_40,
   212					     V2_UD_SEND_WQE_BYTE_40_LBI_S, loopback);
   213	
   214				roce_set_field(ud_sq_wqe->byte_4,
   215					       V2_UD_SEND_WQE_BYTE_4_OPCODE_M,
   216					       V2_UD_SEND_WQE_BYTE_4_OPCODE_S,
   217					       HNS_ROCE_V2_WQE_OP_SEND);
   218	
   219				for (i = 0; i < wr->num_sge; i++)
   220					ud_sq_wqe->msg_len += wr->sg_list[i].length;
   221	
 > 222				ud_sq_wqe->immtdata = send_ieth(wr);
   223	
   224				/* Set sig attr */
   225				roce_set_bit(ud_sq_wqe->byte_4,
   226					   V2_UD_SEND_WQE_BYTE_4_CQE_S,
   227					   (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
   228	
   229				/* Set se attr */
   230				roce_set_bit(ud_sq_wqe->byte_4,
   231					  V2_UD_SEND_WQE_BYTE_4_SE_S,
   232					  (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
   233	
   234				roce_set_bit(ud_sq_wqe->byte_4,
   235					     V2_UD_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
   236	
   237				roce_set_field(ud_sq_wqe->byte_16,
   238					       V2_UD_SEND_WQE_BYTE_16_PD_M,
   239					       V2_UD_SEND_WQE_BYTE_16_PD_S,
   240					       to_hr_pd(ibqp->pd)->pdn);
   241	
   242				roce_set_field(ud_sq_wqe->byte_16,
   243					       V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M,
   244					       V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S,
   245					       wr->num_sge);
   246	
   247				roce_set_field(ud_sq_wqe->byte_20,
   248					     V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
   249					     V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
   250					     sge_ind & (qp->sge.sge_cnt - 1));
   251	
   252				roce_set_field(ud_sq_wqe->byte_24,
   253					       V2_UD_SEND_WQE_BYTE_24_UDPSPN_M,
   254					       V2_UD_SEND_WQE_BYTE_24_UDPSPN_S, 0);
   255				ud_sq_wqe->qkey =
   256				     cpu_to_be32(ud_wr(wr)->remote_qkey & 0x80000000) ?
   257				     qp->qkey : ud_wr(wr)->remote_qkey;
   258				roce_set_field(ud_sq_wqe->byte_32,
   259					       V2_UD_SEND_WQE_BYTE_32_DQPN_M,
   260					       V2_UD_SEND_WQE_BYTE_32_DQPN_S,
   261					       ud_wr(wr)->remote_qpn);
   262	
   263				roce_set_field(ud_sq_wqe->byte_36,
   264					       V2_UD_SEND_WQE_BYTE_36_VLAN_M,
   265					       V2_UD_SEND_WQE_BYTE_36_VLAN_S,
   266					       ah->av.vlan);
   267				roce_set_field(ud_sq_wqe->byte_36,
   268					       V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M,
   269					       V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S,
   270					       ah->av.hop_limit);
   271				roce_set_field(ud_sq_wqe->byte_36,
   272					       V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
   273					       V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
   274					       0);
   275				roce_set_field(ud_sq_wqe->byte_36,
   276					       V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
   277					       V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
   278					       0);
   279				roce_set_field(ud_sq_wqe->byte_40,
   280					       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
   281					       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, 0);
   282				roce_set_field(ud_sq_wqe->byte_40,
   283					       V2_UD_SEND_WQE_BYTE_40_SL_M,
   284					       V2_UD_SEND_WQE_BYTE_40_SL_S,
   285					       ah->av.sl_tclass_flowlabel >>
   286					       HNS_ROCE_SL_SHIFT);
   287				roce_set_field(ud_sq_wqe->byte_40,
   288					       V2_UD_SEND_WQE_BYTE_40_PORTN_M,
   289					       V2_UD_SEND_WQE_BYTE_40_PORTN_S,
   290					       qp->port);
   291	
   292				roce_set_field(ud_sq_wqe->byte_48,
   293					       V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M,
   294					       V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S,
   295					       hns_get_gid_index(hr_dev, qp->phy_port,
   296								 ah->av.gid_index));
   297	
   298				memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0],
   299				       GID_LEN_V2);
   300	
   301				dseg = get_send_extend_sge(qp,
   302						    sge_ind & (qp->sge.sge_cnt - 1));
   303				for (i = 0; i < wr->num_sge; i++) {
   304					set_data_seg_v2(dseg + i, wr->sg_list + i);
   305					sge_ind++;
   306				}
   307	
   308				ind++;
   309			} else if (ibqp->qp_type == IB_QPT_RC) {
   310				rc_sq_wqe = wqe;
   311				memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
   312				for (i = 0; i < wr->num_sge; i++)
   313					rc_sq_wqe->msg_len += wr->sg_list[i].length;
   314	
   315				rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
   316	
   317				roce_set_bit(rc_sq_wqe->byte_4,
   318					     V2_RC_SEND_WQE_BYTE_4_FENCE_S,
   319					     (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
   320	
   321				roce_set_bit(rc_sq_wqe->byte_4,
   322					  V2_RC_SEND_WQE_BYTE_4_SE_S,
   323					  (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
   324	
   325				roce_set_bit(rc_sq_wqe->byte_4,
   326					   V2_RC_SEND_WQE_BYTE_4_CQE_S,
   327					   (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
   328	
   329				roce_set_bit(rc_sq_wqe->byte_4,
   330					     V2_RC_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
   331	
   332				switch (wr->opcode) {
   333				case IB_WR_RDMA_READ:
   334					roce_set_field(rc_sq_wqe->byte_4,
   335						       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   336						       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   337						       HNS_ROCE_V2_WQE_OP_RDMA_READ);
   338					rc_sq_wqe->rkey =
   339						cpu_to_le32(rdma_wr(wr)->rkey);
   340					rc_sq_wqe->va =
   341						cpu_to_le64(rdma_wr(wr)->remote_addr);
   342					break;
   343				case IB_WR_RDMA_WRITE:
   344					roce_set_field(rc_sq_wqe->byte_4,
   345						       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   346						       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   347						       HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
   348					rc_sq_wqe->rkey =
   349						cpu_to_le32(rdma_wr(wr)->rkey);
   350					rc_sq_wqe->va =
   351						cpu_to_le64(rdma_wr(wr)->remote_addr);
   352					break;
   353				case IB_WR_RDMA_WRITE_WITH_IMM:
   354					roce_set_field(rc_sq_wqe->byte_4,
   355					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   356					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   357					       HNS_ROCE_V2_WQE_OP_RDMA_WRITE_WITH_IMM);
   358					rc_sq_wqe->rkey =
   359						cpu_to_le32(rdma_wr(wr)->rkey);
   360					rc_sq_wqe->va =
   361						cpu_to_le64(rdma_wr(wr)->remote_addr);
   362					break;
   363				case IB_WR_SEND:
   364					roce_set_field(rc_sq_wqe->byte_4,
   365						       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   366						       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   367						       HNS_ROCE_V2_WQE_OP_SEND);
   368					break;
   369				case IB_WR_SEND_WITH_INV:
   370					roce_set_field(rc_sq_wqe->byte_4,
   371					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   372					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   373					       HNS_ROCE_V2_WQE_OP_SEND_WITH_INV);
   374					break;
   375				case IB_WR_SEND_WITH_IMM:
   376					roce_set_field(rc_sq_wqe->byte_4,
   377						      V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   378						      V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   379						      HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
   380					break;
   381				case IB_WR_LOCAL_INV:
   382					roce_set_field(rc_sq_wqe->byte_4,
   383						       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   384						       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   385						       HNS_ROCE_V2_WQE_OP_LOCAL_INV);
   386					break;
   387				case IB_WR_ATOMIC_CMP_AND_SWP:
   388					roce_set_field(rc_sq_wqe->byte_4,
   389						  V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   390						  V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   391						  HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
   392					break;
   393				case IB_WR_ATOMIC_FETCH_AND_ADD:
   394					roce_set_field(rc_sq_wqe->byte_4,
   395						 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   396						 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   397						 HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
   398					break;
   399				case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
   400					roce_set_field(rc_sq_wqe->byte_4,
   401					      V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   402					      V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   403					      HNS_ROCE_V2_WQE_OP_ATOM_MSK_CMP_AND_SWAP);
   404					break;
   405				case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
   406					roce_set_field(rc_sq_wqe->byte_4,
   407					     V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   408					     V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   409					     HNS_ROCE_V2_WQE_OP_ATOM_MSK_FETCH_AND_ADD);
   410					break;
   411				default:
   412					roce_set_field(rc_sq_wqe->byte_4,
   413						       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
   414						       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
   415						       HNS_ROCE_V2_WQE_OP_MASK);
   416					break;
   417				}
   418	
   419				wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
   420				dseg = wqe;
   421	
   422				ret = set_rwqe_data_seg(ibqp, wr, rc_sq_wqe, wqe,
   423							&sge_ind, bad_wr);
   424				if (ret)
   425					goto out;
   426				ind++;
   427			} else {
   428				dev_err(dev, "Illegal qp_type(0x%x)\n", ibqp->qp_type);
   429				spin_unlock_irqrestore(&qp->sq.lock, flags);
   430				return -EOPNOTSUPP;
   431			}
   432		}
   433	
   434	out:
   435		if (likely(nreq)) {
   436			qp->sq.head += nreq;
   437			/* Memory barrier */
   438			wmb();
   439	
   440			sq_db.byte_4 = 0;
   441			sq_db.parameter = 0;
   442	
   443			roce_set_field(sq_db.byte_4, V2_DB_BYTE_4_TAG_M,
   444				       V2_DB_BYTE_4_TAG_S, qp->doorbell_qpn);
   445			roce_set_field(sq_db.byte_4, V2_DB_BYTE_4_CMD_M,
   446				       V2_DB_BYTE_4_CMD_S, HNS_ROCE_V2_SQ_DB);
   447			roce_set_field(sq_db.parameter, V2_DB_PARAMETER_CONS_IDX_M,
   448				       V2_DB_PARAMETER_CONS_IDX_S,
   449				       qp->sq.head & ((qp->sq.wqe_cnt << 1) - 1));
   450			roce_set_field(sq_db.parameter, V2_DB_PARAMETER_SL_M,
   451				       V2_DB_PARAMETER_SL_S, qp->sl);
   452	
   453			hns_roce_write64_k((__be32 *)&sq_db, qp->sq.db_reg_l);
   454	
   455			qp->sq_next_wqe = ind;
   456			qp->next_sge = sge_ind;
   457		}
   458	
   459		spin_unlock_irqrestore(&qp->sq.lock, flags);
   460	
   461		return ret;
   462	}
   463	

---
0-DAY kernel test infrastructure                Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all                   Intel Corporation
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type in hip08

[oulijun]

Hi,  Dledford and Jason
  I have awared the warnings by kbuild test robot. It may be not intrdouced
by this patchset. I have doubt that the kernel code may have  the same
question with sparse check when process the userspace according to the following patch:
  See rdma-core commit bffd380cfe56 ("libhns: Make the provider sparse
clean")

I have been planing to fix it in next fixes patch. Could I send a flowup patch separately?

Thanks.
Lijun Ou
在 2018/1/10 14:39, Lijun Ou 写道:
> This patch mainly configure the fileds of sq wqe of ud
> type when posting wr of gsi qp type.
> 
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> V1->V2:
> - return errno code directly and delete unnecessary
>   initialized value of ret from Leon Romanvosky's
>   advice.
> - Add necessary option for free sq lock when qp is
>   illegal qp type.
> 
> V1:
> - The initial submit
> ---
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 446 +++++++++++++++++++----------
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.h |  84 ++++++
>  2 files changed, 385 insertions(+), 145 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> index e53cd7d..cb3ac54 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> @@ -51,26 +51,99 @@ static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
>  	dseg->len  = cpu_to_le32(sg->length);
>  }
>  
> +static int set_rwqe_data_seg(struct ib_qp *ibqp, struct ib_send_wr *wr,
> +			     struct hns_roce_v2_rc_send_wqe *rc_sq_wqe,
> +			     void *wqe, unsigned int *sge_ind,
> +			     struct ib_send_wr **bad_wr)
> +{
> +	struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> +	struct hns_roce_v2_wqe_data_seg *dseg = wqe;
> +	struct hns_roce_qp *qp = to_hr_qp(ibqp);
> +	int i;
> +
> +	if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
> +		if (rc_sq_wqe->msg_len > hr_dev->caps.max_sq_inline) {
> +			*bad_wr = wr;
> +			dev_err(hr_dev->dev, "inline len(1-%d)=%d, illegal",
> +				rc_sq_wqe->msg_len, hr_dev->caps.max_sq_inline);
> +			return -EINVAL;
> +		}
> +
> +		for (i = 0; i < wr->num_sge; i++) {
> +			memcpy(wqe, ((void *)wr->sg_list[i].addr),
> +			       wr->sg_list[i].length);
> +			wqe += wr->sg_list[i].length;
> +		}
> +
> +		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_INLINE_S,
> +			     1);
> +	} else {
> +		if (wr->num_sge <= 2) {
> +			for (i = 0; i < wr->num_sge; i++) {
> +				if (likely(wr->sg_list[i].length)) {
> +					set_data_seg_v2(dseg, wr->sg_list + i);
> +					dseg++;
> +				}
> +			}
> +		} else {
> +			roce_set_field(rc_sq_wqe->byte_20,
> +				     V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
> +				     V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
> +				     (*sge_ind) & (qp->sge.sge_cnt - 1));
> +
> +			for (i = 0; i < 2; i++) {
> +				if (likely(wr->sg_list[i].length)) {
> +					set_data_seg_v2(dseg, wr->sg_list + i);
> +					dseg++;
> +				}
> +			}
> +
> +			dseg = get_send_extend_sge(qp,
> +					    (*sge_ind) & (qp->sge.sge_cnt - 1));
> +
> +			for (i = 0; i < wr->num_sge - 2; i++) {
> +				if (likely(wr->sg_list[i + 2].length)) {
> +					set_data_seg_v2(dseg,
> +							wr->sg_list + 2 + i);
> +					dseg++;
> +					(*sge_ind)++;
> +				}
> +			}
> +		}
> +
> +		roce_set_field(rc_sq_wqe->byte_16,
> +			       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
> +			       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
> +	}
> +
> +	return 0;
> +}
> +
>  static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
>  				 struct ib_send_wr **bad_wr)
>  {
>  	struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> +	struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
> +	struct hns_roce_v2_ud_send_wqe *ud_sq_wqe;
>  	struct hns_roce_v2_rc_send_wqe *rc_sq_wqe;
>  	struct hns_roce_qp *qp = to_hr_qp(ibqp);
>  	struct hns_roce_v2_wqe_data_seg *dseg;
>  	struct device *dev = hr_dev->dev;
>  	struct hns_roce_v2_db sq_db;
>  	unsigned int sge_ind = 0;
> -	unsigned int wqe_sz = 0;
>  	unsigned int owner_bit;
>  	unsigned long flags;
>  	unsigned int ind;
>  	void *wqe = NULL;
> +	bool loopback;
>  	int ret = 0;
> +	u8 *smac;
>  	int nreq;
>  	int i;
>  
> -	if (unlikely(ibqp->qp_type != IB_QPT_RC)) {
> +	if (unlikely(ibqp->qp_type != IB_QPT_RC &&
> +		     ibqp->qp_type != IB_QPT_GSI &&
> +		     ibqp->qp_type != IB_QPT_UD)) {
>  		dev_err(dev, "Not supported QP(0x%x)type!\n", ibqp->qp_type);
>  		*bad_wr = NULL;
>  		return -EOPNOTSUPP;
> @@ -107,172 +180,255 @@ static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
>  								      wr->wr_id;
>  
>  		owner_bit = ~(qp->sq.head >> ilog2(qp->sq.wqe_cnt)) & 0x1;
> -		rc_sq_wqe = wqe;
> -		memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
> -		for (i = 0; i < wr->num_sge; i++)
> -			rc_sq_wqe->msg_len += wr->sg_list[i].length;
>  
> -		rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
> +		/* Corresponding to the QP type, wqe process separately */
> +		if (ibqp->qp_type == IB_QPT_GSI) {
> +			ud_sq_wqe = wqe;
> +			memset(ud_sq_wqe, 0, sizeof(*ud_sq_wqe));
> +
> +			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_0_M,
> +				       V2_UD_SEND_WQE_DMAC_0_S, ah->av.mac[0]);
> +			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_1_M,
> +				       V2_UD_SEND_WQE_DMAC_1_S, ah->av.mac[1]);
> +			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_2_M,
> +				       V2_UD_SEND_WQE_DMAC_2_S, ah->av.mac[2]);
> +			roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_3_M,
> +				       V2_UD_SEND_WQE_DMAC_3_S, ah->av.mac[3]);
> +			roce_set_field(ud_sq_wqe->byte_48,
> +				       V2_UD_SEND_WQE_BYTE_48_DMAC_4_M,
> +				       V2_UD_SEND_WQE_BYTE_48_DMAC_4_S,
> +				       ah->av.mac[4]);
> +			roce_set_field(ud_sq_wqe->byte_48,
> +				       V2_UD_SEND_WQE_BYTE_48_DMAC_5_M,
> +				       V2_UD_SEND_WQE_BYTE_48_DMAC_5_S,
> +				       ah->av.mac[5]);
> +
> +			/* MAC loopback */
> +			smac = (u8 *)hr_dev->dev_addr[qp->port];
> +			loopback = ether_addr_equal_unaligned(ah->av.mac,
> +							      smac) ? 1 : 0;
> +
> +			roce_set_bit(ud_sq_wqe->byte_40,
> +				     V2_UD_SEND_WQE_BYTE_40_LBI_S, loopback);
> +
> +			roce_set_field(ud_sq_wqe->byte_4,
> +				       V2_UD_SEND_WQE_BYTE_4_OPCODE_M,
> +				       V2_UD_SEND_WQE_BYTE_4_OPCODE_S,
> +				       HNS_ROCE_V2_WQE_OP_SEND);
>  
> -		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_FENCE_S,
> -			    (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
> +			for (i = 0; i < wr->num_sge; i++)
> +				ud_sq_wqe->msg_len += wr->sg_list[i].length;
>  
> -		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_SE_S,
> -			    (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
> +			ud_sq_wqe->immtdata = send_ieth(wr);
>  
> -		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_CQE_S,
> -			    (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
> +			/* Set sig attr */
> +			roce_set_bit(ud_sq_wqe->byte_4,
> +				   V2_UD_SEND_WQE_BYTE_4_CQE_S,
> +				   (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
>  
> -		roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_OWNER_S,
> -			     owner_bit);
> +			/* Set se attr */
> +			roce_set_bit(ud_sq_wqe->byte_4,
> +				  V2_UD_SEND_WQE_BYTE_4_SE_S,
> +				  (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
>  
> -		switch (wr->opcode) {
> -		case IB_WR_RDMA_READ:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_RDMA_READ);
> -			rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
> -			rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
> -			break;
> -		case IB_WR_RDMA_WRITE:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
> -			rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
> -			rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
> -			break;
> -		case IB_WR_RDMA_WRITE_WITH_IMM:
> -			roce_set_field(rc_sq_wqe->byte_4,
> +			roce_set_bit(ud_sq_wqe->byte_4,
> +				     V2_UD_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
> +
> +			roce_set_field(ud_sq_wqe->byte_16,
> +				       V2_UD_SEND_WQE_BYTE_16_PD_M,
> +				       V2_UD_SEND_WQE_BYTE_16_PD_S,
> +				       to_hr_pd(ibqp->pd)->pdn);
> +
> +			roce_set_field(ud_sq_wqe->byte_16,
> +				       V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M,
> +				       V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S,
> +				       wr->num_sge);
> +
> +			roce_set_field(ud_sq_wqe->byte_20,
> +				     V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
> +				     V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
> +				     sge_ind & (qp->sge.sge_cnt - 1));
> +
> +			roce_set_field(ud_sq_wqe->byte_24,
> +				       V2_UD_SEND_WQE_BYTE_24_UDPSPN_M,
> +				       V2_UD_SEND_WQE_BYTE_24_UDPSPN_S, 0);
> +			ud_sq_wqe->qkey =
> +			     cpu_to_be32(ud_wr(wr)->remote_qkey & 0x80000000) ?
> +			     qp->qkey : ud_wr(wr)->remote_qkey;
> +			roce_set_field(ud_sq_wqe->byte_32,
> +				       V2_UD_SEND_WQE_BYTE_32_DQPN_M,
> +				       V2_UD_SEND_WQE_BYTE_32_DQPN_S,
> +				       ud_wr(wr)->remote_qpn);
> +
> +			roce_set_field(ud_sq_wqe->byte_36,
> +				       V2_UD_SEND_WQE_BYTE_36_VLAN_M,
> +				       V2_UD_SEND_WQE_BYTE_36_VLAN_S,
> +				       ah->av.vlan);
> +			roce_set_field(ud_sq_wqe->byte_36,
> +				       V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M,
> +				       V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S,
> +				       ah->av.hop_limit);
> +			roce_set_field(ud_sq_wqe->byte_36,
> +				       V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
> +				       V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
> +				       0);
> +			roce_set_field(ud_sq_wqe->byte_36,
> +				       V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
> +				       V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
> +				       0);
> +			roce_set_field(ud_sq_wqe->byte_40,
> +				       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
> +				       V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, 0);
> +			roce_set_field(ud_sq_wqe->byte_40,
> +				       V2_UD_SEND_WQE_BYTE_40_SL_M,
> +				       V2_UD_SEND_WQE_BYTE_40_SL_S,
> +				       ah->av.sl_tclass_flowlabel >>
> +				       HNS_ROCE_SL_SHIFT);
> +			roce_set_field(ud_sq_wqe->byte_40,
> +				       V2_UD_SEND_WQE_BYTE_40_PORTN_M,
> +				       V2_UD_SEND_WQE_BYTE_40_PORTN_S,
> +				       qp->port);
> +
> +			roce_set_field(ud_sq_wqe->byte_48,
> +				       V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M,
> +				       V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S,
> +				       hns_get_gid_index(hr_dev, qp->phy_port,
> +							 ah->av.gid_index));
> +
> +			memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0],
> +			       GID_LEN_V2);
> +
> +			dseg = get_send_extend_sge(qp,
> +					    sge_ind & (qp->sge.sge_cnt - 1));
> +			for (i = 0; i < wr->num_sge; i++) {
> +				set_data_seg_v2(dseg + i, wr->sg_list + i);
> +				sge_ind++;
> +			}
> +
> +			ind++;
> +		} else if (ibqp->qp_type == IB_QPT_RC) {
> +			rc_sq_wqe = wqe;
> +			memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
> +			for (i = 0; i < wr->num_sge; i++)
> +				rc_sq_wqe->msg_len += wr->sg_list[i].length;
> +
> +			rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
> +
> +			roce_set_bit(rc_sq_wqe->byte_4,
> +				     V2_RC_SEND_WQE_BYTE_4_FENCE_S,
> +				     (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
> +
> +			roce_set_bit(rc_sq_wqe->byte_4,
> +				  V2_RC_SEND_WQE_BYTE_4_SE_S,
> +				  (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
> +
> +			roce_set_bit(rc_sq_wqe->byte_4,
> +				   V2_RC_SEND_WQE_BYTE_4_CQE_S,
> +				   (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
> +
> +			roce_set_bit(rc_sq_wqe->byte_4,
> +				     V2_RC_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
> +
> +			switch (wr->opcode) {
> +			case IB_WR_RDMA_READ:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					       HNS_ROCE_V2_WQE_OP_RDMA_READ);
> +				rc_sq_wqe->rkey =
> +					cpu_to_le32(rdma_wr(wr)->rkey);
> +				rc_sq_wqe->va =
> +					cpu_to_le64(rdma_wr(wr)->remote_addr);
> +				break;
> +			case IB_WR_RDMA_WRITE:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					       HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
> +				rc_sq_wqe->rkey =
> +					cpu_to_le32(rdma_wr(wr)->rkey);
> +				rc_sq_wqe->va =
> +					cpu_to_le64(rdma_wr(wr)->remote_addr);
> +				break;
> +			case IB_WR_RDMA_WRITE_WITH_IMM:
> +				roce_set_field(rc_sq_wqe->byte_4,
>  				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
>  				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
>  				       HNS_ROCE_V2_WQE_OP_RDMA_WRITE_WITH_IMM);
> -			rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
> -			rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
> -			break;
> -		case IB_WR_SEND:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_SEND);
> -			break;
> -		case IB_WR_SEND_WITH_INV:
> -			roce_set_field(rc_sq_wqe->byte_4,
> +				rc_sq_wqe->rkey =
> +					cpu_to_le32(rdma_wr(wr)->rkey);
> +				rc_sq_wqe->va =
> +					cpu_to_le64(rdma_wr(wr)->remote_addr);
> +				break;
> +			case IB_WR_SEND:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					       HNS_ROCE_V2_WQE_OP_SEND);
> +				break;
> +			case IB_WR_SEND_WITH_INV:
> +				roce_set_field(rc_sq_wqe->byte_4,
>  				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
>  				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
>  				       HNS_ROCE_V2_WQE_OP_SEND_WITH_INV);
> -			break;
> -		case IB_WR_SEND_WITH_IMM:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
> -			break;
> -		case IB_WR_LOCAL_INV:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_LOCAL_INV);
> -			break;
> -		case IB_WR_ATOMIC_CMP_AND_SWP:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
> -			break;
> -		case IB_WR_ATOMIC_FETCH_AND_ADD:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
> -			break;
> -		case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
> -			roce_set_field(rc_sq_wqe->byte_4,
> +				break;
> +			case IB_WR_SEND_WITH_IMM:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					      V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					      V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					      HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
> +				break;
> +			case IB_WR_LOCAL_INV:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					       HNS_ROCE_V2_WQE_OP_LOCAL_INV);
> +				break;
> +			case IB_WR_ATOMIC_CMP_AND_SWP:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					  V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					  V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					  HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
> +				break;
> +			case IB_WR_ATOMIC_FETCH_AND_ADD:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					 HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
> +				break;
> +			case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
> +				roce_set_field(rc_sq_wqe->byte_4,
>  				      V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
>  				      V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
>  				      HNS_ROCE_V2_WQE_OP_ATOM_MSK_CMP_AND_SWAP);
> -			break;
> -		case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
> -			roce_set_field(rc_sq_wqe->byte_4,
> +				break;
> +			case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
> +				roce_set_field(rc_sq_wqe->byte_4,
>  				     V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
>  				     V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
>  				     HNS_ROCE_V2_WQE_OP_ATOM_MSK_FETCH_AND_ADD);
> -			break;
> -		default:
> -			roce_set_field(rc_sq_wqe->byte_4,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> -				       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> -				       HNS_ROCE_V2_WQE_OP_MASK);
> -			break;
> -		}
> -
> -		wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
> -		dseg = wqe;
> -		if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
> -			if (rc_sq_wqe->msg_len >
> -				hr_dev->caps.max_sq_inline) {
> -				ret = -EINVAL;
> -				*bad_wr = wr;
> -				dev_err(dev, "inline len(1-%d)=%d, illegal",
> -					rc_sq_wqe->msg_len,
> -					hr_dev->caps.max_sq_inline);
> -				goto out;
> +				break;
> +			default:
> +				roce_set_field(rc_sq_wqe->byte_4,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> +					       V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> +					       HNS_ROCE_V2_WQE_OP_MASK);
> +				break;
>  			}
>  
> -			for (i = 0; i < wr->num_sge; i++) {
> -				memcpy(wqe, ((void *)wr->sg_list[i].addr),
> -				       wr->sg_list[i].length);
> -				wqe += wr->sg_list[i].length;
> -				wqe_sz += wr->sg_list[i].length;
> -			}
> +			wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
> +			dseg = wqe;
>  
> -			roce_set_bit(rc_sq_wqe->byte_4,
> -				     V2_RC_SEND_WQE_BYTE_4_INLINE_S, 1);
> +			ret = set_rwqe_data_seg(ibqp, wr, rc_sq_wqe, wqe,
> +						&sge_ind, bad_wr);
> +			if (ret)
> +				goto out;
> +			ind++;
>  		} else {
> -			if (wr->num_sge <= 2) {
> -				for (i = 0; i < wr->num_sge; i++) {
> -					if (likely(wr->sg_list[i].length)) {
> -						set_data_seg_v2(dseg,
> -							       wr->sg_list + i);
> -						dseg++;
> -					}
> -				}
> -			} else {
> -				roce_set_field(rc_sq_wqe->byte_20,
> -				V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
> -				V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
> -				sge_ind & (qp->sge.sge_cnt - 1));
> -
> -				for (i = 0; i < 2; i++) {
> -					if (likely(wr->sg_list[i].length)) {
> -						set_data_seg_v2(dseg,
> -							       wr->sg_list + i);
> -						dseg++;
> -					}
> -				}
> -
> -				dseg = get_send_extend_sge(qp,
> -					sge_ind & (qp->sge.sge_cnt - 1));
> -
> -				for (i = 0; i < wr->num_sge - 2; i++) {
> -					if (likely(wr->sg_list[i + 2].length)) {
> -						set_data_seg_v2(dseg,
> -							   wr->sg_list + 2 + i);
> -						dseg++;
> -						sge_ind++;
> -					}
> -				}
> -			}
> -
> -			roce_set_field(rc_sq_wqe->byte_16,
> -				       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
> -				       V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S,
> -				       wr->num_sge);
> -			wqe_sz += wr->num_sge *
> -				  sizeof(struct hns_roce_v2_wqe_data_seg);
> +			dev_err(dev, "Illegal qp_type(0x%x)\n", ibqp->qp_type);
> +			spin_unlock_irqrestore(&qp->sq.lock, flags);
> +			return -EOPNOTSUPP;
>  		}
> -		ind++;
>  	}
>  
>  out:
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
> index 463edab..c11b253 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
> @@ -916,6 +916,90 @@ struct hns_roce_v2_cq_db {
>  
>  #define V2_CQ_DB_PARAMETER_NOTIFY_S 24
>  
> +struct hns_roce_v2_ud_send_wqe {
> +	u32	byte_4;
> +	u32	msg_len;
> +	u32	immtdata;
> +	u32	byte_16;
> +	u32	byte_20;
> +	u32	byte_24;
> +	u32	qkey;
> +	u32	byte_32;
> +	u32	byte_36;
> +	u32	byte_40;
> +	u32	dmac;
> +	u32	byte_48;
> +	u8	dgid[GID_LEN_V2];
> +
> +};
> +#define	V2_UD_SEND_WQE_BYTE_4_OPCODE_S 0
> +#define V2_UD_SEND_WQE_BYTE_4_OPCODE_M GENMASK(4, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_4_OWNER_S 7
> +
> +#define	V2_UD_SEND_WQE_BYTE_4_CQE_S 8
> +
> +#define	V2_UD_SEND_WQE_BYTE_4_SE_S 11
> +
> +#define	V2_UD_SEND_WQE_BYTE_16_PD_S 0
> +#define V2_UD_SEND_WQE_BYTE_16_PD_M GENMASK(23, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S 24
> +#define V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M GENMASK(31, 24)
> +
> +#define	V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S 0
> +#define V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M GENMASK(23, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_24_UDPSPN_S 16
> +#define V2_UD_SEND_WQE_BYTE_24_UDPSPN_M GENMASK(31, 16)
> +
> +#define	V2_UD_SEND_WQE_BYTE_32_DQPN_S 0
> +#define V2_UD_SEND_WQE_BYTE_32_DQPN_M GENMASK(23, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_36_VLAN_S 0
> +#define V2_UD_SEND_WQE_BYTE_36_VLAN_M GENMASK(15, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S 16
> +#define V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M GENMASK(23, 16)
> +
> +#define	V2_UD_SEND_WQE_BYTE_36_TCLASS_S 24
> +#define V2_UD_SEND_WQE_BYTE_36_TCLASS_M GENMASK(31, 24)
> +
> +#define	V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S 0
> +#define V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M GENMASK(19, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_40_SL_S 20
> +#define V2_UD_SEND_WQE_BYTE_40_SL_M GENMASK(23, 20)
> +
> +#define	V2_UD_SEND_WQE_BYTE_40_PORTN_S 24
> +#define V2_UD_SEND_WQE_BYTE_40_PORTN_M GENMASK(26, 24)
> +
> +#define	V2_UD_SEND_WQE_BYTE_40_LBI_S 31
> +
> +#define	V2_UD_SEND_WQE_DMAC_0_S 0
> +#define V2_UD_SEND_WQE_DMAC_0_M GENMASK(7, 0)
> +
> +#define	V2_UD_SEND_WQE_DMAC_1_S 8
> +#define V2_UD_SEND_WQE_DMAC_1_M GENMASK(15, 8)
> +
> +#define	V2_UD_SEND_WQE_DMAC_2_S 16
> +#define V2_UD_SEND_WQE_DMAC_2_M GENMASK(23, 16)
> +
> +#define	V2_UD_SEND_WQE_DMAC_3_S 24
> +#define V2_UD_SEND_WQE_DMAC_3_M GENMASK(31, 24)
> +
> +#define	V2_UD_SEND_WQE_BYTE_48_DMAC_4_S 0
> +#define V2_UD_SEND_WQE_BYTE_48_DMAC_4_M GENMASK(7, 0)
> +
> +#define	V2_UD_SEND_WQE_BYTE_48_DMAC_5_S 8
> +#define V2_UD_SEND_WQE_BYTE_48_DMAC_5_M GENMASK(15, 8)
> +
> +#define	V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S 16
> +#define V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M GENMASK(23, 16)
> +
> +#define	V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_S 24
> +#define V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_M GENMASK(31, 24)
> +
>  struct hns_roce_v2_rc_send_wqe {
>  	u32		byte_4;
>  	u32		msg_len;
> 


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 5/7] RDMA/hns: Assign zero for pkey_index of wc in hip08

[Lijun Ou]

Because pkey is fixed for hip08 RoCE, it needs to assign zero
for pkey_index of wc. otherwise, it will happen an error when
establishing connection by communication management mechanism.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V2:
- No fixed

V1:
- The initial submit
---
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index cb3ac54..38fb4c7 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1910,6 +1910,9 @@ static int hns_roce_v2_poll_one(struct hns_roce_cq *hr_cq,
 		wc->wc_flags |= (roce_get_bit(cqe->byte_32,
 					      V2_CQE_BYTE_32_GRH_S) ?
 					      IB_WC_GRH : 0);
+		wc->port_num = roce_get_field(cqe->byte_32,
+				V2_CQE_BYTE_32_PORTN_M, V2_CQE_BYTE_32_PORTN_S);
+		wc->pkey_index = 0;
 	}
 
 	return 0;
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 6/7] RDMA/hns: Update the verbs of polling for completion

[Lijun Ou]

If the port is a RoCEv2 port, the remote port address and QP
information which returned for UD will be modified.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V1->V2:
- No fixed

V1:
- The initial submit
---
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++++++++++++
 drivers/infiniband/hw/hns/hns_roce_hw_v2.h |  2 +-
 2 files changed, 13 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 38fb4c7..a1839a3 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1913,6 +1913,18 @@ static int hns_roce_v2_poll_one(struct hns_roce_cq *hr_cq,
 		wc->port_num = roce_get_field(cqe->byte_32,
 				V2_CQE_BYTE_32_PORTN_M, V2_CQE_BYTE_32_PORTN_S);
 		wc->pkey_index = 0;
+		memcpy(wc->smac, cqe->smac, 4);
+		wc->smac[4] = roce_get_field(cqe->byte_28,
+					     V2_CQE_BYTE_28_SMAC_4_M,
+					     V2_CQE_BYTE_28_SMAC_4_S);
+		wc->smac[5] = roce_get_field(cqe->byte_28,
+					     V2_CQE_BYTE_28_SMAC_5_M,
+					     V2_CQE_BYTE_28_SMAC_5_S);
+		wc->vlan_id = 0xffff;
+		wc->wc_flags |= (IB_WC_WITH_VLAN | IB_WC_WITH_SMAC);
+		wc->network_hdr_type = roce_get_field(cqe->byte_28,
+						    V2_CQE_BYTE_28_PORT_TYPE_M,
+						    V2_CQE_BYTE_28_PORT_TYPE_S);
 	}
 
 	return 0;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
index c11b253..ce52c73 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
@@ -766,7 +766,7 @@ struct hns_roce_v2_cqe {
 	u32	byte_12;
 	u32	byte_16;
 	u32	byte_cnt;
-	u32	smac;
+	u8	smac[4];
 	u32	byte_28;
 	u32	byte_32;
 };
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[Lijun Ou]

This patch assgin a guid(Global Unique identifer)
value to the hip08 device.

Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V2:
- Use the correct way to update guid from Leon
  Romanovsky's review.

V1:
- The initial submit
---
 drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index a1839a3..2af4948 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -34,6 +34,7 @@
 #include <linux/etherdevice.h>
 #include <linux/interrupt.h>
 #include <linux/kernel.h>
+#include <net/addrconf.h>
 #include <rdma/ib_umem.h>
 
 #include "hnae3.h"
@@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
 	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
 	hr_dev->iboe.phy_port[0] = 0;
 
+	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
+			    hr_dev->iboe.netdevs[0]->dev_addr);
+
 	for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
 		hr_dev->irq[i] = pci_irq_vector(handle->pdev,
 						i + handle->rinfo.base_vector);
-- 
1.9.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]

On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> This patch assgin a guid(Global Unique identifer)
> value to the hip08 device.
>
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> V2:
> - Use the correct way to update guid from Leon
>   Romanovsky's review.
>
> V1:
> - The initial submit
> ---
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
>  1 file changed, 4 insertions(+)
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> index a1839a3..2af4948 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> @@ -34,6 +34,7 @@
>  #include <linux/etherdevice.h>
>  #include <linux/interrupt.h>
>  #include <linux/kernel.h>
> +#include <net/addrconf.h>
>  #include <rdma/ib_umem.h>
>
>  #include "hnae3.h"
> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
>  	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
>  	hr_dev->iboe.phy_port[0] = 0;
>
> +	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> +			    hr_dev->iboe.netdevs[0]->dev_addr);
> +

Are you sure that it is right thing to do?

ib_dev.node_guid is __be64 and you cast it for u8.

Maybe, it is right, but I don't have ability to test it now.

Thanks

>  	for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
>  		hr_dev->irq[i] = pci_irq_vector(handle->pdev,
>  						i + handle->rinfo.base_vector);
> --
> 1.9.1
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[oulijun]

在 2018/1/10 14:49, Leon Romanovsky 写道:
> On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
>> This patch assgin a guid(Global Unique identifer)
>> value to the hip08 device.
>>
>> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> ---
>> V2:
>> - Use the correct way to update guid from Leon
>>   Romanovsky's review.
>>
>> V1:
>> - The initial submit
>> ---
>>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
>>  1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> index a1839a3..2af4948 100644
>> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> @@ -34,6 +34,7 @@
>>  #include <linux/etherdevice.h>
>>  #include <linux/interrupt.h>
>>  #include <linux/kernel.h>
>> +#include <net/addrconf.h>
>>  #include <rdma/ib_umem.h>
>>
>>  #include "hnae3.h"
>> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
>>  	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
>>  	hr_dev->iboe.phy_port[0] = 0;
>>
>> +	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
>> +			    hr_dev->iboe.netdevs[0]->dev_addr);
>> +
> 
> Are you sure that it is right thing to do?
> 
> ib_dev.node_guid is __be64 and you cast it for u8.
> 
> Maybe, it is right, but I don't have ability to test it now.
> 
> Thanks
Yes, it can be get rightly. I have test it again
The test result as follows:

root@(none)# cat /sys/class/infiniband/hns_0/node_guid
5816:89ff:fe28:a34b

Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
{
	addrconf_addr_eui48_base(eui, addr);
	eui[0] ^= 2;
}

The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
have the same usage.

Thanks
Lijun Ou

> 
>>  	for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
>>  		hr_dev->irq[i] = pci_irq_vector(handle->pdev,
>>  						i + handle->rinfo.base_vector);
>> --
>> 1.9.1
>>


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2869 bytes --]

On Wed, Jan 10, 2018 at 05:23:41PM +0800, oulijun wrote:
> 在 2018/1/10 14:49, Leon Romanovsky 写道:
> > On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> >> This patch assgin a guid(Global Unique identifer)
> >> value to the hip08 device.
> >>
> >> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >> ---
> >> V2:
> >> - Use the correct way to update guid from Leon
> >>   Romanovsky's review.
> >>
> >> V1:
> >> - The initial submit
> >> ---
> >>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
> >>  1 file changed, 4 insertions(+)
> >>
> >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> index a1839a3..2af4948 100644
> >> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> @@ -34,6 +34,7 @@
> >>  #include <linux/etherdevice.h>
> >>  #include <linux/interrupt.h>
> >>  #include <linux/kernel.h>
> >> +#include <net/addrconf.h>
> >>  #include <rdma/ib_umem.h>
> >>
> >>  #include "hnae3.h"
> >> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> >>  	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> >>  	hr_dev->iboe.phy_port[0] = 0;
> >>
> >> +	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> >> +			    hr_dev->iboe.netdevs[0]->dev_addr);
> >> +
> >
> > Are you sure that it is right thing to do?
> >
> > ib_dev.node_guid is __be64 and you cast it for u8.
> >
> > Maybe, it is right, but I don't have ability to test it now.
> >
> > Thanks
> Yes, it can be get rightly. I have test it again
> The test result as follows:
>
> root@(none)# cat /sys/class/infiniband/hns_0/node_guid
> 5816:89ff:fe28:a34b
>
> Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
> static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
> {
> 	addrconf_addr_eui48_base(eui, addr);
> 	eui[0] ^= 2;
> }
>
> The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
> have the same usage.

Indeed the first parameter is u8, but you are converting from __be64 to be u8.

Thanks

>
> Thanks
> Lijun Ou
>
> >
> >>  	for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
> >>  		hr_dev->irq[i] = pci_irq_vector(handle->pdev,
> >>  						i + handle->rinfo.base_vector);
> >> --
> >> 1.9.1
> >>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[Jason Gunthorpe]

On Wed, Jan 10, 2018 at 12:29:02PM +0200, Leon Romanovsky wrote:
> On Wed, Jan 10, 2018 at 05:23:41PM +0800, oulijun wrote:
> > 在 2018/1/10 14:49, Leon Romanovsky 写道:
> > > On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> > >> This patch assgin a guid(Global Unique identifer)
> > >> value to the hip08 device.
> > >>
> > >> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > >> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > >> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > >> V2:
> > >> - Use the correct way to update guid from Leon
> > >>   Romanovsky's review.
> > >>
> > >> V1:
> > >> - The initial submit
> > >>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
> > >>  1 file changed, 4 insertions(+)
> > >>
> > >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> > >> index a1839a3..2af4948 100644
> > >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> > >> @@ -34,6 +34,7 @@
> > >>  #include <linux/etherdevice.h>
> > >>  #include <linux/interrupt.h>
> > >>  #include <linux/kernel.h>
> > >> +#include <net/addrconf.h>
> > >>  #include <rdma/ib_umem.h>
> > >>
> > >>  #include "hnae3.h"
> > >> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> > >>  	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> > >>  	hr_dev->iboe.phy_port[0] = 0;
> > >>
> > >> +	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> > >> +			    hr_dev->iboe.netdevs[0]->dev_addr);
> > >> +
> > >
> > > Are you sure that it is right thing to do?
> > >
> > > ib_dev.node_guid is __be64 and you cast it for u8.
> > >
> > > Maybe, it is right, but I don't have ability to test it now.
> > >
> > > Thanks
> > Yes, it can be get rightly. I have test it again
> > The test result as follows:
> >
> > root@(none)# cat /sys/class/infiniband/hns_0/node_guid
> > 5816:89ff:fe28:a34b
> >
> > Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
> > static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
> > {
> > 	addrconf_addr_eui48_base(eui, addr);
> > 	eui[0] ^= 2;
> > }
> >
> > The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
> > have the same usage.
> 
> Indeed the first parameter is u8, but you are converting from __be64 to be u8.

addrconf_addr_eui48 accepts an array of 6 u8's in big endian byte
order representing the MAC.

If you use a be64 to hold the MAC then it must be unnaturally shifted
<< 16, eg

htobe64(0x1234 << 16);

Will work to set the mac to 00:00:00:00:12:34

So, it is really weird to store a mac in a __be64, but it can be done
correctly...

The *really* sketchy thing is pssing something called the node_guid to
something accepting a eui48. The node_guid is an eui64, and needs
translation before it can become an eui48, so *something* is wrong and
broken here.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[oulijun]

在 2018/1/10 18:29, Leon Romanovsky 写道:
> On Wed, Jan 10, 2018 at 05:23:41PM +0800, oulijun wrote:
>> 在 2018/1/10 14:49, Leon Romanovsky 写道:
>>> On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
>>>> This patch assgin a guid(Global Unique identifer)
>>>> value to the hip08 device.
>>>>
>>>> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>> ---
>>>> V2:
>>>> - Use the correct way to update guid from Leon
>>>>   Romanovsky's review.
>>>>
>>>> V1:
>>>> - The initial submit
>>>> ---
>>>>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
>>>>  1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> index a1839a3..2af4948 100644
>>>> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> @@ -34,6 +34,7 @@
>>>>  #include <linux/etherdevice.h>
>>>>  #include <linux/interrupt.h>
>>>>  #include <linux/kernel.h>
>>>> +#include <net/addrconf.h>
>>>>  #include <rdma/ib_umem.h>
>>>>
>>>>  #include "hnae3.h"
>>>> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
>>>>  	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
>>>>  	hr_dev->iboe.phy_port[0] = 0;
>>>>
>>>> +	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
>>>> +			    hr_dev->iboe.netdevs[0]->dev_addr);
>>>> +
>>>
>>> Are you sure that it is right thing to do?
>>>
>>> ib_dev.node_guid is __be64 and you cast it for u8.
>>>
>>> Maybe, it is right, but I don't have ability to test it now.
>>>
>>> Thanks
>> Yes, it can be get rightly. I have test it again
>> The test result as follows:
>>
>> root@(none)# cat /sys/class/infiniband/hns_0/node_guid
>> 5816:89ff:fe28:a34b
>>
>> Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
>> static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
>> {
>> 	addrconf_addr_eui48_base(eui, addr);
>> 	eui[0] ^= 2;
>> }
>>
>> The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
>> have the same usage.
> 
> Indeed the first parameter is u8, but you are converting from __be64 to be u8.
> 
> Thanks
> 
Yes, the type of node_guid is defined by ib core. the function of addrconf_addr_eui48 is also
defined by ib core. I could not modify them. Did I need to change u8 to unsigned char?

thanks
>>
>> Thanks
>> Lijun Ou
>>
>>>
>>>>  	for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
>>>>  		hr_dev->irq[i] = pci_irq_vector(handle->pdev,
>>>>  						i + handle->rinfo.base_vector);
>>>> --
>>>> 1.9.1
>>>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[Jason Gunthorpe]

On Thu, Jan 11, 2018 at 10:49:57AM +0800, oulijun wrote:
> >>>> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> >>>>  	hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> >>>>  	hr_dev->iboe.phy_port[0] = 0;
> >>>>
> >>>> +	addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> >>>> +			    hr_dev->iboe.netdevs[0]->dev_addr);
> >>>> +
> >>>
> >>> Are you sure that it is right thing to do?
> >>>
> >>> ib_dev.node_guid is __be64 and you cast it for u8.
> >>>
> >>> Maybe, it is right, but I don't have ability to test it now.
> >>>
> >>> Thanks
> >> Yes, it can be get rightly. I have test it again
> >> The test result as follows:
> >>
> >> root@(none)# cat /sys/class/infiniband/hns_0/node_guid
> >> 5816:89ff:fe28:a34b
> >>
> >> Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
> >> static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
> >> {
> >> 	addrconf_addr_eui48_base(eui, addr);
> >> 	eui[0] ^= 2;
> >> }
> >>
> >> The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
> >> have the same usage.
> > 
> > Indeed the first parameter is u8, but you are converting from __be64 to be u8.
> > 
> > Thanks
> > 
> Yes, the type of node_guid is defined by ib core. the function of addrconf_addr_eui48 is also
> defined by ib core. I could not modify them. Did I need to change u8 to unsigned char?

Actually, now that I look more closely, what you have is right.

It is just a very strange and misleading name of a function
'addrconf_addr_eui48' which takes in an eui48 and writes out a IB
GUID. Would have expected 'addrconf_ib_guid_eui48' or something..

Even more confusing because the commit message talks about eui32 and
misses a signed-off-by? Baffling.

.. and there are dragons here, because the IPv6 interface id (eg what
the similar addrconf_ifid_eui48 is handling) uses an inverted U/L bit
to the IEEE EUI-64.

.. and I can't quite recall what the IBTA eventually settled on as the
standard for IB - if it was EUI-64 or modified IPv6 EUI-64 ..

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 687 bytes --]

On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> This patch assgin a guid(Global Unique identifer)
> value to the hip08 device.
>
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> V2:
> - Use the correct way to update guid from Leon
>   Romanovsky's review.
>
> V1:
> - The initial submit
> ---
>  drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
>  1 file changed, 4 insertions(+)
>

Thanks,
Reviewed-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08

[Jason Gunthorpe]

On Wed, Jan 10, 2018 at 02:39:46PM +0800, Lijun Ou wrote:
> This patch series adds the CM(Connection Mangement) support
> to hip08 RoCE driver. Changes done are primarily to add
> support of APIs in IB device and some fixes over the
> origin driver to support RDMA CM. This patch series
> also adjust some codes style in order to fill the
> sq wqe of ud type and fix a bug for tx_cqn field.
> 
> Change from V1:
> - Add a new patch named assign the correct value for
>   tx_cqn splited from the first patchset
> - Fix the comments given by Leon Romanovsky
> - Add the options for free sq lock when qp is illegal
>   qp type
>  
> Lijun Ou (7):
>   RDMA/hns: Assign the correct value for tx_cqn
>   RDMA/hns: Create gsi qp in hip08
>   RDMA/hns: Add gsi qp support for modifying qp in hip08
>   RDMA/hns: Fill sq wqe context of ud type in hip08
>   RDMA/hns: Assign zero for pkey_index of wc in hip08
>   RDMA/hns: Update the verbs of polling for completion
>   RDMA/hns: Set the guid for hip08 RoCE device

Applied to for-next, thanks

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html