* [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08
@ 2018-01-10 6:39 Lijun Ou
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
0 siblings, 1 reply; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
This patch series adds the CM(Connection Mangement) support
to hip08 RoCE driver. Changes done are primarily to add
support of APIs in IB device and some fixes over the
origin driver to support RDMA CM. This patch series
also adjust some codes style in order to fill the
sq wqe of ud type and fix a bug for tx_cqn field.
Change from V1:
- Add a new patch named assign the correct value for
tx_cqn splited from the first patchset
- Fix the comments given by Leon Romanovsky
- Add the options for free sq lock when qp is illegal
qp type
Lijun Ou (7):
RDMA/hns: Assign the correct value for tx_cqn
RDMA/hns: Create gsi qp in hip08
RDMA/hns: Add gsi qp support for modifying qp in hip08
RDMA/hns: Fill sq wqe context of ud type in hip08
RDMA/hns: Assign zero for pkey_index of wc in hip08
RDMA/hns: Update the verbs of polling for completion
RDMA/hns: Set the guid for hip08 RoCE device
drivers/infiniband/hw/hns/hns_roce_device.h | 1 +
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 531 +++++++++++++++++++---------
drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 86 ++++-
drivers/infiniband/hw/hns/hns_roce_qp.c | 18 +-
4 files changed, 471 insertions(+), 165 deletions(-)
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 1/7] RDMA/hns: Assign the correct value for tx_cqn
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
@ 2018-01-10 6:39 ` Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 2/7] RDMA/hns: Create gsi qp in hip08 Lijun Ou
` (6 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
When modifying qp from init to init, it need to
assign the cqn of send cq for tx cqn field of
qp context. Otherwise, it will cause a mistake
when the send and recv cq sizes is the difference.
This patch mainly fixes it.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 2ca35e3..dd50336 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -2342,7 +2342,7 @@ static void modify_qp_init_to_init(struct ib_qp *ibqp,
V2_QPC_BYTE_80_RX_CQN_S, 0);
roce_set_field(context->byte_252_err_txcqn, V2_QPC_BYTE_252_TX_CQN_M,
- V2_QPC_BYTE_252_TX_CQN_S, to_hr_cq(ibqp->recv_cq)->cqn);
+ V2_QPC_BYTE_252_TX_CQN_S, to_hr_cq(ibqp->send_cq)->cqn);
roce_set_field(qpc_mask->byte_252_err_txcqn, V2_QPC_BYTE_252_TX_CQN_M,
V2_QPC_BYTE_252_TX_CQN_S, 0);
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 2/7] RDMA/hns: Create gsi qp in hip08
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 6:39 ` [PATCH V2 for-next 1/7] RDMA/hns: Assign the correct value for tx_cqn Lijun Ou
@ 2018-01-10 6:39 ` Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 3/7] RDMA/hns: Add gsi qp support for modifying " Lijun Ou
` (5 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
The gsi qp and rc qp use the same qp context
structure and the created flow, only
differentiate them by qpn and qp type.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
drivers/infiniband/hw/hns/hns_roce_qp.c | 18 ++++++++++++++++--
1 file changed, 16 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c
index 351fa31..4414cea 100644
--- a/drivers/infiniband/hw/hns/hns_roce_qp.c
+++ b/drivers/infiniband/hw/hns/hns_roce_qp.c
@@ -455,6 +455,13 @@ static int hns_roce_set_kernel_sq_size(struct hns_roce_dev *hr_dev,
hr_qp->sge.sge_shift = 4;
}
+ /* ud sqwqe's sge use extend sge */
+ if (hr_dev->caps.max_sq_sg > 2 && hr_qp->ibqp.qp_type == IB_QPT_GSI) {
+ hr_qp->sge.sge_cnt = roundup_pow_of_two(hr_qp->sq.wqe_cnt *
+ hr_qp->sq.max_gs);
+ hr_qp->sge.sge_shift = 4;
+ }
+
/* Get buf size, SQ and RQ are aligned to PAGE_SIZE */
page_size = 1 << (hr_dev->caps.mtt_buf_pg_sz + PAGE_SHIFT);
hr_qp->sq.offset = 0;
@@ -502,6 +509,8 @@ static int hns_roce_create_qp_common(struct hns_roce_dev *hr_dev,
hr_qp->state = IB_QPS_RESET;
+ hr_qp->ibqp.qp_type = init_attr->qp_type;
+
if (init_attr->sq_sig_type == IB_SIGNAL_ALL_WR)
hr_qp->sq_signal_bits = IB_SIGNAL_ALL_WR;
else
@@ -764,8 +773,13 @@ struct ib_qp *hns_roce_create_qp(struct ib_pd *pd,
hr_qp = &hr_sqp->hr_qp;
hr_qp->port = init_attr->port_num - 1;
hr_qp->phy_port = hr_dev->iboe.phy_port[hr_qp->port];
- hr_qp->ibqp.qp_num = HNS_ROCE_MAX_PORTS +
- hr_dev->iboe.phy_port[hr_qp->port];
+
+ /* when hw version is v1, the sqpn is allocated */
+ if (hr_dev->caps.max_sq_sg <= 2)
+ hr_qp->ibqp.qp_num = HNS_ROCE_MAX_PORTS +
+ hr_dev->iboe.phy_port[hr_qp->port];
+ else
+ hr_qp->ibqp.qp_num = 1;
ret = hns_roce_create_qp_common(hr_dev, pd, init_attr, udata,
hr_qp->ibqp.qp_num, hr_qp);
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 3/7] RDMA/hns: Add gsi qp support for modifying qp in hip08
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 6:39 ` [PATCH V2 for-next 1/7] RDMA/hns: Assign the correct value for tx_cqn Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 2/7] RDMA/hns: Create gsi qp in hip08 Lijun Ou
@ 2018-01-10 6:39 ` Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type " Lijun Ou
` (4 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
It needs to Assign the values for some fields in qp context
when qp type is gsi qp type in hip08.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V2:
- Remove the options of assigning the correct value for
tx cqn field of qp context and generate a separate patch
V1:
- The initial submit
---
drivers/infiniband/hw/hns/hns_roce_device.h | 1 +
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 64 +++++++++++++++++++++--------
2 files changed, 49 insertions(+), 16 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 4afa070..42c3b5a 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -485,6 +485,7 @@ struct hns_roce_qp {
u32 access_flags;
u32 atomic_rd_en;
u32 pkey_index;
+ u32 qkey;
void (*event)(struct hns_roce_qp *,
enum hns_roce_event);
unsigned long qpn;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index dd50336..e53cd7d 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1975,6 +1975,7 @@ static void set_access_flags(struct hns_roce_qp *hr_qp,
static void modify_qp_reset_to_init(struct ib_qp *ibqp,
const struct ib_qp_attr *attr,
+ int attr_mask,
struct hns_roce_v2_qp_context *context,
struct hns_roce_v2_qp_context *qpc_mask)
{
@@ -1991,9 +1992,18 @@ static void modify_qp_reset_to_init(struct ib_qp *ibqp,
roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_TST_M,
V2_QPC_BYTE_4_TST_S, 0);
- roce_set_field(context->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
- V2_QPC_BYTE_4_SGE_SHIFT_S, hr_qp->sq.max_gs > 2 ?
- ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+ if (ibqp->qp_type == IB_QPT_GSI)
+ roce_set_field(context->byte_4_sqpn_tst,
+ V2_QPC_BYTE_4_SGE_SHIFT_M,
+ V2_QPC_BYTE_4_SGE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->sge.sge_cnt));
+ else
+ roce_set_field(context->byte_4_sqpn_tst,
+ V2_QPC_BYTE_4_SGE_SHIFT_M,
+ V2_QPC_BYTE_4_SGE_SHIFT_S,
+ hr_qp->sq.max_gs > 2 ?
+ ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+
roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
V2_QPC_BYTE_4_SGE_SHIFT_S, 0);
@@ -2058,6 +2068,12 @@ static void modify_qp_reset_to_init(struct ib_qp *ibqp,
roce_set_bit(qpc_mask->byte_28_at_fl, V2_QPC_BYTE_28_CNP_TX_FLAG_S, 0);
roce_set_bit(qpc_mask->byte_28_at_fl, V2_QPC_BYTE_28_CE_FLAG_S, 0);
+ if (attr_mask & IB_QP_QKEY) {
+ context->qkey_xrcd = attr->qkey;
+ qpc_mask->qkey_xrcd = 0;
+ hr_qp->qkey = attr->qkey;
+ }
+
roce_set_bit(context->byte_76_srqn_op_en, V2_QPC_BYTE_76_RQIE_S, 1);
roce_set_bit(qpc_mask->byte_76_srqn_op_en, V2_QPC_BYTE_76_RQIE_S, 0);
@@ -2279,9 +2295,17 @@ static void modify_qp_init_to_init(struct ib_qp *ibqp,
roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_TST_M,
V2_QPC_BYTE_4_TST_S, 0);
- roce_set_field(context->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
- V2_QPC_BYTE_4_SGE_SHIFT_S, hr_qp->sq.max_gs > 2 ?
- ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+ if (ibqp->qp_type == IB_QPT_GSI)
+ roce_set_field(context->byte_4_sqpn_tst,
+ V2_QPC_BYTE_4_SGE_SHIFT_M,
+ V2_QPC_BYTE_4_SGE_SHIFT_S,
+ ilog2((unsigned int)hr_qp->sge.sge_cnt));
+ else
+ roce_set_field(context->byte_4_sqpn_tst,
+ V2_QPC_BYTE_4_SGE_SHIFT_M,
+ V2_QPC_BYTE_4_SGE_SHIFT_S, hr_qp->sq.max_gs > 2 ?
+ ilog2((unsigned int)hr_qp->sge.sge_cnt) : 0);
+
roce_set_field(qpc_mask->byte_4_sqpn_tst, V2_QPC_BYTE_4_SGE_SHIFT_M,
V2_QPC_BYTE_4_SGE_SHIFT_S, 0);
@@ -2358,10 +2382,10 @@ static void modify_qp_init_to_init(struct ib_qp *ibqp,
V2_QPC_BYTE_76_SRQN_M, V2_QPC_BYTE_76_SRQN_S, 0);
}
- if (attr_mask & IB_QP_PKEY_INDEX)
- context->qkey_xrcd = attr->pkey_index;
- else
- context->qkey_xrcd = hr_qp->pkey_index;
+ if (attr_mask & IB_QP_QKEY) {
+ context->qkey_xrcd = attr->qkey;
+ qpc_mask->qkey_xrcd = 0;
+ }
roce_set_field(context->byte_4_sqpn_tst, V2_QPC_BYTE_4_SQPN_M,
V2_QPC_BYTE_4_SQPN_S, hr_qp->qpn);
@@ -2457,7 +2481,8 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp,
roce_set_field(context->byte_20_smac_sgid_idx,
V2_QPC_BYTE_20_SGE_HOP_NUM_M,
V2_QPC_BYTE_20_SGE_HOP_NUM_S,
- hr_qp->sq.max_gs > 2 ? hr_dev->caps.mtt_hop_num : 0);
+ ((ibqp->qp_type == IB_QPT_GSI) || hr_qp->sq.max_gs > 2) ?
+ hr_dev->caps.mtt_hop_num : 0);
roce_set_field(qpc_mask->byte_20_smac_sgid_idx,
V2_QPC_BYTE_20_SGE_HOP_NUM_M,
V2_QPC_BYTE_20_SGE_HOP_NUM_S, 0);
@@ -2617,8 +2642,13 @@ static int modify_qp_init_to_rtr(struct ib_qp *ibqp,
roce_set_field(qpc_mask->byte_24_mtu_tc, V2_QPC_BYTE_24_TC_M,
V2_QPC_BYTE_24_TC_S, 0);
- roce_set_field(context->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
- V2_QPC_BYTE_24_MTU_S, attr->path_mtu);
+ if (ibqp->qp_type == IB_QPT_GSI || ibqp->qp_type == IB_QPT_UD)
+ roce_set_field(context->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
+ V2_QPC_BYTE_24_MTU_S, IB_MTU_4096);
+ else
+ roce_set_field(context->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
+ V2_QPC_BYTE_24_MTU_S, attr->path_mtu);
+
roce_set_field(qpc_mask->byte_24_mtu_tc, V2_QPC_BYTE_24_MTU_M,
V2_QPC_BYTE_24_MTU_S, 0);
@@ -2725,13 +2755,14 @@ static int modify_qp_rtr_to_rts(struct ib_qp *ibqp,
V2_QPC_BYTE_168_SQ_CUR_BLK_ADDR_S, 0);
page_size = 1 << (hr_dev->caps.mtt_buf_pg_sz + PAGE_SHIFT);
- context->sq_cur_sge_blk_addr = hr_qp->sq.max_gs > 2 ?
+ context->sq_cur_sge_blk_addr =
+ ((ibqp->qp_type == IB_QPT_GSI) || hr_qp->sq.max_gs > 2) ?
((u32)(mtts[hr_qp->sge.offset / page_size]
>> PAGE_ADDR_SHIFT)) : 0;
roce_set_field(context->byte_184_irrl_idx,
V2_QPC_BYTE_184_SQ_CUR_SGE_BLK_ADDR_M,
V2_QPC_BYTE_184_SQ_CUR_SGE_BLK_ADDR_S,
- hr_qp->sq.max_gs > 2 ?
+ ((ibqp->qp_type == IB_QPT_GSI) || hr_qp->sq.max_gs > 2) ?
(mtts[hr_qp->sge.offset / page_size] >>
(32 + PAGE_ADDR_SHIFT)) : 0);
qpc_mask->sq_cur_sge_blk_addr = 0;
@@ -2902,7 +2933,8 @@ static int hns_roce_v2_modify_qp(struct ib_qp *ibqp,
*/
memset(qpc_mask, 0xff, sizeof(*qpc_mask));
if (cur_state == IB_QPS_RESET && new_state == IB_QPS_INIT) {
- modify_qp_reset_to_init(ibqp, attr, context, qpc_mask);
+ modify_qp_reset_to_init(ibqp, attr, attr_mask, context,
+ qpc_mask);
} else if (cur_state == IB_QPS_INIT && new_state == IB_QPS_INIT) {
modify_qp_init_to_init(ibqp, attr, attr_mask, context,
qpc_mask);
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type in hip08
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (2 preceding siblings ...)
2018-01-10 6:39 ` [PATCH V2 for-next 3/7] RDMA/hns: Add gsi qp support for modifying " Lijun Ou
@ 2018-01-10 6:39 ` Lijun Ou
[not found] ` <1515566393-63888-5-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 6:39 ` [PATCH V2 for-next 5/7] RDMA/hns: Assign zero for pkey_index of wc " Lijun Ou
` (3 subsequent siblings)
7 siblings, 1 reply; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
This patch mainly configure the fileds of sq wqe of ud
type when posting wr of gsi qp type.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V1->V2:
- return errno code directly and delete unnecessary
initialized value of ret from Leon Romanvosky's
advice.
- Add necessary option for free sq lock when qp is
illegal qp type.
V1:
- The initial submit
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 446 +++++++++++++++++++----------
drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 84 ++++++
2 files changed, 385 insertions(+), 145 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index e53cd7d..cb3ac54 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -51,26 +51,99 @@ static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
dseg->len = cpu_to_le32(sg->length);
}
+static int set_rwqe_data_seg(struct ib_qp *ibqp, struct ib_send_wr *wr,
+ struct hns_roce_v2_rc_send_wqe *rc_sq_wqe,
+ void *wqe, unsigned int *sge_ind,
+ struct ib_send_wr **bad_wr)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_v2_wqe_data_seg *dseg = wqe;
+ struct hns_roce_qp *qp = to_hr_qp(ibqp);
+ int i;
+
+ if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
+ if (rc_sq_wqe->msg_len > hr_dev->caps.max_sq_inline) {
+ *bad_wr = wr;
+ dev_err(hr_dev->dev, "inline len(1-%d)=%d, illegal",
+ rc_sq_wqe->msg_len, hr_dev->caps.max_sq_inline);
+ return -EINVAL;
+ }
+
+ for (i = 0; i < wr->num_sge; i++) {
+ memcpy(wqe, ((void *)wr->sg_list[i].addr),
+ wr->sg_list[i].length);
+ wqe += wr->sg_list[i].length;
+ }
+
+ roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_INLINE_S,
+ 1);
+ } else {
+ if (wr->num_sge <= 2) {
+ for (i = 0; i < wr->num_sge; i++) {
+ if (likely(wr->sg_list[i].length)) {
+ set_data_seg_v2(dseg, wr->sg_list + i);
+ dseg++;
+ }
+ }
+ } else {
+ roce_set_field(rc_sq_wqe->byte_20,
+ V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
+ V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+ (*sge_ind) & (qp->sge.sge_cnt - 1));
+
+ for (i = 0; i < 2; i++) {
+ if (likely(wr->sg_list[i].length)) {
+ set_data_seg_v2(dseg, wr->sg_list + i);
+ dseg++;
+ }
+ }
+
+ dseg = get_send_extend_sge(qp,
+ (*sge_ind) & (qp->sge.sge_cnt - 1));
+
+ for (i = 0; i < wr->num_sge - 2; i++) {
+ if (likely(wr->sg_list[i + 2].length)) {
+ set_data_seg_v2(dseg,
+ wr->sg_list + 2 + i);
+ dseg++;
+ (*sge_ind)++;
+ }
+ }
+ }
+
+ roce_set_field(rc_sq_wqe->byte_16,
+ V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
+ V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
+ }
+
+ return 0;
+}
+
static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
struct ib_send_wr **bad_wr)
{
struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
+ struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
+ struct hns_roce_v2_ud_send_wqe *ud_sq_wqe;
struct hns_roce_v2_rc_send_wqe *rc_sq_wqe;
struct hns_roce_qp *qp = to_hr_qp(ibqp);
struct hns_roce_v2_wqe_data_seg *dseg;
struct device *dev = hr_dev->dev;
struct hns_roce_v2_db sq_db;
unsigned int sge_ind = 0;
- unsigned int wqe_sz = 0;
unsigned int owner_bit;
unsigned long flags;
unsigned int ind;
void *wqe = NULL;
+ bool loopback;
int ret = 0;
+ u8 *smac;
int nreq;
int i;
- if (unlikely(ibqp->qp_type != IB_QPT_RC)) {
+ if (unlikely(ibqp->qp_type != IB_QPT_RC &&
+ ibqp->qp_type != IB_QPT_GSI &&
+ ibqp->qp_type != IB_QPT_UD)) {
dev_err(dev, "Not supported QP(0x%x)type!\n", ibqp->qp_type);
*bad_wr = NULL;
return -EOPNOTSUPP;
@@ -107,172 +180,255 @@ static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
wr->wr_id;
owner_bit = ~(qp->sq.head >> ilog2(qp->sq.wqe_cnt)) & 0x1;
- rc_sq_wqe = wqe;
- memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
- for (i = 0; i < wr->num_sge; i++)
- rc_sq_wqe->msg_len += wr->sg_list[i].length;
- rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
+ /* Corresponding to the QP type, wqe process separately */
+ if (ibqp->qp_type == IB_QPT_GSI) {
+ ud_sq_wqe = wqe;
+ memset(ud_sq_wqe, 0, sizeof(*ud_sq_wqe));
+
+ roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_0_M,
+ V2_UD_SEND_WQE_DMAC_0_S, ah->av.mac[0]);
+ roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_1_M,
+ V2_UD_SEND_WQE_DMAC_1_S, ah->av.mac[1]);
+ roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_2_M,
+ V2_UD_SEND_WQE_DMAC_2_S, ah->av.mac[2]);
+ roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_3_M,
+ V2_UD_SEND_WQE_DMAC_3_S, ah->av.mac[3]);
+ roce_set_field(ud_sq_wqe->byte_48,
+ V2_UD_SEND_WQE_BYTE_48_DMAC_4_M,
+ V2_UD_SEND_WQE_BYTE_48_DMAC_4_S,
+ ah->av.mac[4]);
+ roce_set_field(ud_sq_wqe->byte_48,
+ V2_UD_SEND_WQE_BYTE_48_DMAC_5_M,
+ V2_UD_SEND_WQE_BYTE_48_DMAC_5_S,
+ ah->av.mac[5]);
+
+ /* MAC loopback */
+ smac = (u8 *)hr_dev->dev_addr[qp->port];
+ loopback = ether_addr_equal_unaligned(ah->av.mac,
+ smac) ? 1 : 0;
+
+ roce_set_bit(ud_sq_wqe->byte_40,
+ V2_UD_SEND_WQE_BYTE_40_LBI_S, loopback);
+
+ roce_set_field(ud_sq_wqe->byte_4,
+ V2_UD_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_UD_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_SEND);
- roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_FENCE_S,
- (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
+ for (i = 0; i < wr->num_sge; i++)
+ ud_sq_wqe->msg_len += wr->sg_list[i].length;
- roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_SE_S,
- (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
+ ud_sq_wqe->immtdata = send_ieth(wr);
- roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_CQE_S,
- (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
+ /* Set sig attr */
+ roce_set_bit(ud_sq_wqe->byte_4,
+ V2_UD_SEND_WQE_BYTE_4_CQE_S,
+ (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
- roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_OWNER_S,
- owner_bit);
+ /* Set se attr */
+ roce_set_bit(ud_sq_wqe->byte_4,
+ V2_UD_SEND_WQE_BYTE_4_SE_S,
+ (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
- switch (wr->opcode) {
- case IB_WR_RDMA_READ:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_RDMA_READ);
- rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
- rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
- break;
- case IB_WR_RDMA_WRITE:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
- rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
- rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
- break;
- case IB_WR_RDMA_WRITE_WITH_IMM:
- roce_set_field(rc_sq_wqe->byte_4,
+ roce_set_bit(ud_sq_wqe->byte_4,
+ V2_UD_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
+
+ roce_set_field(ud_sq_wqe->byte_16,
+ V2_UD_SEND_WQE_BYTE_16_PD_M,
+ V2_UD_SEND_WQE_BYTE_16_PD_S,
+ to_hr_pd(ibqp->pd)->pdn);
+
+ roce_set_field(ud_sq_wqe->byte_16,
+ V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M,
+ V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S,
+ wr->num_sge);
+
+ roce_set_field(ud_sq_wqe->byte_20,
+ V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
+ V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
+ sge_ind & (qp->sge.sge_cnt - 1));
+
+ roce_set_field(ud_sq_wqe->byte_24,
+ V2_UD_SEND_WQE_BYTE_24_UDPSPN_M,
+ V2_UD_SEND_WQE_BYTE_24_UDPSPN_S, 0);
+ ud_sq_wqe->qkey =
+ cpu_to_be32(ud_wr(wr)->remote_qkey & 0x80000000) ?
+ qp->qkey : ud_wr(wr)->remote_qkey;
+ roce_set_field(ud_sq_wqe->byte_32,
+ V2_UD_SEND_WQE_BYTE_32_DQPN_M,
+ V2_UD_SEND_WQE_BYTE_32_DQPN_S,
+ ud_wr(wr)->remote_qpn);
+
+ roce_set_field(ud_sq_wqe->byte_36,
+ V2_UD_SEND_WQE_BYTE_36_VLAN_M,
+ V2_UD_SEND_WQE_BYTE_36_VLAN_S,
+ ah->av.vlan);
+ roce_set_field(ud_sq_wqe->byte_36,
+ V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M,
+ V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S,
+ ah->av.hop_limit);
+ roce_set_field(ud_sq_wqe->byte_36,
+ V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
+ V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
+ 0);
+ roce_set_field(ud_sq_wqe->byte_36,
+ V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
+ V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
+ 0);
+ roce_set_field(ud_sq_wqe->byte_40,
+ V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
+ V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, 0);
+ roce_set_field(ud_sq_wqe->byte_40,
+ V2_UD_SEND_WQE_BYTE_40_SL_M,
+ V2_UD_SEND_WQE_BYTE_40_SL_S,
+ ah->av.sl_tclass_flowlabel >>
+ HNS_ROCE_SL_SHIFT);
+ roce_set_field(ud_sq_wqe->byte_40,
+ V2_UD_SEND_WQE_BYTE_40_PORTN_M,
+ V2_UD_SEND_WQE_BYTE_40_PORTN_S,
+ qp->port);
+
+ roce_set_field(ud_sq_wqe->byte_48,
+ V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M,
+ V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S,
+ hns_get_gid_index(hr_dev, qp->phy_port,
+ ah->av.gid_index));
+
+ memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0],
+ GID_LEN_V2);
+
+ dseg = get_send_extend_sge(qp,
+ sge_ind & (qp->sge.sge_cnt - 1));
+ for (i = 0; i < wr->num_sge; i++) {
+ set_data_seg_v2(dseg + i, wr->sg_list + i);
+ sge_ind++;
+ }
+
+ ind++;
+ } else if (ibqp->qp_type == IB_QPT_RC) {
+ rc_sq_wqe = wqe;
+ memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
+ for (i = 0; i < wr->num_sge; i++)
+ rc_sq_wqe->msg_len += wr->sg_list[i].length;
+
+ rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
+
+ roce_set_bit(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_FENCE_S,
+ (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
+
+ roce_set_bit(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_SE_S,
+ (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
+
+ roce_set_bit(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_CQE_S,
+ (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
+
+ roce_set_bit(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
+
+ switch (wr->opcode) {
+ case IB_WR_RDMA_READ:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_RDMA_READ);
+ rc_sq_wqe->rkey =
+ cpu_to_le32(rdma_wr(wr)->rkey);
+ rc_sq_wqe->va =
+ cpu_to_le64(rdma_wr(wr)->remote_addr);
+ break;
+ case IB_WR_RDMA_WRITE:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
+ rc_sq_wqe->rkey =
+ cpu_to_le32(rdma_wr(wr)->rkey);
+ rc_sq_wqe->va =
+ cpu_to_le64(rdma_wr(wr)->remote_addr);
+ break;
+ case IB_WR_RDMA_WRITE_WITH_IMM:
+ roce_set_field(rc_sq_wqe->byte_4,
V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
HNS_ROCE_V2_WQE_OP_RDMA_WRITE_WITH_IMM);
- rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
- rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
- break;
- case IB_WR_SEND:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_SEND);
- break;
- case IB_WR_SEND_WITH_INV:
- roce_set_field(rc_sq_wqe->byte_4,
+ rc_sq_wqe->rkey =
+ cpu_to_le32(rdma_wr(wr)->rkey);
+ rc_sq_wqe->va =
+ cpu_to_le64(rdma_wr(wr)->remote_addr);
+ break;
+ case IB_WR_SEND:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_SEND);
+ break;
+ case IB_WR_SEND_WITH_INV:
+ roce_set_field(rc_sq_wqe->byte_4,
V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
HNS_ROCE_V2_WQE_OP_SEND_WITH_INV);
- break;
- case IB_WR_SEND_WITH_IMM:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
- break;
- case IB_WR_LOCAL_INV:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_LOCAL_INV);
- break;
- case IB_WR_ATOMIC_CMP_AND_SWP:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
- break;
- case IB_WR_ATOMIC_FETCH_AND_ADD:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
- break;
- case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
- roce_set_field(rc_sq_wqe->byte_4,
+ break;
+ case IB_WR_SEND_WITH_IMM:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
+ break;
+ case IB_WR_LOCAL_INV:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_LOCAL_INV);
+ break;
+ case IB_WR_ATOMIC_CMP_AND_SWP:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
+ break;
+ case IB_WR_ATOMIC_FETCH_AND_ADD:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
+ break;
+ case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
+ roce_set_field(rc_sq_wqe->byte_4,
V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
HNS_ROCE_V2_WQE_OP_ATOM_MSK_CMP_AND_SWAP);
- break;
- case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
- roce_set_field(rc_sq_wqe->byte_4,
+ break;
+ case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
+ roce_set_field(rc_sq_wqe->byte_4,
V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
HNS_ROCE_V2_WQE_OP_ATOM_MSK_FETCH_AND_ADD);
- break;
- default:
- roce_set_field(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
- V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
- HNS_ROCE_V2_WQE_OP_MASK);
- break;
- }
-
- wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
- dseg = wqe;
- if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
- if (rc_sq_wqe->msg_len >
- hr_dev->caps.max_sq_inline) {
- ret = -EINVAL;
- *bad_wr = wr;
- dev_err(dev, "inline len(1-%d)=%d, illegal",
- rc_sq_wqe->msg_len,
- hr_dev->caps.max_sq_inline);
- goto out;
+ break;
+ default:
+ roce_set_field(rc_sq_wqe->byte_4,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
+ V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
+ HNS_ROCE_V2_WQE_OP_MASK);
+ break;
}
- for (i = 0; i < wr->num_sge; i++) {
- memcpy(wqe, ((void *)wr->sg_list[i].addr),
- wr->sg_list[i].length);
- wqe += wr->sg_list[i].length;
- wqe_sz += wr->sg_list[i].length;
- }
+ wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
+ dseg = wqe;
- roce_set_bit(rc_sq_wqe->byte_4,
- V2_RC_SEND_WQE_BYTE_4_INLINE_S, 1);
+ ret = set_rwqe_data_seg(ibqp, wr, rc_sq_wqe, wqe,
+ &sge_ind, bad_wr);
+ if (ret)
+ goto out;
+ ind++;
} else {
- if (wr->num_sge <= 2) {
- for (i = 0; i < wr->num_sge; i++) {
- if (likely(wr->sg_list[i].length)) {
- set_data_seg_v2(dseg,
- wr->sg_list + i);
- dseg++;
- }
- }
- } else {
- roce_set_field(rc_sq_wqe->byte_20,
- V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
- V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
- sge_ind & (qp->sge.sge_cnt - 1));
-
- for (i = 0; i < 2; i++) {
- if (likely(wr->sg_list[i].length)) {
- set_data_seg_v2(dseg,
- wr->sg_list + i);
- dseg++;
- }
- }
-
- dseg = get_send_extend_sge(qp,
- sge_ind & (qp->sge.sge_cnt - 1));
-
- for (i = 0; i < wr->num_sge - 2; i++) {
- if (likely(wr->sg_list[i + 2].length)) {
- set_data_seg_v2(dseg,
- wr->sg_list + 2 + i);
- dseg++;
- sge_ind++;
- }
- }
- }
-
- roce_set_field(rc_sq_wqe->byte_16,
- V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
- V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S,
- wr->num_sge);
- wqe_sz += wr->num_sge *
- sizeof(struct hns_roce_v2_wqe_data_seg);
+ dev_err(dev, "Illegal qp_type(0x%x)\n", ibqp->qp_type);
+ spin_unlock_irqrestore(&qp->sq.lock, flags);
+ return -EOPNOTSUPP;
}
- ind++;
}
out:
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
index 463edab..c11b253 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
@@ -916,6 +916,90 @@ struct hns_roce_v2_cq_db {
#define V2_CQ_DB_PARAMETER_NOTIFY_S 24
+struct hns_roce_v2_ud_send_wqe {
+ u32 byte_4;
+ u32 msg_len;
+ u32 immtdata;
+ u32 byte_16;
+ u32 byte_20;
+ u32 byte_24;
+ u32 qkey;
+ u32 byte_32;
+ u32 byte_36;
+ u32 byte_40;
+ u32 dmac;
+ u32 byte_48;
+ u8 dgid[GID_LEN_V2];
+
+};
+#define V2_UD_SEND_WQE_BYTE_4_OPCODE_S 0
+#define V2_UD_SEND_WQE_BYTE_4_OPCODE_M GENMASK(4, 0)
+
+#define V2_UD_SEND_WQE_BYTE_4_OWNER_S 7
+
+#define V2_UD_SEND_WQE_BYTE_4_CQE_S 8
+
+#define V2_UD_SEND_WQE_BYTE_4_SE_S 11
+
+#define V2_UD_SEND_WQE_BYTE_16_PD_S 0
+#define V2_UD_SEND_WQE_BYTE_16_PD_M GENMASK(23, 0)
+
+#define V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S 24
+#define V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M GENMASK(31, 24)
+
+#define V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S 0
+#define V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M GENMASK(23, 0)
+
+#define V2_UD_SEND_WQE_BYTE_24_UDPSPN_S 16
+#define V2_UD_SEND_WQE_BYTE_24_UDPSPN_M GENMASK(31, 16)
+
+#define V2_UD_SEND_WQE_BYTE_32_DQPN_S 0
+#define V2_UD_SEND_WQE_BYTE_32_DQPN_M GENMASK(23, 0)
+
+#define V2_UD_SEND_WQE_BYTE_36_VLAN_S 0
+#define V2_UD_SEND_WQE_BYTE_36_VLAN_M GENMASK(15, 0)
+
+#define V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S 16
+#define V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M GENMASK(23, 16)
+
+#define V2_UD_SEND_WQE_BYTE_36_TCLASS_S 24
+#define V2_UD_SEND_WQE_BYTE_36_TCLASS_M GENMASK(31, 24)
+
+#define V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S 0
+#define V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M GENMASK(19, 0)
+
+#define V2_UD_SEND_WQE_BYTE_40_SL_S 20
+#define V2_UD_SEND_WQE_BYTE_40_SL_M GENMASK(23, 20)
+
+#define V2_UD_SEND_WQE_BYTE_40_PORTN_S 24
+#define V2_UD_SEND_WQE_BYTE_40_PORTN_M GENMASK(26, 24)
+
+#define V2_UD_SEND_WQE_BYTE_40_LBI_S 31
+
+#define V2_UD_SEND_WQE_DMAC_0_S 0
+#define V2_UD_SEND_WQE_DMAC_0_M GENMASK(7, 0)
+
+#define V2_UD_SEND_WQE_DMAC_1_S 8
+#define V2_UD_SEND_WQE_DMAC_1_M GENMASK(15, 8)
+
+#define V2_UD_SEND_WQE_DMAC_2_S 16
+#define V2_UD_SEND_WQE_DMAC_2_M GENMASK(23, 16)
+
+#define V2_UD_SEND_WQE_DMAC_3_S 24
+#define V2_UD_SEND_WQE_DMAC_3_M GENMASK(31, 24)
+
+#define V2_UD_SEND_WQE_BYTE_48_DMAC_4_S 0
+#define V2_UD_SEND_WQE_BYTE_48_DMAC_4_M GENMASK(7, 0)
+
+#define V2_UD_SEND_WQE_BYTE_48_DMAC_5_S 8
+#define V2_UD_SEND_WQE_BYTE_48_DMAC_5_M GENMASK(15, 8)
+
+#define V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S 16
+#define V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M GENMASK(23, 16)
+
+#define V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_S 24
+#define V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_M GENMASK(31, 24)
+
struct hns_roce_v2_rc_send_wqe {
u32 byte_4;
u32 msg_len;
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 5/7] RDMA/hns: Assign zero for pkey_index of wc in hip08
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (3 preceding siblings ...)
2018-01-10 6:39 ` [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type " Lijun Ou
@ 2018-01-10 6:39 ` Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 6/7] RDMA/hns: Update the verbs of polling for completion Lijun Ou
` (2 subsequent siblings)
7 siblings, 0 replies; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
Because pkey is fixed for hip08 RoCE, it needs to assign zero
for pkey_index of wc. otherwise, it will happen an error when
establishing connection by communication management mechanism.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V2:
- No fixed
V1:
- The initial submit
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index cb3ac54..38fb4c7 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1910,6 +1910,9 @@ static int hns_roce_v2_poll_one(struct hns_roce_cq *hr_cq,
wc->wc_flags |= (roce_get_bit(cqe->byte_32,
V2_CQE_BYTE_32_GRH_S) ?
IB_WC_GRH : 0);
+ wc->port_num = roce_get_field(cqe->byte_32,
+ V2_CQE_BYTE_32_PORTN_M, V2_CQE_BYTE_32_PORTN_S);
+ wc->pkey_index = 0;
}
return 0;
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 6/7] RDMA/hns: Update the verbs of polling for completion
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (4 preceding siblings ...)
2018-01-10 6:39 ` [PATCH V2 for-next 5/7] RDMA/hns: Assign zero for pkey_index of wc " Lijun Ou
@ 2018-01-10 6:39 ` Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device Lijun Ou
2018-01-17 3:50 ` [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08 Jason Gunthorpe
7 siblings, 0 replies; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
If the port is a RoCEv2 port, the remote port address and QP
information which returned for UD will be modified.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V1->V2:
- No fixed
V1:
- The initial submit
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 12 ++++++++++++
drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 2 +-
2 files changed, 13 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 38fb4c7..a1839a3 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1913,6 +1913,18 @@ static int hns_roce_v2_poll_one(struct hns_roce_cq *hr_cq,
wc->port_num = roce_get_field(cqe->byte_32,
V2_CQE_BYTE_32_PORTN_M, V2_CQE_BYTE_32_PORTN_S);
wc->pkey_index = 0;
+ memcpy(wc->smac, cqe->smac, 4);
+ wc->smac[4] = roce_get_field(cqe->byte_28,
+ V2_CQE_BYTE_28_SMAC_4_M,
+ V2_CQE_BYTE_28_SMAC_4_S);
+ wc->smac[5] = roce_get_field(cqe->byte_28,
+ V2_CQE_BYTE_28_SMAC_5_M,
+ V2_CQE_BYTE_28_SMAC_5_S);
+ wc->vlan_id = 0xffff;
+ wc->wc_flags |= (IB_WC_WITH_VLAN | IB_WC_WITH_SMAC);
+ wc->network_hdr_type = roce_get_field(cqe->byte_28,
+ V2_CQE_BYTE_28_PORT_TYPE_M,
+ V2_CQE_BYTE_28_PORT_TYPE_S);
}
return 0;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
index c11b253..ce52c73 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
@@ -766,7 +766,7 @@ struct hns_roce_v2_cqe {
u32 byte_12;
u32 byte_16;
u32 byte_cnt;
- u32 smac;
+ u8 smac[4];
u32 byte_28;
u32 byte_32;
};
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (5 preceding siblings ...)
2018-01-10 6:39 ` [PATCH V2 for-next 6/7] RDMA/hns: Update the verbs of polling for completion Lijun Ou
@ 2018-01-10 6:39 ` Lijun Ou
[not found] ` <1515566393-63888-8-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-17 3:50 ` [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08 Jason Gunthorpe
7 siblings, 1 reply; 18+ messages in thread
From: Lijun Ou @ 2018-01-10 6:39 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
This patch assgin a guid(Global Unique identifer)
value to the hip08 device.
Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
---
V2:
- Use the correct way to update guid from Leon
Romanovsky's review.
V1:
- The initial submit
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
1 file changed, 4 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index a1839a3..2af4948 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -34,6 +34,7 @@
#include <linux/etherdevice.h>
#include <linux/interrupt.h>
#include <linux/kernel.h>
+#include <net/addrconf.h>
#include <rdma/ib_umem.h>
#include "hnae3.h"
@@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
hr_dev->iboe.phy_port[0] = 0;
+ addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
+ hr_dev->iboe.netdevs[0]->dev_addr);
+
for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
hr_dev->irq[i] = pci_irq_vector(handle->pdev,
i + handle->rinfo.base_vector);
--
1.9.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <1515566393-63888-8-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
@ 2018-01-10 6:49 ` Leon Romanovsky
[not found] ` <20180110064937.GE7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2018-01-11 5:46 ` Leon Romanovsky
1 sibling, 1 reply; 18+ messages in thread
From: Leon Romanovsky @ 2018-01-10 6:49 UTC (permalink / raw)
To: Lijun Ou
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
[-- Attachment #1: Type: text/plain, Size: 1640 bytes --]
On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> This patch assgin a guid(Global Unique identifer)
> value to the hip08 device.
>
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> V2:
> - Use the correct way to update guid from Leon
> Romanovsky's review.
>
> V1:
> - The initial submit
> ---
> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> index a1839a3..2af4948 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> @@ -34,6 +34,7 @@
> #include <linux/etherdevice.h>
> #include <linux/interrupt.h>
> #include <linux/kernel.h>
> +#include <net/addrconf.h>
> #include <rdma/ib_umem.h>
>
> #include "hnae3.h"
> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> hr_dev->iboe.phy_port[0] = 0;
>
> + addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> + hr_dev->iboe.netdevs[0]->dev_addr);
> +
Are you sure that it is right thing to do?
ib_dev.node_guid is __be64 and you cast it for u8.
Maybe, it is right, but I don't have ability to test it now.
Thanks
> for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
> hr_dev->irq[i] = pci_irq_vector(handle->pdev,
> i + handle->rinfo.base_vector);
> --
> 1.9.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <20180110064937.GE7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
@ 2018-01-10 9:23 ` oulijun
[not found] ` <f2c33ef8-2457-bccc-e6ac-80711ccec25f-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
0 siblings, 1 reply; 18+ messages in thread
From: oulijun @ 2018-01-10 9:23 UTC (permalink / raw)
To: Leon Romanovsky
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
在 2018/1/10 14:49, Leon Romanovsky 写道:
> On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
>> This patch assgin a guid(Global Unique identifer)
>> value to the hip08 device.
>>
>> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>> ---
>> V2:
>> - Use the correct way to update guid from Leon
>> Romanovsky's review.
>>
>> V1:
>> - The initial submit
>> ---
>> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
>> 1 file changed, 4 insertions(+)
>>
>> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> index a1839a3..2af4948 100644
>> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>> @@ -34,6 +34,7 @@
>> #include <linux/etherdevice.h>
>> #include <linux/interrupt.h>
>> #include <linux/kernel.h>
>> +#include <net/addrconf.h>
>> #include <rdma/ib_umem.h>
>>
>> #include "hnae3.h"
>> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
>> hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
>> hr_dev->iboe.phy_port[0] = 0;
>>
>> + addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
>> + hr_dev->iboe.netdevs[0]->dev_addr);
>> +
>
> Are you sure that it is right thing to do?
>
> ib_dev.node_guid is __be64 and you cast it for u8.
>
> Maybe, it is right, but I don't have ability to test it now.
>
> Thanks
Yes, it can be get rightly. I have test it again
The test result as follows:
root@(none)# cat /sys/class/infiniband/hns_0/node_guid
5816:89ff:fe28:a34b
Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
{
addrconf_addr_eui48_base(eui, addr);
eui[0] ^= 2;
}
The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
have the same usage.
Thanks
Lijun Ou
>
>> for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
>> hr_dev->irq[i] = pci_irq_vector(handle->pdev,
>> i + handle->rinfo.base_vector);
>> --
>> 1.9.1
>>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <f2c33ef8-2457-bccc-e6ac-80711ccec25f-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
@ 2018-01-10 10:29 ` Leon Romanovsky
[not found] ` <20180110102902.GI7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
0 siblings, 1 reply; 18+ messages in thread
From: Leon Romanovsky @ 2018-01-10 10:29 UTC (permalink / raw)
To: oulijun
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
[-- Attachment #1: Type: text/plain, Size: 2869 bytes --]
On Wed, Jan 10, 2018 at 05:23:41PM +0800, oulijun wrote:
> 在 2018/1/10 14:49, Leon Romanovsky 写道:
> > On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> >> This patch assgin a guid(Global Unique identifer)
> >> value to the hip08 device.
> >>
> >> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> >> ---
> >> V2:
> >> - Use the correct way to update guid from Leon
> >> Romanovsky's review.
> >>
> >> V1:
> >> - The initial submit
> >> ---
> >> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
> >> 1 file changed, 4 insertions(+)
> >>
> >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> index a1839a3..2af4948 100644
> >> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> >> @@ -34,6 +34,7 @@
> >> #include <linux/etherdevice.h>
> >> #include <linux/interrupt.h>
> >> #include <linux/kernel.h>
> >> +#include <net/addrconf.h>
> >> #include <rdma/ib_umem.h>
> >>
> >> #include "hnae3.h"
> >> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> >> hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> >> hr_dev->iboe.phy_port[0] = 0;
> >>
> >> + addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> >> + hr_dev->iboe.netdevs[0]->dev_addr);
> >> +
> >
> > Are you sure that it is right thing to do?
> >
> > ib_dev.node_guid is __be64 and you cast it for u8.
> >
> > Maybe, it is right, but I don't have ability to test it now.
> >
> > Thanks
> Yes, it can be get rightly. I have test it again
> The test result as follows:
>
> root@(none)# cat /sys/class/infiniband/hns_0/node_guid
> 5816:89ff:fe28:a34b
>
> Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
> static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
> {
> addrconf_addr_eui48_base(eui, addr);
> eui[0] ^= 2;
> }
>
> The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
> have the same usage.
Indeed the first parameter is u8, but you are converting from __be64 to be u8.
Thanks
>
> Thanks
> Lijun Ou
>
> >
> >> for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
> >> hr_dev->irq[i] = pci_irq_vector(handle->pdev,
> >> i + handle->rinfo.base_vector);
> >> --
> >> 1.9.1
> >>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <20180110102902.GI7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
@ 2018-01-10 21:01 ` Jason Gunthorpe
2018-01-11 2:49 ` oulijun
1 sibling, 0 replies; 18+ messages in thread
From: Jason Gunthorpe @ 2018-01-10 21:01 UTC (permalink / raw)
To: Leon Romanovsky
Cc: oulijun, dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Wed, Jan 10, 2018 at 12:29:02PM +0200, Leon Romanovsky wrote:
> On Wed, Jan 10, 2018 at 05:23:41PM +0800, oulijun wrote:
> > 在 2018/1/10 14:49, Leon Romanovsky 写道:
> > > On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> > >> This patch assgin a guid(Global Unique identifer)
> > >> value to the hip08 device.
> > >>
> > >> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > >> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > >> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> > >> V2:
> > >> - Use the correct way to update guid from Leon
> > >> Romanovsky's review.
> > >>
> > >> V1:
> > >> - The initial submit
> > >> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
> > >> 1 file changed, 4 insertions(+)
> > >>
> > >> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> > >> index a1839a3..2af4948 100644
> > >> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> > >> @@ -34,6 +34,7 @@
> > >> #include <linux/etherdevice.h>
> > >> #include <linux/interrupt.h>
> > >> #include <linux/kernel.h>
> > >> +#include <net/addrconf.h>
> > >> #include <rdma/ib_umem.h>
> > >>
> > >> #include "hnae3.h"
> > >> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> > >> hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> > >> hr_dev->iboe.phy_port[0] = 0;
> > >>
> > >> + addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> > >> + hr_dev->iboe.netdevs[0]->dev_addr);
> > >> +
> > >
> > > Are you sure that it is right thing to do?
> > >
> > > ib_dev.node_guid is __be64 and you cast it for u8.
> > >
> > > Maybe, it is right, but I don't have ability to test it now.
> > >
> > > Thanks
> > Yes, it can be get rightly. I have test it again
> > The test result as follows:
> >
> > root@(none)# cat /sys/class/infiniband/hns_0/node_guid
> > 5816:89ff:fe28:a34b
> >
> > Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
> > static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
> > {
> > addrconf_addr_eui48_base(eui, addr);
> > eui[0] ^= 2;
> > }
> >
> > The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
> > have the same usage.
>
> Indeed the first parameter is u8, but you are converting from __be64 to be u8.
addrconf_addr_eui48 accepts an array of 6 u8's in big endian byte
order representing the MAC.
If you use a be64 to hold the MAC then it must be unnaturally shifted
<< 16, eg
htobe64(0x1234 << 16);
Will work to set the mac to 00:00:00:00:12:34
So, it is really weird to store a mac in a __be64, but it can be done
correctly...
The *really* sketchy thing is pssing something called the node_guid to
something accepting a eui48. The node_guid is an eui64, and needs
translation before it can become an eui48, so *something* is wrong and
broken here.
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <20180110102902.GI7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2018-01-10 21:01 ` Jason Gunthorpe
@ 2018-01-11 2:49 ` oulijun
[not found] ` <71771a6a-ef08-1750-69c3-f2fca8294751-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
1 sibling, 1 reply; 18+ messages in thread
From: oulijun @ 2018-01-11 2:49 UTC (permalink / raw)
To: Leon Romanovsky
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
在 2018/1/10 18:29, Leon Romanovsky 写道:
> On Wed, Jan 10, 2018 at 05:23:41PM +0800, oulijun wrote:
>> 在 2018/1/10 14:49, Leon Romanovsky 写道:
>>> On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
>>>> This patch assgin a guid(Global Unique identifer)
>>>> value to the hip08 device.
>>>>
>>>> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
>>>> ---
>>>> V2:
>>>> - Use the correct way to update guid from Leon
>>>> Romanovsky's review.
>>>>
>>>> V1:
>>>> - The initial submit
>>>> ---
>>>> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
>>>> 1 file changed, 4 insertions(+)
>>>>
>>>> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> index a1839a3..2af4948 100644
>>>> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
>>>> @@ -34,6 +34,7 @@
>>>> #include <linux/etherdevice.h>
>>>> #include <linux/interrupt.h>
>>>> #include <linux/kernel.h>
>>>> +#include <net/addrconf.h>
>>>> #include <rdma/ib_umem.h>
>>>>
>>>> #include "hnae3.h"
>>>> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
>>>> hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
>>>> hr_dev->iboe.phy_port[0] = 0;
>>>>
>>>> + addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
>>>> + hr_dev->iboe.netdevs[0]->dev_addr);
>>>> +
>>>
>>> Are you sure that it is right thing to do?
>>>
>>> ib_dev.node_guid is __be64 and you cast it for u8.
>>>
>>> Maybe, it is right, but I don't have ability to test it now.
>>>
>>> Thanks
>> Yes, it can be get rightly. I have test it again
>> The test result as follows:
>>
>> root@(none)# cat /sys/class/infiniband/hns_0/node_guid
>> 5816:89ff:fe28:a34b
>>
>> Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
>> static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
>> {
>> addrconf_addr_eui48_base(eui, addr);
>> eui[0] ^= 2;
>> }
>>
>> The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
>> have the same usage.
>
> Indeed the first parameter is u8, but you are converting from __be64 to be u8.
>
> Thanks
>
Yes, the type of node_guid is defined by ib core. the function of addrconf_addr_eui48 is also
defined by ib core. I could not modify them. Did I need to change u8 to unsigned char?
thanks
>>
>> Thanks
>> Lijun Ou
>>
>>>
>>>> for (i = 0; i < HNS_ROCE_V2_MAX_IRQ_NUM; i++)
>>>> hr_dev->irq[i] = pci_irq_vector(handle->pdev,
>>>> i + handle->rinfo.base_vector);
>>>> --
>>>> 1.9.1
>>>>
>>
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <71771a6a-ef08-1750-69c3-f2fca8294751-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
@ 2018-01-11 5:31 ` Jason Gunthorpe
0 siblings, 0 replies; 18+ messages in thread
From: Jason Gunthorpe @ 2018-01-11 5:31 UTC (permalink / raw)
To: oulijun
Cc: Leon Romanovsky, dledford-H+wXaHxf7aLQT0dZR+AlfA,
linux-rdma-u79uwXL29TY76Z2rM5mHXA, Yuval Shaia
On Thu, Jan 11, 2018 at 10:49:57AM +0800, oulijun wrote:
> >>>> @@ -4679,6 +4680,9 @@ static int hns_roce_hw_v2_get_cfg(struct hns_roce_dev *hr_dev,
> >>>> hr_dev->iboe.netdevs[0] = handle->rinfo.netdev;
> >>>> hr_dev->iboe.phy_port[0] = 0;
> >>>>
> >>>> + addrconf_addr_eui48((u8 *)&hr_dev->ib_dev.node_guid,
> >>>> + hr_dev->iboe.netdevs[0]->dev_addr);
> >>>> +
> >>>
> >>> Are you sure that it is right thing to do?
> >>>
> >>> ib_dev.node_guid is __be64 and you cast it for u8.
> >>>
> >>> Maybe, it is right, but I don't have ability to test it now.
> >>>
> >>> Thanks
> >> Yes, it can be get rightly. I have test it again
> >> The test result as follows:
> >>
> >> root@(none)# cat /sys/class/infiniband/hns_0/node_guid
> >> 5816:89ff:fe28:a34b
> >>
> >> Besides, I confirm the orign definition for addrconf_addr_eui48, it defined as follows:
> >> static inline void addrconf_addr_eui48(u8 *eui, const char *const addr)
> >> {
> >> addrconf_addr_eui48_base(eui, addr);
> >> eui[0] ^= 2;
> >> }
> >>
> >> The first input parameter type is u8 * and the patch [v4] {net,IB}/{rxe,usnic}: Utilize generic mac to eui32 function
> >> have the same usage.
> >
> > Indeed the first parameter is u8, but you are converting from __be64 to be u8.
> >
> > Thanks
> >
> Yes, the type of node_guid is defined by ib core. the function of addrconf_addr_eui48 is also
> defined by ib core. I could not modify them. Did I need to change u8 to unsigned char?
Actually, now that I look more closely, what you have is right.
It is just a very strange and misleading name of a function
'addrconf_addr_eui48' which takes in an eui48 and writes out a IB
GUID. Would have expected 'addrconf_ib_guid_eui48' or something..
Even more confusing because the commit message talks about eui32 and
misses a signed-off-by? Baffling.
.. and there are dragons here, because the IPv6 interface id (eg what
the similar addrconf_ifid_eui48 is handling) uses an inverted U/L bit
to the IEEE EUI-64.
.. and I can't quite recall what the IBTA eventually settled on as the
standard for IB - if it was EUI-64 or modified IPv6 EUI-64 ..
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device
[not found] ` <1515566393-63888-8-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 6:49 ` Leon Romanovsky
@ 2018-01-11 5:46 ` Leon Romanovsky
1 sibling, 0 replies; 18+ messages in thread
From: Leon Romanovsky @ 2018-01-11 5:46 UTC (permalink / raw)
To: Lijun Ou
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
[-- Attachment #1: Type: text/plain, Size: 687 bytes --]
On Wed, Jan 10, 2018 at 02:39:53PM +0800, Lijun Ou wrote:
> This patch assgin a guid(Global Unique identifer)
> value to the hip08 device.
>
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> V2:
> - Use the correct way to update guid from Leon
> Romanovsky's review.
>
> V1:
> - The initial submit
> ---
> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++++
> 1 file changed, 4 insertions(+)
>
Thanks,
Reviewed-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type in hip08
[not found] ` <1515566393-63888-5-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
@ 2018-01-12 21:22 ` kbuild test robot
2018-01-16 9:00 ` oulijun
1 sibling, 0 replies; 18+ messages in thread
From: kbuild test robot @ 2018-01-12 21:22 UTC (permalink / raw)
To: Lijun Ou
Cc: kbuild-all-JC7UmRfGjtg, dledford-H+wXaHxf7aLQT0dZR+AlfA,
jgg-uk2M96/98Pc, leon-DgEjT+Ai2ygdnm+yROfE0A,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
Hi Lijun,
I love your patch! Perhaps something to improve:
[auto build test WARNING on rdma/for-next]
[also build test WARNING on next-20180112]
[cannot apply to linus/master v4.15-rc7]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
url: https://github.com/0day-ci/linux/commits/Lijun-Ou/Add-CM-and-a-bugfix-to-hip08/20180112-215900
base: https://git.kernel.org/pub/scm/linux/kernel/git/rdma/rdma.git for-next
reproduce:
# apt-get install sparse
make ARCH=x86_64 allmodconfig
make C=1 CF=-D__CHECK_ENDIAN__
sparse warnings: (new ones prefixed by >>)
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:49:20: sparse: incorrect type in assignment (different base types) @@ expected restricted __be32 lkey @@ got restricted __be32 lkey @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:49:20: expected restricted __be32 lkey
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:49:20: got restricted __le32 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:50:20: sparse: incorrect type in assignment (different base types) @@ expected restricted __be64 addr @@ got restricted __be64 addr @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:50:20: expected restricted __be64 addr
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:50:20: got restricted __le64 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:51:20: sparse: incorrect type in assignment (different base types) @@ expected restricted __be32 len @@ got restricted __be32 len @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:51:20: expected restricted __be32 len
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:51:20: got restricted __le32 <noident>
>> drivers/infiniband/hw/hns/hns_roce_hw_v2.c:222:45: sparse: incorrect type in assignment (different base types) @@ expected unsigned int immtdata @@ got gned] immtdata @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:222:45: expected unsigned int immtdata
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:222:45: got restricted __be32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:263:25: sparse: cast from restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:282:25: sparse: restricted __le32 degrades to integer
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:315:53: sparse: incorrect type in assignment (different base types) @@ expected unsigned int inv_key_immtdata @@ got gned] inv_key_immtdata @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:315:53: expected unsigned int inv_key_immtdata
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:315:53: got restricted __be32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:338:49: sparse: incorrect type in assignment (different base types) @@ expected unsigned int rkey @@ got ed int rkey @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:338:49: expected unsigned int rkey
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:338:49: got restricted __le32 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:340:47: sparse: incorrect type in assignment (different base types) @@ expected unsigned long long va @@ got g long va @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:340:47: expected unsigned long long va
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:340:47: got restricted __le64 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:348:49: sparse: incorrect type in assignment (different base types) @@ expected unsigned int rkey @@ got ed int rkey @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:348:49: expected unsigned int rkey
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:348:49: got restricted __le32 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:350:47: sparse: incorrect type in assignment (different base types) @@ expected unsigned long long va @@ got g long va @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:350:47: expected unsigned long long va
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:350:47: got restricted __le64 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:358:49: sparse: incorrect type in assignment (different base types) @@ expected unsigned int rkey @@ got ed int rkey @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:358:49: expected unsigned int rkey
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:358:49: got restricted __le32 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:360:47: sparse: incorrect type in assignment (different base types) @@ expected unsigned long long va @@ got g long va @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:360:47: expected unsigned long long va
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:360:47: got restricted __le64 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:443:17: sparse: cast from restricted __le64
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:692:22: sparse: incorrect type in assignment (different base types) @@ expected unsigned short opcode @@ got short opcode @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:692:22: expected unsigned short opcode
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:692:22: got restricted __le16 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:693:20: sparse: incorrect type in assignment (different base types) @@ expected unsigned short flag @@ got short flag @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:693:20: expected unsigned short flag
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:693:20: got restricted __le16 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:696:28: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:696:28: left side has type unsigned short
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:696:28: right side has type restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:698:28: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:698:28: left side has type unsigned short
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:698:28: right side has type restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:832:26: sparse: cast to restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:832:26: sparse: cast from restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:833:29: sparse: cast to restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:833:29: sparse: cast from restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:848:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:851:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:870:38: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:870:38: left side has type unsigned short
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:870:38: right side has type restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:872:38: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:872:38: left side has type unsigned short
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:872:38: right side has type fouled restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:881:35: sparse: restricted __le32 degrades to integer
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:884:36: sparse: restricted __le32 degrades to integer
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:887:35: sparse: restricted __le32 degrades to integer
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:890:35: sparse: restricted __le32 degrades to integer
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:913:38: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:913:38: left side has type unsigned short
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:913:38: right side has type restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:915:38: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:915:38: left side has type unsigned short
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:915:38: right side has type fouled restricted __le16
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1202:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned long val @@ got restunsigned long val @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1202:22: expected unsigned long val
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1202:22: got restricted __le64 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1203:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned long val @@ got restunsigned long val @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1203:22: expected unsigned long val
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1203:22: got restricted __le64 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1208:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned int val @@ got restrunsigned int val @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1208:22: expected unsigned int val
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1208:22: got restricted __le32 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1209:22: sparse: incorrect type in argument 1 (different base types) @@ expected unsigned int val @@ got restrunsigned int val @@
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1209:22: expected unsigned int val
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1209:22: got restricted __le32 <noident>
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1321:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1323:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: sparse: invalid assignment: &=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: right side has type unsigned long
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: sparse: invalid assignment: |=
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: left side has type restricted __le32
drivers/infiniband/hw/hns/hns_roce_hw_v2.c:1326:9: right side has type unsigned long
vim +222 drivers/infiniband/hw/hns/hns_roce_hw_v2.c
45
46 static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
47 struct ib_sge *sg)
48 {
> 49 dseg->lkey = cpu_to_le32(sg->lkey);
50 dseg->addr = cpu_to_le64(sg->addr);
51 dseg->len = cpu_to_le32(sg->length);
52 }
53
54 static int set_rwqe_data_seg(struct ib_qp *ibqp, struct ib_send_wr *wr,
55 struct hns_roce_v2_rc_send_wqe *rc_sq_wqe,
56 void *wqe, unsigned int *sge_ind,
57 struct ib_send_wr **bad_wr)
58 {
59 struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
60 struct hns_roce_v2_wqe_data_seg *dseg = wqe;
61 struct hns_roce_qp *qp = to_hr_qp(ibqp);
62 int i;
63
64 if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
65 if (rc_sq_wqe->msg_len > hr_dev->caps.max_sq_inline) {
66 *bad_wr = wr;
67 dev_err(hr_dev->dev, "inline len(1-%d)=%d, illegal",
68 rc_sq_wqe->msg_len, hr_dev->caps.max_sq_inline);
69 return -EINVAL;
70 }
71
72 for (i = 0; i < wr->num_sge; i++) {
73 memcpy(wqe, ((void *)wr->sg_list[i].addr),
74 wr->sg_list[i].length);
75 wqe += wr->sg_list[i].length;
76 }
77
78 roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_INLINE_S,
79 1);
80 } else {
81 if (wr->num_sge <= 2) {
82 for (i = 0; i < wr->num_sge; i++) {
83 if (likely(wr->sg_list[i].length)) {
84 set_data_seg_v2(dseg, wr->sg_list + i);
85 dseg++;
86 }
87 }
88 } else {
89 roce_set_field(rc_sq_wqe->byte_20,
90 V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
91 V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
92 (*sge_ind) & (qp->sge.sge_cnt - 1));
93
94 for (i = 0; i < 2; i++) {
95 if (likely(wr->sg_list[i].length)) {
96 set_data_seg_v2(dseg, wr->sg_list + i);
97 dseg++;
98 }
99 }
100
101 dseg = get_send_extend_sge(qp,
102 (*sge_ind) & (qp->sge.sge_cnt - 1));
103
104 for (i = 0; i < wr->num_sge - 2; i++) {
105 if (likely(wr->sg_list[i + 2].length)) {
106 set_data_seg_v2(dseg,
107 wr->sg_list + 2 + i);
108 dseg++;
109 (*sge_ind)++;
110 }
111 }
112 }
113
114 roce_set_field(rc_sq_wqe->byte_16,
115 V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
116 V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
117 }
118
119 return 0;
120 }
121
122 static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
123 struct ib_send_wr **bad_wr)
124 {
125 struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
126 struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
127 struct hns_roce_v2_ud_send_wqe *ud_sq_wqe;
128 struct hns_roce_v2_rc_send_wqe *rc_sq_wqe;
129 struct hns_roce_qp *qp = to_hr_qp(ibqp);
130 struct hns_roce_v2_wqe_data_seg *dseg;
131 struct device *dev = hr_dev->dev;
132 struct hns_roce_v2_db sq_db;
133 unsigned int sge_ind = 0;
134 unsigned int owner_bit;
135 unsigned long flags;
136 unsigned int ind;
137 void *wqe = NULL;
138 bool loopback;
139 int ret = 0;
140 u8 *smac;
141 int nreq;
142 int i;
143
144 if (unlikely(ibqp->qp_type != IB_QPT_RC &&
145 ibqp->qp_type != IB_QPT_GSI &&
146 ibqp->qp_type != IB_QPT_UD)) {
147 dev_err(dev, "Not supported QP(0x%x)type!\n", ibqp->qp_type);
148 *bad_wr = NULL;
149 return -EOPNOTSUPP;
150 }
151
152 if (unlikely(qp->state == IB_QPS_RESET || qp->state == IB_QPS_INIT ||
153 qp->state == IB_QPS_RTR)) {
154 dev_err(dev, "Post WQE fail, QP state %d err!\n", qp->state);
155 *bad_wr = wr;
156 return -EINVAL;
157 }
158
159 spin_lock_irqsave(&qp->sq.lock, flags);
160 ind = qp->sq_next_wqe;
161 sge_ind = qp->next_sge;
162
163 for (nreq = 0; wr; ++nreq, wr = wr->next) {
164 if (hns_roce_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)) {
165 ret = -ENOMEM;
166 *bad_wr = wr;
167 goto out;
168 }
169
170 if (unlikely(wr->num_sge > qp->sq.max_gs)) {
171 dev_err(dev, "num_sge=%d > qp->sq.max_gs=%d\n",
172 wr->num_sge, qp->sq.max_gs);
173 ret = -EINVAL;
174 *bad_wr = wr;
175 goto out;
176 }
177
178 wqe = get_send_wqe(qp, ind & (qp->sq.wqe_cnt - 1));
179 qp->sq.wrid[(qp->sq.head + nreq) & (qp->sq.wqe_cnt - 1)] =
180 wr->wr_id;
181
182 owner_bit = ~(qp->sq.head >> ilog2(qp->sq.wqe_cnt)) & 0x1;
183
184 /* Corresponding to the QP type, wqe process separately */
185 if (ibqp->qp_type == IB_QPT_GSI) {
186 ud_sq_wqe = wqe;
187 memset(ud_sq_wqe, 0, sizeof(*ud_sq_wqe));
188
189 roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_0_M,
190 V2_UD_SEND_WQE_DMAC_0_S, ah->av.mac[0]);
191 roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_1_M,
192 V2_UD_SEND_WQE_DMAC_1_S, ah->av.mac[1]);
193 roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_2_M,
194 V2_UD_SEND_WQE_DMAC_2_S, ah->av.mac[2]);
195 roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_3_M,
196 V2_UD_SEND_WQE_DMAC_3_S, ah->av.mac[3]);
197 roce_set_field(ud_sq_wqe->byte_48,
198 V2_UD_SEND_WQE_BYTE_48_DMAC_4_M,
199 V2_UD_SEND_WQE_BYTE_48_DMAC_4_S,
200 ah->av.mac[4]);
201 roce_set_field(ud_sq_wqe->byte_48,
202 V2_UD_SEND_WQE_BYTE_48_DMAC_5_M,
203 V2_UD_SEND_WQE_BYTE_48_DMAC_5_S,
204 ah->av.mac[5]);
205
206 /* MAC loopback */
207 smac = (u8 *)hr_dev->dev_addr[qp->port];
208 loopback = ether_addr_equal_unaligned(ah->av.mac,
209 smac) ? 1 : 0;
210
211 roce_set_bit(ud_sq_wqe->byte_40,
212 V2_UD_SEND_WQE_BYTE_40_LBI_S, loopback);
213
214 roce_set_field(ud_sq_wqe->byte_4,
215 V2_UD_SEND_WQE_BYTE_4_OPCODE_M,
216 V2_UD_SEND_WQE_BYTE_4_OPCODE_S,
217 HNS_ROCE_V2_WQE_OP_SEND);
218
219 for (i = 0; i < wr->num_sge; i++)
220 ud_sq_wqe->msg_len += wr->sg_list[i].length;
221
> 222 ud_sq_wqe->immtdata = send_ieth(wr);
223
224 /* Set sig attr */
225 roce_set_bit(ud_sq_wqe->byte_4,
226 V2_UD_SEND_WQE_BYTE_4_CQE_S,
227 (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
228
229 /* Set se attr */
230 roce_set_bit(ud_sq_wqe->byte_4,
231 V2_UD_SEND_WQE_BYTE_4_SE_S,
232 (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
233
234 roce_set_bit(ud_sq_wqe->byte_4,
235 V2_UD_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
236
237 roce_set_field(ud_sq_wqe->byte_16,
238 V2_UD_SEND_WQE_BYTE_16_PD_M,
239 V2_UD_SEND_WQE_BYTE_16_PD_S,
240 to_hr_pd(ibqp->pd)->pdn);
241
242 roce_set_field(ud_sq_wqe->byte_16,
243 V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M,
244 V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S,
245 wr->num_sge);
246
247 roce_set_field(ud_sq_wqe->byte_20,
248 V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
249 V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
250 sge_ind & (qp->sge.sge_cnt - 1));
251
252 roce_set_field(ud_sq_wqe->byte_24,
253 V2_UD_SEND_WQE_BYTE_24_UDPSPN_M,
254 V2_UD_SEND_WQE_BYTE_24_UDPSPN_S, 0);
255 ud_sq_wqe->qkey =
256 cpu_to_be32(ud_wr(wr)->remote_qkey & 0x80000000) ?
257 qp->qkey : ud_wr(wr)->remote_qkey;
258 roce_set_field(ud_sq_wqe->byte_32,
259 V2_UD_SEND_WQE_BYTE_32_DQPN_M,
260 V2_UD_SEND_WQE_BYTE_32_DQPN_S,
261 ud_wr(wr)->remote_qpn);
262
263 roce_set_field(ud_sq_wqe->byte_36,
264 V2_UD_SEND_WQE_BYTE_36_VLAN_M,
265 V2_UD_SEND_WQE_BYTE_36_VLAN_S,
266 ah->av.vlan);
267 roce_set_field(ud_sq_wqe->byte_36,
268 V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M,
269 V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S,
270 ah->av.hop_limit);
271 roce_set_field(ud_sq_wqe->byte_36,
272 V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
273 V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
274 0);
275 roce_set_field(ud_sq_wqe->byte_36,
276 V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
277 V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
278 0);
279 roce_set_field(ud_sq_wqe->byte_40,
280 V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
281 V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, 0);
282 roce_set_field(ud_sq_wqe->byte_40,
283 V2_UD_SEND_WQE_BYTE_40_SL_M,
284 V2_UD_SEND_WQE_BYTE_40_SL_S,
285 ah->av.sl_tclass_flowlabel >>
286 HNS_ROCE_SL_SHIFT);
287 roce_set_field(ud_sq_wqe->byte_40,
288 V2_UD_SEND_WQE_BYTE_40_PORTN_M,
289 V2_UD_SEND_WQE_BYTE_40_PORTN_S,
290 qp->port);
291
292 roce_set_field(ud_sq_wqe->byte_48,
293 V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M,
294 V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S,
295 hns_get_gid_index(hr_dev, qp->phy_port,
296 ah->av.gid_index));
297
298 memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0],
299 GID_LEN_V2);
300
301 dseg = get_send_extend_sge(qp,
302 sge_ind & (qp->sge.sge_cnt - 1));
303 for (i = 0; i < wr->num_sge; i++) {
304 set_data_seg_v2(dseg + i, wr->sg_list + i);
305 sge_ind++;
306 }
307
308 ind++;
309 } else if (ibqp->qp_type == IB_QPT_RC) {
310 rc_sq_wqe = wqe;
311 memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
312 for (i = 0; i < wr->num_sge; i++)
313 rc_sq_wqe->msg_len += wr->sg_list[i].length;
314
315 rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
316
317 roce_set_bit(rc_sq_wqe->byte_4,
318 V2_RC_SEND_WQE_BYTE_4_FENCE_S,
319 (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
320
321 roce_set_bit(rc_sq_wqe->byte_4,
322 V2_RC_SEND_WQE_BYTE_4_SE_S,
323 (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
324
325 roce_set_bit(rc_sq_wqe->byte_4,
326 V2_RC_SEND_WQE_BYTE_4_CQE_S,
327 (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
328
329 roce_set_bit(rc_sq_wqe->byte_4,
330 V2_RC_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
331
332 switch (wr->opcode) {
333 case IB_WR_RDMA_READ:
334 roce_set_field(rc_sq_wqe->byte_4,
335 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
336 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
337 HNS_ROCE_V2_WQE_OP_RDMA_READ);
338 rc_sq_wqe->rkey =
339 cpu_to_le32(rdma_wr(wr)->rkey);
340 rc_sq_wqe->va =
341 cpu_to_le64(rdma_wr(wr)->remote_addr);
342 break;
343 case IB_WR_RDMA_WRITE:
344 roce_set_field(rc_sq_wqe->byte_4,
345 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
346 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
347 HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
348 rc_sq_wqe->rkey =
349 cpu_to_le32(rdma_wr(wr)->rkey);
350 rc_sq_wqe->va =
351 cpu_to_le64(rdma_wr(wr)->remote_addr);
352 break;
353 case IB_WR_RDMA_WRITE_WITH_IMM:
354 roce_set_field(rc_sq_wqe->byte_4,
355 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
356 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
357 HNS_ROCE_V2_WQE_OP_RDMA_WRITE_WITH_IMM);
358 rc_sq_wqe->rkey =
359 cpu_to_le32(rdma_wr(wr)->rkey);
360 rc_sq_wqe->va =
361 cpu_to_le64(rdma_wr(wr)->remote_addr);
362 break;
363 case IB_WR_SEND:
364 roce_set_field(rc_sq_wqe->byte_4,
365 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
366 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
367 HNS_ROCE_V2_WQE_OP_SEND);
368 break;
369 case IB_WR_SEND_WITH_INV:
370 roce_set_field(rc_sq_wqe->byte_4,
371 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
372 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
373 HNS_ROCE_V2_WQE_OP_SEND_WITH_INV);
374 break;
375 case IB_WR_SEND_WITH_IMM:
376 roce_set_field(rc_sq_wqe->byte_4,
377 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
378 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
379 HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
380 break;
381 case IB_WR_LOCAL_INV:
382 roce_set_field(rc_sq_wqe->byte_4,
383 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
384 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
385 HNS_ROCE_V2_WQE_OP_LOCAL_INV);
386 break;
387 case IB_WR_ATOMIC_CMP_AND_SWP:
388 roce_set_field(rc_sq_wqe->byte_4,
389 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
390 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
391 HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
392 break;
393 case IB_WR_ATOMIC_FETCH_AND_ADD:
394 roce_set_field(rc_sq_wqe->byte_4,
395 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
396 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
397 HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
398 break;
399 case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
400 roce_set_field(rc_sq_wqe->byte_4,
401 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
402 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
403 HNS_ROCE_V2_WQE_OP_ATOM_MSK_CMP_AND_SWAP);
404 break;
405 case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
406 roce_set_field(rc_sq_wqe->byte_4,
407 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
408 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
409 HNS_ROCE_V2_WQE_OP_ATOM_MSK_FETCH_AND_ADD);
410 break;
411 default:
412 roce_set_field(rc_sq_wqe->byte_4,
413 V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
414 V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
415 HNS_ROCE_V2_WQE_OP_MASK);
416 break;
417 }
418
419 wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
420 dseg = wqe;
421
422 ret = set_rwqe_data_seg(ibqp, wr, rc_sq_wqe, wqe,
423 &sge_ind, bad_wr);
424 if (ret)
425 goto out;
426 ind++;
427 } else {
428 dev_err(dev, "Illegal qp_type(0x%x)\n", ibqp->qp_type);
429 spin_unlock_irqrestore(&qp->sq.lock, flags);
430 return -EOPNOTSUPP;
431 }
432 }
433
434 out:
435 if (likely(nreq)) {
436 qp->sq.head += nreq;
437 /* Memory barrier */
438 wmb();
439
440 sq_db.byte_4 = 0;
441 sq_db.parameter = 0;
442
443 roce_set_field(sq_db.byte_4, V2_DB_BYTE_4_TAG_M,
444 V2_DB_BYTE_4_TAG_S, qp->doorbell_qpn);
445 roce_set_field(sq_db.byte_4, V2_DB_BYTE_4_CMD_M,
446 V2_DB_BYTE_4_CMD_S, HNS_ROCE_V2_SQ_DB);
447 roce_set_field(sq_db.parameter, V2_DB_PARAMETER_CONS_IDX_M,
448 V2_DB_PARAMETER_CONS_IDX_S,
449 qp->sq.head & ((qp->sq.wqe_cnt << 1) - 1));
450 roce_set_field(sq_db.parameter, V2_DB_PARAMETER_SL_M,
451 V2_DB_PARAMETER_SL_S, qp->sl);
452
453 hns_roce_write64_k((__be32 *)&sq_db, qp->sq.db_reg_l);
454
455 qp->sq_next_wqe = ind;
456 qp->next_sge = sge_ind;
457 }
458
459 spin_unlock_irqrestore(&qp->sq.lock, flags);
460
461 return ret;
462 }
463
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type in hip08
[not found] ` <1515566393-63888-5-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-12 21:22 ` kbuild test robot
@ 2018-01-16 9:00 ` oulijun
1 sibling, 0 replies; 18+ messages in thread
From: oulijun @ 2018-01-16 9:00 UTC (permalink / raw)
To: dledford-H+wXaHxf7aLQT0dZR+AlfA, jgg-uk2M96/98Pc
Cc: leon-DgEjT+Ai2ygdnm+yROfE0A, linux-rdma-u79uwXL29TY76Z2rM5mHXA
Hi, Dledford and Jason
I have awared the warnings by kbuild test robot. It may be not intrdouced
by this patchset. I have doubt that the kernel code may have the same
question with sparse check when process the userspace according to the following patch:
See rdma-core commit bffd380cfe56 ("libhns: Make the provider sparse
clean")
I have been planing to fix it in next fixes patch. Could I send a flowup patch separately?
Thanks.
Lijun Ou
在 2018/1/10 14:39, Lijun Ou 写道:
> This patch mainly configure the fileds of sq wqe of ud
> type when posting wr of gsi qp type.
>
> Signed-off-by: Lijun Ou <oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Yixian Liu <liuyixian-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Wei Hu (Xavier) <xavier.huwei-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
> ---
> V1->V2:
> - return errno code directly and delete unnecessary
> initialized value of ret from Leon Romanvosky's
> advice.
> - Add necessary option for free sq lock when qp is
> illegal qp type.
>
> V1:
> - The initial submit
> ---
> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 446 +++++++++++++++++++----------
> drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 84 ++++++
> 2 files changed, 385 insertions(+), 145 deletions(-)
>
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> index e53cd7d..cb3ac54 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
> @@ -51,26 +51,99 @@ static void set_data_seg_v2(struct hns_roce_v2_wqe_data_seg *dseg,
> dseg->len = cpu_to_le32(sg->length);
> }
>
> +static int set_rwqe_data_seg(struct ib_qp *ibqp, struct ib_send_wr *wr,
> + struct hns_roce_v2_rc_send_wqe *rc_sq_wqe,
> + void *wqe, unsigned int *sge_ind,
> + struct ib_send_wr **bad_wr)
> +{
> + struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> + struct hns_roce_v2_wqe_data_seg *dseg = wqe;
> + struct hns_roce_qp *qp = to_hr_qp(ibqp);
> + int i;
> +
> + if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
> + if (rc_sq_wqe->msg_len > hr_dev->caps.max_sq_inline) {
> + *bad_wr = wr;
> + dev_err(hr_dev->dev, "inline len(1-%d)=%d, illegal",
> + rc_sq_wqe->msg_len, hr_dev->caps.max_sq_inline);
> + return -EINVAL;
> + }
> +
> + for (i = 0; i < wr->num_sge; i++) {
> + memcpy(wqe, ((void *)wr->sg_list[i].addr),
> + wr->sg_list[i].length);
> + wqe += wr->sg_list[i].length;
> + }
> +
> + roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_INLINE_S,
> + 1);
> + } else {
> + if (wr->num_sge <= 2) {
> + for (i = 0; i < wr->num_sge; i++) {
> + if (likely(wr->sg_list[i].length)) {
> + set_data_seg_v2(dseg, wr->sg_list + i);
> + dseg++;
> + }
> + }
> + } else {
> + roce_set_field(rc_sq_wqe->byte_20,
> + V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
> + V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
> + (*sge_ind) & (qp->sge.sge_cnt - 1));
> +
> + for (i = 0; i < 2; i++) {
> + if (likely(wr->sg_list[i].length)) {
> + set_data_seg_v2(dseg, wr->sg_list + i);
> + dseg++;
> + }
> + }
> +
> + dseg = get_send_extend_sge(qp,
> + (*sge_ind) & (qp->sge.sge_cnt - 1));
> +
> + for (i = 0; i < wr->num_sge - 2; i++) {
> + if (likely(wr->sg_list[i + 2].length)) {
> + set_data_seg_v2(dseg,
> + wr->sg_list + 2 + i);
> + dseg++;
> + (*sge_ind)++;
> + }
> + }
> + }
> +
> + roce_set_field(rc_sq_wqe->byte_16,
> + V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
> + V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S, wr->num_sge);
> + }
> +
> + return 0;
> +}
> +
> static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
> struct ib_send_wr **bad_wr)
> {
> struct hns_roce_dev *hr_dev = to_hr_dev(ibqp->device);
> + struct hns_roce_ah *ah = to_hr_ah(ud_wr(wr)->ah);
> + struct hns_roce_v2_ud_send_wqe *ud_sq_wqe;
> struct hns_roce_v2_rc_send_wqe *rc_sq_wqe;
> struct hns_roce_qp *qp = to_hr_qp(ibqp);
> struct hns_roce_v2_wqe_data_seg *dseg;
> struct device *dev = hr_dev->dev;
> struct hns_roce_v2_db sq_db;
> unsigned int sge_ind = 0;
> - unsigned int wqe_sz = 0;
> unsigned int owner_bit;
> unsigned long flags;
> unsigned int ind;
> void *wqe = NULL;
> + bool loopback;
> int ret = 0;
> + u8 *smac;
> int nreq;
> int i;
>
> - if (unlikely(ibqp->qp_type != IB_QPT_RC)) {
> + if (unlikely(ibqp->qp_type != IB_QPT_RC &&
> + ibqp->qp_type != IB_QPT_GSI &&
> + ibqp->qp_type != IB_QPT_UD)) {
> dev_err(dev, "Not supported QP(0x%x)type!\n", ibqp->qp_type);
> *bad_wr = NULL;
> return -EOPNOTSUPP;
> @@ -107,172 +180,255 @@ static int hns_roce_v2_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
> wr->wr_id;
>
> owner_bit = ~(qp->sq.head >> ilog2(qp->sq.wqe_cnt)) & 0x1;
> - rc_sq_wqe = wqe;
> - memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
> - for (i = 0; i < wr->num_sge; i++)
> - rc_sq_wqe->msg_len += wr->sg_list[i].length;
>
> - rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
> + /* Corresponding to the QP type, wqe process separately */
> + if (ibqp->qp_type == IB_QPT_GSI) {
> + ud_sq_wqe = wqe;
> + memset(ud_sq_wqe, 0, sizeof(*ud_sq_wqe));
> +
> + roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_0_M,
> + V2_UD_SEND_WQE_DMAC_0_S, ah->av.mac[0]);
> + roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_1_M,
> + V2_UD_SEND_WQE_DMAC_1_S, ah->av.mac[1]);
> + roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_2_M,
> + V2_UD_SEND_WQE_DMAC_2_S, ah->av.mac[2]);
> + roce_set_field(ud_sq_wqe->dmac, V2_UD_SEND_WQE_DMAC_3_M,
> + V2_UD_SEND_WQE_DMAC_3_S, ah->av.mac[3]);
> + roce_set_field(ud_sq_wqe->byte_48,
> + V2_UD_SEND_WQE_BYTE_48_DMAC_4_M,
> + V2_UD_SEND_WQE_BYTE_48_DMAC_4_S,
> + ah->av.mac[4]);
> + roce_set_field(ud_sq_wqe->byte_48,
> + V2_UD_SEND_WQE_BYTE_48_DMAC_5_M,
> + V2_UD_SEND_WQE_BYTE_48_DMAC_5_S,
> + ah->av.mac[5]);
> +
> + /* MAC loopback */
> + smac = (u8 *)hr_dev->dev_addr[qp->port];
> + loopback = ether_addr_equal_unaligned(ah->av.mac,
> + smac) ? 1 : 0;
> +
> + roce_set_bit(ud_sq_wqe->byte_40,
> + V2_UD_SEND_WQE_BYTE_40_LBI_S, loopback);
> +
> + roce_set_field(ud_sq_wqe->byte_4,
> + V2_UD_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_UD_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_SEND);
>
> - roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_FENCE_S,
> - (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
> + for (i = 0; i < wr->num_sge; i++)
> + ud_sq_wqe->msg_len += wr->sg_list[i].length;
>
> - roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_SE_S,
> - (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
> + ud_sq_wqe->immtdata = send_ieth(wr);
>
> - roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_CQE_S,
> - (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
> + /* Set sig attr */
> + roce_set_bit(ud_sq_wqe->byte_4,
> + V2_UD_SEND_WQE_BYTE_4_CQE_S,
> + (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
>
> - roce_set_bit(rc_sq_wqe->byte_4, V2_RC_SEND_WQE_BYTE_4_OWNER_S,
> - owner_bit);
> + /* Set se attr */
> + roce_set_bit(ud_sq_wqe->byte_4,
> + V2_UD_SEND_WQE_BYTE_4_SE_S,
> + (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
>
> - switch (wr->opcode) {
> - case IB_WR_RDMA_READ:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_RDMA_READ);
> - rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
> - rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
> - break;
> - case IB_WR_RDMA_WRITE:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
> - rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
> - rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
> - break;
> - case IB_WR_RDMA_WRITE_WITH_IMM:
> - roce_set_field(rc_sq_wqe->byte_4,
> + roce_set_bit(ud_sq_wqe->byte_4,
> + V2_UD_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
> +
> + roce_set_field(ud_sq_wqe->byte_16,
> + V2_UD_SEND_WQE_BYTE_16_PD_M,
> + V2_UD_SEND_WQE_BYTE_16_PD_S,
> + to_hr_pd(ibqp->pd)->pdn);
> +
> + roce_set_field(ud_sq_wqe->byte_16,
> + V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M,
> + V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S,
> + wr->num_sge);
> +
> + roce_set_field(ud_sq_wqe->byte_20,
> + V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
> + V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
> + sge_ind & (qp->sge.sge_cnt - 1));
> +
> + roce_set_field(ud_sq_wqe->byte_24,
> + V2_UD_SEND_WQE_BYTE_24_UDPSPN_M,
> + V2_UD_SEND_WQE_BYTE_24_UDPSPN_S, 0);
> + ud_sq_wqe->qkey =
> + cpu_to_be32(ud_wr(wr)->remote_qkey & 0x80000000) ?
> + qp->qkey : ud_wr(wr)->remote_qkey;
> + roce_set_field(ud_sq_wqe->byte_32,
> + V2_UD_SEND_WQE_BYTE_32_DQPN_M,
> + V2_UD_SEND_WQE_BYTE_32_DQPN_S,
> + ud_wr(wr)->remote_qpn);
> +
> + roce_set_field(ud_sq_wqe->byte_36,
> + V2_UD_SEND_WQE_BYTE_36_VLAN_M,
> + V2_UD_SEND_WQE_BYTE_36_VLAN_S,
> + ah->av.vlan);
> + roce_set_field(ud_sq_wqe->byte_36,
> + V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M,
> + V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S,
> + ah->av.hop_limit);
> + roce_set_field(ud_sq_wqe->byte_36,
> + V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
> + V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
> + 0);
> + roce_set_field(ud_sq_wqe->byte_36,
> + V2_UD_SEND_WQE_BYTE_36_TCLASS_M,
> + V2_UD_SEND_WQE_BYTE_36_TCLASS_S,
> + 0);
> + roce_set_field(ud_sq_wqe->byte_40,
> + V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M,
> + V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S, 0);
> + roce_set_field(ud_sq_wqe->byte_40,
> + V2_UD_SEND_WQE_BYTE_40_SL_M,
> + V2_UD_SEND_WQE_BYTE_40_SL_S,
> + ah->av.sl_tclass_flowlabel >>
> + HNS_ROCE_SL_SHIFT);
> + roce_set_field(ud_sq_wqe->byte_40,
> + V2_UD_SEND_WQE_BYTE_40_PORTN_M,
> + V2_UD_SEND_WQE_BYTE_40_PORTN_S,
> + qp->port);
> +
> + roce_set_field(ud_sq_wqe->byte_48,
> + V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M,
> + V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S,
> + hns_get_gid_index(hr_dev, qp->phy_port,
> + ah->av.gid_index));
> +
> + memcpy(&ud_sq_wqe->dgid[0], &ah->av.dgid[0],
> + GID_LEN_V2);
> +
> + dseg = get_send_extend_sge(qp,
> + sge_ind & (qp->sge.sge_cnt - 1));
> + for (i = 0; i < wr->num_sge; i++) {
> + set_data_seg_v2(dseg + i, wr->sg_list + i);
> + sge_ind++;
> + }
> +
> + ind++;
> + } else if (ibqp->qp_type == IB_QPT_RC) {
> + rc_sq_wqe = wqe;
> + memset(rc_sq_wqe, 0, sizeof(*rc_sq_wqe));
> + for (i = 0; i < wr->num_sge; i++)
> + rc_sq_wqe->msg_len += wr->sg_list[i].length;
> +
> + rc_sq_wqe->inv_key_immtdata = send_ieth(wr);
> +
> + roce_set_bit(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_FENCE_S,
> + (wr->send_flags & IB_SEND_FENCE) ? 1 : 0);
> +
> + roce_set_bit(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_SE_S,
> + (wr->send_flags & IB_SEND_SOLICITED) ? 1 : 0);
> +
> + roce_set_bit(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_CQE_S,
> + (wr->send_flags & IB_SEND_SIGNALED) ? 1 : 0);
> +
> + roce_set_bit(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OWNER_S, owner_bit);
> +
> + switch (wr->opcode) {
> + case IB_WR_RDMA_READ:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_RDMA_READ);
> + rc_sq_wqe->rkey =
> + cpu_to_le32(rdma_wr(wr)->rkey);
> + rc_sq_wqe->va =
> + cpu_to_le64(rdma_wr(wr)->remote_addr);
> + break;
> + case IB_WR_RDMA_WRITE:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_RDMA_WRITE);
> + rc_sq_wqe->rkey =
> + cpu_to_le32(rdma_wr(wr)->rkey);
> + rc_sq_wqe->va =
> + cpu_to_le64(rdma_wr(wr)->remote_addr);
> + break;
> + case IB_WR_RDMA_WRITE_WITH_IMM:
> + roce_set_field(rc_sq_wqe->byte_4,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> HNS_ROCE_V2_WQE_OP_RDMA_WRITE_WITH_IMM);
> - rc_sq_wqe->rkey = cpu_to_le32(rdma_wr(wr)->rkey);
> - rc_sq_wqe->va = cpu_to_le64(rdma_wr(wr)->remote_addr);
> - break;
> - case IB_WR_SEND:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_SEND);
> - break;
> - case IB_WR_SEND_WITH_INV:
> - roce_set_field(rc_sq_wqe->byte_4,
> + rc_sq_wqe->rkey =
> + cpu_to_le32(rdma_wr(wr)->rkey);
> + rc_sq_wqe->va =
> + cpu_to_le64(rdma_wr(wr)->remote_addr);
> + break;
> + case IB_WR_SEND:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_SEND);
> + break;
> + case IB_WR_SEND_WITH_INV:
> + roce_set_field(rc_sq_wqe->byte_4,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> HNS_ROCE_V2_WQE_OP_SEND_WITH_INV);
> - break;
> - case IB_WR_SEND_WITH_IMM:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
> - break;
> - case IB_WR_LOCAL_INV:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_LOCAL_INV);
> - break;
> - case IB_WR_ATOMIC_CMP_AND_SWP:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
> - break;
> - case IB_WR_ATOMIC_FETCH_AND_ADD:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
> - break;
> - case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
> - roce_set_field(rc_sq_wqe->byte_4,
> + break;
> + case IB_WR_SEND_WITH_IMM:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_SEND_WITH_IMM);
> + break;
> + case IB_WR_LOCAL_INV:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_LOCAL_INV);
> + break;
> + case IB_WR_ATOMIC_CMP_AND_SWP:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_ATOM_CMP_AND_SWAP);
> + break;
> + case IB_WR_ATOMIC_FETCH_AND_ADD:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_ATOM_FETCH_AND_ADD);
> + break;
> + case IB_WR_MASKED_ATOMIC_CMP_AND_SWP:
> + roce_set_field(rc_sq_wqe->byte_4,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> HNS_ROCE_V2_WQE_OP_ATOM_MSK_CMP_AND_SWAP);
> - break;
> - case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
> - roce_set_field(rc_sq_wqe->byte_4,
> + break;
> + case IB_WR_MASKED_ATOMIC_FETCH_AND_ADD:
> + roce_set_field(rc_sq_wqe->byte_4,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> HNS_ROCE_V2_WQE_OP_ATOM_MSK_FETCH_AND_ADD);
> - break;
> - default:
> - roce_set_field(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> - V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> - HNS_ROCE_V2_WQE_OP_MASK);
> - break;
> - }
> -
> - wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
> - dseg = wqe;
> - if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
> - if (rc_sq_wqe->msg_len >
> - hr_dev->caps.max_sq_inline) {
> - ret = -EINVAL;
> - *bad_wr = wr;
> - dev_err(dev, "inline len(1-%d)=%d, illegal",
> - rc_sq_wqe->msg_len,
> - hr_dev->caps.max_sq_inline);
> - goto out;
> + break;
> + default:
> + roce_set_field(rc_sq_wqe->byte_4,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_M,
> + V2_RC_SEND_WQE_BYTE_4_OPCODE_S,
> + HNS_ROCE_V2_WQE_OP_MASK);
> + break;
> }
>
> - for (i = 0; i < wr->num_sge; i++) {
> - memcpy(wqe, ((void *)wr->sg_list[i].addr),
> - wr->sg_list[i].length);
> - wqe += wr->sg_list[i].length;
> - wqe_sz += wr->sg_list[i].length;
> - }
> + wqe += sizeof(struct hns_roce_v2_rc_send_wqe);
> + dseg = wqe;
>
> - roce_set_bit(rc_sq_wqe->byte_4,
> - V2_RC_SEND_WQE_BYTE_4_INLINE_S, 1);
> + ret = set_rwqe_data_seg(ibqp, wr, rc_sq_wqe, wqe,
> + &sge_ind, bad_wr);
> + if (ret)
> + goto out;
> + ind++;
> } else {
> - if (wr->num_sge <= 2) {
> - for (i = 0; i < wr->num_sge; i++) {
> - if (likely(wr->sg_list[i].length)) {
> - set_data_seg_v2(dseg,
> - wr->sg_list + i);
> - dseg++;
> - }
> - }
> - } else {
> - roce_set_field(rc_sq_wqe->byte_20,
> - V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M,
> - V2_RC_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S,
> - sge_ind & (qp->sge.sge_cnt - 1));
> -
> - for (i = 0; i < 2; i++) {
> - if (likely(wr->sg_list[i].length)) {
> - set_data_seg_v2(dseg,
> - wr->sg_list + i);
> - dseg++;
> - }
> - }
> -
> - dseg = get_send_extend_sge(qp,
> - sge_ind & (qp->sge.sge_cnt - 1));
> -
> - for (i = 0; i < wr->num_sge - 2; i++) {
> - if (likely(wr->sg_list[i + 2].length)) {
> - set_data_seg_v2(dseg,
> - wr->sg_list + 2 + i);
> - dseg++;
> - sge_ind++;
> - }
> - }
> - }
> -
> - roce_set_field(rc_sq_wqe->byte_16,
> - V2_RC_SEND_WQE_BYTE_16_SGE_NUM_M,
> - V2_RC_SEND_WQE_BYTE_16_SGE_NUM_S,
> - wr->num_sge);
> - wqe_sz += wr->num_sge *
> - sizeof(struct hns_roce_v2_wqe_data_seg);
> + dev_err(dev, "Illegal qp_type(0x%x)\n", ibqp->qp_type);
> + spin_unlock_irqrestore(&qp->sq.lock, flags);
> + return -EOPNOTSUPP;
> }
> - ind++;
> }
>
> out:
> diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
> index 463edab..c11b253 100644
> --- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
> +++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
> @@ -916,6 +916,90 @@ struct hns_roce_v2_cq_db {
>
> #define V2_CQ_DB_PARAMETER_NOTIFY_S 24
>
> +struct hns_roce_v2_ud_send_wqe {
> + u32 byte_4;
> + u32 msg_len;
> + u32 immtdata;
> + u32 byte_16;
> + u32 byte_20;
> + u32 byte_24;
> + u32 qkey;
> + u32 byte_32;
> + u32 byte_36;
> + u32 byte_40;
> + u32 dmac;
> + u32 byte_48;
> + u8 dgid[GID_LEN_V2];
> +
> +};
> +#define V2_UD_SEND_WQE_BYTE_4_OPCODE_S 0
> +#define V2_UD_SEND_WQE_BYTE_4_OPCODE_M GENMASK(4, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_4_OWNER_S 7
> +
> +#define V2_UD_SEND_WQE_BYTE_4_CQE_S 8
> +
> +#define V2_UD_SEND_WQE_BYTE_4_SE_S 11
> +
> +#define V2_UD_SEND_WQE_BYTE_16_PD_S 0
> +#define V2_UD_SEND_WQE_BYTE_16_PD_M GENMASK(23, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_16_SGE_NUM_S 24
> +#define V2_UD_SEND_WQE_BYTE_16_SGE_NUM_M GENMASK(31, 24)
> +
> +#define V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_S 0
> +#define V2_UD_SEND_WQE_BYTE_20_MSG_START_SGE_IDX_M GENMASK(23, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_24_UDPSPN_S 16
> +#define V2_UD_SEND_WQE_BYTE_24_UDPSPN_M GENMASK(31, 16)
> +
> +#define V2_UD_SEND_WQE_BYTE_32_DQPN_S 0
> +#define V2_UD_SEND_WQE_BYTE_32_DQPN_M GENMASK(23, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_36_VLAN_S 0
> +#define V2_UD_SEND_WQE_BYTE_36_VLAN_M GENMASK(15, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_S 16
> +#define V2_UD_SEND_WQE_BYTE_36_HOPLIMIT_M GENMASK(23, 16)
> +
> +#define V2_UD_SEND_WQE_BYTE_36_TCLASS_S 24
> +#define V2_UD_SEND_WQE_BYTE_36_TCLASS_M GENMASK(31, 24)
> +
> +#define V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_S 0
> +#define V2_UD_SEND_WQE_BYTE_40_FLOW_LABEL_M GENMASK(19, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_40_SL_S 20
> +#define V2_UD_SEND_WQE_BYTE_40_SL_M GENMASK(23, 20)
> +
> +#define V2_UD_SEND_WQE_BYTE_40_PORTN_S 24
> +#define V2_UD_SEND_WQE_BYTE_40_PORTN_M GENMASK(26, 24)
> +
> +#define V2_UD_SEND_WQE_BYTE_40_LBI_S 31
> +
> +#define V2_UD_SEND_WQE_DMAC_0_S 0
> +#define V2_UD_SEND_WQE_DMAC_0_M GENMASK(7, 0)
> +
> +#define V2_UD_SEND_WQE_DMAC_1_S 8
> +#define V2_UD_SEND_WQE_DMAC_1_M GENMASK(15, 8)
> +
> +#define V2_UD_SEND_WQE_DMAC_2_S 16
> +#define V2_UD_SEND_WQE_DMAC_2_M GENMASK(23, 16)
> +
> +#define V2_UD_SEND_WQE_DMAC_3_S 24
> +#define V2_UD_SEND_WQE_DMAC_3_M GENMASK(31, 24)
> +
> +#define V2_UD_SEND_WQE_BYTE_48_DMAC_4_S 0
> +#define V2_UD_SEND_WQE_BYTE_48_DMAC_4_M GENMASK(7, 0)
> +
> +#define V2_UD_SEND_WQE_BYTE_48_DMAC_5_S 8
> +#define V2_UD_SEND_WQE_BYTE_48_DMAC_5_M GENMASK(15, 8)
> +
> +#define V2_UD_SEND_WQE_BYTE_48_SGID_INDX_S 16
> +#define V2_UD_SEND_WQE_BYTE_48_SGID_INDX_M GENMASK(23, 16)
> +
> +#define V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_S 24
> +#define V2_UD_SEND_WQE_BYTE_48_SMAC_INDX_M GENMASK(31, 24)
> +
> struct hns_roce_v2_rc_send_wqe {
> u32 byte_4;
> u32 msg_len;
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
` (6 preceding siblings ...)
2018-01-10 6:39 ` [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device Lijun Ou
@ 2018-01-17 3:50 ` Jason Gunthorpe
7 siblings, 0 replies; 18+ messages in thread
From: Jason Gunthorpe @ 2018-01-17 3:50 UTC (permalink / raw)
To: Lijun Ou
Cc: dledford-H+wXaHxf7aLQT0dZR+AlfA, leon-DgEjT+Ai2ygdnm+yROfE0A,
linux-rdma-u79uwXL29TY76Z2rM5mHXA
On Wed, Jan 10, 2018 at 02:39:46PM +0800, Lijun Ou wrote:
> This patch series adds the CM(Connection Mangement) support
> to hip08 RoCE driver. Changes done are primarily to add
> support of APIs in IB device and some fixes over the
> origin driver to support RDMA CM. This patch series
> also adjust some codes style in order to fill the
> sq wqe of ud type and fix a bug for tx_cqn field.
>
> Change from V1:
> - Add a new patch named assign the correct value for
> tx_cqn splited from the first patchset
> - Fix the comments given by Leon Romanovsky
> - Add the options for free sq lock when qp is illegal
> qp type
>
> Lijun Ou (7):
> RDMA/hns: Assign the correct value for tx_cqn
> RDMA/hns: Create gsi qp in hip08
> RDMA/hns: Add gsi qp support for modifying qp in hip08
> RDMA/hns: Fill sq wqe context of ud type in hip08
> RDMA/hns: Assign zero for pkey_index of wc in hip08
> RDMA/hns: Update the verbs of polling for completion
> RDMA/hns: Set the guid for hip08 RoCE device
Applied to for-next, thanks
Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2018-01-17 3:50 UTC | newest]
Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-01-10 6:39 [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08 Lijun Ou
[not found] ` <1515566393-63888-1-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 6:39 ` [PATCH V2 for-next 1/7] RDMA/hns: Assign the correct value for tx_cqn Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 2/7] RDMA/hns: Create gsi qp in hip08 Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 3/7] RDMA/hns: Add gsi qp support for modifying " Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 4/7] RDMA/hns: Fill sq wqe context of ud type " Lijun Ou
[not found] ` <1515566393-63888-5-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-12 21:22 ` kbuild test robot
2018-01-16 9:00 ` oulijun
2018-01-10 6:39 ` [PATCH V2 for-next 5/7] RDMA/hns: Assign zero for pkey_index of wc " Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 6/7] RDMA/hns: Update the verbs of polling for completion Lijun Ou
2018-01-10 6:39 ` [PATCH V2 for-next 7/7] RDMA/hns: Set the guid for hip08 RoCE device Lijun Ou
[not found] ` <1515566393-63888-8-git-send-email-oulijun-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 6:49 ` Leon Romanovsky
[not found] ` <20180110064937.GE7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2018-01-10 9:23 ` oulijun
[not found] ` <f2c33ef8-2457-bccc-e6ac-80711ccec25f-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-10 10:29 ` Leon Romanovsky
[not found] ` <20180110102902.GI7368-U/DQcQFIOTAAJjI8aNfphQ@public.gmane.org>
2018-01-10 21:01 ` Jason Gunthorpe
2018-01-11 2:49 ` oulijun
[not found] ` <71771a6a-ef08-1750-69c3-f2fca8294751-hv44wF8Li93QT0dZR+AlfA@public.gmane.org>
2018-01-11 5:31 ` Jason Gunthorpe
2018-01-11 5:46 ` Leon Romanovsky
2018-01-17 3:50 ` [PATCH V2 for-next 0/7] Add CM and a bugfix to hip08 Jason Gunthorpe
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.