* [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver
@ 2016-09-13 19:49 Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 1/9] IB/hns: Register HNS RoCE Driver get_netdev() with IB Core Salil Mehta
` (8 more replies)
0 siblings, 9 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
This patch-set adds the CM(Connection Manager) support to the
HNS RoCE driver. Changes done are primarily to add support of
APIs in IB device and some fixes over the base driver to
support RDMA Connection Manager. This patch-set also updates
the Device binding document as new parameter node-guid was
added to the DT.
All of the patches have been authored by Oulijun and reviewed
by Huwei.
Change Log:
V1->V2: Addressed the comments from Leon Romanovsky. Please refer
individual patch for details.
Lijun Ou (9):
IB/hns: Register HNS RoCE Driver get_netdev() with IB Core
IB/hns: Add & initialize "node_guid" parameter for RDMA CM
IB/hns: Fix the value of device_cap_flags
IB/hns: Fix two possible bugs for rdma cm
IB/hns: Add phy_port for computing GSI/QPN
IB/hns: Change the logic for allocating uar registers
IB/hns: Fix the bug of rdma cm connecting on user mode
IB/hns: Fix two bugs for rdma cm connecting
IB/hns: Add node_guid definition to the bindings document
.../bindings/infiniband/hisilicon-hns-roce.txt | 2 +
drivers/infiniband/hw/hns/hns_roce_device.h | 4 +-
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 80 +++++++++-------------
drivers/infiniband/hw/hns/hns_roce_main.c | 32 ++++++++-
drivers/infiniband/hw/hns/hns_roce_pd.c | 4 +-
drivers/infiniband/hw/hns/hns_roce_qp.c | 13 ++--
6 files changed, 76 insertions(+), 59 deletions(-)
--
1.9.1
^ permalink raw reply [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 1/9] IB/hns: Register HNS RoCE Driver get_netdev() with IB Core
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 2/9] IB/hns: Add & initialize "node_guid" parameter for RDMA CM Salil Mehta
` (7 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
This patch adds get_netdev() function to the IB device. This shall be
used to fetch netdev corresponding to the port number. This function
would be called by IB core(Generic CM Agent) for example, when the
RDMA connection is being established.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_main.c | 20 ++++++++++++++++++++
1 file changed, 20 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index f64f0dd..39e69c3 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -372,6 +372,25 @@ static int hns_roce_query_device(struct ib_device *ib_dev,
return 0;
}
+static struct net_device *hns_roce_get_netdev(struct ib_device *ib_dev,
+ u8 port_num)
+{
+ struct hns_roce_dev *hr_dev = to_hr_dev(ib_dev);
+ struct net_device *ndev;
+
+ if (port_num < 1 || port_num > hr_dev->caps.num_ports)
+ return NULL;
+
+ rcu_read_lock();
+
+ ndev = hr_dev->iboe.netdevs[port_num - 1];
+ if (ndev)
+ dev_hold(ndev);
+
+ rcu_read_unlock();
+ return ndev;
+}
+
static int hns_roce_query_port(struct ib_device *ib_dev, u8 port_num,
struct ib_port_attr *props)
{
@@ -618,6 +637,7 @@ static int hns_roce_register_device(struct hns_roce_dev *hr_dev)
ib_dev->query_port = hns_roce_query_port;
ib_dev->modify_port = hns_roce_modify_port;
ib_dev->get_link_layer = hns_roce_get_link_layer;
+ ib_dev->get_netdev = hns_roce_get_netdev;
ib_dev->query_gid = hns_roce_query_gid;
ib_dev->query_pkey = hns_roce_query_pkey;
ib_dev->alloc_ucontext = hns_roce_alloc_ucontext;
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 2/9] IB/hns: Add & initialize "node_guid" parameter for RDMA CM
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 1/9] IB/hns: Register HNS RoCE Driver get_netdev() with IB Core Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 3/9] IB/hns: Fix the value of device_cap_flags Salil Mehta
` (6 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
According to the Infiniband spec, NodeGUID uniquely identifies a
node. This must be initialized to some unique value. This patch
adds the support to the HNS RoCE driver to fetch the NodeGUID
value from DT or ACPI and then use this value to initialize the
node_guid parameter of IB device. This value shall be used by
RDMA CM.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
PATCH V2: Addressed the comment by Leon Romanovsky
* Link: https://lkml.org/lkml/2016/9/12/309
PATCH V1: Initial Patch
---
drivers/infiniband/hw/hns/hns_roce_device.h | 1 +
drivers/infiniband/hw/hns/hns_roce_main.c | 9 +++++++++
2 files changed, 10 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index ea73580..e943b98 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -74,6 +74,7 @@
#define MR_TYPE_DMA 0x03
#define PKEY_ID 0xffff
+#define GUID_LEN 8
#define NODE_DESC_SIZE 64
#define SERV_TYPE_RC 0
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index 39e69c3..f0700f4 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -797,6 +797,15 @@ static int hns_roce_get_cfg(struct hns_roce_dev *hr_dev)
if (IS_ERR(hr_dev->reg_base))
return PTR_ERR(hr_dev->reg_base);
+ /* read the node_guid of IB device from the DT or ACPI */
+ ret = device_property_read_u8_array(dev, "node-guid",
+ (u8 *)&hr_dev->ib_dev.node_guid,
+ GUID_LEN);
+ if (ret) {
+ dev_err(dev, "couldn't get node_guid from DT or ACPI!\n");
+ return ret;
+ }
+
/* get the RoCE associated ethernet ports or netdevices */
for (i = 0; i < HNS_ROCE_MAX_PORTS; i++) {
if (dev_of_node(dev)) {
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 3/9] IB/hns: Fix the value of device_cap_flags
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 1/9] IB/hns: Register HNS RoCE Driver get_netdev() with IB Core Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 2/9] IB/hns: Add & initialize "node_guid" parameter for RDMA CM Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 4/9] IB/hns: Fix two possible bugs for rdma cm Salil Mehta
` (5 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
In the latest IB core version, it has some known issues
with memory registration using the local_dma_lkey.
Thus RoCE don't expose support for it, and remove
device->local_dma_lkey which is introduced to working systems.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_main.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_main.c b/drivers/infiniband/hw/hns/hns_roce_main.c
index f0700f4..a9960ba 100644
--- a/drivers/infiniband/hw/hns/hns_roce_main.c
+++ b/drivers/infiniband/hw/hns/hns_roce_main.c
@@ -355,8 +355,7 @@ static int hns_roce_query_device(struct ib_device *ib_dev,
props->max_qp = hr_dev->caps.num_qps;
props->max_qp_wr = hr_dev->caps.max_wqes;
props->device_cap_flags = IB_DEVICE_PORT_ACTIVE_EVENT |
- IB_DEVICE_RC_RNR_NAK_GEN |
- IB_DEVICE_LOCAL_DMA_LKEY;
+ IB_DEVICE_RC_RNR_NAK_GEN;
props->max_sge = hr_dev->caps.max_sq_sg;
props->max_sge_rd = 1;
props->max_cq = hr_dev->caps.num_cqs;
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 4/9] IB/hns: Fix two possible bugs for rdma cm
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
` (2 preceding siblings ...)
2016-09-13 19:49 ` [PATCH V2 for-next 3/9] IB/hns: Fix the value of device_cap_flags Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 5/9] IB/hns: Add phy_port for computing GSI/QPN Salil Mehta
` (4 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
Fix the length of wqe that maybe lead to an error and
write the end bytes of QP1C into the register.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 10 ++++------
1 file changed, 4 insertions(+), 6 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 399f5de..aaea95c 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -205,8 +205,7 @@ int hns_roce_v1_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
(wr->send_flags & IB_SEND_FENCE ?
(cpu_to_le32(HNS_ROCE_WQE_FENCE)) : 0);
- wqe = (struct hns_roce_wqe_ctrl_seg *)wqe +
- sizeof(struct hns_roce_wqe_ctrl_seg);
+ wqe += sizeof(struct hns_roce_wqe_ctrl_seg);
switch (wr->opcode) {
case IB_WR_RDMA_READ:
@@ -235,8 +234,7 @@ int hns_roce_v1_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
break;
}
ctrl->flag |= cpu_to_le32(ps_opcode);
- wqe = (struct hns_roce_wqe_raddr_seg *)wqe +
- sizeof(struct hns_roce_wqe_raddr_seg);
+ wqe += sizeof(struct hns_roce_wqe_raddr_seg);
dseg = wqe;
if (wr->send_flags & IB_SEND_INLINE && wr->num_sge) {
@@ -253,8 +251,7 @@ int hns_roce_v1_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
memcpy(wqe, ((void *) (uintptr_t)
wr->sg_list[i].addr),
wr->sg_list[i].length);
- wqe = (struct hns_roce_wqe_raddr_seg *)
- wqe + wr->sg_list[i].length;
+ wqe += wr->sg_list[i].length;
}
ctrl->flag |= HNS_ROCE_WQE_INLINE;
} else {
@@ -1795,6 +1792,7 @@ static int hns_roce_v1_m_sqp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
writel(context->qp1c_bytes_28, addr + 6);
writel(context->qp1c_bytes_32, addr + 7);
writel(context->cur_sq_wqe_ba_l, addr + 8);
+ writel(context->qp1c_bytes_40, addr + 9);
}
/* Modify QP1C status */
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 5/9] IB/hns: Add phy_port for computing GSI/QPN
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
` (3 preceding siblings ...)
2016-09-13 19:49 ` [PATCH V2 for-next 4/9] IB/hns: Fix two possible bugs for rdma cm Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 6/9] IB/hns: Change the logic for allocating uar registers Salil Mehta
` (3 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
This patch mainly adds phy_port to HNS RoCE QP. This shall be
used in calculating the GSI QPN for the port.
Initally when RDMA is being established, all IB ports share a
QPN which later needs to be re-assigned to a particular GSI/QPN
and which is per-port.
This also fixes a bug in base driver where iboe port was being
used instead of phy_port at some places. This values might not
be same always.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 1 +
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 30 +++++++++++++++--------------
drivers/infiniband/hw/hns/hns_roce_qp.c | 13 ++++++-------
3 files changed, 23 insertions(+), 21 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index e943b98..833742b 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -409,6 +409,7 @@ struct hns_roce_qp {
u32 buff_size;
struct mutex mutex;
u8 port;
+ u8 phy_port;
u8 sl;
u8 resp_depth;
u8 state;
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index aaea95c..581e542 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -162,7 +162,7 @@ int hns_roce_v1_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
roce_set_field(ud_sq_wqe->u32_36,
UD_SEND_WQE_U32_36_SGID_INDEX_M,
UD_SEND_WQE_U32_36_SGID_INDEX_S,
- hns_get_gid_index(hr_dev, qp->port,
+ hns_get_gid_index(hr_dev, qp->phy_port,
ah->av.gid_index));
roce_set_field(ud_sq_wqe->u32_40,
@@ -282,7 +282,7 @@ out:
SQ_DOORBELL_U32_4_SQ_HEAD_S,
(qp->sq.head & ((qp->sq.wqe_cnt << 1) - 1)));
roce_set_field(sq_db.u32_4, SQ_DOORBELL_U32_4_PORT_M,
- SQ_DOORBELL_U32_4_PORT_S, qp->port);
+ SQ_DOORBELL_U32_4_PORT_S, qp->phy_port);
roce_set_field(sq_db.u32_8, SQ_DOORBELL_U32_8_QPN_M,
SQ_DOORBELL_U32_8_QPN_S, qp->doorbell_qpn);
roce_set_bit(sq_db.u32_8, SQ_DOORBELL_HW_SYNC_S, 1);
@@ -362,14 +362,14 @@ out:
/* SW update GSI rq header */
reg_val = roce_read(to_hr_dev(ibqp->device),
ROCEE_QP1C_CFG3_0_REG +
- QP1C_CFGN_OFFSET * hr_qp->port);
+ QP1C_CFGN_OFFSET * hr_qp->phy_port);
roce_set_field(reg_val,
ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_M,
ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S,
hr_qp->rq.head);
roce_write(to_hr_dev(ibqp->device),
ROCEE_QP1C_CFG3_0_REG +
- QP1C_CFGN_OFFSET * hr_qp->port, reg_val);
+ QP1C_CFGN_OFFSET * hr_qp->phy_port, reg_val);
} else {
rq_db.u32_4 = 0;
rq_db.u32_8 = 0;
@@ -1730,7 +1730,7 @@ static int hns_roce_v1_m_sqp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
roce_set_field(context->qp1c_bytes_16, QP1C_BYTES_16_RQ_HEAD_M,
QP1C_BYTES_16_RQ_HEAD_S, hr_qp->rq.head);
roce_set_field(context->qp1c_bytes_16, QP1C_BYTES_16_PORT_NUM_M,
- QP1C_BYTES_16_PORT_NUM_S, hr_qp->port);
+ QP1C_BYTES_16_PORT_NUM_S, hr_qp->phy_port);
roce_set_bit(context->qp1c_bytes_16,
QP1C_BYTES_16_SIGNALING_TYPE_S,
hr_qp->sq_signal_bits);
@@ -1781,7 +1781,7 @@ static int hns_roce_v1_m_sqp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
/* Copy context to QP1C register */
addr = (u32 *)(hr_dev->reg_base + ROCEE_QP1C_CFG0_0_REG +
- hr_qp->port * sizeof(*context));
+ hr_qp->phy_port * sizeof(*context));
writel(context->qp1c_bytes_4, addr);
writel(context->sq_rq_bt_l, addr + 1);
@@ -1797,11 +1797,11 @@ static int hns_roce_v1_m_sqp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
/* Modify QP1C status */
reg_val = roce_read(hr_dev, ROCEE_QP1C_CFG0_0_REG +
- hr_qp->port * sizeof(*context));
+ hr_qp->phy_port * sizeof(*context));
roce_set_field(reg_val, ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_M,
ROCEE_QP1C_CFG0_0_ROCEE_QP1C_QP_ST_S, new_state);
roce_write(hr_dev, ROCEE_QP1C_CFG0_0_REG +
- hr_qp->port * sizeof(*context), reg_val);
+ hr_qp->phy_port * sizeof(*context), reg_val);
hr_qp->state = new_state;
if (new_state == IB_QPS_RESET) {
@@ -2184,7 +2184,7 @@ static int hns_roce_v1_m_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
roce_set_field(context->qpc_bytes_156,
QP_CONTEXT_QPC_BYTES_156_PORT_NUM_M,
QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S,
- hr_qp->port);
+ hr_qp->phy_port);
roce_set_field(context->qpc_bytes_156,
QP_CONTEXT_QPC_BYTES_156_SL_M,
QP_CONTEXT_QPC_BYTES_156_SL_S, attr->ah_attr.sl);
@@ -2290,7 +2290,7 @@ static int hns_roce_v1_m_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
roce_set_field(context->qpc_bytes_156,
QP_CONTEXT_QPC_BYTES_156_PORT_NUM_M,
QP_CONTEXT_QPC_BYTES_156_PORT_NUM_S,
- hr_qp->port);
+ hr_qp->phy_port);
roce_set_field(context->qpc_bytes_156,
QP_CONTEXT_QPC_BYTES_156_SL_M,
QP_CONTEXT_QPC_BYTES_156_SL_S, attr->ah_attr.sl);
@@ -2398,13 +2398,13 @@ static int hns_roce_v1_m_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
if (hr_qp->ibqp.qp_type == IB_QPT_GSI) {
/* SW update GSI rq header */
reg_val = roce_read(hr_dev, ROCEE_QP1C_CFG3_0_REG +
- QP1C_CFGN_OFFSET * hr_qp->port);
+ QP1C_CFGN_OFFSET * hr_qp->phy_port);
roce_set_field(reg_val,
ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_M,
ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S,
hr_qp->rq.head);
roce_write(hr_dev, ROCEE_QP1C_CFG3_0_REG +
- QP1C_CFGN_OFFSET * hr_qp->port, reg_val);
+ QP1C_CFGN_OFFSET * hr_qp->phy_port, reg_val);
} else {
rq_db.u32_4 = 0;
rq_db.u32_8 = 0;
@@ -2430,8 +2430,10 @@ static int hns_roce_v1_m_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
if (attr_mask & IB_QP_MAX_DEST_RD_ATOMIC)
hr_qp->resp_depth = attr->max_dest_rd_atomic;
- if (attr_mask & IB_QP_PORT)
- hr_qp->port = (attr->port_num - 1);
+ if (attr_mask & IB_QP_PORT) {
+ hr_qp->port = attr->port_num - 1;
+ hr_qp->phy_port = hr_dev->iboe.phy_port[hr_qp->port];
+ }
if (new_state == IB_QPS_RESET && !ibqp->uobject) {
hns_roce_v1_cq_clean(to_hr_cq(ibqp->recv_cq), hr_qp->qpn,
diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c
index 645c18d..089da7f 100644
--- a/drivers/infiniband/hw/hns/hns_roce_qp.c
+++ b/drivers/infiniband/hw/hns/hns_roce_qp.c
@@ -617,21 +617,20 @@ struct ib_qp *hns_roce_create_qp(struct ib_pd *pd,
return ERR_PTR(-ENOMEM);
hr_qp = &hr_sqp->hr_qp;
+ hr_qp->port = init_attr->port_num - 1;
+ hr_qp->phy_port = hr_dev->iboe.phy_port[hr_qp->port];
+ hr_qp->ibqp.qp_num = hr_dev->caps.sqp_start +
+ HNS_ROCE_MAX_PORTS +
+ hr_dev->iboe.phy_port[hr_qp->port];
ret = hns_roce_create_qp_common(hr_dev, pd, init_attr, udata,
- hr_dev->caps.sqp_start +
- hr_dev->caps.num_ports +
- init_attr->port_num - 1, hr_qp);
+ hr_qp->ibqp.qp_num, hr_qp);
if (ret) {
dev_err(dev, "Create GSI QP failed!\n");
kfree(hr_sqp);
return ERR_PTR(ret);
}
- hr_qp->port = (init_attr->port_num - 1);
- hr_qp->ibqp.qp_num = hr_dev->caps.sqp_start +
- hr_dev->caps.num_ports +
- init_attr->port_num - 1;
break;
}
default:{
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 6/9] IB/hns: Change the logic for allocating uar registers
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
` (4 preceding siblings ...)
2016-09-13 19:49 ` [PATCH V2 for-next 5/9] IB/hns: Add phy_port for computing GSI/QPN Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 7/9] IB/hns: Fix the bug of rdma cm connecting on user mode Salil Mehta
` (2 subsequent siblings)
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
This patch mainly modifies the logic for allocating uar registers.
In HiP06 SoC, HW has 8 group of uar registers for kernel and
user space application. The uar index is assigned as follows:
0 ------ for kernel
1~7 ------ for user space application
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_pd.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_pd.c b/drivers/infiniband/hw/hns/hns_roce_pd.c
index 16271b5..4109f74 100644
--- a/drivers/infiniband/hw/hns/hns_roce_pd.c
+++ b/drivers/infiniband/hw/hns/hns_roce_pd.c
@@ -117,7 +117,9 @@ int hns_roce_uar_alloc(struct hns_roce_dev *hr_dev, struct hns_roce_uar *uar)
if (ret == -1)
return -ENOMEM;
- uar->index = (uar->index - 1) % hr_dev->caps.phy_num_uars + 1;
+ if (uar->index > 0)
+ uar->index = (uar->index - 1) %
+ (hr_dev->caps.phy_num_uars - 1) + 1;
res = platform_get_resource(hr_dev->pdev, IORESOURCE_MEM, 0);
uar->pfn = ((res->start) >> PAGE_SHIFT) + uar->index;
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 7/9] IB/hns: Fix the bug of rdma cm connecting on user mode
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
` (5 preceding siblings ...)
2016-09-13 19:49 ` [PATCH V2 for-next 6/9] IB/hns: Change the logic for allocating uar registers Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-14 6:41 ` kbuild test robot
2016-09-13 19:49 ` [PATCH V2 for-next 8/9] IB/hns: Fix two bugs for rdma cm connecting Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 9/9] IB/hns: Add node_guid definition to the bindings document Salil Mehta
8 siblings, 1 reply; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
Fix bug of modify qp from init to init on user mode. Otherwise,
it will oops when rmda cm established.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 41 ++++++++++--------------------
1 file changed, 14 insertions(+), 27 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 581e542..206957b 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -2395,35 +2395,22 @@ static int hns_roce_v1_m_qp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
if (cur_state == IB_QPS_INIT && new_state == IB_QPS_INIT) {
/* Memory barrier */
wmb();
- if (hr_qp->ibqp.qp_type == IB_QPT_GSI) {
- /* SW update GSI rq header */
- reg_val = roce_read(hr_dev, ROCEE_QP1C_CFG3_0_REG +
- QP1C_CFGN_OFFSET * hr_qp->phy_port);
- roce_set_field(reg_val,
- ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_M,
- ROCEE_QP1C_CFG3_0_ROCEE_QP1C_RQ_HEAD_S,
- hr_qp->rq.head);
- roce_write(hr_dev, ROCEE_QP1C_CFG3_0_REG +
- QP1C_CFGN_OFFSET * hr_qp->phy_port, reg_val);
- } else {
- rq_db.u32_4 = 0;
- rq_db.u32_8 = 0;
- roce_set_field(rq_db.u32_4, RQ_DOORBELL_U32_4_RQ_HEAD_M,
- RQ_DOORBELL_U32_4_RQ_HEAD_S,
- hr_qp->rq.head);
- roce_set_field(rq_db.u32_8, RQ_DOORBELL_U32_8_QPN_M,
- RQ_DOORBELL_U32_8_QPN_S, hr_qp->qpn);
- roce_set_field(rq_db.u32_8, RQ_DOORBELL_U32_8_CMD_M,
- RQ_DOORBELL_U32_8_CMD_S, 1);
- roce_set_bit(rq_db.u32_8, RQ_DOORBELL_U32_8_HW_SYNC_S,
- 1);
-
- doorbell[0] = rq_db.u32_4;
- doorbell[1] = rq_db.u32_8;
-
- hns_roce_write64_k(doorbell, hr_qp->rq.db_reg_l);
+ roce_set_field(doorbell[0], RQ_DOORBELL_U32_4_RQ_HEAD_M,
+ RQ_DOORBELL_U32_4_RQ_HEAD_S, hr_qp->rq.head);
+ roce_set_field(doorbell[1], RQ_DOORBELL_U32_8_QPN_M,
+ RQ_DOORBELL_U32_8_QPN_S, hr_qp->qpn);
+ roce_set_field(doorbell[1], RQ_DOORBELL_U32_8_CMD_M,
+ RQ_DOORBELL_U32_8_CMD_S, 1);
+ roce_set_bit(doorbell[1], RQ_DOORBELL_U32_8_HW_SYNC_S, 1);
+
+ if (ibqp->uobject) {
+ hr_qp->rq.db_reg_l = hr_dev->reg_base +
+ ROCEE_DB_OTHERS_L_0_REG +
+ DB_REG_OFFSET * hr_dev->priv_uar.index;
}
+
+ hns_roce_write64_k(doorbell, hr_qp->rq.db_reg_l);
}
hr_qp->state = new_state;
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 8/9] IB/hns: Fix two bugs for rdma cm connecting
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
` (6 preceding siblings ...)
2016-09-13 19:49 ` [PATCH V2 for-next 7/9] IB/hns: Fix the bug of rdma cm connecting on user mode Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 9/9] IB/hns: Add node_guid definition to the bindings document Salil Mehta
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
This patch mainly modify the value of HNS_ROCE_SL_SHIFT
and delete the lines for assigning for the field of
local_enable_e2e_credit in QP1C.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 2 +-
drivers/infiniband/hw/hns/hns_roce_hw_v1.c | 3 ---
2 files changed, 1 insertion(+), 4 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 833742b..066bb3e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -62,7 +62,7 @@
#define HNS_ROCE_AEQE_OF_VEC_NUM 1
/* 4G/4K = 1M */
-#define HNS_ROCE_SL_SHIFT 29
+#define HNS_ROCE_SL_SHIFT 28
#define HNS_ROCE_TCLASS_SHIFT 20
#define HNS_ROCE_FLOW_LABLE_MASK 0xfffff
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
index 206957b..3feac8a 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v1.c
@@ -1734,9 +1734,6 @@ static int hns_roce_v1_m_sqp(struct ib_qp *ibqp, const struct ib_qp_attr *attr,
roce_set_bit(context->qp1c_bytes_16,
QP1C_BYTES_16_SIGNALING_TYPE_S,
hr_qp->sq_signal_bits);
- roce_set_bit(context->qp1c_bytes_16,
- QP1C_BYTES_16_LOCAL_ENABLE_E2E_CREDIT_S,
- hr_qp->sq_signal_bits);
roce_set_bit(context->qp1c_bytes_16, QP1C_BYTES_16_RQ_BA_FLG_S,
1);
roce_set_bit(context->qp1c_bytes_16, QP1C_BYTES_16_SQ_BA_FLG_S,
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* [PATCH V2 for-next 9/9] IB/hns: Add node_guid definition to the bindings document
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
` (7 preceding siblings ...)
2016-09-13 19:49 ` [PATCH V2 for-next 8/9] IB/hns: Fix two bugs for rdma cm connecting Salil Mehta
@ 2016-09-13 19:49 ` Salil Mehta
8 siblings, 0 replies; 11+ messages in thread
From: Salil Mehta @ 2016-09-13 19:49 UTC (permalink / raw)
To: dledford
Cc: salil.mehta, xavier.huwei, oulijun, yisen.zhuang,
mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel, linuxarm
From: Lijun Ou <oulijun@huawei.com>
This patch adds node_guid definition in bindings document.
The value of node_guid will be used during RDMA connection.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Reviewed-by: Wei Hu <xavier.huwei@huawei.com>
Signed-off-by: Salil Mehta <salil.mehta@huawei.com>
---
Documentation/devicetree/bindings/infiniband/hisilicon-hns-roce.txt | 2 ++
1 file changed, 2 insertions(+)
diff --git a/Documentation/devicetree/bindings/infiniband/hisilicon-hns-roce.txt b/Documentation/devicetree/bindings/infiniband/hisilicon-hns-roce.txt
index f97993b..d3b273e 100644
--- a/Documentation/devicetree/bindings/infiniband/hisilicon-hns-roce.txt
+++ b/Documentation/devicetree/bindings/infiniband/hisilicon-hns-roce.txt
@@ -14,6 +14,7 @@ length of memory mapped region.
representing a ethernet device.
- dsaf-handle: phandle, specifies a reference to a node
representing a dsaf device.
+- node_guid: a number that uniquely identifies a device or component
- #address-cells: must be 2
- #size-cells: must be 2
Optional properties:
@@ -32,6 +33,7 @@ Example:
dma-coherent;
eth-handle = <ð2 ð3 ð4 ð5 ð6 ð7>;
dsaf-handle = <&soc0_dsa>;
+ node-guid = [00 9A CD 00 00 01 02 03];
#address-cells = <2>;
#size-cells = <2>;
interrupt-parent = <&mbigen_dsa>;
--
1.9.1
^ permalink raw reply related [flat|nested] 11+ messages in thread
* Re: [PATCH V2 for-next 7/9] IB/hns: Fix the bug of rdma cm connecting on user mode
2016-09-13 19:49 ` [PATCH V2 for-next 7/9] IB/hns: Fix the bug of rdma cm connecting on user mode Salil Mehta
@ 2016-09-14 6:41 ` kbuild test robot
0 siblings, 0 replies; 11+ messages in thread
From: kbuild test robot @ 2016-09-14 6:41 UTC (permalink / raw)
To: Salil Mehta
Cc: kbuild-all, dledford, salil.mehta, xavier.huwei, oulijun,
yisen.zhuang, mehta.salil.lnk, xuwei5, linux-rdma, linux-kernel,
linuxarm
[-- Attachment #1: Type: text/plain, Size: 2347 bytes --]
Hi Lijun,
[auto build test ERROR on next-20160913]
[cannot apply to rdma/master robh/for-next v4.8-rc6 v4.8-rc5 v4.8-rc4 v4.8-rc6]
[if your patch is applied to the wrong git tree, please drop us a note to help improve the system]
[Suggest to use git(>=2.9.0) format-patch --base=<commit> (or --base=auto for convenience) to record what (public, well-known) commit your patch series was built on]
[Check https://git-scm.com/docs/git-format-patch for more information]
url: https://github.com/0day-ci/linux/commits/Salil-Mehta/IB-hns-Add-CM-Connection-Manager-Support-to-HNS-RoCe-Driver/20160914-035830
config: arm64-allmodconfig (attached as .config)
compiler: aarch64-linux-gnu-gcc (Debian 5.4.0-6) 5.4.0 20160609
reproduce:
wget https://git.kernel.org/cgit/linux/kernel/git/wfg/lkp-tests.git/plain/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# save the attached .config to linux build tree
make.cross ARCH=arm64
All error/warnings (new ones prefixed by >>):
drivers/infiniband/hw/hns/hns_roce_hw_v1.c: In function 'hns_roce_v1_m_qp':
>> drivers/infiniband/hw/hns/hns_roce_hw_v1.c:2410:10: error: 'DB_REG_OFFSET' undeclared (first use in this function)
DB_REG_OFFSET * hr_dev->priv_uar.index;
^
drivers/infiniband/hw/hns/hns_roce_hw_v1.c:2410:10: note: each undeclared identifier is reported only once for each function it appears in
>> drivers/infiniband/hw/hns/hns_roce_hw_v1.c:1842:6: warning: unused variable 'reg_val' [-Wunused-variable]
u32 reg_val = 0;
^
>> drivers/infiniband/hw/hns/hns_roce_hw_v1.c:1837:24: warning: unused variable 'rq_db' [-Wunused-variable]
struct hns_roce_rq_db rq_db;
^
vim +/DB_REG_OFFSET +2410 drivers/infiniband/hw/hns/hns_roce_hw_v1.c
2404 RQ_DOORBELL_U32_8_CMD_S, 1);
2405 roce_set_bit(doorbell[1], RQ_DOORBELL_U32_8_HW_SYNC_S, 1);
2406
2407 if (ibqp->uobject) {
2408 hr_qp->rq.db_reg_l = hr_dev->reg_base +
2409 ROCEE_DB_OTHERS_L_0_REG +
> 2410 DB_REG_OFFSET * hr_dev->priv_uar.index;
2411 }
2412
2413 hns_roce_write64_k(doorbell, hr_qp->rq.db_reg_l);
---
0-DAY kernel test infrastructure Open Source Technology Center
https://lists.01.org/pipermail/kbuild-all Intel Corporation
[-- Attachment #2: .config.gz --]
[-- Type: application/octet-stream, Size: 51907 bytes --]
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2016-09-14 6:41 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2016-09-13 19:49 [PATCH V2 for-next 0/9] IB/hns: Add CM(Connection Manager) Support to HNS RoCe Driver Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 1/9] IB/hns: Register HNS RoCE Driver get_netdev() with IB Core Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 2/9] IB/hns: Add & initialize "node_guid" parameter for RDMA CM Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 3/9] IB/hns: Fix the value of device_cap_flags Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 4/9] IB/hns: Fix two possible bugs for rdma cm Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 5/9] IB/hns: Add phy_port for computing GSI/QPN Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 6/9] IB/hns: Change the logic for allocating uar registers Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 7/9] IB/hns: Fix the bug of rdma cm connecting on user mode Salil Mehta
2016-09-14 6:41 ` kbuild test robot
2016-09-13 19:49 ` [PATCH V2 for-next 8/9] IB/hns: Fix two bugs for rdma cm connecting Salil Mehta
2016-09-13 19:49 ` [PATCH V2 for-next 9/9] IB/hns: Add node_guid definition to the bindings document Salil Mehta
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).