* [PATCH for-next 1/9] RDMA/hns: Bugfix for querying qkey
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 2/9] RDMA/hns: Fix cmdq parameter of querying pf timer resource Weihang Li
` (8 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
From: Lijun Ou <oulijun@huawei.com>
The qkey queried through the query ud qp verb is a fixed value and it
should be read from qp context.
Fixes: 926a01dc000d ("RDMA/hns: Add QP operations support for hip08 SoC")
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index ad9a11a..61098b0 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -4680,7 +4680,7 @@ static int hns_roce_v2_query_qp(struct ib_qp *ibqp, struct ib_qp_attr *qp_attr,
qp_attr->path_mig_state = IB_MIG_ARMED;
qp_attr->ah_attr.type = RDMA_AH_ATTR_TYPE_ROCE;
if (hr_qp->ibqp.qp_type == IB_QPT_UD)
- qp_attr->qkey = V2_QKEY_VAL;
+ qp_attr->qkey = le32_to_cpu(context.qkey_xrcd);
qp_attr->rq_psn = roce_get_field(context.byte_108_rx_reqepsn,
V2_QPC_BYTE_108_RX_REQ_EPSN_M,
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 2/9] RDMA/hns: Fix cmdq parameter of querying pf timer resource
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
2020-05-08 9:45 ` [PATCH for-next 1/9] RDMA/hns: Bugfix for querying qkey Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 3/9] RDMA/hns: Fix assignment to ba_pg_sz of eqe Weihang Li
` (7 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
From: Lang Cheng <chenglang@huawei.com>
The firmware has reduced the number of descriptions of command
HNS_ROCE_OPC_QUERY_PF_TIMER_RES to 1. The driver needs to adapt, otherwise
the hardware will report error 4(CMD_NEXT_ERR).
Fixes: 0e40dc2f70cd ("RDMA/hns: Add timer allocation support for hip08")
Signed-off-by: Lang Cheng <chenglang@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 32 +++++++++++-------------------
1 file changed, 12 insertions(+), 20 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 61098b0..81abab0 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -1450,34 +1450,26 @@ static int hns_roce_query_pf_resource(struct hns_roce_dev *hr_dev)
static int hns_roce_query_pf_timer_resource(struct hns_roce_dev *hr_dev)
{
struct hns_roce_pf_timer_res_a *req_a;
- struct hns_roce_cmq_desc desc[2];
- int ret, i;
+ struct hns_roce_cmq_desc desc;
+ int ret;
- for (i = 0; i < 2; i++) {
- hns_roce_cmq_setup_basic_desc(&desc[i],
- HNS_ROCE_OPC_QUERY_PF_TIMER_RES,
- true);
+ hns_roce_cmq_setup_basic_desc(&desc, HNS_ROCE_OPC_QUERY_PF_TIMER_RES,
+ true);
- if (i == 0)
- desc[i].flag |= cpu_to_le16(HNS_ROCE_CMD_FLAG_NEXT);
- else
- desc[i].flag &= ~cpu_to_le16(HNS_ROCE_CMD_FLAG_NEXT);
- }
-
- ret = hns_roce_cmq_send(hr_dev, desc, 2);
+ ret = hns_roce_cmq_send(hr_dev, &desc, 1);
if (ret)
return ret;
- req_a = (struct hns_roce_pf_timer_res_a *)desc[0].data;
+ req_a = (struct hns_roce_pf_timer_res_a *)desc.data;
hr_dev->caps.qpc_timer_bt_num =
- roce_get_field(req_a->qpc_timer_bt_idx_num,
- PF_RES_DATA_1_PF_QPC_TIMER_BT_NUM_M,
- PF_RES_DATA_1_PF_QPC_TIMER_BT_NUM_S);
+ roce_get_field(req_a->qpc_timer_bt_idx_num,
+ PF_RES_DATA_1_PF_QPC_TIMER_BT_NUM_M,
+ PF_RES_DATA_1_PF_QPC_TIMER_BT_NUM_S);
hr_dev->caps.cqc_timer_bt_num =
- roce_get_field(req_a->cqc_timer_bt_idx_num,
- PF_RES_DATA_2_PF_CQC_TIMER_BT_NUM_M,
- PF_RES_DATA_2_PF_CQC_TIMER_BT_NUM_S);
+ roce_get_field(req_a->cqc_timer_bt_idx_num,
+ PF_RES_DATA_2_PF_CQC_TIMER_BT_NUM_M,
+ PF_RES_DATA_2_PF_CQC_TIMER_BT_NUM_S);
return 0;
}
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 3/9] RDMA/hns: Fix assignment to ba_pg_sz of eqe
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
2020-05-08 9:45 ` [PATCH for-next 1/9] RDMA/hns: Bugfix for querying qkey Weihang Li
2020-05-08 9:45 ` [PATCH for-next 2/9] RDMA/hns: Fix cmdq parameter of querying pf timer resource Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 4/9] RDMA/hns: Fix wrong assignment of SRQ's max_wr Weihang Li
` (6 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
From: Wenpeng Liang <liangwenpeng@huawei.com>
When allocating eq buffer, the size of base address page should be defined
by eqe_ba_pg_sz instead of srqwqe_ba_pg_sz.
Fixes: 477a0a387072 ("RDMA/hns: Optimize 0 hop addressing for EQE buffer")
Signed-off-by: Wenpeng Liang <liangwenpeng@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 81abab0..3039896 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -5612,7 +5612,7 @@ static int alloc_eq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_eq *eq)
buf_attr.fixed_page = true;
err = hns_roce_mtr_create(hr_dev, &eq->mtr, &buf_attr,
- hr_dev->caps.srqwqe_ba_pg_sz +
+ hr_dev->caps.eqe_ba_pg_sz +
PAGE_ADDR_SHIFT, NULL, 0);
if (err)
dev_err(hr_dev->dev, "Failed to alloc EQE mtr, err %d\n", err);
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 4/9] RDMA/hns: Fix wrong assignment of SRQ's max_wr
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (2 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 3/9] RDMA/hns: Fix assignment to ba_pg_sz of eqe Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 5/9] RDMA/hns: Fix error with to_hr_hem_entries_count() Weihang Li
` (5 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
srq's attribute max_wr should be 1 less than the total count of wqe.
Fixes: ffb1308b88b6 ("RDMA/hns: Move SRQ code to the reasonable place")
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++--
1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 3039896..61c6bb3 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -5041,8 +5041,8 @@ static int hns_roce_v2_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
SRQC_BYTE_8_SRQ_LIMIT_WL_S);
attr->srq_limit = limit_wl;
- attr->max_wr = srq->wqe_cnt;
- attr->max_sge = srq->max_gs;
+ attr->max_wr = srq->wqe_cnt - 1;
+ attr->max_sge = srq->max_gs;
memcpy(srq_context, mailbox->buf, sizeof(*srq_context));
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 5/9] RDMA/hns: Fix error with to_hr_hem_entries_count()
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (3 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 4/9] RDMA/hns: Fix wrong assignment of SRQ's max_wr Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 6/9] RDMA/hns: Store mr len information into mr obj Weihang Li
` (4 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
For ilog2(x), if x is 0 and not a constant variable, it will return -1. And
there will be an error as below:
hns3 0000:7d:00.0 hns_0: Local work queue 0x8 catast error, sub_event type is: 2
So modify to_hr_hem_entries_shift() to return 0 if conut is 0.
Fixes: 54d6638765b0 ("RDMA/hns: Optimize WQE buffer size calculating process")
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 3 +++
1 file changed, 3 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 5cac14d..5564773 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -1111,6 +1111,9 @@ static inline u32 to_hr_hem_entries_count(u32 count, u32 buf_shift)
static inline u32 to_hr_hem_entries_shift(u32 count, u32 buf_shift)
{
+ if (!count)
+ return 0;
+
return ilog2(to_hr_hem_entries_count(count, buf_shift));
}
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 6/9] RDMA/hns: Store mr len information into mr obj
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (4 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 5/9] RDMA/hns: Fix error with to_hr_hem_entries_count() Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 7/9] RDMA/hns: Remove redundant memcpy() Weihang Li
` (3 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
From: Lang Cheng <chenglang@huawei.com>
The length information should be stored in the struct ib_mr object,
otherwise the length value of a valid mr object would always be 0.
Signed-off-by: Lang Cheng <chenglang@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_mr.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/drivers/infiniband/hw/hns/hns_roce_mr.c b/drivers/infiniband/hw/hns/hns_roce_mr.c
index ecd7675..f727b18 100644
--- a/drivers/infiniband/hw/hns/hns_roce_mr.c
+++ b/drivers/infiniband/hw/hns/hns_roce_mr.c
@@ -285,6 +285,7 @@ struct ib_mr *hns_roce_reg_user_mr(struct ib_pd *pd, u64 start, u64 length,
goto err_alloc_pbl;
mr->ibmr.rkey = mr->ibmr.lkey = mr->key;
+ mr->ibmr.length = length;
return &mr->ibmr;
@@ -451,6 +452,7 @@ struct ib_mr *hns_roce_alloc_mr(struct ib_pd *pd, enum ib_mr_type mr_type,
goto err_pbl;
mr->ibmr.rkey = mr->ibmr.lkey = mr->key;
+ mr->ibmr.length = length;
return &mr->ibmr;
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 7/9] RDMA/hns: Remove redundant memcpy()
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (5 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 6/9] RDMA/hns: Store mr len information into mr obj Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 8/9] RDMA/hns: Rename macro for defining hns hardware page size Weihang Li
` (2 subsequent siblings)
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
srq_context is a local variables and is only used to get some fields from
buffer of mailbox. It's meaningless to copy mailbox's buffer's contents
back to it.
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index 61c6bb3..d137abc 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -5044,8 +5044,6 @@ static int hns_roce_v2_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
attr->max_wr = srq->wqe_cnt - 1;
attr->max_sge = srq->max_gs;
- memcpy(srq_context, mailbox->buf, sizeof(*srq_context));
-
out:
hns_roce_free_cmd_mailbox(hr_dev, mailbox);
return ret;
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 8/9] RDMA/hns: Rename macro for defining hns hardware page size
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (6 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 7/9] RDMA/hns: Remove redundant memcpy() Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-08 9:45 ` [PATCH for-next 9/9] RDMA/hns: Reserve one sge in order to avoid local length error Weihang Li
2020-05-22 7:39 ` [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups liweihang
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
From: Xi Wang <wangxi11@huawei.com>
Rename the PAGE_ADDR_SHIFT as HNS_HW_PAGE_SHIFT to make code more
readable.
Signed-off-by: Xi Wang <wangxi11@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_alloc.c | 6 +++---
drivers/infiniband/hw/hns/hns_roce_cq.c | 4 ++--
drivers/infiniband/hw/hns/hns_roce_device.h | 10 ++++++----
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 4 ++--
drivers/infiniband/hw/hns/hns_roce_mr.c | 6 +++---
drivers/infiniband/hw/hns/hns_roce_qp.c | 4 ++--
drivers/infiniband/hw/hns/hns_roce_srq.c | 8 ++++----
7 files changed, 22 insertions(+), 20 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_alloc.c b/drivers/infiniband/hw/hns/hns_roce_alloc.c
index 365e7db..742aee8 100644
--- a/drivers/infiniband/hw/hns/hns_roce_alloc.c
+++ b/drivers/infiniband/hw/hns/hns_roce_alloc.c
@@ -189,8 +189,8 @@ int hns_roce_buf_alloc(struct hns_roce_dev *hr_dev, u32 size, u32 max_direct,
u32 page_size;
int i;
- /* The minimum shift of the page accessed by hw is PAGE_ADDR_SHIFT */
- buf->page_shift = max_t(int, PAGE_ADDR_SHIFT, page_shift);
+ /* The minimum shift of the page accessed by hw is HNS_HW_PAGE_SHIFT */
+ buf->page_shift = max_t(int, HNS_HW_PAGE_SHIFT, page_shift);
page_size = 1 << buf->page_shift;
buf->npages = DIV_ROUND_UP(size, page_size);
@@ -261,7 +261,7 @@ int hns_roce_get_umem_bufs(struct hns_roce_dev *hr_dev, dma_addr_t *bufs,
int idx = 0;
u64 addr;
- if (page_shift < PAGE_ADDR_SHIFT) {
+ if (page_shift < HNS_HW_PAGE_SHIFT) {
dev_err(hr_dev->dev, "Failed to check umem page shift %d!\n",
page_shift);
return -EINVAL;
diff --git a/drivers/infiniband/hw/hns/hns_roce_cq.c b/drivers/infiniband/hw/hns/hns_roce_cq.c
index d2d7074..6dd8dea 100644
--- a/drivers/infiniband/hw/hns/hns_roce_cq.c
+++ b/drivers/infiniband/hw/hns/hns_roce_cq.c
@@ -149,14 +149,14 @@ static int alloc_cq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_cq *hr_cq,
struct hns_roce_buf_attr buf_attr = {};
int err;
- buf_attr.page_shift = hr_dev->caps.cqe_buf_pg_sz + PAGE_ADDR_SHIFT;
+ buf_attr.page_shift = hr_dev->caps.cqe_buf_pg_sz + HNS_HW_PAGE_SHIFT;
buf_attr.region[0].size = hr_cq->cq_depth * hr_dev->caps.cq_entry_sz;
buf_attr.region[0].hopnum = hr_dev->caps.cqe_hop_num;
buf_attr.region_count = 1;
buf_attr.fixed_page = true;
err = hns_roce_mtr_create(hr_dev, &hr_cq->mtr, &buf_attr,
- hr_dev->caps.cqe_ba_pg_sz + PAGE_ADDR_SHIFT,
+ hr_dev->caps.cqe_ba_pg_sz + HNS_HW_PAGE_SHIFT,
udata, addr);
if (err)
ibdev_err(ibdev, "Failed to alloc CQ mtr, err %d\n", err);
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 5564773..543c37a 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -260,7 +260,9 @@ enum {
#define HNS_ROCE_PORT_DOWN 0
#define HNS_ROCE_PORT_UP 1
-#define PAGE_ADDR_SHIFT 12
+/* The minimum page size is 4K for hardware */
+#define HNS_HW_PAGE_SHIFT 12
+#define HNS_HW_PAGE_SIZE (1 << HNS_HW_PAGE_SHIFT)
/* The minimum page count for hardware access page directly. */
#define HNS_HW_DIRECT_PAGE_COUNT 2
@@ -1079,16 +1081,16 @@ static inline dma_addr_t hns_roce_buf_page(struct hns_roce_buf *buf, int idx)
return buf->page_list[idx].map;
}
-#define hr_hw_page_align(x) ALIGN(x, 1 << PAGE_ADDR_SHIFT)
+#define hr_hw_page_align(x) ALIGN(x, 1 << HNS_HW_PAGE_SHIFT)
static inline u64 to_hr_hw_page_addr(u64 addr)
{
- return addr >> PAGE_ADDR_SHIFT;
+ return addr >> HNS_HW_PAGE_SHIFT;
}
static inline u32 to_hr_hw_page_shift(u32 page_shift)
{
- return page_shift - PAGE_ADDR_SHIFT;
+ return page_shift - HNS_HW_PAGE_SHIFT;
}
static inline u32 to_hr_hem_hopnum(u32 hopnum, u32 count)
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index d137abc..e7ebe31 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -5603,7 +5603,7 @@ static int alloc_eq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_eq *eq)
else
eq->hop_num = hr_dev->caps.eqe_hop_num;
- buf_attr.page_shift = hr_dev->caps.eqe_buf_pg_sz + PAGE_ADDR_SHIFT;
+ buf_attr.page_shift = hr_dev->caps.eqe_buf_pg_sz + HNS_HW_PAGE_SHIFT;
buf_attr.region[0].size = eq->entries * eq->eqe_size;
buf_attr.region[0].hopnum = eq->hop_num;
buf_attr.region_count = 1;
@@ -5611,7 +5611,7 @@ static int alloc_eq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_eq *eq)
err = hns_roce_mtr_create(hr_dev, &eq->mtr, &buf_attr,
hr_dev->caps.eqe_ba_pg_sz +
- PAGE_ADDR_SHIFT, NULL, 0);
+ HNS_HW_PAGE_SHIFT, NULL, 0);
if (err)
dev_err(hr_dev->dev, "Failed to alloc EQE mtr, err %d\n", err);
diff --git a/drivers/infiniband/hw/hns/hns_roce_mr.c b/drivers/infiniband/hw/hns/hns_roce_mr.c
index f727b18..3075e845 100644
--- a/drivers/infiniband/hw/hns/hns_roce_mr.c
+++ b/drivers/infiniband/hw/hns/hns_roce_mr.c
@@ -120,7 +120,7 @@ static int alloc_mr_pbl(struct hns_roce_dev *hr_dev, struct hns_roce_mr *mr,
mr->pbl_hop_num = is_fast ? 1 : hr_dev->caps.pbl_hop_num;
buf_attr.page_shift = is_fast ? PAGE_SHIFT :
- hr_dev->caps.pbl_buf_pg_sz + PAGE_ADDR_SHIFT;
+ hr_dev->caps.pbl_buf_pg_sz + HNS_HW_PAGE_SHIFT;
buf_attr.region[0].size = length;
buf_attr.region[0].hopnum = mr->pbl_hop_num;
buf_attr.region_count = 1;
@@ -130,7 +130,7 @@ static int alloc_mr_pbl(struct hns_roce_dev *hr_dev, struct hns_roce_mr *mr,
buf_attr.mtt_only = is_fast;
err = hns_roce_mtr_create(hr_dev, &mr->pbl_mtr, &buf_attr,
- hr_dev->caps.pbl_ba_pg_sz + PAGE_ADDR_SHIFT,
+ hr_dev->caps.pbl_ba_pg_sz + HNS_HW_PAGE_SHIFT,
udata, start);
if (err)
ibdev_err(ibdev, "failed to alloc pbl mtr, ret = %d.\n", err);
@@ -819,7 +819,7 @@ static int mtr_alloc_bufs(struct hns_roce_dev *hr_dev, struct hns_roce_mtr *mtr,
}
/* must bigger than minimum hardware page shift */
- if (best_pg_shift < PAGE_ADDR_SHIFT || all_pg_count < 1) {
+ if (best_pg_shift < HNS_HW_PAGE_SHIFT || all_pg_count < 1) {
ret = -EINVAL;
ibdev_err(ibdev, "Failed to check mtr page shift %d count %d\n",
best_pg_shift, all_pg_count);
diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c
index b570759..c8334d7 100644
--- a/drivers/infiniband/hw/hns/hns_roce_qp.c
+++ b/drivers/infiniband/hw/hns/hns_roce_qp.c
@@ -546,7 +546,7 @@ static int split_wqe_buf_region(struct hns_roce_dev *hr_dev,
if (hr_qp->buff_size < 1)
return -EINVAL;
- buf_attr->page_shift = PAGE_ADDR_SHIFT + hr_dev->caps.mtt_buf_pg_sz;
+ buf_attr->page_shift = HNS_HW_PAGE_SHIFT + hr_dev->caps.mtt_buf_pg_sz;
buf_attr->fixed_page = true;
buf_attr->region_count = idx;
@@ -681,7 +681,7 @@ static int alloc_qp_buf(struct hns_roce_dev *hr_dev, struct hns_roce_qp *hr_qp,
goto err_inline;
}
ret = hns_roce_mtr_create(hr_dev, &hr_qp->mtr, &buf_attr,
- PAGE_ADDR_SHIFT + hr_dev->caps.mtt_ba_pg_sz,
+ HNS_HW_PAGE_SHIFT + hr_dev->caps.mtt_ba_pg_sz,
udata, addr);
if (ret) {
ibdev_err(ibdev, "failed to create WQE mtr, ret = %d.\n", ret);
diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c
index 6e5a2ad..03b76e6 100644
--- a/drivers/infiniband/hw/hns/hns_roce_srq.c
+++ b/drivers/infiniband/hw/hns/hns_roce_srq.c
@@ -187,7 +187,7 @@ static int alloc_srq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq,
HNS_ROCE_SGE_SIZE *
srq->max_gs)));
- buf_attr.page_shift = hr_dev->caps.srqwqe_buf_pg_sz + PAGE_ADDR_SHIFT;
+ buf_attr.page_shift = hr_dev->caps.srqwqe_buf_pg_sz + HNS_HW_PAGE_SHIFT;
buf_attr.region[0].size = to_hr_hem_entries_size(srq->wqe_cnt,
srq->wqe_shift);
buf_attr.region[0].hopnum = hr_dev->caps.srqwqe_hop_num;
@@ -196,7 +196,7 @@ static int alloc_srq_buf(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq,
err = hns_roce_mtr_create(hr_dev, &srq->buf_mtr, &buf_attr,
hr_dev->caps.srqwqe_ba_pg_sz +
- PAGE_ADDR_SHIFT, udata, addr);
+ HNS_HW_PAGE_SHIFT, udata, addr);
if (err)
ibdev_err(ibdev, "Failed to alloc SRQ buf mtr, err %d\n", err);
@@ -218,7 +218,7 @@ static int alloc_srq_idx(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq,
srq->idx_que.entry_shift = ilog2(HNS_ROCE_IDX_QUE_ENTRY_SZ);
- buf_attr.page_shift = hr_dev->caps.idx_buf_pg_sz + PAGE_ADDR_SHIFT;
+ buf_attr.page_shift = hr_dev->caps.idx_buf_pg_sz + HNS_HW_PAGE_SHIFT;
buf_attr.region[0].size = to_hr_hem_entries_size(srq->wqe_cnt,
srq->idx_que.entry_shift);
buf_attr.region[0].hopnum = hr_dev->caps.idx_hop_num;
@@ -226,7 +226,7 @@ static int alloc_srq_idx(struct hns_roce_dev *hr_dev, struct hns_roce_srq *srq,
buf_attr.fixed_page = true;
err = hns_roce_mtr_create(hr_dev, &idx_que->mtr, &buf_attr,
- hr_dev->caps.idx_ba_pg_sz + PAGE_ADDR_SHIFT,
+ hr_dev->caps.idx_ba_pg_sz + HNS_HW_PAGE_SHIFT,
udata, addr);
if (err) {
ibdev_err(ibdev, "Failed to alloc SRQ idx mtr, err %d\n", err);
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* [PATCH for-next 9/9] RDMA/hns: Reserve one sge in order to avoid local length error
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (7 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 8/9] RDMA/hns: Rename macro for defining hns hardware page size Weihang Li
@ 2020-05-08 9:45 ` Weihang Li
2020-05-22 7:39 ` [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups liweihang
9 siblings, 0 replies; 13+ messages in thread
From: Weihang Li @ 2020-05-08 9:45 UTC (permalink / raw)
To: dledford, jgg; +Cc: leon, linux-rdma, linuxarm
From: Lijun Ou <oulijun@huawei.com>
When rq/srq sge length is smaller than sq sge length, it will produce a
local length error and may cause the bus to hang. Therefore, for rq wqe
and srq wqe, one reserved sge pointing to a reserved mr is used to avoid
this error.
Signed-off-by: Lijun Ou <oulijun@huawei.com>
Signed-off-by: Weihang Li <liweihang@huawei.com>
---
drivers/infiniband/hw/hns/hns_roce_device.h | 2 ++
drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 9 +++++----
drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 4 +++-
drivers/infiniband/hw/hns/hns_roce_qp.c | 5 +++--
drivers/infiniband/hw/hns/hns_roce_srq.c | 2 +-
5 files changed, 14 insertions(+), 8 deletions(-)
diff --git a/drivers/infiniband/hw/hns/hns_roce_device.h b/drivers/infiniband/hw/hns/hns_roce_device.h
index 543c37a..2f804c18 100644
--- a/drivers/infiniband/hw/hns/hns_roce_device.h
+++ b/drivers/infiniband/hw/hns/hns_roce_device.h
@@ -66,6 +66,8 @@
#define HNS_ROCE_CQE_WCMD_EMPTY_BIT 0x2
#define HNS_ROCE_MIN_CQE_CNT 16
+#define HNS_ROCE_RESERVED_SGE 1
+
#define HNS_ROCE_MAX_IRQ_NUM 128
#define HNS_ROCE_SGE_IN_WQE 2
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
index e7ebe31..6877c1e 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.c
@@ -629,7 +629,7 @@ static int hns_roce_v2_post_recv(struct ib_qp *ibqp,
wqe_idx = (hr_qp->rq.head + nreq) & (hr_qp->rq.wqe_cnt - 1);
- if (unlikely(wr->num_sge > hr_qp->rq.max_gs)) {
+ if (unlikely(wr->num_sge >= hr_qp->rq.max_gs)) {
ibdev_err(ibdev, "rq:num_sge=%d >= qp->sq.max_gs=%d\n",
wr->num_sge, hr_qp->rq.max_gs);
ret = -EINVAL;
@@ -649,6 +649,7 @@ static int hns_roce_v2_post_recv(struct ib_qp *ibqp,
if (i < hr_qp->rq.max_gs) {
dseg->lkey = cpu_to_le32(HNS_ROCE_INVALID_LKEY);
dseg->addr = 0;
+ dseg->len = cpu_to_le32(HNS_ROCE_INVALID_SGE_LENGTH);
}
/* rq support inline data */
@@ -782,8 +783,8 @@ static int hns_roce_v2_post_srq_recv(struct ib_srq *ibsrq,
}
if (i < srq->max_gs) {
- dseg[i].len = 0;
- dseg[i].lkey = cpu_to_le32(0x100);
+ dseg[i].len = cpu_to_le32(HNS_ROCE_INVALID_SGE_LENGTH);
+ dseg[i].lkey = cpu_to_le32(HNS_ROCE_INVALID_LKEY);
dseg[i].addr = 0;
}
@@ -5042,7 +5043,7 @@ static int hns_roce_v2_query_srq(struct ib_srq *ibsrq, struct ib_srq_attr *attr)
attr->srq_limit = limit_wl;
attr->max_wr = srq->wqe_cnt - 1;
- attr->max_sge = srq->max_gs;
+ attr->max_sge = srq->max_gs - HNS_ROCE_RESERVED_SGE;
out:
hns_roce_free_cmd_mailbox(hr_dev, mailbox);
diff --git a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
index 05bfe07..a135b09 100644
--- a/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
+++ b/drivers/infiniband/hw/hns/hns_roce_hw_v2.h
@@ -92,7 +92,9 @@
#define HNS_ROCE_V2_CQC_TIMER_ENTRY_SZ PAGE_SIZE
#define HNS_ROCE_V2_PAGE_SIZE_SUPPORTED 0xFFFFF000
#define HNS_ROCE_V2_MAX_INNER_MTPT_NUM 2
-#define HNS_ROCE_INVALID_LKEY 0x100
+#define HNS_ROCE_INVALID_LKEY 0x0
+#define HNS_ROCE_INVALID_SGE_LENGTH 0x80000000
+
#define HNS_ROCE_CMQ_TX_TIMEOUT 30000
#define HNS_ROCE_V2_UC_RC_SGE_NUM_IN_WQE 2
#define HNS_ROCE_V2_RSV_QPS 8
diff --git a/drivers/infiniband/hw/hns/hns_roce_qp.c b/drivers/infiniband/hw/hns/hns_roce_qp.c
index c8334d7..6e5df46 100644
--- a/drivers/infiniband/hw/hns/hns_roce_qp.c
+++ b/drivers/infiniband/hw/hns/hns_roce_qp.c
@@ -386,7 +386,8 @@ static int set_rq_size(struct hns_roce_dev *hr_dev, struct ib_qp_cap *cap,
return -EINVAL;
}
- hr_qp->rq.max_gs = roundup_pow_of_two(max(1U, cap->max_recv_sge));
+ hr_qp->rq.max_gs = roundup_pow_of_two(max(1U, cap->max_recv_sge) +
+ HNS_ROCE_RESERVED_SGE);
if (hr_dev->caps.max_rq_sg <= HNS_ROCE_SGE_IN_WQE)
hr_qp->rq.wqe_shift = ilog2(hr_dev->caps.max_rq_desc_sz);
@@ -401,7 +402,7 @@ static int set_rq_size(struct hns_roce_dev *hr_dev, struct ib_qp_cap *cap,
hr_qp->rq_inl_buf.wqe_cnt = 0;
cap->max_recv_wr = cnt;
- cap->max_recv_sge = hr_qp->rq.max_gs;
+ cap->max_recv_sge = hr_qp->rq.max_gs - HNS_ROCE_RESERVED_SGE;
return 0;
}
diff --git a/drivers/infiniband/hw/hns/hns_roce_srq.c b/drivers/infiniband/hw/hns/hns_roce_srq.c
index 03b76e6..3018c98 100644
--- a/drivers/infiniband/hw/hns/hns_roce_srq.c
+++ b/drivers/infiniband/hw/hns/hns_roce_srq.c
@@ -297,7 +297,7 @@ int hns_roce_create_srq(struct ib_srq *ib_srq,
spin_lock_init(&srq->lock);
srq->wqe_cnt = roundup_pow_of_two(init_attr->attr.max_wr + 1);
- srq->max_gs = init_attr->attr.max_sge;
+ srq->max_gs = init_attr->attr.max_sge + HNS_ROCE_RESERVED_SGE;
if (udata) {
ret = ib_copy_from_udata(&ucmd, udata, sizeof(ucmd));
--
2.8.1
^ permalink raw reply related [flat|nested] 13+ messages in thread* Re: [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups
2020-05-08 9:45 [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups Weihang Li
` (8 preceding siblings ...)
2020-05-08 9:45 ` [PATCH for-next 9/9] RDMA/hns: Reserve one sge in order to avoid local length error Weihang Li
@ 2020-05-22 7:39 ` liweihang
2020-05-22 9:01 ` liweihang
2020-05-22 14:28 ` Jason Gunthorpe
9 siblings, 2 replies; 13+ messages in thread
From: liweihang @ 2020-05-22 7:39 UTC (permalink / raw)
To: jgg; +Cc: linux-rdma, Linuxarm
On 2020/5/8 17:46, Weihang Li wrote:
> This series contains the following:
> - #1 ~ #2 are fixes to solve issues found from previous versions.
> - #3 ~ #5 are fixes for recent refactoring codes to 5.8.
> - #6 ~ #9 are various cleanups.
>
> Lang Cheng (2):
> RDMA/hns: Fix cmdq parameter of querying pf timer resource
> RDMA/hns: Store mr len information into mr obj
>
> Lijun Ou (2):
> RDMA/hns: Bugfix for querying qkey
> RDMA/hns: Reserve one sge in order to avoid local length error
>
> Weihang Li (3):
> RDMA/hns: Fix wrong assignment of SRQ's max_wr
> RDMA/hns: Fix error with to_hr_hem_entries_count()
> RDMA/hns: Remove redundant memcpy()
>
> Wenpeng Liang (1):
> RDMA/hns: Fix assignment to ba_pg_sz of eqe
>
> Xi Wang (1):
> RDMA/hns: Rename macro for defining hns hardware page size
>
> drivers/infiniband/hw/hns/hns_roce_alloc.c | 6 ++--
> drivers/infiniband/hw/hns/hns_roce_cq.c | 4 +--
> drivers/infiniband/hw/hns/hns_roce_device.h | 15 +++++---
> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 53 ++++++++++++-----------------
> drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 4 ++-
> drivers/infiniband/hw/hns/hns_roce_mr.c | 8 +++--
> drivers/infiniband/hw/hns/hns_roce_qp.c | 9 ++---
> drivers/infiniband/hw/hns/hns_roce_srq.c | 10 +++---
> 8 files changed, 56 insertions(+), 53 deletions(-)
>
Hi Jason,
I notice that this series has been marked as "Accepted" on patchwork,
but I can't find them on your for-next branch. Maybe there is something
wrong :)
Thank you
Weihang
^ permalink raw reply [flat|nested] 13+ messages in thread* Re: [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups
2020-05-22 7:39 ` [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups liweihang
@ 2020-05-22 9:01 ` liweihang
2020-05-22 14:28 ` Jason Gunthorpe
1 sibling, 0 replies; 13+ messages in thread
From: liweihang @ 2020-05-22 9:01 UTC (permalink / raw)
To: jgg; +Cc: linux-rdma, Linuxarm
On 2020/5/22 15:39, liweihang wrote:
> On 2020/5/8 17:46, Weihang Li wrote:
>> This series contains the following:
>> - #1 ~ #2 are fixes to solve issues found from previous versions.
>> - #3 ~ #5 are fixes for recent refactoring codes to 5.8.
>> - #6 ~ #9 are various cleanups.
>>
>> Lang Cheng (2):
>> RDMA/hns: Fix cmdq parameter of querying pf timer resource
>> RDMA/hns: Store mr len information into mr obj
>>
>> Lijun Ou (2):
>> RDMA/hns: Bugfix for querying qkey
>> RDMA/hns: Reserve one sge in order to avoid local length error
>>
>> Weihang Li (3):
>> RDMA/hns: Fix wrong assignment of SRQ's max_wr
>> RDMA/hns: Fix error with to_hr_hem_entries_count()
>> RDMA/hns: Remove redundant memcpy()
>>
>> Wenpeng Liang (1):
>> RDMA/hns: Fix assignment to ba_pg_sz of eqe
>>
>> Xi Wang (1):
>> RDMA/hns: Rename macro for defining hns hardware page size
>>
>> drivers/infiniband/hw/hns/hns_roce_alloc.c | 6 ++--
>> drivers/infiniband/hw/hns/hns_roce_cq.c | 4 +--
>> drivers/infiniband/hw/hns/hns_roce_device.h | 15 +++++---
>> drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 53 ++++++++++++-----------------
>> drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 4 ++-
>> drivers/infiniband/hw/hns/hns_roce_mr.c | 8 +++--
>> drivers/infiniband/hw/hns/hns_roce_qp.c | 9 ++---
>> drivers/infiniband/hw/hns/hns_roce_srq.c | 10 +++---
>> 8 files changed, 56 insertions(+), 53 deletions(-)
>>
>
> Hi Jason,
>
> I notice that this series has been marked as "Accepted" on patchwork,
> but I can't find them on your for-next branch. Maybe there is something
> wrong :)
>
Sorry, I just saw them on your branch.
Thanks
Weihang
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups
2020-05-22 7:39 ` [PATCH for-next 0/9] RDMA/hns: Various fixes and cleanups liweihang
2020-05-22 9:01 ` liweihang
@ 2020-05-22 14:28 ` Jason Gunthorpe
1 sibling, 0 replies; 13+ messages in thread
From: Jason Gunthorpe @ 2020-05-22 14:28 UTC (permalink / raw)
To: liweihang; +Cc: linux-rdma, Linuxarm
On Fri, May 22, 2020 at 07:39:31AM +0000, liweihang wrote:
> On 2020/5/8 17:46, Weihang Li wrote:
> > This series contains the following:
> > - #1 ~ #2 are fixes to solve issues found from previous versions.
> > - #3 ~ #5 are fixes for recent refactoring codes to 5.8.
> > - #6 ~ #9 are various cleanups.
> >
> > Lang Cheng (2):
> > RDMA/hns: Fix cmdq parameter of querying pf timer resource
> > RDMA/hns: Store mr len information into mr obj
> >
> > Lijun Ou (2):
> > RDMA/hns: Bugfix for querying qkey
> > RDMA/hns: Reserve one sge in order to avoid local length error
> >
> > Weihang Li (3):
> > RDMA/hns: Fix wrong assignment of SRQ's max_wr
> > RDMA/hns: Fix error with to_hr_hem_entries_count()
> > RDMA/hns: Remove redundant memcpy()
> >
> > Wenpeng Liang (1):
> > RDMA/hns: Fix assignment to ba_pg_sz of eqe
> >
> > Xi Wang (1):
> > RDMA/hns: Rename macro for defining hns hardware page size
> >
> > drivers/infiniband/hw/hns/hns_roce_alloc.c | 6 ++--
> > drivers/infiniband/hw/hns/hns_roce_cq.c | 4 +--
> > drivers/infiniband/hw/hns/hns_roce_device.h | 15 +++++---
> > drivers/infiniband/hw/hns/hns_roce_hw_v2.c | 53 ++++++++++++-----------------
> > drivers/infiniband/hw/hns/hns_roce_hw_v2.h | 4 ++-
> > drivers/infiniband/hw/hns/hns_roce_mr.c | 8 +++--
> > drivers/infiniband/hw/hns/hns_roce_qp.c | 9 ++---
> > drivers/infiniband/hw/hns/hns_roce_srq.c | 10 +++---
> > 8 files changed, 56 insertions(+), 53 deletions(-)
> >
>
> Hi Jason,
>
> I notice that this series has been marked as "Accepted" on patchwork,
> but I can't find them on your for-next branch. Maybe there is something
> wrong :)
It is in the wip branch waiting on 0-day to test compile it
Jason
^ permalink raw reply [flat|nested] 13+ messages in thread