[PATCH rdma-next 0/8] RDMA/qedr: various fixes

[PATCH rdma-next 0/8] RDMA/qedr: various fixes

[Michal Kalderon]

This set addresses several issues that were observed
and reproduced on different test and production configurations.

Dave, Jason,
There is one qede patch which is related to the mtu change notify.
This is a small change and required for the qede_rdma.h interface
change. Please consider applying this to rdma tree.

thanks,
Michal

Michal Kalderon (8):
  RDMA/qedr: Fix qp structure memory leak
  RDMA/qedr: Fix doorbell setting
  RDMA/qedr: Fix use of uninitialized field
  RDMA/qedr: Fix return code if accept is called on a destroyed qp
  qede: Notify qedr when mtu has changed
  RDMA/qedr: Fix iWARP active mtu display
  RDMA/qedr: Fix inline size returned for iWARP
  RDMA/qedr: Fix function prototype parameters alignment

 drivers/infiniband/hw/qedr/main.c               |  9 ++++++++-
 drivers/infiniband/hw/qedr/qedr_iw_cm.c         |  6 ++++--
 drivers/infiniband/hw/qedr/verbs.c              | 13 ++++++++-----
 drivers/net/ethernet/qlogic/qede/qede_ethtool.c |  4 +++-
 drivers/net/ethernet/qlogic/qede/qede_rdma.c    | 17 +++++++++++++++++
 include/linux/qed/qede_rdma.h                   |  4 +++-
 6 files changed, 43 insertions(+), 10 deletions(-)

-- 
2.14.5

[PATCH rdma-next 1/8] RDMA/qedr: Fix qp structure memory leak

[Michal Kalderon]

The qedr_qp structure wasn't freed when the protocol was RoCE.
kmemleak output when running basic RoCE scenario.

unreferenced object 0xffff927ad7e22c00 (size 1024):
  comm "ib_send_bw", pid 7082, jiffies 4384133693 (age 274.698s)
  hex dump (first 32 bytes):
    00 b0 cd a2 79 92 ff ff 00 3f a1 a2 79 92 ff ff  ....y....?..y...
    00 ee 5c dd 80 92 ff ff 00 f6 5c dd 80 92 ff ff  ..\.......\.....
  backtrace:
    [<00000000b2ba0f35>] qedr_create_qp+0xb3/0x6c0 [qedr]
    [<00000000e85a43dd>] ib_uverbs_handler_UVERBS_METHOD_QP_CREATE+0x555/0xad0 [ib_uverbs]
    [<00000000fee4d029>] ib_uverbs_cmd_verbs+0xa5a/0xb80 [ib_uverbs]
    [<000000005d622660>] ib_uverbs_ioctl+0xa4/0x110 [ib_uverbs]
    [<00000000eb4cdc71>] ksys_ioctl+0x87/0xc0
    [<00000000abe6b23a>] __x64_sys_ioctl+0x16/0x20
    [<0000000046e7cef4>] do_syscall_64+0x4d/0x90
    [<00000000c6948f76>] entry_SYSCALL_64_after_hwframe+0x44/0xa9

Fixes: 1212767e23bb ("qedr: Add wrapping generic structure for qpidr and
adjust idr routines")
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/verbs.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index b49bef94637e..e35f2a20bfdf 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -2752,6 +2752,8 @@ int qedr_destroy_qp(struct ib_qp *ibqp, struct ib_udata *udata)
 
 	if (rdma_protocol_iwarp(&dev->ibdev, 1))
 		qedr_iw_qp_rem_ref(&qp->ibqp);
+	else
+		kfree(qp);
 
 	return 0;
 }
-- 
2.14.5

[PATCH rdma-next 2/8] RDMA/qedr: Fix doorbell setting

[Michal Kalderon]

Change the doorbell setting so that the maximum value between
the last and current value is set. This is to avoid doorbells
being lost.

Fixes: a7efd7773e31 ("qedr: Add support for PD,PKEY and CQ verbs")
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/verbs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index e35f2a20bfdf..62fe9fc40f42 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -999,7 +999,7 @@ int qedr_create_cq(struct ib_cq *ibcq, const struct ib_cq_init_attr *attr,
 		/* Generate doorbell address. */
 		cq->db.data.icid = cq->icid;
 		cq->db_addr = dev->db_addr + db_offset;
-		cq->db.data.params = DB_AGG_CMD_SET <<
+		cq->db.data.params = DB_AGG_CMD_MAX <<
 		    RDMA_PWM_VAL32_DATA_AGG_CMD_SHIFT;
 
 		/* point to the very last element, passing it we will toggle */
-- 
2.14.5

[PATCH rdma-next 3/8] RDMA/qedr: Fix use of uninitialized field

[Michal Kalderon]

dev->attr.page_size_caps was used uninitialized
when setting device attributes

Fixes: ec72fce401c6 ("qedr: Add support for RoCE HW init")
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/main.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/qedr/main.c b/drivers/infiniband/hw/qedr/main.c
index d85f992bac29..8e1365951fb6 100644
--- a/drivers/infiniband/hw/qedr/main.c
+++ b/drivers/infiniband/hw/qedr/main.c
@@ -602,7 +602,7 @@ static int qedr_set_device_attr(struct qedr_dev *dev)
 	qed_attr = dev->ops->rdma_query_device(dev->rdma_ctx);
 
 	/* Part 2 - check capabilities */
-	page_size = ~dev->attr.page_size_caps + 1;
+	page_size = ~qed_attr->page_size_caps + 1;
 	if (page_size > PAGE_SIZE) {
 		DP_ERR(dev,
 		       "Kernel PAGE_SIZE is %ld which is smaller than minimum page size (%d) required by qedr\n",
-- 
2.14.5

[PATCH rdma-next 4/8] RDMA/qedr: Fix return code if accept is called on a destroyed qp

[Michal Kalderon]

In iWARP, accept could be called after a QP is already destroyed.
In this case an error should be returned and not success.

Fixes: 82af6d19d8d9 ("RDMA/qedr: Fix synchronization methods and memory leaks in qedr")
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/qedr_iw_cm.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/qedr/qedr_iw_cm.c b/drivers/infiniband/hw/qedr/qedr_iw_cm.c
index 97fc7dd353b0..c7169d2c69e5 100644
--- a/drivers/infiniband/hw/qedr/qedr_iw_cm.c
+++ b/drivers/infiniband/hw/qedr/qedr_iw_cm.c
@@ -736,7 +736,7 @@ int qedr_iw_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param)
 	struct qedr_dev *dev = ep->dev;
 	struct qedr_qp *qp;
 	struct qed_iwarp_accept_in params;
-	int rc = 0;
+	int rc;
 
 	DP_DEBUG(dev, QEDR_MSG_IWARP, "Accept on qpid=%d\n", conn_param->qpn);
 
@@ -759,8 +759,10 @@ int qedr_iw_accept(struct iw_cm_id *cm_id, struct iw_cm_conn_param *conn_param)
 	params.ord = conn_param->ord;
 
 	if (test_and_set_bit(QEDR_IWARP_CM_WAIT_FOR_CONNECT,
-			     &qp->iwarp_cm_flags))
+			     &qp->iwarp_cm_flags)) {
+		rc = -EINVAL;
 		goto err; /* QP already destroyed */
+	}
 
 	rc = dev->ops->iwarp_accept(dev->rdma_ctx, &params);
 	if (rc) {
-- 
2.14.5

[PATCH rdma-next 5/8] qede: Notify qedr when mtu has changed

[Michal Kalderon]

MTU change on ethtool is currently not supported for iWARP.
Notify qedr driver so that appropriate logging can take
place.

Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/net/ethernet/qlogic/qede/qede_ethtool.c |  4 +++-
 drivers/net/ethernet/qlogic/qede/qede_rdma.c    | 17 +++++++++++++++++
 include/linux/qed/qede_rdma.h                   |  4 +++-
 3 files changed, 23 insertions(+), 2 deletions(-)

diff --git a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
index b9aa6384563b..bedbb85a179a 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_ethtool.c
@@ -1026,7 +1026,9 @@ int qede_change_mtu(struct net_device *ndev, int new_mtu)
 	args.u.mtu = new_mtu;
 	args.func = &qede_update_mtu;
 	qede_reload(edev, &args, false);
-
+#if IS_ENABLED(CONFIG_QED_RDMA)
+	qede_rdma_event_change_mtu(edev);
+#endif
 	edev->ops->common->update_mtu(edev->cdev, new_mtu);
 
 	return 0;
diff --git a/drivers/net/ethernet/qlogic/qede/qede_rdma.c b/drivers/net/ethernet/qlogic/qede/qede_rdma.c
index 769ec2f4d0b7..2f6598086d9b 100644
--- a/drivers/net/ethernet/qlogic/qede/qede_rdma.c
+++ b/drivers/net/ethernet/qlogic/qede/qede_rdma.c
@@ -234,6 +234,15 @@ static void qede_rdma_changeaddr(struct qede_dev *edev)
 		qedr_drv->notify(edev->rdma_info.qedr_dev, QEDE_CHANGE_ADDR);
 }
 
+static void qede_rdma_change_mtu(struct qede_dev *edev)
+{
+	if (qede_rdma_supported(edev)) {
+		if (qedr_drv && edev->rdma_info.qedr_dev && qedr_drv->notify)
+			qedr_drv->notify(edev->rdma_info.qedr_dev,
+					 QEDE_CHANGE_MTU);
+	}
+}
+
 static struct qede_rdma_event_work *
 qede_rdma_get_free_event_node(struct qede_dev *edev)
 {
@@ -287,6 +296,9 @@ static void qede_rdma_handle_event(struct work_struct *work)
 	case QEDE_CHANGE_ADDR:
 		qede_rdma_changeaddr(edev);
 		break;
+	case QEDE_CHANGE_MTU:
+		qede_rdma_change_mtu(edev);
+		break;
 	default:
 		DP_NOTICE(edev, "Invalid rdma event %d", event);
 	}
@@ -338,3 +350,8 @@ void qede_rdma_event_changeaddr(struct qede_dev *edev)
 {
 	qede_rdma_add_event(edev, QEDE_CHANGE_ADDR);
 }
+
+void qede_rdma_event_change_mtu(struct qede_dev *edev)
+{
+	qede_rdma_add_event(edev, QEDE_CHANGE_MTU);
+}
diff --git a/include/linux/qed/qede_rdma.h b/include/linux/qed/qede_rdma.h
index 072da2f6da37..0d5564a59a59 100644
--- a/include/linux/qed/qede_rdma.h
+++ b/include/linux/qed/qede_rdma.h
@@ -20,7 +20,8 @@ enum qede_rdma_event {
 	QEDE_UP,
 	QEDE_DOWN,
 	QEDE_CHANGE_ADDR,
-	QEDE_CLOSE
+	QEDE_CLOSE,
+	QEDE_CHANGE_MTU,
 };
 
 struct qede_rdma_event_work {
@@ -54,6 +55,7 @@ void qede_rdma_dev_event_open(struct qede_dev *dev);
 void qede_rdma_dev_event_close(struct qede_dev *dev);
 void qede_rdma_dev_remove(struct qede_dev *dev, bool recovery);
 void qede_rdma_event_changeaddr(struct qede_dev *edr);
+void qede_rdma_event_change_mtu(struct qede_dev *edev);
 
 #else
 static inline int qede_rdma_dev_add(struct qede_dev *dev,
-- 
2.14.5

[PATCH rdma-next 6/8] RDMA/qedr: Fix iWARP active mtu display

[Michal Kalderon]

Currently iWARP does not support mtu-change. Notify user when
MTU changes that reload of qedr is required for mtu to change.
Display the correct active mtu.

Fixes: f5b1b1775be6 ("RDMA/qedr: Add iWARP support in existing verbs")
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/main.c  | 7 +++++++
 drivers/infiniband/hw/qedr/verbs.c | 3 ++-
 2 files changed, 9 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/qedr/main.c b/drivers/infiniband/hw/qedr/main.c
index 8e1365951fb6..1f32e8b8cc11 100644
--- a/drivers/infiniband/hw/qedr/main.c
+++ b/drivers/infiniband/hw/qedr/main.c
@@ -1026,6 +1026,13 @@ static void qedr_notify(struct qedr_dev *dev, enum qede_rdma_event event)
 	case QEDE_CHANGE_ADDR:
 		qedr_mac_address_change(dev);
 		break;
+	case QEDE_CHANGE_MTU:
+		if (rdma_protocol_iwarp(&dev->ibdev, 1))
+			if (dev->ndev->mtu != dev->iwarp_max_mtu)
+				DP_NOTICE(dev,
+					  "Mtu was changed from %d to %d. This will not take affect for iWARP until qedr is reloaded\n",
+					  dev->iwarp_max_mtu, dev->ndev->mtu);
+		break;
 	default:
 		pr_err("Event not supported\n");
 	}
diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index 62fe9fc40f42..4ec2d05f63d1 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -231,15 +231,16 @@ int qedr_query_port(struct ib_device *ibdev, u8 port, struct ib_port_attr *attr)
 		attr->phys_state = IB_PORT_PHYS_STATE_DISABLED;
 	}
 	attr->max_mtu = IB_MTU_4096;
-	attr->active_mtu = iboe_get_mtu(dev->ndev->mtu);
 	attr->lid = 0;
 	attr->lmc = 0;
 	attr->sm_lid = 0;
 	attr->sm_sl = 0;
 	attr->ip_gids = true;
 	if (rdma_protocol_iwarp(&dev->ibdev, 1)) {
+		attr->active_mtu = iboe_get_mtu(dev->iwarp_max_mtu);
 		attr->gid_tbl_len = 1;
 	} else {
+		attr->active_mtu = iboe_get_mtu(dev->ndev->mtu);
 		attr->gid_tbl_len = QEDR_MAX_SGID;
 		attr->pkey_tbl_len = QEDR_ROCE_PKEY_TABLE_LEN;
 	}
-- 
2.14.5

[PATCH rdma-next 7/8] RDMA/qedr: Fix inline size returned for iWARP

[Michal Kalderon]

commit 59e8970b3798 ("RDMA/qedr: Return max inline data in QP query
result")
changed query_qp max_inline size to return the max roce inline size.
When iwarp was introduced, this should have been modified
to return the max inline size based on protocol.
This size is cached in the device attributes

Fixes: 69ad0e7fe845 ("RDMA/qedr: Add support for iWARP in user space")
Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/verbs.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index 4ec2d05f63d1..3b19afdf71b8 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -2637,7 +2637,7 @@ int qedr_query_qp(struct ib_qp *ibqp,
 	qp_attr->cap.max_recv_wr = qp->rq.max_wr;
 	qp_attr->cap.max_send_sge = qp->sq.max_sges;
 	qp_attr->cap.max_recv_sge = qp->rq.max_sges;
-	qp_attr->cap.max_inline_data = ROCE_REQ_MAX_INLINE_DATA_SIZE;
+	qp_attr->cap.max_inline_data = dev->attr.max_inline;
 	qp_init_attr->cap = qp_attr->cap;
 
 	qp_attr->ah_attr.type = RDMA_AH_ATTR_TYPE_ROCE;
-- 
2.14.5

[PATCH rdma-next 8/8] RDMA/qedr: Fix function prototype parameters alignment

[Michal Kalderon]

Alignment of parameters was off by one

Signed-off-by: Michal Kalderon <michal.kalderon@marvell.com>
---
 drivers/infiniband/hw/qedr/verbs.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/hw/qedr/verbs.c b/drivers/infiniband/hw/qedr/verbs.c
index 3b19afdf71b8..0227540838ee 100644
--- a/drivers/infiniband/hw/qedr/verbs.c
+++ b/drivers/infiniband/hw/qedr/verbs.c
@@ -1249,8 +1249,8 @@ static int qedr_copy_srq_uresp(struct qedr_dev *dev,
 }
 
 static void qedr_copy_rq_uresp(struct qedr_dev *dev,
-			      struct qedr_create_qp_uresp *uresp,
-			      struct qedr_qp *qp)
+			       struct qedr_create_qp_uresp *uresp,
+			       struct qedr_qp *qp)
 {
 	/* iWARP requires two doorbells per RQ. */
 	if (rdma_protocol_iwarp(&dev->ibdev, 1)) {
-- 
2.14.5

Re: [PATCH rdma-next 0/8] RDMA/qedr: various fixes

[Jason Gunthorpe]

On Wed, Sep 02, 2020 at 07:57:33PM +0300, Michal Kalderon wrote:
> This set addresses several issues that were observed
> and reproduced on different test and production configurations.
> 
> Dave, Jason,
> There is one qede patch which is related to the mtu change notify.
> This is a small change and required for the qede_rdma.h interface
> change. Please consider applying this to rdma tree.

Ok, it is up to Marvell to ensure no conflicting patches land in net..

> Michal Kalderon (8):
>   RDMA/qedr: Fix qp structure memory leak
>   RDMA/qedr: Fix doorbell setting
>   RDMA/qedr: Fix use of uninitialized field
>   RDMA/qedr: Fix return code if accept is called on a destroyed qp
>   qede: Notify qedr when mtu has changed
>   RDMA/qedr: Fix iWARP active mtu display
>   RDMA/qedr: Fix inline size returned for iWARP
>   RDMA/qedr: Fix function prototype parameters alignment

Applied to rdma for-next, thanks

Jason

RE: [EXT] Re: [PATCH rdma-next 0/8] RDMA/qedr: various fixes

[Michal Kalderon]

> From: Jason Gunthorpe <jgg@nvidia.com>
> Sent: Wednesday, September 9, 2020 10:15 PM
> On Wed, Sep 02, 2020 at 07:57:33PM +0300, Michal Kalderon wrote:
> > This set addresses several issues that were observed and reproduced on
> > different test and production configurations.
> >
> > Dave, Jason,
> > There is one qede patch which is related to the mtu change notify.
> > This is a small change and required for the qede_rdma.h interface
> > change. Please consider applying this to rdma tree.
> 
> Ok, it is up to Marvell to ensure no conflicting patches land in net..
Sure, thanks.

> 
> > Michal Kalderon (8):
> >   RDMA/qedr: Fix qp structure memory leak
> >   RDMA/qedr: Fix doorbell setting
> >   RDMA/qedr: Fix use of uninitialized field
> >   RDMA/qedr: Fix return code if accept is called on a destroyed qp
> >   qede: Notify qedr when mtu has changed
> >   RDMA/qedr: Fix iWARP active mtu display
> >   RDMA/qedr: Fix inline size returned for iWARP
> >   RDMA/qedr: Fix function prototype parameters alignment
> 
> Applied to rdma for-next, thanks
> 
> Jason