[PATCH rdma-rc 0/9] RDMA fixes for 4.8

[PATCH rdma-rc 0/9] RDMA fixes for 4.8

[Leon Romanovsky]

Hi Doug,

This is compilation of fixes to RDMA core, IPoIB, Mellanox mlx4
and mlx5 drivers all together. Most of the fixes are one liners and
for easy handling, I'm sending them in one patchset.

Thanks

Chuck Lever (1):
  IB/mlx5: Return EINVAL when caller specifies too many SGEs

Erez Shitrit (2):
  IB/core: Fix use after free in send_leave function
  IB/ipoib: Fix memory corruption during ipoib cm connection
    establishment

Leon Romanovsky (5):
  Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"
  IB/mlx4: Don't return errors from poll_cq
  IB/mlx5: Simplify code by removing return variable
  IB/mlx5: Add VERBOSITY Kconfig option
  IB/mlx5: Don't return errors from poll_cq

Yishai Hadas (1):
  IB/mlx5: Use TIR number based on selector

 drivers/infiniband/core/multicast.c       | 13 ++-----------
 drivers/infiniband/hw/mlx4/cq.c           | 26 ++------------------------
 drivers/infiniband/hw/mlx5/Kconfig        |  6 ++++++
 drivers/infiniband/hw/mlx5/cq.c           | 22 ++--------------------
 drivers/infiniband/hw/mlx5/main.c         |  6 +++++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h      |  1 +
 drivers/infiniband/hw/mlx5/qp.c           | 15 ++++++---------
 drivers/infiniband/ulp/ipoib/ipoib.h      |  1 +
 drivers/infiniband/ulp/ipoib/ipoib_cm.c   | 16 ++++++++++++++++
 drivers/infiniband/ulp/ipoib/ipoib_main.c |  2 +-
 10 files changed, 42 insertions(+), 66 deletions(-)

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 1/9] IB/core: Fix use after free in send_leave function

[Leon Romanovsky]

From: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

The function send_leave sets the member: group->query_id
(group->query_id = ret) after calling the sa_query, but leave_handler
can be executed before the setting and it might delete the group object,
and will get a memory corruption.

Additionally, this patch gets rid of group->query_id variable which is
not used.

Fixes: faec2f7b96b5 ('IB/sa: Track multicast join/leave requests')
Signed-off-by: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/core/multicast.c | 13 ++-----------
 1 file changed, 2 insertions(+), 11 deletions(-)

diff --git a/drivers/infiniband/core/multicast.c b/drivers/infiniband/core/multicast.c
index 3a3c5d7..51c79b2 100644
--- a/drivers/infiniband/core/multicast.c
+++ b/drivers/infiniband/core/multicast.c
@@ -106,7 +106,6 @@ struct mcast_group {
 	atomic_t		refcount;
 	enum mcast_group_state	state;
 	struct ib_sa_query	*query;
-	int			query_id;
 	u16			pkey_index;
 	u8			leave_state;
 	int			retries;
@@ -340,11 +339,7 @@ static int send_join(struct mcast_group *group, struct mcast_member *member)
 				       member->multicast.comp_mask,
 				       3000, GFP_KERNEL, join_handler, group,
 				       &group->query);
-	if (ret >= 0) {
-		group->query_id = ret;
-		ret = 0;
-	}
-	return ret;
+	return (ret > 0) ? 0 : ret;
 }
 
 static int send_leave(struct mcast_group *group, u8 leave_state)
@@ -364,11 +359,7 @@ static int send_leave(struct mcast_group *group, u8 leave_state)
 				       IB_SA_MCMEMBER_REC_JOIN_STATE,
 				       3000, GFP_KERNEL, leave_handler,
 				       group, &group->query);
-	if (ret >= 0) {
-		group->query_id = ret;
-		ret = 0;
-	}
-	return ret;
+	return (ret > 0) ? 0 : ret;
 }
 
 static void join_group(struct mcast_group *group, struct mcast_member *member,
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 2/9] IB/ipoib: Fix memory corruption during ipoib cm connection establishment

[Leon Romanovsky]

From: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

When a new CM connection is being requested, ipoib driver copies data
from the path pointer in the CM/tx object, the path object might be
invalid at the point and memory corruption will happened later when now
the CM driver will try using that data.

The next scenario demonstrates it:
	neigh_add_path --> ipoib_cm_create_tx -->
	queue_work (pointer to path is in the cm/tx struct)
	#while the work is still in the queue,
	#the port goes down and causes the ipoib_flush_paths:
	ipoib_flush_paths --> path_free --> kfree(path)
	#at this point the work scheduled starts.
	ipoib_cm_tx_start --> copy from the (invalid)path pointer:
	(memcpy(&pathrec, &p->path->pathrec, sizeof pathrec);)
	 -> memory corruption.

To fix that the driver now starts the CM/tx connection only if that
specific path exists in the general paths database.
This check is protected with the relevant locks, and uses the gid from
the neigh member in the CM/tx object which is valid according to the ref
count that was taken by the CM/tx.

Fixes: 839fcaba35 ('IPoIB: Connected mode experimental support')
Signed-off-by: Erez Shitrit <erezsh-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/ulp/ipoib/ipoib.h      |  1 +
 drivers/infiniband/ulp/ipoib/ipoib_cm.c   | 16 ++++++++++++++++
 drivers/infiniband/ulp/ipoib/ipoib_main.c |  2 +-
 3 files changed, 18 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h
index 4f7d9b4..9dbfcc0 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -478,6 +478,7 @@ void ipoib_send(struct net_device *dev, struct sk_buff *skb,
 		struct ipoib_ah *address, u32 qpn);
 void ipoib_reap_ah(struct work_struct *work);
 
+struct ipoib_path *__path_find(struct net_device *dev, void *gid);
 void ipoib_mark_paths_invalid(struct net_device *dev);
 void ipoib_flush_paths(struct net_device *dev);
 int ipoib_check_sm_sendonly_fullmember_support(struct ipoib_dev_priv *priv);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 951d9ab..4ad297d 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -1318,6 +1318,8 @@ void ipoib_cm_destroy_tx(struct ipoib_cm_tx *tx)
 	}
 }
 
+#define QPN_AND_OPTIONS_OFFSET	4
+
 static void ipoib_cm_tx_start(struct work_struct *work)
 {
 	struct ipoib_dev_priv *priv = container_of(work, struct ipoib_dev_priv,
@@ -1326,6 +1328,7 @@ static void ipoib_cm_tx_start(struct work_struct *work)
 	struct ipoib_neigh *neigh;
 	struct ipoib_cm_tx *p;
 	unsigned long flags;
+	struct ipoib_path *path;
 	int ret;
 
 	struct ib_sa_path_rec pathrec;
@@ -1338,7 +1341,19 @@ static void ipoib_cm_tx_start(struct work_struct *work)
 		p = list_entry(priv->cm.start_list.next, typeof(*p), list);
 		list_del_init(&p->list);
 		neigh = p->neigh;
+
 		qpn = IPOIB_QPN(neigh->daddr);
+		/*
+		 * As long as the search is with these 2 locks,
+		 * path existence indicates its validity.
+		 */
+		path = __path_find(dev, neigh->daddr + QPN_AND_OPTIONS_OFFSET);
+		if (!path) {
+			pr_info("%s ignore not valid path %pI6\n",
+				__func__,
+				neigh->daddr + QPN_AND_OPTIONS_OFFSET);
+			goto free_neigh;
+		}
 		memcpy(&pathrec, &p->path->pathrec, sizeof pathrec);
 
 		spin_unlock_irqrestore(&priv->lock, flags);
@@ -1350,6 +1365,7 @@ static void ipoib_cm_tx_start(struct work_struct *work)
 		spin_lock_irqsave(&priv->lock, flags);
 
 		if (ret) {
+free_neigh:
 			neigh = p->neigh;
 			if (neigh) {
 				neigh->cm = NULL;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 74bcaa0..cc1c1b0 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -485,7 +485,7 @@ int ipoib_set_mode(struct net_device *dev, const char *buf)
 	return -EINVAL;
 }
 
-static struct ipoib_path *__path_find(struct net_device *dev, void *gid)
+struct ipoib_path *__path_find(struct net_device *dev, void *gid)
 {
 	struct ipoib_dev_priv *priv = netdev_priv(dev);
 	struct rb_node *n = priv->path_tree.rb_node;
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

By Mellanox HW design and SW implementation poll_cq never
fails and returns errors, so all these prints are to catch ULP bugs.

In case of such bug, the reverted patch patch will cause to reentry
(EAGAIN) and kprints storm again and again.

It is undesired and misleading behaviour.

This reverts commit 5412352fcd8f ("IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one")

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx4/cq.c | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)

diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 006db64..15b6289 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -690,7 +690,7 @@ repoll:
 	if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_OPCODE_NOP &&
 		     is_send)) {
 		pr_warn("Completion for NOP opcode detected!\n");
-		return -EAGAIN;
+		return -EINVAL;
 	}

 	/* Resize CQ in progress */
@@ -721,7 +721,7 @@ repoll:
 		if (unlikely(!mqp)) {
 			pr_warn("CQ %06x with entry for unknown QPN %06x\n",
 			       cq->mcq.cqn, be32_to_cpu(cqe->vlan_my_qpn) & MLX4_CQE_QPN_MASK);
-			return -EAGAIN;
+			return -EINVAL;
 		}

 		*cur_qp = to_mibqp(mqp);
@@ -739,7 +739,7 @@ repoll:
 		if (unlikely(!msrq)) {
 			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
 				cq->mcq.cqn, srq_num);
-			return -EAGAIN;
+			return -EINVAL;
 		}
 	}

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 4/9] IB/mlx4: Don't return errors from poll_cq

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Remove returning errors from mlx4 poll_cq function. Polling CQ
operation in kernel never fails by Mellanox HCA architecture and
respective driver design.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx4/cq.c | 26 ++------------------------
 1 file changed, 2 insertions(+), 24 deletions(-)

diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
index 15b6289..5df63da 100644
--- a/drivers/infiniband/hw/mlx4/cq.c
+++ b/drivers/infiniband/hw/mlx4/cq.c
@@ -687,12 +687,6 @@ repoll:
 	is_error = (cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) ==
 		MLX4_CQE_OPCODE_ERROR;

-	if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_OPCODE_NOP &&
-		     is_send)) {
-		pr_warn("Completion for NOP opcode detected!\n");
-		return -EINVAL;
-	}
-
 	/* Resize CQ in progress */
 	if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_CQE_OPCODE_RESIZE)) {
 		if (cq->resize_buf) {
@@ -718,12 +712,6 @@ repoll:
 		 */
 		mqp = __mlx4_qp_lookup(to_mdev(cq->ibcq.device)->dev,
 				       be32_to_cpu(cqe->vlan_my_qpn));
-		if (unlikely(!mqp)) {
-			pr_warn("CQ %06x with entry for unknown QPN %06x\n",
-			       cq->mcq.cqn, be32_to_cpu(cqe->vlan_my_qpn) & MLX4_CQE_QPN_MASK);
-			return -EINVAL;
-		}
-
 		*cur_qp = to_mibqp(mqp);
 	}

@@ -736,11 +724,6 @@ repoll:
 		/* SRQ is also in the radix tree */
 		msrq = mlx4_srq_lookup(to_mdev(cq->ibcq.device)->dev,
 				       srq_num);
-		if (unlikely(!msrq)) {
-			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
-				cq->mcq.cqn, srq_num);
-			return -EINVAL;
-		}
 	}

 	if (is_send) {
@@ -891,7 +874,6 @@ int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 	struct mlx4_ib_qp *cur_qp = NULL;
 	unsigned long flags;
 	int npolled;
-	int err = 0;
 	struct mlx4_ib_dev *mdev = to_mdev(cq->ibcq.device);

 	spin_lock_irqsave(&cq->lock, flags);
@@ -901,8 +883,7 @@ int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 	}

 	for (npolled = 0; npolled < num_entries; ++npolled) {
-		err = mlx4_ib_poll_one(cq, &cur_qp, wc + npolled);
-		if (err)
+		if (mlx4_ib_poll_one(cq, &cur_qp, wc + npolled))
 			break;
 	}

@@ -911,10 +892,7 @@ int mlx4_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 out:
 	spin_unlock_irqrestore(&cq->lock, flags);

-	if (err == 0 || err == -EAGAIN)
-		return npolled;
-	else
-		return err;
+	return npolled;
 }

 int mlx4_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 5/9] IB/mlx5: Return EINVAL when caller specifies too many SGEs

[Leon Romanovsky]

From: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>

The returned value should be EINVAL, because it is caused by wrong
caller and not by internal overflow event.

Signed-off-by: Chuck Lever <chuck.lever-QHcLZuEGTsvQT0dZR+AlfA@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/qp.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 0dd7d93..acb3b72 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -3758,7 +3758,7 @@ int mlx5_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
 		num_sge = wr->num_sge;
 		if (unlikely(num_sge > qp->sq.max_gs)) {
 			mlx5_ib_warn(dev, "\n");
-			err = -ENOMEM;
+			err = -EINVAL;
 			*bad_wr = wr;
 			goto out;
 		}
--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 6/9] IB/mlx5: Simplify code by removing return variable

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Return variable was set in a line before the
actual return was called in begin_wqe function.

This patch removes such variable and simplifies the code.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/qp.c | 10 +++-------
 1 file changed, 3 insertions(+), 7 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index acb3b72..174d09b 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -3658,12 +3658,8 @@ static int begin_wqe(struct mlx5_ib_qp *qp, void **seg,
 		     struct ib_send_wr *wr, unsigned *idx,
 		     int *size, int nreq)
 {
-	int err = 0;
-
-	if (unlikely(mlx5_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq))) {
-		err = -ENOMEM;
-		return err;
-	}
+	if (unlikely(mlx5_wq_overflow(&qp->sq, nreq, qp->ibqp.send_cq)))
+		return -ENOMEM;

 	*idx = qp->sq.cur_post & (qp->sq.wqe_cnt - 1);
 	*seg = mlx5_get_send_wqe(qp, *idx);
@@ -3679,7 +3675,7 @@ static int begin_wqe(struct mlx5_ib_qp *qp, void **seg,
 	*seg += sizeof(**ctrl);
 	*size = sizeof(**ctrl) / 16;

-	return err;
+	return 0;
 }

 static void finish_wqe(struct mlx5_ib_qp *qp,
--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 7/9] IB/mlx5: Add VERBOSITY Kconfig option

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

This patch introduces new MLX5_INFINIBAND_VERBOSE Kconfig
option to enable the output of dump_wqe() which was never
printed.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/Kconfig | 6 ++++++
 drivers/infiniband/hw/mlx5/qp.c    | 2 +-
 2 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/Kconfig b/drivers/infiniband/hw/mlx5/Kconfig
index bce263b..789fef4 100644
--- a/drivers/infiniband/hw/mlx5/Kconfig
+++ b/drivers/infiniband/hw/mlx5/Kconfig
@@ -6,3 +6,9 @@ config MLX5_INFINIBAND
 	  Mellanox Connect-IB PCI Express host channel adapters (HCAs).
 	  This is required to use InfiniBand protocols such as
 	  IP-over-IB or SRP with these devices.
+
+config MLX5_INFINIBAND_VERBOSE
+	bool "Enable ConnectX-4/Connect-IB verbosity"
+	depends on MLX5_INFINIBAND
+	---help---
+	  This is a configuration option to enable verbose output to easy debug.
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 174d09b..67b58f7 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -3984,7 +3984,7 @@ int mlx5_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
 			   get_fence(fence, wr), next_fence,
 			   mlx5_ib_opcode[wr->opcode]);
 skip_psv:
-		if (0)
+		if (IS_BUILTIN(CONFIG_MLX5_INFINIBAND_VERBOSE))
 			dump_wqe(qp, idx, size);
 	}

--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 8/9] IB/mlx5: Use TIR number based on selector

[Leon Romanovsky]

From: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Use TIR number based on selector, it should be done to differentiate
between RSS QP to RAW one.

Reported-by: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
Signed-off-by: Yishai Hadas <yishaih-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Signed-off-by: Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/main.c    | 6 +++++-
 drivers/infiniband/hw/mlx5/mlx5_ib.h | 1 +
 drivers/infiniband/hw/mlx5/qp.c      | 1 +
 3 files changed, 7 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/hw/mlx5/main.c b/drivers/infiniband/hw/mlx5/main.c
index 1b4094b..8150ea3 100644
--- a/drivers/infiniband/hw/mlx5/main.c
+++ b/drivers/infiniband/hw/mlx5/main.c
@@ -1849,6 +1849,7 @@ static struct ib_flow *mlx5_ib_create_flow(struct ib_qp *qp,
 					   int domain)
 {
 	struct mlx5_ib_dev *dev = to_mdev(qp->device);
+	struct mlx5_ib_qp *mqp = to_mqp(qp);
 	struct mlx5_ib_flow_handler *handler = NULL;
 	struct mlx5_flow_destination *dst = NULL;
 	struct mlx5_ib_flow_prio *ft_prio;
@@ -1875,7 +1876,10 @@ static struct ib_flow *mlx5_ib_create_flow(struct ib_qp *qp,
 	}
 
 	dst->type = MLX5_FLOW_DESTINATION_TYPE_TIR;
-	dst->tir_num = to_mqp(qp)->raw_packet_qp.rq.tirn;
+	if (mqp->flags & MLX5_IB_QP_RSS)
+		dst->tir_num = mqp->rss_qp.tirn;
+	else
+		dst->tir_num = mqp->raw_packet_qp.rq.tirn;
 
 	if (flow_attr->type == IB_FLOW_ATTR_NORMAL) {
 		if (flow_attr->flags & IB_FLOW_ATTR_FLAGS_DONT_TRAP)  {
diff --git a/drivers/infiniband/hw/mlx5/mlx5_ib.h b/drivers/infiniband/hw/mlx5/mlx5_ib.h
index 372385d..95146f4 100644
--- a/drivers/infiniband/hw/mlx5/mlx5_ib.h
+++ b/drivers/infiniband/hw/mlx5/mlx5_ib.h
@@ -402,6 +402,7 @@ enum mlx5_ib_qp_flags {
 	/* QP uses 1 as its source QP number */
 	MLX5_IB_QP_SQPN_QP1			= 1 << 6,
 	MLX5_IB_QP_CAP_SCATTER_FCS		= 1 << 7,
+	MLX5_IB_QP_RSS				= 1 << 8,
 };
 
 struct mlx5_umr_wr {
diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
index 67b58f7..fc5030f 100644
--- a/drivers/infiniband/hw/mlx5/qp.c
+++ b/drivers/infiniband/hw/mlx5/qp.c
@@ -1449,6 +1449,7 @@ create_tir:
 	kvfree(in);
 	/* qpn is reserved for that QP */
 	qp->trans_qp.base.mqp.qpn = 0;
+	qp->flags |= MLX5_IB_QP_RSS;
 	return 0;
 
 err:
-- 
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH rdma-rc 9/9] IB/mlx5: Don't return errors from poll_cq

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Remove returning errors from mlx5 poll_cq function. Polling CQ
operation in kernel never fails by Mellanox HCA architecture and
respective driver design.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/infiniband/hw/mlx5/cq.c | 22 ++--------------------
 1 file changed, 2 insertions(+), 20 deletions(-)

diff --git a/drivers/infiniband/hw/mlx5/cq.c b/drivers/infiniband/hw/mlx5/cq.c
index 308a358..e4fac929 100644
--- a/drivers/infiniband/hw/mlx5/cq.c
+++ b/drivers/infiniband/hw/mlx5/cq.c
@@ -553,12 +553,6 @@ repoll:
 		 * from the table.
 		 */
 		mqp = __mlx5_qp_lookup(dev->mdev, qpn);
-		if (unlikely(!mqp)) {
-			mlx5_ib_warn(dev, "CQE@CQ %06x for unknown QPN %6x\n",
-				     cq->mcq.cqn, qpn);
-			return -EINVAL;
-		}
-
 		*cur_qp = to_mibqp(mqp);
 	}

@@ -619,13 +613,6 @@ repoll:
 		read_lock(&dev->mdev->priv.mkey_table.lock);
 		mmkey = __mlx5_mr_lookup(dev->mdev,
 					 mlx5_base_mkey(be32_to_cpu(sig_err_cqe->mkey)));
-		if (unlikely(!mmkey)) {
-			read_unlock(&dev->mdev->priv.mkey_table.lock);
-			mlx5_ib_warn(dev, "CQE@CQ %06x for unknown MR %6x\n",
-				     cq->mcq.cqn, be32_to_cpu(sig_err_cqe->mkey));
-			return -EINVAL;
-		}
-
 		mr = to_mibmr(mmkey);
 		get_sig_err_item(sig_err_cqe, &mr->sig->err_item);
 		mr->sig->sig_err_exists = true;
@@ -676,7 +663,6 @@ int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 	unsigned long flags;
 	int soft_polled = 0;
 	int npolled;
-	int err = 0;

 	spin_lock_irqsave(&cq->lock, flags);
 	if (mdev->state == MLX5_DEVICE_STATE_INTERNAL_ERROR) {
@@ -688,8 +674,7 @@ int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 		soft_polled = poll_soft_wc(cq, num_entries, wc);

 	for (npolled = 0; npolled < num_entries - soft_polled; npolled++) {
-		err = mlx5_poll_one(cq, &cur_qp, wc + soft_polled + npolled);
-		if (err)
+		if (mlx5_poll_one(cq, &cur_qp, wc + soft_polled + npolled))
 			break;
 	}

@@ -698,10 +683,7 @@ int mlx5_ib_poll_cq(struct ib_cq *ibcq, int num_entries, struct ib_wc *wc)
 out:
 	spin_unlock_irqrestore(&cq->lock, flags);

-	if (err == 0 || err == -EAGAIN)
-		return soft_polled + npolled;
-	else
-		return err;
+	return soft_polled + npolled;
 }

 int mlx5_ib_arm_cq(struct ib_cq *ibcq, enum ib_cq_notify_flags flags)
--
2.7.4

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Yuval Shaia]

On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> By Mellanox HW design and SW implementation poll_cq never
> fails and returns errors, so all these prints are to catch ULP bugs.
> 
> In case of such bug, the reverted patch patch will cause to reentry

s/patch patch/atch

> (EAGAIN) and kprints storm again and again.
> 
> It is undesired and misleading behaviour.
> 
> This reverts commit 5412352fcd8f ("IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one")
> 
> Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/infiniband/hw/mlx4/cq.c | 6 +++---
>  1 file changed, 3 insertions(+), 3 deletions(-)
> 
> diff --git a/drivers/infiniband/hw/mlx4/cq.c b/drivers/infiniband/hw/mlx4/cq.c
> index 006db64..15b6289 100644
> --- a/drivers/infiniband/hw/mlx4/cq.c
> +++ b/drivers/infiniband/hw/mlx4/cq.c
> @@ -690,7 +690,7 @@ repoll:
>  	if (unlikely((cqe->owner_sr_opcode & MLX4_CQE_OPCODE_MASK) == MLX4_OPCODE_NOP &&
>  		     is_send)) {
>  		pr_warn("Completion for NOP opcode detected!\n");
> -		return -EAGAIN;
> +		return -EINVAL;
>  	}
> 
>  	/* Resize CQ in progress */
> @@ -721,7 +721,7 @@ repoll:
>  		if (unlikely(!mqp)) {
>  			pr_warn("CQ %06x with entry for unknown QPN %06x\n",
>  			       cq->mcq.cqn, be32_to_cpu(cqe->vlan_my_qpn) & MLX4_CQE_QPN_MASK);
> -			return -EAGAIN;
> +			return -EINVAL;
>  		}
> 
>  		*cur_qp = to_mibqp(mqp);
> @@ -739,7 +739,7 @@ repoll:
>  		if (unlikely(!msrq)) {
>  			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
>  				cq->mcq.cqn, srq_num);
> -			return -EAGAIN;
> +			return -EINVAL;
>  		}
>  	}
> 
> --
> 2.7.4
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 485 bytes --]

On Sun, Aug 28, 2016 at 11:09:55AM +0300, Yuval Shaia wrote:
> On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >
> > By Mellanox HW design and SW implementation poll_cq never
> > fails and returns errors, so all these prints are to catch ULP bugs.
> >
> > In case of such bug, the reverted patch patch will cause to reentry
>
> s/patch patch/atch

Doug,
Do you want me to respin it?

Thanks

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 8/9] IB/mlx5: Use TIR number based on selector

[Sagi Grimberg]

This does fix my problem ;)

Tested-by: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>

Thanks Yishai!
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 4/9] IB/mlx4: Don't return errors from poll_cq

[Sagi Grimberg]

>  		/* SRQ is also in the radix tree */
>  		msrq = mlx4_srq_lookup(to_mdev(cq->ibcq.device)->dev,
>  				       srq_num);
> -		if (unlikely(!msrq)) {
> -			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
> -				cq->mcq.cqn, srq_num);
> -			return -EINVAL;
> -		}
>  	}

BTW, this is completely unrelated to this patch, but the current
implementation of shared receive queues in Mellanox drivers is
*very* inefficient. Each completion that relates to a srq the
mlx4/mlx5 drivers perform a device-wide locked srq lookup which
is pretty bad... (it kills consumers that want to use more
then one SRQ)

Other drivers that support kernel SRQs that I've looked at are ocrdma 
and i40iw and they don't seem to have this lock everything approach.

At the very-least we should try to make it a rcu + percpu_ref
instead of a killer device-wide lock. It'd be even better if
we use refcounting in the IB core and have the drivers not worry
about the kernel consumers destroying SRQs while processing IO
(i.e. when all the related QPs and CQs are destroyed).

I have some code in the works on this but it's not high on my todo
list at the moment. Mellanox folks, any thoughts on this?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Jason Gunthorpe]

On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> By Mellanox HW design and SW implementation poll_cq never
> fails and returns errors, so all these prints are to catch ULP bugs.

Eh? How can a ULP cause poll_cq to get errors?

Are you sure these are not driver bugs?

Why can't you just print and discard the broken CQ entry?

What should use ULP do when it get EINVAL? You say poll again is
not correct, so you suggest a full QP tear down?

The entire point was to make the ULP interface sane. IMHO if the
driver has bugged itself then it should do something sane internally
and not just punt random error codes to the user.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 684 bytes --]

On Sun, Aug 28, 2016 at 11:17:58AM -0600, Jason Gunthorpe wrote:
> On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >
> > By Mellanox HW design and SW implementation poll_cq never
> > fails and returns errors, so all these prints are to catch ULP bugs.
>
> Eh? How can a ULP cause poll_cq to get errors?
>
> Are you sure these are not driver bugs?
>
> Why can't you just print and discard the broken CQ entry?
>
> What should use ULP do when it get EINVAL? You say poll again is
> not correct, so you suggest a full QP tear down?

See patches 4 and 6, they completely removed these EINVALs.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Leon Romanovsky]

On Sun, Aug 28, 2016 at 9:26 PM, Leon Romanovsky <leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org> wrote:
> On Sun, Aug 28, 2016 at 11:17:58AM -0600, Jason Gunthorpe wrote:
>> On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
>> > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>> >
>> > By Mellanox HW design and SW implementation poll_cq never
>> > fails and returns errors, so all these prints are to catch ULP bugs.
>>
>> Eh? How can a ULP cause poll_cq to get errors?
>>
>> Are you sure these are not driver bugs?
>>
>> Why can't you just print and discard the broken CQ entry?
>>
>> What should use ULP do when it get EINVAL? You say poll again is
>> not correct, so you suggest a full QP tear down?
>
> See patches 4 and 6, they completely removed these EINVALs.

Sorry, 4 and 9
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Jason Gunthorpe]

On Sun, Aug 28, 2016 at 09:26:13PM +0300, Leon Romanovsky wrote:
> On Sun, Aug 28, 2016 at 11:17:58AM -0600, Jason Gunthorpe wrote:
> > On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> > > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > >
> > > By Mellanox HW design and SW implementation poll_cq never
> > > fails and returns errors, so all these prints are to catch ULP bugs.
> >
> > Eh? How can a ULP cause poll_cq to get errors?
> >
> > Are you sure these are not driver bugs?
> >
> > Why can't you just print and discard the broken CQ entry?
> >
> > What should use ULP do when it get EINVAL? You say poll again is
> > not correct, so you suggest a full QP tear down?
> 
> See patches 4 and 6, they completely removed these EINVALs.

So the commit message is still wrong.

Why do we need this revert? Just squash it and mark it fixup the
original.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1369 bytes --]

On Sun, Aug 28, 2016 at 12:28:13PM -0600, Jason Gunthorpe wrote:
> On Sun, Aug 28, 2016 at 09:26:13PM +0300, Leon Romanovsky wrote:
> > On Sun, Aug 28, 2016 at 11:17:58AM -0600, Jason Gunthorpe wrote:
> > > On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> > > > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > > >
> > > > By Mellanox HW design and SW implementation poll_cq never
> > > > fails and returns errors, so all these prints are to catch ULP bugs.
> > >
> > > Eh? How can a ULP cause poll_cq to get errors?
> > >
> > > Are you sure these are not driver bugs?
> > >
> > > Why can't you just print and discard the broken CQ entry?
> > >
> > > What should use ULP do when it get EINVAL? You say poll again is
> > > not correct, so you suggest a full QP tear down?
> >
> > See patches 4 and 6, they completely removed these EINVALs.
>
> So the commit message is still wrong.
>
> Why do we need this revert? Just squash it and mark it fixup the
> original.

We need this revert, because the original commit is wrong and as was presented
by Sasha Levin in his talk about automatic creation of stable trees [1], he needs
this information to ensure that this commit won't be in stable tree by mistake.

[1]
https://lcccna2016.sched.org/event/7JW4/automating-the-creation-of-stable-trees-sasha-levin-verizon-labs

>
> Jason

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Jason Gunthorpe]

On Sun, Aug 28, 2016 at 09:35:00PM +0300, Leon Romanovsky wrote:

> We need this revert, because the original commit is wrong and as was
> presented

Sure, but it was also wrong to return EINVAL at all, the only correct
outcome I can see is to apply your whole series. Reverting doesn't
make the big picture any better, just differently wrong.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH rdma-rc 4/9] IB/mlx4: Don't return errors from poll_cq

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2136 bytes --]

On Sun, Aug 28, 2016 at 07:05:51PM +0300, Sagi Grimberg wrote:
>
> > 		/* SRQ is also in the radix tree */
> > 		msrq = mlx4_srq_lookup(to_mdev(cq->ibcq.device)->dev,
> > 				       srq_num);
> >-		if (unlikely(!msrq)) {
> >-			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
> >-				cq->mcq.cqn, srq_num);
> >-			return -EINVAL;
> >-		}
> > 	}
>
> BTW, this is completely unrelated to this patch, but the current
> implementation of shared receive queues in Mellanox drivers is
> *very* inefficient. Each completion that relates to a srq the
> mlx4/mlx5 drivers perform a device-wide locked srq lookup which
> is pretty bad... (it kills consumers that want to use more
> then one SRQ)
>
> Other drivers that support kernel SRQs that I've looked at are ocrdma and
> i40iw and they don't seem to have this lock everything approach.
>
> At the very-least we should try to make it a rcu + percpu_ref
> instead of a killer device-wide lock. It'd be even better if
> we use refcounting in the IB core and have the drivers not worry
> about the kernel consumers destroying SRQs while processing IO
> (i.e. when all the related QPs and CQs are destroyed).

This can be as a beginning.
diff --git a/drivers/net/ethernet/mellanox/mlx4/srq.c b/drivers/net/ethernet/mellanox/mlx4/srq.c
index 6714662..e53d366 100644
--- a/drivers/net/ethernet/mellanox/mlx4/srq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/srq.c
@@ -303,10 +303,10 @@ struct mlx4_srq *mlx4_srq_lookup(struct mlx4_dev *dev, u32 srqn)
 	struct mlx4_srq *srq;
 	unsigned long flags;

-	spin_lock_irqsave(&srq_table->lock, flags);
+	rcu_read_lock();
 	srq = radix_tree_lookup(&srq_table->tree,
 				srqn & (dev->caps.num_srqs - 1));
-	spin_unlock_irqrestore(&srq_table->lock, flags);
+	rcu_read_unlock();

 	return srq;
 }

>
> I have some code in the works on this but it's not high on my todo
> list at the moment. Mellanox folks, any thoughts on this?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 4/9] IB/mlx4: Don't return errors from poll_cq

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 4215 bytes --]

On Mon, Aug 29, 2016 at 12:41:19PM +0300, Leon Romanovsky wrote:
> On Sun, Aug 28, 2016 at 07:05:51PM +0300, Sagi Grimberg wrote:
> >
> > > 		/* SRQ is also in the radix tree */
> > > 		msrq = mlx4_srq_lookup(to_mdev(cq->ibcq.device)->dev,
> > > 				       srq_num);
> > >-		if (unlikely(!msrq)) {
> > >-			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
> > >-				cq->mcq.cqn, srq_num);
> > >-			return -EINVAL;
> > >-		}
> > > 	}
> >
> > BTW, this is completely unrelated to this patch, but the current
> > implementation of shared receive queues in Mellanox drivers is
> > *very* inefficient. Each completion that relates to a srq the
> > mlx4/mlx5 drivers perform a device-wide locked srq lookup which
> > is pretty bad... (it kills consumers that want to use more
> > then one SRQ)
> >
> > Other drivers that support kernel SRQs that I've looked at are ocrdma and
> > i40iw and they don't seem to have this lock everything approach.
> >
> > At the very-least we should try to make it a rcu + percpu_ref
> > instead of a killer device-wide lock. It'd be even better if
> > we use refcounting in the IB core and have the drivers not worry
> > about the kernel consumers destroying SRQs while processing IO
> > (i.e. when all the related QPs and CQs are destroyed).
>
> This can be as a beginning.
> diff --git a/drivers/net/ethernet/mellanox/mlx4/srq.c b/drivers/net/ethernet/mellanox/mlx4/srq.c
> index 6714662..e53d366 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/srq.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/srq.c
> @@ -303,10 +303,10 @@ struct mlx4_srq *mlx4_srq_lookup(struct mlx4_dev *dev, u32 srqn)
>  	struct mlx4_srq *srq;
>  	unsigned long flags;
>
> -	spin_lock_irqsave(&srq_table->lock, flags);
> +	rcu_read_lock();
>  	srq = radix_tree_lookup(&srq_table->tree,
>  				srqn & (dev->caps.num_srqs - 1));
> -	spin_unlock_irqrestore(&srq_table->lock, flags);
> +	rcu_read_unlock();
>
>  	return srq;
>  }

Or more complete patch (untested),

From 8695e5b807610e08de055b04ddd16caa6a5b61f2 Mon Sep 17 00:00:00 2001
From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Date: Mon, 29 Aug 2016 12:42:24 +0300
Subject: [PATCH] IB/mlx4: Use RCU to perform radix tree lookup for SRQ

Radix tree lookup can be performed without locking.

Suggested-by: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 drivers/net/ethernet/mellanox/mlx4/srq.c | 14 +++++---------
 1 file changed, 5 insertions(+), 9 deletions(-)

diff --git a/drivers/net/ethernet/mellanox/mlx4/srq.c b/drivers/net/ethernet/mellanox/mlx4/srq.c
index 6714662..f44d089 100644
--- a/drivers/net/ethernet/mellanox/mlx4/srq.c
+++ b/drivers/net/ethernet/mellanox/mlx4/srq.c
@@ -45,15 +45,12 @@ void mlx4_srq_event(struct mlx4_dev *dev, u32 srqn, int event_type)
 	struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table;
 	struct mlx4_srq *srq;

-	spin_lock(&srq_table->lock);
-
+	rcu_read_lock();
 	srq = radix_tree_lookup(&srq_table->tree, srqn & (dev->caps.num_srqs - 1));
+	rcu_read_unlock();
 	if (srq)
 		atomic_inc(&srq->refcount);
-
-	spin_unlock(&srq_table->lock);
-
-	if (!srq) {
+	else {
 		mlx4_warn(dev, "Async event for bogus SRQ %08x\n", srqn);
 		return;
 	}
@@ -301,12 +298,11 @@ struct mlx4_srq *mlx4_srq_lookup(struct mlx4_dev *dev, u32 srqn)
 {
 	struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table;
 	struct mlx4_srq *srq;
-	unsigned long flags;

-	spin_lock_irqsave(&srq_table->lock, flags);
+	rcu_read_lock();
 	srq = radix_tree_lookup(&srq_table->tree,
 				srqn & (dev->caps.num_srqs - 1));
-	spin_unlock_irqrestore(&srq_table->lock, flags);
+	rcu_read_unlock();

 	return srq;
 }
--
2.7.4


>
> >
> > I have some code in the works on this but it's not high on my todo
> > list at the moment. Mellanox folks, any thoughts on this?
> > --
> > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > More majordomo info at  http://vger.kernel.org/majordomo-info.html



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 7/9] IB/mlx5: Add VERBOSITY Kconfig option

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 2066 bytes --]

On 8/28/2016 3:58 AM, Leon Romanovsky wrote:
> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> 
> This patch introduces new MLX5_INFINIBAND_VERBOSE Kconfig
> option to enable the output of dump_wqe() which was never
> printed.
> 
> Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/infiniband/hw/mlx5/Kconfig | 6 ++++++
>  drivers/infiniband/hw/mlx5/qp.c    | 2 +-
>  2 files changed, 7 insertions(+), 1 deletion(-)
> 
> diff --git a/drivers/infiniband/hw/mlx5/Kconfig b/drivers/infiniband/hw/mlx5/Kconfig
> index bce263b..789fef4 100644
> --- a/drivers/infiniband/hw/mlx5/Kconfig
> +++ b/drivers/infiniband/hw/mlx5/Kconfig
> @@ -6,3 +6,9 @@ config MLX5_INFINIBAND
>  	  Mellanox Connect-IB PCI Express host channel adapters (HCAs).
>  	  This is required to use InfiniBand protocols such as
>  	  IP-over-IB or SRP with these devices.
> +
> +config MLX5_INFINIBAND_VERBOSE
> +	bool "Enable ConnectX-4/Connect-IB verbosity"
> +	depends on MLX5_INFINIBAND
> +	---help---
> +	  This is a configuration option to enable verbose output to easy debug.
> diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
> index 174d09b..67b58f7 100644
> --- a/drivers/infiniband/hw/mlx5/qp.c
> +++ b/drivers/infiniband/hw/mlx5/qp.c
> @@ -3984,7 +3984,7 @@ int mlx5_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
>  			   get_fence(fence, wr), next_fence,
>  			   mlx5_ib_opcode[wr->opcode]);
>  skip_psv:
> -		if (0)
> +		if (IS_BUILTIN(CONFIG_MLX5_INFINIBAND_VERBOSE))
>  			dump_wqe(qp, idx, size);
>  	}

This patch is just a big old bunch of no.  There is no way I'm sending
in Kconfig changes in a late -rc.  Further, there is no way I'm sending
in a Kconfig option for a single debug print function.  Third, there is
no way I'm sending in a Kconfig setting for what should be a runtime
debug switch or something like that.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: 0E572FDD


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 1817 bytes --]

On 8/28/2016 2:35 PM, Leon Romanovsky wrote:
> On Sun, Aug 28, 2016 at 12:28:13PM -0600, Jason Gunthorpe wrote:
>> On Sun, Aug 28, 2016 at 09:26:13PM +0300, Leon Romanovsky wrote:
>>> On Sun, Aug 28, 2016 at 11:17:58AM -0600, Jason Gunthorpe wrote:
>>>> On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
>>>>> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
>>>>>
>>>>> By Mellanox HW design and SW implementation poll_cq never
>>>>> fails and returns errors, so all these prints are to catch ULP bugs.
>>>>
>>>> Eh? How can a ULP cause poll_cq to get errors?
>>>>
>>>> Are you sure these are not driver bugs?
>>>>
>>>> Why can't you just print and discard the broken CQ entry?
>>>>
>>>> What should use ULP do when it get EINVAL? You say poll again is
>>>> not correct, so you suggest a full QP tear down?
>>>
>>> See patches 4 and 6, they completely removed these EINVALs.
>>
>> So the commit message is still wrong.
>>
>> Why do we need this revert? Just squash it and mark it fixup the
>> original.
> 
> We need this revert, because the original commit is wrong and as was presented
> by Sasha Levin in his talk about automatic creation of stable trees [1], he needs
> this information to ensure that this commit won't be in stable tree by mistake.
> 
> [1]
> https://lcccna2016.sched.org/event/7JW4/automating-the-creation-of-stable-trees-sasha-levin-verizon-labs

I didn't see the presentation at this link, but I'm not sure I agree
with your statement here.  There should be no difference, effectively,
between a commit to revert a bad commit, and a commit to fix a bad
commit that uses a Fixes: tag referencing the original commit.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: 0E572FDD


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [PATCH rdma-rc 0/9] RDMA fixes for 4.8

[Doug Ledford]

[-- Attachment #1.1: Type: text/plain, Size: 1876 bytes --]

On 8/28/2016 3:58 AM, Leon Romanovsky wrote:
> Hi Doug,
> 
> This is compilation of fixes to RDMA core, IPoIB, Mellanox mlx4
> and mlx5 drivers all together. Most of the fixes are one liners and
> for easy handling, I'm sending them in one patchset.
> 
> Thanks
> 
> Chuck Lever (1):
>   IB/mlx5: Return EINVAL when caller specifies too many SGEs
> 
> Erez Shitrit (2):
>   IB/core: Fix use after free in send_leave function
>   IB/ipoib: Fix memory corruption during ipoib cm connection
>     establishment
> 
> Leon Romanovsky (5):
>   Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"
>   IB/mlx4: Don't return errors from poll_cq
>   IB/mlx5: Simplify code by removing return variable
>   IB/mlx5: Add VERBOSITY Kconfig option
>   IB/mlx5: Don't return errors from poll_cq
> 
> Yishai Hadas (1):
>   IB/mlx5: Use TIR number based on selector
> 
>  drivers/infiniband/core/multicast.c       | 13 ++-----------
>  drivers/infiniband/hw/mlx4/cq.c           | 26 ++------------------------
>  drivers/infiniband/hw/mlx5/Kconfig        |  6 ++++++
>  drivers/infiniband/hw/mlx5/cq.c           | 22 ++--------------------
>  drivers/infiniband/hw/mlx5/main.c         |  6 +++++-
>  drivers/infiniband/hw/mlx5/mlx5_ib.h      |  1 +
>  drivers/infiniband/hw/mlx5/qp.c           | 15 ++++++---------
>  drivers/infiniband/ulp/ipoib/ipoib.h      |  1 +
>  drivers/infiniband/ulp/ipoib/ipoib_cm.c   | 16 ++++++++++++++++
>  drivers/infiniband/ulp/ipoib/ipoib_main.c |  2 +-
>  10 files changed, 42 insertions(+), 66 deletions(-)

I took everything except patch 7.  That one had to go.  It would take a
very strong argument about why that needs a Kconfig option instead of a
runtime config switch to make me change my mind.


-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG Key ID: 0E572FDD


[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 884 bytes --]

Re: [PATCH rdma-rc 3/9] Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2012 bytes --]

On Fri, Sep 02, 2016 at 02:03:13PM -0400, Doug Ledford wrote:
> On 8/28/2016 2:35 PM, Leon Romanovsky wrote:
> > On Sun, Aug 28, 2016 at 12:28:13PM -0600, Jason Gunthorpe wrote:
> >> On Sun, Aug 28, 2016 at 09:26:13PM +0300, Leon Romanovsky wrote:
> >>> On Sun, Aug 28, 2016 at 11:17:58AM -0600, Jason Gunthorpe wrote:
> >>>> On Sun, Aug 28, 2016 at 10:58:32AM +0300, Leon Romanovsky wrote:
> >>>>> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >>>>>
> >>>>> By Mellanox HW design and SW implementation poll_cq never
> >>>>> fails and returns errors, so all these prints are to catch ULP bugs.
> >>>>
> >>>> Eh? How can a ULP cause poll_cq to get errors?
> >>>>
> >>>> Are you sure these are not driver bugs?
> >>>>
> >>>> Why can't you just print and discard the broken CQ entry?
> >>>>
> >>>> What should use ULP do when it get EINVAL? You say poll again is
> >>>> not correct, so you suggest a full QP tear down?
> >>>
> >>> See patches 4 and 6, they completely removed these EINVALs.
> >>
> >> So the commit message is still wrong.
> >>
> >> Why do we need this revert? Just squash it and mark it fixup the
> >> original.
> >
> > We need this revert, because the original commit is wrong and as was presented
> > by Sasha Levin in his talk about automatic creation of stable trees [1], he needs
> > this information to ensure that this commit won't be in stable tree by mistake.
> >
> > [1]
> > https://lcccna2016.sched.org/event/7JW4/automating-the-creation-of-stable-trees-sasha-levin-verizon-labs
>
> I didn't see the presentation at this link, but I'm not sure I agree
> with your statement here.  There should be no difference, effectively,
> between a commit to revert a bad commit, and a commit to fix a bad
> commit that uses a Fixes: tag referencing the original commit.

The final code will look the same, but it will be much easier for the
tools to pickup them.

>
>
> --
> Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>     GPG Key ID: 0E572FDD
>




[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 4/9] IB/mlx4: Don't return errors from poll_cq

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 4463 bytes --]

On Mon, Aug 29, 2016 at 01:04:34PM +0300, Leon Romanovsky wrote:
> On Mon, Aug 29, 2016 at 12:41:19PM +0300, Leon Romanovsky wrote:
> > On Sun, Aug 28, 2016 at 07:05:51PM +0300, Sagi Grimberg wrote:
> > >
> > > > 		/* SRQ is also in the radix tree */
> > > > 		msrq = mlx4_srq_lookup(to_mdev(cq->ibcq.device)->dev,
> > > > 				       srq_num);
> > > >-		if (unlikely(!msrq)) {
> > > >-			pr_warn("CQ %06x with entry for unknown SRQN %06x\n",
> > > >-				cq->mcq.cqn, srq_num);
> > > >-			return -EINVAL;
> > > >-		}
> > > > 	}
> > >
> > > BTW, this is completely unrelated to this patch, but the current
> > > implementation of shared receive queues in Mellanox drivers is
> > > *very* inefficient. Each completion that relates to a srq the
> > > mlx4/mlx5 drivers perform a device-wide locked srq lookup which
> > > is pretty bad... (it kills consumers that want to use more
> > > then one SRQ)
> > >
> > > Other drivers that support kernel SRQs that I've looked at are ocrdma and
> > > i40iw and they don't seem to have this lock everything approach.
> > >
> > > At the very-least we should try to make it a rcu + percpu_ref
> > > instead of a killer device-wide lock. It'd be even better if
> > > we use refcounting in the IB core and have the drivers not worry
> > > about the kernel consumers destroying SRQs while processing IO
> > > (i.e. when all the related QPs and CQs are destroyed).
> >
> > This can be as a beginning.
> > diff --git a/drivers/net/ethernet/mellanox/mlx4/srq.c b/drivers/net/ethernet/mellanox/mlx4/srq.c
> > index 6714662..e53d366 100644
> > --- a/drivers/net/ethernet/mellanox/mlx4/srq.c
> > +++ b/drivers/net/ethernet/mellanox/mlx4/srq.c
> > @@ -303,10 +303,10 @@ struct mlx4_srq *mlx4_srq_lookup(struct mlx4_dev *dev, u32 srqn)
> >  	struct mlx4_srq *srq;
> >  	unsigned long flags;
> >
> > -	spin_lock_irqsave(&srq_table->lock, flags);
> > +	rcu_read_lock();
> >  	srq = radix_tree_lookup(&srq_table->tree,
> >  				srqn & (dev->caps.num_srqs - 1));
> > -	spin_unlock_irqrestore(&srq_table->lock, flags);
> > +	rcu_read_unlock();
> >
> >  	return srq;
> >  }
>
> Or more complete patch (untested),
>
> From 8695e5b807610e08de055b04ddd16caa6a5b61f2 Mon Sep 17 00:00:00 2001
> From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> Date: Mon, 29 Aug 2016 12:42:24 +0300
> Subject: [PATCH] IB/mlx4: Use RCU to perform radix tree lookup for SRQ

As a followup, it was tested and it will be submitted by Tariq to net/mlx4.

>
> Radix tree lookup can be performed without locking.
>
> Suggested-by: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
> Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> ---
>  drivers/net/ethernet/mellanox/mlx4/srq.c | 14 +++++---------
>  1 file changed, 5 insertions(+), 9 deletions(-)
>
> diff --git a/drivers/net/ethernet/mellanox/mlx4/srq.c b/drivers/net/ethernet/mellanox/mlx4/srq.c
> index 6714662..f44d089 100644
> --- a/drivers/net/ethernet/mellanox/mlx4/srq.c
> +++ b/drivers/net/ethernet/mellanox/mlx4/srq.c
> @@ -45,15 +45,12 @@ void mlx4_srq_event(struct mlx4_dev *dev, u32 srqn, int event_type)
>  	struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table;
>  	struct mlx4_srq *srq;
>
> -	spin_lock(&srq_table->lock);
> -
> +	rcu_read_lock();
>  	srq = radix_tree_lookup(&srq_table->tree, srqn & (dev->caps.num_srqs - 1));
> +	rcu_read_unlock();
>  	if (srq)
>  		atomic_inc(&srq->refcount);
> -
> -	spin_unlock(&srq_table->lock);
> -
> -	if (!srq) {
> +	else {
>  		mlx4_warn(dev, "Async event for bogus SRQ %08x\n", srqn);
>  		return;
>  	}
> @@ -301,12 +298,11 @@ struct mlx4_srq *mlx4_srq_lookup(struct mlx4_dev *dev, u32 srqn)
>  {
>  	struct mlx4_srq_table *srq_table = &mlx4_priv(dev)->srq_table;
>  	struct mlx4_srq *srq;
> -	unsigned long flags;
>
> -	spin_lock_irqsave(&srq_table->lock, flags);
> +	rcu_read_lock();
>  	srq = radix_tree_lookup(&srq_table->tree,
>  				srqn & (dev->caps.num_srqs - 1));
> -	spin_unlock_irqrestore(&srq_table->lock, flags);
> +	rcu_read_unlock();
>
>  	return srq;
>  }
> --
> 2.7.4
>
>
> >
> > >
> > > I have some code in the works on this but it's not high on my todo
> > > list at the moment. Mellanox folks, any thoughts on this?
> > > --
> > > To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> > > the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> > > More majordomo info at  http://vger.kernel.org/majordomo-info.html
>
>



[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 7/9] IB/mlx5: Add VERBOSITY Kconfig option

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2994 bytes --]

On Fri, Sep 02, 2016 at 01:52:59PM -0400, Doug Ledford wrote:
> On 8/28/2016 3:58 AM, Leon Romanovsky wrote:
> > From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> >
> > This patch introduces new MLX5_INFINIBAND_VERBOSE Kconfig
> > option to enable the output of dump_wqe() which was never
> > printed.
> >
> > Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
> > ---
> >  drivers/infiniband/hw/mlx5/Kconfig | 6 ++++++
> >  drivers/infiniband/hw/mlx5/qp.c    | 2 +-
> >  2 files changed, 7 insertions(+), 1 deletion(-)
> >
> > diff --git a/drivers/infiniband/hw/mlx5/Kconfig b/drivers/infiniband/hw/mlx5/Kconfig
> > index bce263b..789fef4 100644
> > --- a/drivers/infiniband/hw/mlx5/Kconfig
> > +++ b/drivers/infiniband/hw/mlx5/Kconfig
> > @@ -6,3 +6,9 @@ config MLX5_INFINIBAND
> >  	  Mellanox Connect-IB PCI Express host channel adapters (HCAs).
> >  	  This is required to use InfiniBand protocols such as
> >  	  IP-over-IB or SRP with these devices.
> > +
> > +config MLX5_INFINIBAND_VERBOSE
> > +	bool "Enable ConnectX-4/Connect-IB verbosity"
> > +	depends on MLX5_INFINIBAND
> > +	---help---
> > +	  This is a configuration option to enable verbose output to easy debug.
> > diff --git a/drivers/infiniband/hw/mlx5/qp.c b/drivers/infiniband/hw/mlx5/qp.c
> > index 174d09b..67b58f7 100644
> > --- a/drivers/infiniband/hw/mlx5/qp.c
> > +++ b/drivers/infiniband/hw/mlx5/qp.c
> > @@ -3984,7 +3984,7 @@ int mlx5_ib_post_send(struct ib_qp *ibqp, struct ib_send_wr *wr,
> >  			   get_fence(fence, wr), next_fence,
> >  			   mlx5_ib_opcode[wr->opcode]);
> >  skip_psv:
> > -		if (0)
> > +		if (IS_BUILTIN(CONFIG_MLX5_INFINIBAND_VERBOSE))
> >  			dump_wqe(qp, idx, size);
> >  	}
>
> This patch is just a big old bunch of no.  There is no way I'm sending
> in Kconfig changes in a late -rc.

Totally understandable, it was a combination of bad luck with my faults.
1. This patch was prepared before merge window.
2. Shared code was posted late.
3. The whole series was targeted to be sent for 4.9 and not for RCs.
Yuval's patch caused to change in my plans.

> Further, there is no way I'm sending
> in a Kconfig option for a single debug print function.

The goal of this config option is to preserver this print function which
is extremely useful for debug and development. Currently it is under if(0)
and can be removed by mistake.

> Third, there is
> no way I'm sending in a Kconfig setting for what should be a runtime
> debug switch or something like that.

It is in data-path and I wanted to ensure that no change in performance
will be observed. Dynamic configuration won't allow to achieve it.

Will it be acceptable by you to take this patch for for-next to ensure
no one is removing it by mistake? And i will explore the commonalities
in the different drivers to create common VERBOSE flag for whole
subsystem?

Thanks


>
>
> --
> Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>     GPG Key ID: 0E572FDD
>




[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]

Re: [PATCH rdma-rc 0/9] RDMA fixes for 4.8

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2015 bytes --]

On Fri, Sep 02, 2016 at 02:13:27PM -0400, Doug Ledford wrote:
> On 8/28/2016 3:58 AM, Leon Romanovsky wrote:
> > Hi Doug,
> >
> > This is compilation of fixes to RDMA core, IPoIB, Mellanox mlx4
> > and mlx5 drivers all together. Most of the fixes are one liners and
> > for easy handling, I'm sending them in one patchset.
> >
> > Thanks
> >
> > Chuck Lever (1):
> >   IB/mlx5: Return EINVAL when caller specifies too many SGEs
> >
> > Erez Shitrit (2):
> >   IB/core: Fix use after free in send_leave function
> >   IB/ipoib: Fix memory corruption during ipoib cm connection
> >     establishment
> >
> > Leon Romanovsky (5):
> >   Revert "IB/mlx4: Return EAGAIN for any error in mlx4_ib_poll_one"
> >   IB/mlx4: Don't return errors from poll_cq
> >   IB/mlx5: Simplify code by removing return variable
> >   IB/mlx5: Add VERBOSITY Kconfig option
> >   IB/mlx5: Don't return errors from poll_cq
> >
> > Yishai Hadas (1):
> >   IB/mlx5: Use TIR number based on selector
> >
> >  drivers/infiniband/core/multicast.c       | 13 ++-----------
> >  drivers/infiniband/hw/mlx4/cq.c           | 26 ++------------------------
> >  drivers/infiniband/hw/mlx5/Kconfig        |  6 ++++++
> >  drivers/infiniband/hw/mlx5/cq.c           | 22 ++--------------------
> >  drivers/infiniband/hw/mlx5/main.c         |  6 +++++-
> >  drivers/infiniband/hw/mlx5/mlx5_ib.h      |  1 +
> >  drivers/infiniband/hw/mlx5/qp.c           | 15 ++++++---------
> >  drivers/infiniband/ulp/ipoib/ipoib.h      |  1 +
> >  drivers/infiniband/ulp/ipoib/ipoib_cm.c   | 16 ++++++++++++++++
> >  drivers/infiniband/ulp/ipoib/ipoib_main.c |  2 +-
> >  10 files changed, 42 insertions(+), 66 deletions(-)
>
> I took everything except patch 7.  That one had to go.  It would take a
> very strong argument about why that needs a Kconfig option instead of a
> runtime config switch to make me change my mind.

Thanks,
I responded to that thread.

>
>
> --
> Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
>     GPG Key ID: 0E572FDD
>




[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 819 bytes --]