[PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Håkon Bugge]

ib_modify_qp() is an expensive operation on some HCAs running
virtualized. This series removes two ib_modify_qp() calls from RDS.

I am sending this as a v3, even though it is the first sent to
net. This because the IB Core commit has reach v3.

Håkon Bugge (2):
  IB/cma: Introduce rdma_set_min_rnr_timer()
  rds: ib: Remove two ib_modify_qp() calls

 drivers/infiniband/core/cma.c      | 41 ++++++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/cma_priv.h |  2 ++
 include/rdma/rdma_cm.h             |  2 ++
 net/rds/ib_cm.c                    | 35 +-------------------------------
 net/rds/rdma_transport.c           |  1 +
 5 files changed, 47 insertions(+), 34 deletions(-)

--
1.8.3.1

[PATCH for-next v3 1/2] IB/cma: Introduce rdma_set_min_rnr_timer()

[Håkon Bugge]

Introduce the ability for kernel ULPs to adjust the minimum RNR Retry
timer. The INIT -> RTR transition executed by RDMA CM will be used for
this adjustment. This avoids an additional ib_modify_qp() call.

rdma_set_min_rnr_timer() must be called before the call to
rdma_connect() on the active side and before the call to rdma_accept()
on the passive side.

The default value of RNR Retry timer is zero, which translates to 655
ms. When the receiver is not ready to accept a send messages, it
encodes the RNR Retry timer value in the NAK. The requestor will then
wait at least the specified time value before retrying the send.

The 5-bit value to be supplied to the rdma_set_min_rnr_timer() is
documented in IBTA Table 45: "Encoding for RNR NAK Timer Field".

Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
Acked-by: Jason Gunthorpe <jgg@nvidia.com>
---
 drivers/infiniband/core/cma.c      | 41 ++++++++++++++++++++++++++++++++++++++
 drivers/infiniband/core/cma_priv.h |  2 ++
 include/rdma/rdma_cm.h             |  2 ++
 3 files changed, 45 insertions(+)

diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 9409651..5ce097d 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -852,6 +852,7 @@ static void cma_id_put(struct rdma_id_private *id_priv)
 	id_priv->id.qp_type = qp_type;
 	id_priv->tos_set = false;
 	id_priv->timeout_set = false;
+	id_priv->min_rnr_timer_set = false;
 	id_priv->gid_type = IB_GID_TYPE_IB;
 	spin_lock_init(&id_priv->lock);
 	mutex_init(&id_priv->qp_mutex);
@@ -1141,6 +1142,9 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
 	if ((*qp_attr_mask & IB_QP_TIMEOUT) && id_priv->timeout_set)
 		qp_attr->timeout = id_priv->timeout;
 
+	if ((*qp_attr_mask & IB_QP_MIN_RNR_TIMER) && id_priv->min_rnr_timer_set)
+		qp_attr->min_rnr_timer = id_priv->min_rnr_timer;
+
 	return ret;
 }
 EXPORT_SYMBOL(rdma_init_qp_attr);
@@ -2615,6 +2619,43 @@ int rdma_set_ack_timeout(struct rdma_cm_id *id, u8 timeout)
 }
 EXPORT_SYMBOL(rdma_set_ack_timeout);
 
+/**
+ * rdma_set_min_rnr_timer() - Set the minimum RNR Retry timer of the
+ *			      QP associated with a connection identifier.
+ * @id: Communication identifier to associated with service type.
+ * @min_rnr_timer: 5-bit value encoded as Table 45: "Encoding for RNR NAK
+ *		   Timer Field" in the IBTA specification.
+ *
+ * This function should be called before rdma_connect() on active
+ * side, and on passive side before rdma_accept(). The timer value
+ * will be associated with the local QP. When it receives a send it is
+ * not read to handle, typically if the receive queue is empty, an RNR
+ * Retry NAK is returned to the requester with the min_rnr_timer
+ * encoded. The requester will then wait at least the time specified
+ * in the NAK before retrying. The default is zero, which translates
+ * to a minimum RNR Timer value of 655 ms.
+ *
+ * Return: 0 for success
+ */
+int rdma_set_min_rnr_timer(struct rdma_cm_id *id, u8 min_rnr_timer)
+{
+	struct rdma_id_private *id_priv;
+
+	/* It is a five-bit value */
+	if (min_rnr_timer & 0xe0)
+		return -EINVAL;
+
+	if (id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT)
+		return -EINVAL;
+
+	id_priv = container_of(id, struct rdma_id_private, id);
+	id_priv->min_rnr_timer = min_rnr_timer;
+	id_priv->min_rnr_timer_set = true;
+
+	return 0;
+}
+EXPORT_SYMBOL(rdma_set_min_rnr_timer);
+
 static void cma_query_handler(int status, struct sa_path_rec *path_rec,
 			      void *context)
 {
diff --git a/drivers/infiniband/core/cma_priv.h b/drivers/infiniband/core/cma_priv.h
index caece96..bf83d32 100644
--- a/drivers/infiniband/core/cma_priv.h
+++ b/drivers/infiniband/core/cma_priv.h
@@ -86,9 +86,11 @@ struct rdma_id_private {
 	u8			tos;
 	u8			tos_set:1;
 	u8                      timeout_set:1;
+	u8			min_rnr_timer_set:1;
 	u8			reuseaddr;
 	u8			afonly;
 	u8			timeout;
+	u8			min_rnr_timer;
 	enum ib_gid_type	gid_type;
 
 	/*
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index 32a67af..8b0f66e 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -331,6 +331,8 @@ int rdma_join_multicast(struct rdma_cm_id *id, struct sockaddr *addr,
 int rdma_set_afonly(struct rdma_cm_id *id, int afonly);
 
 int rdma_set_ack_timeout(struct rdma_cm_id *id, u8 timeout);
+
+int rdma_set_min_rnr_timer(struct rdma_cm_id *id, u8 min_rnr_timer);
  /**
  * rdma_get_service_id - Return the IB service ID for a specified address.
  * @id: Communication identifier associated with the address.
-- 
1.8.3.1

Re: [PATCH for-next v3 1/2] IB/cma: Introduce rdma_set_min_rnr_timer()

[Leon Romanovsky]

On Wed, Mar 31, 2021 at 08:43:13PM +0200, Håkon Bugge wrote:
> Introduce the ability for kernel ULPs to adjust the minimum RNR Retry
> timer. The INIT -> RTR transition executed by RDMA CM will be used for
> this adjustment. This avoids an additional ib_modify_qp() call.
> 
> rdma_set_min_rnr_timer() must be called before the call to
> rdma_connect() on the active side and before the call to rdma_accept()
> on the passive side.
> 
> The default value of RNR Retry timer is zero, which translates to 655
> ms. When the receiver is not ready to accept a send messages, it
> encodes the RNR Retry timer value in the NAK. The requestor will then
> wait at least the specified time value before retrying the send.
> 
> The 5-bit value to be supplied to the rdma_set_min_rnr_timer() is
> documented in IBTA Table 45: "Encoding for RNR NAK Timer Field".
> 
> Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
> Acked-by: Jason Gunthorpe <jgg@nvidia.com>
> ---
>  drivers/infiniband/core/cma.c      | 41 ++++++++++++++++++++++++++++++++++++++
>  drivers/infiniband/core/cma_priv.h |  2 ++
>  include/rdma/rdma_cm.h             |  2 ++
>  3 files changed, 45 insertions(+)
> 
> diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> index 9409651..5ce097d 100644
> --- a/drivers/infiniband/core/cma.c
> +++ b/drivers/infiniband/core/cma.c
> @@ -852,6 +852,7 @@ static void cma_id_put(struct rdma_id_private *id_priv)
>  	id_priv->id.qp_type = qp_type;
>  	id_priv->tos_set = false;
>  	id_priv->timeout_set = false;
> +	id_priv->min_rnr_timer_set = false;
>  	id_priv->gid_type = IB_GID_TYPE_IB;
>  	spin_lock_init(&id_priv->lock);
>  	mutex_init(&id_priv->qp_mutex);
> @@ -1141,6 +1142,9 @@ int rdma_init_qp_attr(struct rdma_cm_id *id, struct ib_qp_attr *qp_attr,
>  	if ((*qp_attr_mask & IB_QP_TIMEOUT) && id_priv->timeout_set)
>  		qp_attr->timeout = id_priv->timeout;
>  
> +	if ((*qp_attr_mask & IB_QP_MIN_RNR_TIMER) && id_priv->min_rnr_timer_set)
> +		qp_attr->min_rnr_timer = id_priv->min_rnr_timer;
> +
>  	return ret;
>  }
>  EXPORT_SYMBOL(rdma_init_qp_attr);
> @@ -2615,6 +2619,43 @@ int rdma_set_ack_timeout(struct rdma_cm_id *id, u8 timeout)
>  }
>  EXPORT_SYMBOL(rdma_set_ack_timeout);
>  
> +/**
> + * rdma_set_min_rnr_timer() - Set the minimum RNR Retry timer of the
> + *			      QP associated with a connection identifier.
> + * @id: Communication identifier to associated with service type.
> + * @min_rnr_timer: 5-bit value encoded as Table 45: "Encoding for RNR NAK
> + *		   Timer Field" in the IBTA specification.
> + *
> + * This function should be called before rdma_connect() on active
> + * side, and on passive side before rdma_accept(). The timer value
> + * will be associated with the local QP. When it receives a send it is
> + * not read to handle, typically if the receive queue is empty, an RNR
> + * Retry NAK is returned to the requester with the min_rnr_timer
> + * encoded. The requester will then wait at least the time specified
> + * in the NAK before retrying. The default is zero, which translates
> + * to a minimum RNR Timer value of 655 ms.
> + *
> + * Return: 0 for success
> + */
> +int rdma_set_min_rnr_timer(struct rdma_cm_id *id, u8 min_rnr_timer)
> +{
> +	struct rdma_id_private *id_priv;
> +
> +	/* It is a five-bit value */
> +	if (min_rnr_timer & 0xe0)
> +		return -EINVAL;
> +
> +	if (id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT)
> +		return -EINVAL;

This is in-kernel API and safe to use WARN_ON() instead of returning
error which RDS is not checking anyway.

Thanks

[PATCH for-next v3 2/2] rds: ib: Remove two ib_modify_qp() calls

[Håkon Bugge]

For some HCAs, ib_modify_qp() is an expensive operation running
virtualized.

For both the active and passive side, the QP returned by the CM has
the state set to RTS, so no need for this excess RTS -> RTS
transition. With IB Core's ability to set the RNR Retry timer, we use
this interface to shave off another ib_modify_qp().

Fixes: ec16227e1414 ("RDS/IB: Infiniband transport")
Signed-off-by: Håkon Bugge <haakon.bugge@oracle.com>
---
 net/rds/ib_cm.c          | 35 +----------------------------------
 net/rds/rdma_transport.c |  1 +
 2 files changed, 2 insertions(+), 34 deletions(-)

diff --git a/net/rds/ib_cm.c b/net/rds/ib_cm.c
index f5cbe96..26b069e 100644
--- a/net/rds/ib_cm.c
+++ b/net/rds/ib_cm.c
@@ -68,31 +68,6 @@ static void rds_ib_set_flow_control(struct rds_connection *conn, u32 credits)
 }
 
 /*
- * Tune RNR behavior. Without flow control, we use a rather
- * low timeout, but not the absolute minimum - this should
- * be tunable.
- *
- * We already set the RNR retry count to 7 (which is the
- * smallest infinite number :-) above.
- * If flow control is off, we want to change this back to 0
- * so that we learn quickly when our credit accounting is
- * buggy.
- *
- * Caller passes in a qp_attr pointer - don't waste stack spacv
- * by allocation this twice.
- */
-static void
-rds_ib_tune_rnr(struct rds_ib_connection *ic, struct ib_qp_attr *attr)
-{
-	int ret;
-
-	attr->min_rnr_timer = IB_RNR_TIMER_000_32;
-	ret = ib_modify_qp(ic->i_cm_id->qp, attr, IB_QP_MIN_RNR_TIMER);
-	if (ret)
-		printk(KERN_NOTICE "ib_modify_qp(IB_QP_MIN_RNR_TIMER): err=%d\n", -ret);
-}
-
-/*
  * Connection established.
  * We get here for both outgoing and incoming connection.
  */
@@ -100,7 +75,6 @@ void rds_ib_cm_connect_complete(struct rds_connection *conn, struct rdma_cm_even
 {
 	struct rds_ib_connection *ic = conn->c_transport_data;
 	const union rds_ib_conn_priv *dp = NULL;
-	struct ib_qp_attr qp_attr;
 	__be64 ack_seq = 0;
 	__be32 credit = 0;
 	u8 major = 0;
@@ -168,14 +142,6 @@ void rds_ib_cm_connect_complete(struct rds_connection *conn, struct rdma_cm_even
 	 * the posted credit count. */
 	rds_ib_recv_refill(conn, 1, GFP_KERNEL);
 
-	/* Tune RNR behavior */
-	rds_ib_tune_rnr(ic, &qp_attr);
-
-	qp_attr.qp_state = IB_QPS_RTS;
-	err = ib_modify_qp(ic->i_cm_id->qp, &qp_attr, IB_QP_STATE);
-	if (err)
-		printk(KERN_NOTICE "ib_modify_qp(IB_QP_STATE, RTS): err=%d\n", err);
-
 	/* update ib_device with this local ipaddr */
 	err = rds_ib_update_ipaddr(ic->rds_ibdev, &conn->c_laddr);
 	if (err)
@@ -947,6 +913,7 @@ int rds_ib_cm_handle_connect(struct rdma_cm_id *cm_id,
 				  event->param.conn.responder_resources,
 				  event->param.conn.initiator_depth, isv6);
 
+	rdma_set_min_rnr_timer(cm_id, IB_RNR_TIMER_000_32);
 	/* rdma_accept() calls rdma_reject() internally if it fails */
 	if (rdma_accept(cm_id, &conn_param))
 		rds_ib_conn_error(conn, "rdma_accept failed\n");
diff --git a/net/rds/rdma_transport.c b/net/rds/rdma_transport.c
index 5f741e5..a9e4ff9 100644
--- a/net/rds/rdma_transport.c
+++ b/net/rds/rdma_transport.c
@@ -87,6 +87,7 @@ static int rds_rdma_cm_event_handler_cmn(struct rdma_cm_id *cm_id,
 
 	case RDMA_CM_EVENT_ADDR_RESOLVED:
 		rdma_set_service_type(cm_id, conn->c_tos);
+		rdma_set_min_rnr_timer(cm_id, IB_RNR_TIMER_000_32);
 		/* XXX do we need to clean up if this fails? */
 		ret = rdma_resolve_route(cm_id,
 					 RDS_RDMA_RESOLVE_TIMEOUT_MS);
-- 
1.8.3.1

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Santosh Shilimkar]

[...]

Thanks Haakon. Patchset looks fine by me.
Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Jason Gunthorpe]

On Wed, Mar 31, 2021 at 07:54:17PM +0000, Santosh Shilimkar wrote:
> [...]
> 
> Thanks Haakon. Patchset looks fine by me.
> Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>

Jakub/Dave are you OK if I take this RDS patch rdma to rdma's tree?

Thanks,
Jason

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Haakon Bugge]

> On 1 Apr 2021, at 19:51, Jason Gunthorpe <jgg@nvidia.com> wrote:
> 
> On Wed, Mar 31, 2021 at 07:54:17PM +0000, Santosh Shilimkar wrote:
>> [...]
>> 
>> Thanks Haakon. Patchset looks fine by me.
>> Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
> 
> Jakub/Dave are you OK if I take this RDS patch rdma to rdma's tree?

Let me know if this is lingering due to Leon's comment about using WARN_ON() instead of error returns.


Håkon

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Haakon Bugge]

> On 7 Apr 2021, at 18:41, Haakon Bugge <haakon.bugge@oracle.com> wrote:
> 
> 
> 
>> On 1 Apr 2021, at 19:51, Jason Gunthorpe <jgg@nvidia.com> wrote:
>> 
>> On Wed, Mar 31, 2021 at 07:54:17PM +0000, Santosh Shilimkar wrote:
>>> [...]
>>> 
>>> Thanks Haakon. Patchset looks fine by me.
>>> Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
>> 
>> Jakub/Dave are you OK if I take this RDS patch rdma to rdma's tree?
> 
> Let me know if this is lingering due to Leon's comment about using WARN_ON() instead of error returns.

A gentle ping.

Håkon

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Jason Gunthorpe]

On Mon, Apr 12, 2021 at 06:35:35PM +0000, Haakon Bugge wrote:
> 
> 
> > On 7 Apr 2021, at 18:41, Haakon Bugge <haakon.bugge@oracle.com> wrote:
> > 
> > 
> > 
> >> On 1 Apr 2021, at 19:51, Jason Gunthorpe <jgg@nvidia.com> wrote:
> >> 
> >> On Wed, Mar 31, 2021 at 07:54:17PM +0000, Santosh Shilimkar wrote:
> >>> [...]
> >>> 
> >>> Thanks Haakon. Patchset looks fine by me.
> >>> Acked-by: Santosh Shilimkar <santosh.shilimkar@oracle.com>
> >> 
> >> Jakub/Dave are you OK if I take this RDS patch rdma to rdma's tree?
> > 
> > Let me know if this is lingering due to Leon's comment about using WARN_ON() instead of error returns.
> 
> A gentle ping.

I will take it with Santos' ack.

Jason

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Jason Gunthorpe]

On Wed, Mar 31, 2021 at 08:43:12PM +0200, Håkon Bugge wrote:
> ib_modify_qp() is an expensive operation on some HCAs running
> virtualized. This series removes two ib_modify_qp() calls from RDS.
> 
> I am sending this as a v3, even though it is the first sent to
> net. This because the IB Core commit has reach v3.
> 
> Håkon Bugge (2):
>   IB/cma: Introduce rdma_set_min_rnr_timer()
>   rds: ib: Remove two ib_modify_qp() calls

Applied to rdma for-next, thanks

Jason

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Leon Romanovsky]

On Mon, Apr 12, 2021 at 07:58:47PM -0300, Jason Gunthorpe wrote:
> On Wed, Mar 31, 2021 at 08:43:12PM +0200, Håkon Bugge wrote:
> > ib_modify_qp() is an expensive operation on some HCAs running
> > virtualized. This series removes two ib_modify_qp() calls from RDS.
> > 
> > I am sending this as a v3, even though it is the first sent to
> > net. This because the IB Core commit has reach v3.
> > 
> > Håkon Bugge (2):
> >   IB/cma: Introduce rdma_set_min_rnr_timer()
> >   rds: ib: Remove two ib_modify_qp() calls
> 
> Applied to rdma for-next, thanks

Jason,

It should be 
+	WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT);

and not
+	if (WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT))
+		return -EINVAL;

Thanks

> 
> Jason

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Haakon Bugge]

> On 13 Apr 2021, at 08:29, Leon Romanovsky <leon@kernel.org> wrote:
> 
> On Mon, Apr 12, 2021 at 07:58:47PM -0300, Jason Gunthorpe wrote:
>> On Wed, Mar 31, 2021 at 08:43:12PM +0200, Håkon Bugge wrote:
>>> ib_modify_qp() is an expensive operation on some HCAs running
>>> virtualized. This series removes two ib_modify_qp() calls from RDS.
>>> 
>>> I am sending this as a v3, even though it is the first sent to
>>> net. This because the IB Core commit has reach v3.
>>> 
>>> Håkon Bugge (2):
>>>  IB/cma: Introduce rdma_set_min_rnr_timer()
>>>  rds: ib: Remove two ib_modify_qp() calls
>> 
>> Applied to rdma for-next, thanks
> 
> Jason,
> 
> It should be 
> +	WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT);

With no return you will arm the setting of the timer and subsequently get an error from the modify_qp later.


Håkon

> 
> and not
> +	if (WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT))
> +		return -EINVAL;
> 
> Thanks
> 
>> 
>> Jason

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Leon Romanovsky]

On Tue, Apr 13, 2021 at 11:13:38AM +0000, Haakon Bugge wrote:
> 
> 
> > On 13 Apr 2021, at 08:29, Leon Romanovsky <leon@kernel.org> wrote:
> > 
> > On Mon, Apr 12, 2021 at 07:58:47PM -0300, Jason Gunthorpe wrote:
> >> On Wed, Mar 31, 2021 at 08:43:12PM +0200, Håkon Bugge wrote:
> >>> ib_modify_qp() is an expensive operation on some HCAs running
> >>> virtualized. This series removes two ib_modify_qp() calls from RDS.
> >>> 
> >>> I am sending this as a v3, even though it is the first sent to
> >>> net. This because the IB Core commit has reach v3.
> >>> 
> >>> Håkon Bugge (2):
> >>>  IB/cma: Introduce rdma_set_min_rnr_timer()
> >>>  rds: ib: Remove two ib_modify_qp() calls
> >> 
> >> Applied to rdma for-next, thanks
> > 
> > Jason,
> > 
> > It should be 
> > +	WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT);
> 
> With no return you will arm the setting of the timer and subsequently get an error from the modify_qp later.

The addition of WARN_ON() means that this is programmer error to get
such input. Historically, in-kernel API doesn't need to have protection
from other kernel developers.

Thanks

> 
> 
> Håkon
> 
> > 
> > and not
> > +	if (WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT))
> > +		return -EINVAL;
> > 
> > Thanks
> > 
> >> 
> >> Jason
> 

Re: [PATCH for-next v3 0/2] Introduce rdma_set_min_rnr_timer() and use it in RDS

[Jason Gunthorpe]

On Tue, Apr 13, 2021 at 09:29:41AM +0300, Leon Romanovsky wrote:
> On Mon, Apr 12, 2021 at 07:58:47PM -0300, Jason Gunthorpe wrote:
> > On Wed, Mar 31, 2021 at 08:43:12PM +0200, Håkon Bugge wrote:
> > > ib_modify_qp() is an expensive operation on some HCAs running
> > > virtualized. This series removes two ib_modify_qp() calls from RDS.
> > > 
> > > I am sending this as a v3, even though it is the first sent to
> > > net. This because the IB Core commit has reach v3.
> > > 
> > > Håkon Bugge (2):
> > >   IB/cma: Introduce rdma_set_min_rnr_timer()
> > >   rds: ib: Remove two ib_modify_qp() calls
> > 
> > Applied to rdma for-next, thanks
> 
> Jason,
> 
> It should be 
> +	WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT);
> 
> and not
> +	if (WARN_ON(id->qp_type != IB_QPT_RC && id->qp_type != IB_QPT_XRC_TGT))
> +		return -EINVAL;

Unless we can completely remove the return code the if statement is a
reasonable way to use the WARN_ON here

Jason