[PATCH v2] IB/ipoib: improve latency in ipoib/cm connection formation

[Manjunath Patil <manjunath.b.patil@oracle.com>]

Currently ipoib connected mode queries the device[HCA] to get pkey table
entry during connection formation. This will increase the time taken to
form the connection, especially when limited pkeys are in use.  This
gets worse when multiple connection attempts are done in parallel.

Since ipoib interfaces are locked to a single pkey, use the pkey index
that was determined at link up time instead of searching anything.

This improved the latency from 500ms to 1ms on an internal setup.

Suggested-by: Jason Gunthorpe <jgg@nvidia.com>
Signed-off-by: Manjunath Patil <manjunath.b.patil@oracle.com>
---
v2: v1 used the cached version[ib_find_cached_pkey()] to get the pkey table
entry. Following the Jason's comments for v1, I switched to pkey index that was
determined at link up time in v2.

 drivers/infiniband/ulp/ipoib/ipoib_cm.c |    6 +-----
 1 files changed, 1 insertions(+), 5 deletions(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index d5d592b..9dbc85a 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -1122,12 +1122,8 @@ static int ipoib_cm_modify_tx_init(struct net_device *dev,
 	struct ipoib_dev_priv *priv = ipoib_priv(dev);
 	struct ib_qp_attr qp_attr;
 	int qp_attr_mask, ret;
-	ret = ib_find_pkey(priv->ca, priv->port, priv->pkey, &qp_attr.pkey_index);
-	if (ret) {
-		ipoib_warn(priv, "pkey 0x%x not found: %d\n", priv->pkey, ret);
-		return ret;
-	}
 
+	qp_attr.pkey_index = priv->pkey_index;
 	qp_attr.qp_state = IB_QPS_INIT;
 	qp_attr.qp_access_flags = IB_ACCESS_LOCAL_WRITE;
 	qp_attr.port_num = priv->port;
-- 
1.7.1