[PATCH v2 00/11] IB/core: Add 32 bit LID support

[PATCH v2 00/11] IB/core: Add 32 bit LID support

[Dasaratharaman Chandramouli]

OPA devices can support more than 48K LIDs in the fabric. A node with a LID
greater than 0xbfff is called an 'extended lid'. In order to support verbs with
extended LIDs it is necessary to modify some of the RDMA data structures where
LIDs are currently only 16 bits in length.

This patch series follows on what was presented at the OFA Workshop.  Rather
than breaking the current UABI we propose to extend the LID address space by
sending a 'special' GID value down the verbs stack that has the 32-bit LID
programmed in it. By having a means to differentiate a regular GID from our
'special' GID, the underlying OPA device driver is able to retrieve the 32-bit
LIDs from the GID fields instead of picking them up from the 16 bit lid fields.

Internal to the kernel data structures such as struct ib_wc, struct
ib_port_attr and related ones have been modified to use 32 bit LID fields.
These changes are specific to the kernel and do not break the current UABI.


Node <-> SM interaction in getting extended LID information
----------------------------------------------------------------------------
1. Source application determines the GID of the destination through standard
   means and send a pathrecord query to the SM.
2. SM (which is OPA specific) recognizes that one or more nodes in the
   pathrecord request uses extended LIDs.
3. SM issues a pathrecord response. The SGID and DGID fields in the pathrecord
   response is the specially formulated GID.
4. Additionally, SM sets the hoplimit field of the pathrecord to 1.
5. Source receives the response and can determine the actual LID of the
   destination, if needed, from the response.

Source Node <-> Destination Node interaction in using extended LID information
-------------------------------------------------------------------------------
1. Source uses the pathrecord response from the SM to create an address handle
   to the destination (either at user or kernel space).
2. Since hoplimit field in the pathrecord is > 0, GRH fields are enabled in the
   address handle.
3. Address handle information is now passed down through the RDMA stack and
   reaches the driver.
4. Driver looks at the GRH fields in the address handle and determines that the
   GID in the GRH is actually a special GID.
5. Driver retrieves LID from GID field and uses 16B packets to send data
   on the wire.
6. Driver at the receiving side determines that a GRH needs to be added to the
   address handle before passing it on to the destination application.
7. Destination now receives the packet and can send back the response using the
   same address handle information.

There are some obvious limitations with this scheme:
----------------------------------------------------
1. Multicast packets which always need a GRH cannot use this scheme.
   Essentially multicast LIDs cannot be extended.
2. Subnet routed packets which also need a GRH cannot fully use this scheme.
   Specifically the LID of the router itself cannot be extended.
   The actual destination can still be extended.
3. Applications will need to use pathrecords to get destination address
   information. Any other out-of-band mechanisms are not guaranteed to work.
4. As an extension to 3, applications that 'validate' pathrecord responses need
   to be careful not to treat 0 LID field as an error condition.

Changes from V1:
1. Increase ah_attr.dlid from 16 to 32 bits

Dasaratharaman Chandramouli (9):
  IB/core: Add rdma_cap_opa_ah to expose opa address handles
  IB/core: Change port_attr.sm_lid from 16 to 32 bits
  IB/core: Change ah_attr.dlid from 16 to 32 bits
  IB/core: Change port_attr.lid size from 16 to 32 bits
  IB/mad: Change slid in RMPP recv from 16 to 32 bits
  IB/SA: Program extended LID in SM Address handle
  IB/IPoIB: Retrieve 32 bit LIDs from path records when running on OPA
    devices
  IB/IPoIB: Modify ipoib_get_net_dev_by_params to lookup gid table
  IB/srpt: Increase lid and sm_lid to 32 bits

Don Hiatt (2):
  IB/core: Change wc.slid from 16 to 32 bits
  IB/mad: Ensure DR MADs are correctly specified when using OPA devices

 drivers/infiniband/core/cm.c              |   4 +-
 drivers/infiniband/core/mad.c             | 104 ++++++++++++++++++++++++++----
 drivers/infiniband/core/mad_rmpp.c        |   2 +-
 drivers/infiniband/core/sa_query.c        |   8 ++-
 drivers/infiniband/core/user_mad.c        |   2 +-
 drivers/infiniband/core/uverbs_cmd.c      |  23 +++++--
 drivers/infiniband/core/uverbs_marshall.c |   2 +-
 drivers/infiniband/hw/hfi1/driver.c       |   4 +-
 drivers/infiniband/hw/hfi1/mad.c          |   2 +-
 drivers/infiniband/hw/hfi1/rc.c           |   2 +-
 drivers/infiniband/hw/hfi1/ruc.c          |  19 +++---
 drivers/infiniband/hw/hfi1/ud.c           |  10 +--
 drivers/infiniband/hw/hfi1/verbs.c        |   4 +-
 drivers/infiniband/hw/mlx4/ah.c           |   2 +-
 drivers/infiniband/hw/mlx4/alias_GUID.c   |   2 +-
 drivers/infiniband/hw/mlx4/mad.c          |   8 +--
 drivers/infiniband/hw/mlx4/qp.c           |   2 +-
 drivers/infiniband/hw/mlx5/ah.c           |   2 +-
 drivers/infiniband/hw/mlx5/mad.c          |   2 +-
 drivers/infiniband/hw/mthca/mthca_av.c    |   2 +-
 drivers/infiniband/hw/mthca/mthca_cmd.c   |   4 +-
 drivers/infiniband/hw/mthca/mthca_mad.c   |   4 +-
 drivers/infiniband/hw/mthca/mthca_qp.c    |   2 +-
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c  |   2 +-
 drivers/infiniband/hw/qib/qib_rc.c        |   2 +-
 drivers/infiniband/hw/qib/qib_ruc.c       |   9 +--
 drivers/infiniband/hw/qib/qib_ud.c        |   8 +--
 drivers/infiniband/sw/rdmavt/cq.c         |   2 +-
 drivers/infiniband/ulp/ipoib/ipoib.h      |   4 +-
 drivers/infiniband/ulp/ipoib/ipoib_cm.c   |  11 ++++
 drivers/infiniband/ulp/ipoib/ipoib_main.c |  63 +++++++++++++++++-
 drivers/infiniband/ulp/srpt/ib_srpt.h     |   4 +-
 include/rdma/ib_verbs.h                   |  29 +++++++--
 include/rdma/opa_addr.h                   |  68 +++++++++++++++++++
 34 files changed, 340 insertions(+), 78 deletions(-)
 create mode 100644 include/rdma/opa_addr.h

-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 01/11] IB/core: Add rdma_cap_opa_ah to expose opa address handles

[Dasaratharaman Chandramouli]

rdma_cap_opa_ah(..) enables core components to check if the
corresponding port supports extended addresses

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 include/rdma/ib_verbs.h | 21 +++++++++++++++++++++
 1 file changed, 21 insertions(+)

diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 5ad43a4..30fa96e 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -479,6 +479,7 @@ static inline struct rdma_hw_stats *rdma_alloc_hw_stats_struct(
 /* Address format                       0x000FF000 */
 #define RDMA_CORE_CAP_AF_IB             0x00001000
 #define RDMA_CORE_CAP_ETH_AH            0x00002000
+#define RDMA_CORE_CAP_OPA_AH            0x00004000
 
 /* Protocol                             0xFFF00000 */
 #define RDMA_CORE_CAP_PROT_IB           0x00100000
@@ -2472,6 +2473,26 @@ static inline bool rdma_cap_eth_ah(const struct ib_device *device, u8 port_num)
 }
 
 /**
+ * rdma_cap_opa_ah - Check if the port of device has the capability
+ * OPA Address handle
+ * @device: Device to check
+ * @port_num: Port number to check
+ *
+ * OPA Address handles enable use of 32 bit LIDs by using a specially
+ * formatted GID field to carry the LID. This check enables kernel
+ * components to identify such a scheme so that they can then try
+ * to make use of the LID in the GID field.
+ *
+ * Return: true if we are running as a OPA device which enables
+ * 32 bit LIDs to be used in the fabric.
+ */
+static inline bool rdma_cap_opa_ah(struct ib_device *device, u8 port_num)
+{
+	return (device->port_immutable[port_num].core_cap_flags &
+		RDMA_CORE_CAP_OPA_AH) == RDMA_CORE_CAP_OPA_AH;
+}
+
+/**
  * rdma_max_mad_size - Return the max MAD size required by this RDMA Port.
  *
  * @device: Device
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 02/11] IB/core: Change port_attr.sm_lid from 16 to 32 bits

[Dasaratharaman Chandramouli]

sm_lid field in port_attr is increased to 32 bits. This
enables core components to use the larger addresses if needed.
The user ABI is unchanged and return 16 bit values when queried.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/sa_query.c    |  2 +-
 drivers/infiniband/core/uverbs_cmd.c  |  6 +++++-
 drivers/infiniband/ulp/srpt/ib_srpt.c |  2 +-
 include/rdma/ib_verbs.h               |  2 +-
 include/rdma/opa_addr.h               | 38 +++++++++++++++++++++++++++++++++++
 5 files changed, 46 insertions(+), 4 deletions(-)
 create mode 100644 include/rdma/opa_addr.h

diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 81b742c..0b0dc43 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -958,7 +958,7 @@ static void update_sm_ah(struct work_struct *work)
 		pr_err("Couldn't find index for default PKey\n");
 
 	memset(&ah_attr, 0, sizeof ah_attr);
-	ah_attr.dlid     = port_attr.sm_lid;
+	ah_attr.dlid     = (u16)port_attr.sm_lid;
 	ah_attr.sl       = port_attr.sm_sl;
 	ah_attr.port_num = port->port_num;
 	if (port_attr.grh_required) {
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index cb3f515a..7630c92 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -39,6 +39,7 @@
 #include <linux/sched.h>
 
 #include <asm/uaccess.h>
+#include <rdma/opa_addr.h>
 
 #include "uverbs.h"
 #include "core_priv.h"
@@ -515,7 +516,10 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file *file,
 	resp.qkey_viol_cntr  = attr.qkey_viol_cntr;
 	resp.pkey_tbl_len    = attr.pkey_tbl_len;
 	resp.lid 	     = attr.lid;
-	resp.sm_lid 	     = attr.sm_lid;
+	if (rdma_cap_opa_ah(ib_dev, cmd.port_num))
+		resp.sm_lid  = OPA_TO_IB_UCAST_LID(attr.sm_lid);
+	else
+		resp.sm_lid  = (u16)attr.sm_lid;
 	resp.lmc 	     = attr.lmc;
 	resp.max_vl_num      = attr.max_vl_num;
 	resp.sm_sl 	     = attr.sm_sl;
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 0b1f69e..c6d0c47 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -514,7 +514,7 @@ static int srpt_refresh_port(struct srpt_port *sport)
 	if (ret)
 		goto err_query_port;
 
-	sport->sm_lid = port_attr.sm_lid;
+	sport->sm_lid = (u16)port_attr.sm_lid;
 	sport->lid = port_attr.lid;
 
 	ret = ib_query_gid(sport->sdev->device, sport->port, 0, &sport->gid,
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 30fa96e..52216f6 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -521,7 +521,7 @@ struct ib_port_attr {
 	u32			qkey_viol_cntr;
 	u16			pkey_tbl_len;
 	u16			lid;
-	u16			sm_lid;
+	u32			sm_lid;
 	u8			lmc;
 	u8			max_vl_num;
 	u8			sm_sl;
diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
new file mode 100644
index 0000000..142b327
--- /dev/null
+++ b/include/rdma/opa_addr.h
@@ -0,0 +1,38 @@
+/*
+ * Copyright (c) 2016 Intel Corporation.  All rights reserved.
+ *
+ * This software is available to you under a choice of one of two
+ * licenses.  You may choose to be licensed under the terms of the GNU
+ * General Public License (GPL) Version 2, available from the file
+ * COPYING in the main directory of this source tree, or the
+ * OpenIB.org BSD license below:
+ *
+ *     Redistribution and use in source and binary forms, with or
+ *     without modification, are permitted provided that the following
+ *     conditions are met:
+ *
+ *      - Redistributions of source code must retain the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer.
+ *
+ *      - Redistributions in binary form must reproduce the above
+ *        copyright notice, this list of conditions and the following
+ *        disclaimer in the documentation and/or other materials
+ *        provided with the distribution.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+ * EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+ * MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+ * NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS
+ * BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN
+ * ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN
+ * CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+ * SOFTWARE.
+ */
+
+#if !defined(OPA_ADDR_H)
+#define OPA_ADDR_H
+
+#define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
+				 ? 0 : x)
+#endif /* OPA_ADDR_H */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Dasaratharaman Chandramouli]

dlid field in ah_attr is increased to 32 bits. This
enables core components to use the larger addresses if needed.
The user ABI is unchanged and userspace applications can use
16 bit lids when creating and modifying address handles.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/mad_rmpp.c        |  2 +-
 drivers/infiniband/core/sa_query.c        |  2 +-
 drivers/infiniband/core/uverbs_cmd.c      | 11 +++++++++--
 drivers/infiniband/core/uverbs_marshall.c |  2 +-
 drivers/infiniband/hw/hfi1/driver.c       |  4 ++--
 drivers/infiniband/hw/hfi1/rc.c           |  4 ++--
 drivers/infiniband/hw/hfi1/ruc.c          | 21 +++++++++++----------
 drivers/infiniband/hw/hfi1/uc.c           |  2 +-
 drivers/infiniband/hw/hfi1/ud.c           | 10 +++++-----
 drivers/infiniband/hw/hfi1/verbs.c        |  4 ++--
 drivers/infiniband/hw/mlx4/ah.c           |  2 +-
 drivers/infiniband/hw/mlx4/mcg.c          |  2 +-
 drivers/infiniband/hw/mlx4/qp.c           |  2 +-
 drivers/infiniband/hw/mlx5/ah.c           |  2 +-
 drivers/infiniband/hw/mthca/mthca_av.c    |  2 +-
 drivers/infiniband/hw/mthca/mthca_qp.c    |  2 +-
 drivers/infiniband/hw/ocrdma/ocrdma_ah.c  |  2 +-
 drivers/infiniband/hw/qib/qib_rc.c        |  4 ++--
 drivers/infiniband/hw/qib/qib_ruc.c       | 11 ++++++-----
 drivers/infiniband/hw/qib/qib_uc.c        |  2 +-
 drivers/infiniband/hw/qib/qib_ud.c        |  8 ++++----
 include/rdma/ib_verbs.h                   |  2 +-
 22 files changed, 56 insertions(+), 47 deletions(-)

diff --git a/drivers/infiniband/core/mad_rmpp.c b/drivers/infiniband/core/mad_rmpp.c
index 382941b..c34dca3 100644
--- a/drivers/infiniband/core/mad_rmpp.c
+++ b/drivers/infiniband/core/mad_rmpp.c
@@ -870,7 +870,7 @@ static int init_newwin(struct ib_mad_send_wr_private *mad_send_wr)
 		if (ib_query_ah(mad_send_wr->send_buf.ah, &ah_attr))
 			continue;
 
-		if (rmpp_recv->slid == ah_attr.dlid) {
+		if (rmpp_recv->slid == (u16)ah_attr.dlid) {
 			newwin = rmpp_recv->repwin;
 			break;
 		}
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 0b0dc43..81b742c 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -958,7 +958,7 @@ static void update_sm_ah(struct work_struct *work)
 		pr_err("Couldn't find index for default PKey\n");
 
 	memset(&ah_attr, 0, sizeof ah_attr);
-	ah_attr.dlid     = (u16)port_attr.sm_lid;
+	ah_attr.dlid     = port_attr.sm_lid;
 	ah_attr.sl       = port_attr.sm_sl;
 	ah_attr.port_num = port->port_num;
 	if (port_attr.grh_required) {
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 7630c92..36f5bc1 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -2281,7 +2281,10 @@ ssize_t ib_uverbs_query_qp(struct ib_uverbs_file *file,
 	resp.dest.sgid_index        = attr->ah_attr.grh.sgid_index;
 	resp.dest.hop_limit         = attr->ah_attr.grh.hop_limit;
 	resp.dest.traffic_class     = attr->ah_attr.grh.traffic_class;
-	resp.dest.dlid              = attr->ah_attr.dlid;
+	if (rdma_cap_opa_ah(ib_dev, attr->ah_attr.port_num))
+		resp.dest.dlid	    = OPA_TO_IB_UCAST_LID(attr->ah_attr.dlid);
+	else
+		resp.dest.dlid	    = (u16)attr->ah_attr.dlid;
 	resp.dest.sl                = attr->ah_attr.sl;
 	resp.dest.src_path_bits     = attr->ah_attr.src_path_bits;
 	resp.dest.static_rate       = attr->ah_attr.static_rate;
@@ -2293,7 +2296,11 @@ ssize_t ib_uverbs_query_qp(struct ib_uverbs_file *file,
 	resp.alt_dest.sgid_index    = attr->alt_ah_attr.grh.sgid_index;
 	resp.alt_dest.hop_limit     = attr->alt_ah_attr.grh.hop_limit;
 	resp.alt_dest.traffic_class = attr->alt_ah_attr.grh.traffic_class;
-	resp.alt_dest.dlid          = attr->alt_ah_attr.dlid;
+	if (rdma_cap_opa_ah(ib_dev, attr->alt_ah_attr.port_num))
+		resp.alt_dest.dlid  =
+			OPA_TO_IB_UCAST_LID(attr->alt_ah_attr.dlid);
+	else
+		resp.alt_dest.dlid  = (u16)attr->alt_ah_attr.dlid;
 	resp.alt_dest.sl            = attr->alt_ah_attr.sl;
 	resp.alt_dest.src_path_bits = attr->alt_ah_attr.src_path_bits;
 	resp.alt_dest.static_rate   = attr->alt_ah_attr.static_rate;
diff --git a/drivers/infiniband/core/uverbs_marshall.c b/drivers/infiniband/core/uverbs_marshall.c
index af020f8..f8c9008 100644
--- a/drivers/infiniband/core/uverbs_marshall.c
+++ b/drivers/infiniband/core/uverbs_marshall.c
@@ -42,7 +42,7 @@ void ib_copy_ah_attr_to_user(struct ib_uverbs_ah_attr *dst,
 	dst->grh.hop_limit         = src->grh.hop_limit;
 	dst->grh.traffic_class     = src->grh.traffic_class;
 	memset(&dst->grh.reserved, 0, sizeof(dst->grh.reserved));
-	dst->dlid 	    	   = src->dlid;
+	dst->dlid		   = (u16)src->dlid;
 	dst->sl   	    	   = src->sl;
 	dst->src_path_bits 	   = src->src_path_bits;
 	dst->static_rate   	   = src->static_rate;
diff --git a/drivers/infiniband/hw/hfi1/driver.c b/drivers/infiniband/hw/hfi1/driver.c
index 6563e4d..4195ba8 100644
--- a/drivers/infiniband/hw/hfi1/driver.c
+++ b/drivers/infiniband/hw/hfi1/driver.c
@@ -473,12 +473,12 @@ void hfi1_process_ecn_slowpath(struct rvt_qp *qp, struct hfi1_packet *pkt,
 			(dlid != be16_to_cpu(IB_LID_PERMISSIVE));
 		break;
 	case IB_QPT_UC:
-		rlid = qp->remote_ah_attr.dlid;
+		rlid = (u16)qp->remote_ah_attr.dlid;
 		rqpn = qp->remote_qpn;
 		svc_type = IB_CC_SVCTYPE_UC;
 		break;
 	case IB_QPT_RC:
-		rlid = qp->remote_ah_attr.dlid;
+		rlid = (u16)qp->remote_ah_attr.dlid;
 		rqpn = qp->remote_qpn;
 		svc_type = IB_CC_SVCTYPE_RC;
 		break;
diff --git a/drivers/infiniband/hw/hfi1/rc.c b/drivers/infiniband/hw/hfi1/rc.c
index 8bc5013..caca6f5 100644
--- a/drivers/infiniband/hw/hfi1/rc.c
+++ b/drivers/infiniband/hw/hfi1/rc.c
@@ -890,7 +890,7 @@ void hfi1_send_rc_ack(struct hfi1_ctxtdata *rcd, struct rvt_qp *qp,
 	pbc_flags |= ((!!(sc5 & 0x10)) << PBC_DC_INFO_SHIFT);
 	lrh0 |= (sc5 & 0xf) << 12 | (qp->remote_ah_attr.sl & 0xf) << 4;
 	hdr.lrh[0] = cpu_to_be16(lrh0);
-	hdr.lrh[1] = cpu_to_be16(qp->remote_ah_attr.dlid);
+	hdr.lrh[1] = cpu_to_be16((u16)qp->remote_ah_attr.dlid);
 	hdr.lrh[2] = cpu_to_be16(hwords + SIZE_OF_CRC);
 	hdr.lrh[3] = cpu_to_be16(ppd->lid | qp->remote_ah_attr.src_path_bits);
 	ohdr->bth[0] = cpu_to_be32(bth0);
@@ -2306,7 +2306,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)
 			wc.opcode = IB_WC_RECV;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = qp->remote_ah_attr.dlid;
+		wc.slid = (u16)qp->remote_ah_attr.dlid;
 		/*
 		 * It seems that IB mandates the presence of an SL in a
 		 * work completion only for the UD transport (see section
diff --git a/drivers/infiniband/hw/hfi1/ruc.c b/drivers/infiniband/hw/hfi1/ruc.c
index a1576ae..2fe2b2f 100644
--- a/drivers/infiniband/hw/hfi1/ruc.c
+++ b/drivers/infiniband/hw/hfi1/ruc.c
@@ -297,7 +297,7 @@ int hfi1_ruc_check_hdr(struct hfi1_ibport *ibp, struct ib_header *hdr,
 			goto err;
 		}
 		/* Validate the SLID. See Ch. 9.6.1.5 and 17.2.8 */
-		if (be16_to_cpu(hdr->lrh[3]) != qp->alt_ah_attr.dlid ||
+		if (be16_to_cpu(hdr->lrh[3]) != (u16)qp->alt_ah_attr.dlid ||
 		    ppd_from_ibp(ibp)->port != qp->alt_ah_attr.port_num)
 			goto err;
 		spin_lock_irqsave(&qp->s_lock, flags);
@@ -332,7 +332,7 @@ int hfi1_ruc_check_hdr(struct hfi1_ibport *ibp, struct ib_header *hdr,
 			goto err;
 		}
 		/* Validate the SLID. See Ch. 9.6.1.5 */
-		if (be16_to_cpu(hdr->lrh[3]) != qp->remote_ah_attr.dlid ||
+		if (be16_to_cpu(hdr->lrh[3]) != (u16)qp->remote_ah_attr.dlid ||
 		    ppd_from_ibp(ibp)->port != qp->port_num)
 			goto err;
 		if (qp->s_mig_state == IB_MIG_REARM &&
@@ -591,7 +591,7 @@ static void ruc_loopback(struct rvt_qp *sqp)
 	wc.byte_len = wqe->length;
 	wc.qp = &qp->ibqp;
 	wc.src_qp = qp->remote_qpn;
-	wc.slid = qp->remote_ah_attr.dlid;
+	wc.slid = (u16)qp->remote_ah_attr.dlid;
 	wc.sl = qp->remote_ah_attr.sl;
 	wc.port_num = 1;
 	/* Signal completion event if the solicited bit is set. */
@@ -812,7 +812,8 @@ void hfi1_make_ruc_header(struct rvt_qp *qp, struct ib_other_headers *ohdr,
 	else
 		qp->s_flags &= ~RVT_S_AHG_VALID;
 	ps->s_txreq->phdr.hdr.lrh[0] = cpu_to_be16(lrh0);
-	ps->s_txreq->phdr.hdr.lrh[1] = cpu_to_be16(qp->remote_ah_attr.dlid);
+	ps->s_txreq->phdr.hdr.lrh[1] =
+		cpu_to_be16((u16)qp->remote_ah_attr.dlid);
 	ps->s_txreq->phdr.hdr.lrh[2] =
 		cpu_to_be16(qp->s_hdrwords + nwords + SIZE_OF_CRC);
 	ps->s_txreq->phdr.hdr.lrh[3] = cpu_to_be16(ppd_from_ibp(ibp)->lid |
@@ -864,9 +865,9 @@ void hfi1_do_send(struct rvt_qp *qp)
 
 	switch (qp->ibqp.qp_type) {
 	case IB_QPT_RC:
-		if (!loopback && ((qp->remote_ah_attr.dlid & ~((1 << ps.ppd->lmc
-								) - 1)) ==
-				 ps.ppd->lid)) {
+		if (!loopback && (((u16)qp->remote_ah_attr.dlid &
+				   ~((1 << ps.ppd->lmc) - 1)) ==
+				  ps.ppd->lid)) {
 			ruc_loopback(qp);
 			return;
 		}
@@ -874,9 +875,9 @@ void hfi1_do_send(struct rvt_qp *qp)
 		timeout_int = (qp->timeout_jiffies);
 		break;
 	case IB_QPT_UC:
-		if (!loopback && ((qp->remote_ah_attr.dlid & ~((1 << ps.ppd->lmc
-								) - 1)) ==
-				 ps.ppd->lid)) {
+		if (!loopback && (((u16)qp->remote_ah_attr.dlid &
+				   ~((1 << ps.ppd->lmc) - 1)) ==
+				  ps.ppd->lid)) {
 			ruc_loopback(qp);
 			return;
 		}
diff --git a/drivers/infiniband/hw/hfi1/uc.c b/drivers/infiniband/hw/hfi1/uc.c
index 5e6d1ba..0572dc7 100644
--- a/drivers/infiniband/hw/hfi1/uc.c
+++ b/drivers/infiniband/hw/hfi1/uc.c
@@ -451,7 +451,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet)
 		wc.status = IB_WC_SUCCESS;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = qp->remote_ah_attr.dlid;
+		wc.slid = (u16)qp->remote_ah_attr.dlid;
 		/*
 		 * It seems that IB mandates the presence of an SL in a
 		 * work completion only for the UD transport (see section
diff --git a/drivers/infiniband/hw/hfi1/ud.c b/drivers/infiniband/hw/hfi1/ud.c
index 97ae24b..8a27ab1 100644
--- a/drivers/infiniband/hw/hfi1/ud.c
+++ b/drivers/infiniband/hw/hfi1/ud.c
@@ -113,7 +113,7 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)
 			hfi1_bad_pqkey(ibp, OPA_TRAP_BAD_P_KEY, pkey,
 				       ah_attr->sl,
 				       sqp->ibqp.qp_num, qp->ibqp.qp_num,
-				       slid, ah_attr->dlid);
+				       slid, (u16)ah_attr->dlid);
 			goto drop;
 		}
 	}
@@ -137,7 +137,7 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)
 				       ah_attr->sl,
 				       sqp->ibqp.qp_num, qp->ibqp.qp_num,
 				       lid,
-				       ah_attr->dlid);
+				       (u16)ah_attr->dlid);
 			goto drop;
 		}
 	}
@@ -248,7 +248,7 @@ static void ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)
 	if (wc.slid == 0 && sqp->ibqp.qp_type == IB_QPT_GSI)
 		wc.slid = be16_to_cpu(IB_LID_PERMISSIVE);
 	wc.sl = ah_attr->sl;
-	wc.dlid_path_bits = ah_attr->dlid & ((1 << ppd->lmc) - 1);
+	wc.dlid_path_bits = (u16)ah_attr->dlid & ((1 << ppd->lmc) - 1);
 	wc.port_num = qp->port_num;
 	/* Signal completion event if the solicited bit is set. */
 	rvt_cq_enter(ibcq_to_rvtcq(qp->ibqp.recv_cq), &wc,
@@ -321,7 +321,7 @@ int hfi1_make_ud_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)
 	ah_attr = &ibah_to_rvtah(wqe->ud_wr.ah)->attr;
 	if (ah_attr->dlid < be16_to_cpu(IB_MULTICAST_LID_BASE) ||
 	    ah_attr->dlid == be16_to_cpu(IB_LID_PERMISSIVE)) {
-		lid = ah_attr->dlid & ~((1 << ppd->lmc) - 1);
+		lid = (u16)ah_attr->dlid & ~((1 << ppd->lmc) - 1);
 		if (unlikely(!loopback &&
 			     (lid == ppd->lid ||
 			      (lid == be16_to_cpu(IB_LID_PERMISSIVE) &&
@@ -402,7 +402,7 @@ int hfi1_make_ud_req(struct rvt_qp *qp, struct hfi1_pkt_state *ps)
 	priv->s_sendcontext = qp_to_send_context(qp, priv->s_sc);
 	ps->s_txreq->psc = priv->s_sendcontext;
 	ps->s_txreq->phdr.hdr.lrh[0] = cpu_to_be16(lrh0);
-	ps->s_txreq->phdr.hdr.lrh[1] = cpu_to_be16(ah_attr->dlid);
+	ps->s_txreq->phdr.hdr.lrh[1] = cpu_to_be16((u16)ah_attr->dlid);
 	ps->s_txreq->phdr.hdr.lrh[2] =
 		cpu_to_be16(qp->s_hdrwords + nwords + SIZE_OF_CRC);
 	if (ah_attr->dlid == be16_to_cpu(IB_LID_PERMISSIVE)) {
diff --git a/drivers/infiniband/hw/hfi1/verbs.c b/drivers/infiniband/hw/hfi1/verbs.c
index 4b7a16c..a13bfdf 100644
--- a/drivers/infiniband/hw/hfi1/verbs.c
+++ b/drivers/infiniband/hw/hfi1/verbs.c
@@ -1781,12 +1781,12 @@ void hfi1_cnp_rcv(struct hfi1_packet *packet)
 
 	switch (packet->qp->ibqp.qp_type) {
 	case IB_QPT_UC:
-		rlid = qp->remote_ah_attr.dlid;
+		rlid = (u16)qp->remote_ah_attr.dlid;
 		rqpn = qp->remote_qpn;
 		svc_type = IB_CC_SVCTYPE_UC;
 		break;
 	case IB_QPT_RC:
-		rlid = qp->remote_ah_attr.dlid;
+		rlid = (u16)qp->remote_ah_attr.dlid;
 		rqpn = qp->remote_qpn;
 		svc_type = IB_CC_SVCTYPE_RC;
 		break;
diff --git a/drivers/infiniband/hw/mlx4/ah.c b/drivers/infiniband/hw/mlx4/ah.c
index 5fc6233..772789f 100644
--- a/drivers/infiniband/hw/mlx4/ah.c
+++ b/drivers/infiniband/hw/mlx4/ah.c
@@ -58,7 +58,7 @@ static struct ib_ah *create_ib_ah(struct ib_pd *pd, struct ib_ah_attr *ah_attr,
 		memcpy(ah->av.ib.dgid, ah_attr->grh.dgid.raw, 16);
 	}
 
-	ah->av.ib.dlid    = cpu_to_be16(ah_attr->dlid);
+	ah->av.ib.dlid    = cpu_to_be16((u16)ah_attr->dlid);
 	if (ah_attr->static_rate) {
 		ah->av.ib.stat_rate = ah_attr->static_rate + MLX4_STAT_RATE_OFFSET;
 		while (ah->av.ib.stat_rate > IB_RATE_2_5_GBPS + MLX4_STAT_RATE_OFFSET &&
diff --git a/drivers/infiniband/hw/mlx4/mcg.c b/drivers/infiniband/hw/mlx4/mcg.c
index a21d37f..d46a847 100644
--- a/drivers/infiniband/hw/mlx4/mcg.c
+++ b/drivers/infiniband/hw/mlx4/mcg.c
@@ -244,7 +244,7 @@ static int send_mad_to_slave(int slave, struct mlx4_ib_demux_ctx *ctx,
 	wc.sl = 0;
 	wc.dlid_path_bits = 0;
 	wc.port_num = ctx->port;
-	wc.slid = ah_attr.dlid;  /* opensm lid */
+	wc.slid = (u16)ah_attr.dlid;  /* opensm lid */
 	wc.src_qp = 1;
 	return mlx4_ib_send_to_slave(dev, slave, ctx->port, IB_QPT_GSI, &wc, NULL, mad);
 }
diff --git a/drivers/infiniband/hw/mlx4/qp.c b/drivers/infiniband/hw/mlx4/qp.c
index 570bc86..ada1ecc 100644
--- a/drivers/infiniband/hw/mlx4/qp.c
+++ b/drivers/infiniband/hw/mlx4/qp.c
@@ -1396,7 +1396,7 @@ static int _mlx4_set_path(struct mlx4_ib_dev *dev, const struct ib_ah_attr *ah,
 
 
 	path->grh_mylmc     = ah->src_path_bits & 0x7f;
-	path->rlid	    = cpu_to_be16(ah->dlid);
+	path->rlid	    = cpu_to_be16((u16)ah->dlid);
 	if (ah->static_rate) {
 		path->static_rate = ah->static_rate + MLX4_STAT_RATE_OFFSET;
 		while (path->static_rate > IB_RATE_2_5_GBPS + MLX4_STAT_RATE_OFFSET &&
diff --git a/drivers/infiniband/hw/mlx5/ah.c b/drivers/infiniband/hw/mlx5/ah.c
index 745efa4..6248542 100644
--- a/drivers/infiniband/hw/mlx5/ah.c
+++ b/drivers/infiniband/hw/mlx5/ah.c
@@ -56,7 +56,7 @@ static struct ib_ah *create_ib_ah(struct mlx5_ib_dev *dev,
 						ah_attr->grh.sgid_index);
 		ah->av.stat_rate_sl |= (ah_attr->sl & 0x7) << 1;
 	} else {
-		ah->av.rlid = cpu_to_be16(ah_attr->dlid);
+		ah->av.rlid = cpu_to_be16((u16)ah_attr->dlid);
 		ah->av.fl_mlid = ah_attr->src_path_bits & 0x7f;
 		ah->av.stat_rate_sl |= (ah_attr->sl & 0xf);
 	}
diff --git a/drivers/infiniband/hw/mthca/mthca_av.c b/drivers/infiniband/hw/mthca/mthca_av.c
index bcac294..9e0c5c8 100644
--- a/drivers/infiniband/hw/mthca/mthca_av.c
+++ b/drivers/infiniband/hw/mthca/mthca_av.c
@@ -200,7 +200,7 @@ int mthca_create_ah(struct mthca_dev *dev,
 
 	av->port_pd = cpu_to_be32(pd->pd_num | (ah_attr->port_num << 24));
 	av->g_slid  = ah_attr->src_path_bits;
-	av->dlid    = cpu_to_be16(ah_attr->dlid);
+	av->dlid    = cpu_to_be16((u16)ah_attr->dlid);
 	av->msg_sr  = (3 << 4) | /* 2K message */
 		mthca_get_rate(dev, ah_attr->static_rate, ah_attr->port_num);
 	av->sl_tclass_flowlabel = cpu_to_be32(ah_attr->sl << 28);
diff --git a/drivers/infiniband/hw/mthca/mthca_qp.c b/drivers/infiniband/hw/mthca/mthca_qp.c
index 96e5fb9..32d000f 100644
--- a/drivers/infiniband/hw/mthca/mthca_qp.c
+++ b/drivers/infiniband/hw/mthca/mthca_qp.c
@@ -516,7 +516,7 @@ static int mthca_path_set(struct mthca_dev *dev, const struct ib_ah_attr *ah,
 			  struct mthca_qp_path *path, u8 port)
 {
 	path->g_mylmc     = ah->src_path_bits & 0x7f;
-	path->rlid        = cpu_to_be16(ah->dlid);
+	path->rlid        = cpu_to_be16((u16)ah->dlid);
 	path->static_rate = mthca_get_rate(dev, ah->static_rate, port);
 
 	if (ah->ah_flags & IB_AH_GRH) {
diff --git a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
index 797362a..ea90304 100644
--- a/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
+++ b/drivers/infiniband/hw/ocrdma/ocrdma_ah.c
@@ -215,7 +215,7 @@ struct ib_ah *ocrdma_create_ah(struct ib_pd *ibpd, struct ib_ah_attr *attr)
 
 	/* if pd is for the user process, pass the ah_id to user space */
 	if ((pd->uctx) && (pd->uctx->ah_tbl.va)) {
-		ahid_addr = pd->uctx->ah_tbl.va + attr->dlid;
+		ahid_addr = pd->uctx->ah_tbl.va + (u16)attr->dlid;
 		*ahid_addr = 0;
 		*ahid_addr |= ah->id & OCRDMA_AH_ID_MASK;
 		if (ocrdma_is_udp_encap_supported(dev)) {
diff --git a/drivers/infiniband/hw/qib/qib_rc.c b/drivers/infiniband/hw/qib/qib_rc.c
index 2097512..91f0d08 100644
--- a/drivers/infiniband/hw/qib/qib_rc.c
+++ b/drivers/infiniband/hw/qib/qib_rc.c
@@ -665,7 +665,7 @@ void qib_send_rc_ack(struct rvt_qp *qp)
 	lrh0 |= ibp->sl_to_vl[qp->remote_ah_attr.sl] << 12 |
 		qp->remote_ah_attr.sl << 4;
 	hdr.lrh[0] = cpu_to_be16(lrh0);
-	hdr.lrh[1] = cpu_to_be16(qp->remote_ah_attr.dlid);
+	hdr.lrh[1] = cpu_to_be16((u16)qp->remote_ah_attr.dlid);
 	hdr.lrh[2] = cpu_to_be16(hwords + SIZE_OF_CRC);
 	hdr.lrh[3] = cpu_to_be16(ppd->lid | qp->remote_ah_attr.src_path_bits);
 	ohdr->bth[0] = cpu_to_be32(bth0);
@@ -2018,7 +2018,7 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr,
 			wc.opcode = IB_WC_RECV;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = qp->remote_ah_attr.dlid;
+		wc.slid = (u16)qp->remote_ah_attr.dlid;
 		wc.sl = qp->remote_ah_attr.sl;
 		/* zero fields that are N/A */
 		wc.vendor_err = 0;
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c b/drivers/infiniband/hw/qib/qib_ruc.c
index de1bde5..0b0620f 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -297,7 +297,7 @@ int qib_ruc_check_hdr(struct qib_ibport *ibp, struct ib_header *hdr,
 			goto err;
 		}
 		/* Validate the SLID. See Ch. 9.6.1.5 and 17.2.8 */
-		if (be16_to_cpu(hdr->lrh[3]) != qp->alt_ah_attr.dlid ||
+		if (be16_to_cpu(hdr->lrh[3]) != (u16)qp->alt_ah_attr.dlid ||
 		    ppd_from_ibp(ibp)->port != qp->alt_ah_attr.port_num)
 			goto err;
 		spin_lock_irqsave(&qp->s_lock, flags);
@@ -330,7 +330,7 @@ int qib_ruc_check_hdr(struct qib_ibport *ibp, struct ib_header *hdr,
 			goto err;
 		}
 		/* Validate the SLID. See Ch. 9.6.1.5 */
-		if (be16_to_cpu(hdr->lrh[3]) != qp->remote_ah_attr.dlid ||
+		if (be16_to_cpu(hdr->lrh[3]) != (u16)qp->remote_ah_attr.dlid ||
 		    ppd_from_ibp(ibp)->port != qp->port_num)
 			goto err;
 		if (qp->s_mig_state == IB_MIG_REARM &&
@@ -566,7 +566,7 @@ static void qib_ruc_loopback(struct rvt_qp *sqp)
 	wc.byte_len = wqe->length;
 	wc.qp = &qp->ibqp;
 	wc.src_qp = qp->remote_qpn;
-	wc.slid = qp->remote_ah_attr.dlid;
+	wc.slid = (u16)qp->remote_ah_attr.dlid;
 	wc.sl = qp->remote_ah_attr.sl;
 	wc.port_num = 1;
 	/* Signal completion event if the solicited bit is set. */
@@ -702,7 +702,7 @@ void qib_make_ruc_header(struct rvt_qp *qp, struct ib_other_headers *ohdr,
 	lrh0 |= ibp->sl_to_vl[qp->remote_ah_attr.sl] << 12 |
 		qp->remote_ah_attr.sl << 4;
 	priv->s_hdr->lrh[0] = cpu_to_be16(lrh0);
-	priv->s_hdr->lrh[1] = cpu_to_be16(qp->remote_ah_attr.dlid);
+	priv->s_hdr->lrh[1] = cpu_to_be16((u16)qp->remote_ah_attr.dlid);
 	priv->s_hdr->lrh[2] =
 			cpu_to_be16(qp->s_hdrwords + nwords + SIZE_OF_CRC);
 	priv->s_hdr->lrh[3] = cpu_to_be16(ppd_from_ibp(ibp)->lid |
@@ -744,7 +744,8 @@ void qib_do_send(struct rvt_qp *qp)
 
 	if ((qp->ibqp.qp_type == IB_QPT_RC ||
 	     qp->ibqp.qp_type == IB_QPT_UC) &&
-	    (qp->remote_ah_attr.dlid & ~((1 << ppd->lmc) - 1)) == ppd->lid) {
+	    (((u16)qp->remote_ah_attr.dlid) & ~((1 << ppd->lmc) - 1))
+	    == ppd->lid) {
 		qib_ruc_loopback(qp);
 		return;
 	}
diff --git a/drivers/infiniband/hw/qib/qib_uc.c b/drivers/infiniband/hw/qib/qib_uc.c
index 5b2d483..7a10748 100644
--- a/drivers/infiniband/hw/qib/qib_uc.c
+++ b/drivers/infiniband/hw/qib/qib_uc.c
@@ -403,7 +403,7 @@ void qib_uc_rcv(struct qib_ibport *ibp, struct ib_header *hdr,
 		wc.status = IB_WC_SUCCESS;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = qp->remote_ah_attr.dlid;
+		wc.slid = (u16)qp->remote_ah_attr.dlid;
 		wc.sl = qp->remote_ah_attr.sl;
 		/* zero fields that are N/A */
 		wc.vendor_err = 0;
diff --git a/drivers/infiniband/hw/qib/qib_ud.c b/drivers/infiniband/hw/qib/qib_ud.c
index f45cad1..37cd128 100644
--- a/drivers/infiniband/hw/qib/qib_ud.c
+++ b/drivers/infiniband/hw/qib/qib_ud.c
@@ -98,7 +98,7 @@ static void qib_ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)
 				      ah_attr->sl,
 				      sqp->ibqp.qp_num, qp->ibqp.qp_num,
 				      cpu_to_be16(lid),
-				      cpu_to_be16(ah_attr->dlid));
+				      cpu_to_be16((u16)ah_attr->dlid));
 			goto drop;
 		}
 	}
@@ -122,7 +122,7 @@ static void qib_ud_loopback(struct rvt_qp *sqp, struct rvt_swqe *swqe)
 				      ah_attr->sl,
 				      sqp->ibqp.qp_num, qp->ibqp.qp_num,
 				      cpu_to_be16(lid),
-				      cpu_to_be16(ah_attr->dlid));
+				      cpu_to_be16((u16)ah_attr->dlid));
 			goto drop;
 		}
 	}
@@ -296,7 +296,7 @@ int qib_make_ud_req(struct rvt_qp *qp, unsigned long *flags)
 			this_cpu_inc(ibp->pmastats->n_unicast_xmit);
 	} else {
 		this_cpu_inc(ibp->pmastats->n_unicast_xmit);
-		lid = ah_attr->dlid & ~((1 << ppd->lmc) - 1);
+		lid = ((u16)ah_attr->dlid) & ~((1 << ppd->lmc) - 1);
 		if (unlikely(lid == ppd->lid)) {
 			unsigned long tflags = *flags;
 			/*
@@ -363,7 +363,7 @@ int qib_make_ud_req(struct rvt_qp *qp, unsigned long *flags)
 	else
 		lrh0 |= ibp->sl_to_vl[ah_attr->sl] << 12;
 	priv->s_hdr->lrh[0] = cpu_to_be16(lrh0);
-	priv->s_hdr->lrh[1] = cpu_to_be16(ah_attr->dlid);  /* DEST LID */
+	priv->s_hdr->lrh[1] = cpu_to_be16((u16)ah_attr->dlid);  /* DEST LID */
 	priv->s_hdr->lrh[2] =
 			cpu_to_be16(qp->s_hdrwords + nwords + SIZE_OF_CRC);
 	lid = ppd->lid;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 52216f6..294d3ed 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -814,7 +814,7 @@ struct ib_mr_status {
 
 struct ib_ah_attr {
 	struct ib_global_route	grh;
-	u16			dlid;
+	u32			dlid;
 	u8			sl;
 	u8			src_path_bits;
 	u8			static_rate;
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 04/11] IB/core: Change port_attr.lid size from 16 to 32 bits

[Dasaratharaman Chandramouli]

lid field in port_attr is increased to 32 bits. This enables core
components to use the larger addresses if needed.
The user ABI is unchanged and return 16 bit values when queried.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/uverbs_cmd.c           | 8 +++++---
 drivers/infiniband/hw/mlx4/alias_GUID.c        | 2 +-
 drivers/infiniband/hw/mlx4/mad.c               | 2 +-
 drivers/infiniband/hw/mthca/mthca_mad.c        | 2 +-
 drivers/infiniband/ulp/ipoib/ipoib_multicast.c | 2 +-
 drivers/infiniband/ulp/srpt/ib_srpt.c          | 2 +-
 include/rdma/ib_verbs.h                        | 2 +-
 7 files changed, 11 insertions(+), 9 deletions(-)

diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index 36f5bc1..e135c08 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -515,11 +515,13 @@ ssize_t ib_uverbs_query_port(struct ib_uverbs_file *file,
 	resp.bad_pkey_cntr   = attr.bad_pkey_cntr;
 	resp.qkey_viol_cntr  = attr.qkey_viol_cntr;
 	resp.pkey_tbl_len    = attr.pkey_tbl_len;
-	resp.lid 	     = attr.lid;
-	if (rdma_cap_opa_ah(ib_dev, cmd.port_num))
+	if (rdma_cap_opa_ah(ib_dev, cmd.port_num)) {
 		resp.sm_lid  = OPA_TO_IB_UCAST_LID(attr.sm_lid);
-	else
+		resp.lid  = OPA_TO_IB_UCAST_LID(attr.lid);
+	} else {
 		resp.sm_lid  = (u16)attr.sm_lid;
+		resp.lid     = (u16)attr.lid;
+	}
 	resp.lmc 	     = attr.lmc;
 	resp.max_vl_num      = attr.max_vl_num;
 	resp.sm_sl 	     = attr.sm_sl;
diff --git a/drivers/infiniband/hw/mlx4/alias_GUID.c b/drivers/infiniband/hw/mlx4/alias_GUID.c
index 5e99390..7fa64e6 100644
--- a/drivers/infiniband/hw/mlx4/alias_GUID.c
+++ b/drivers/infiniband/hw/mlx4/alias_GUID.c
@@ -527,7 +527,7 @@ static int set_guid_rec(struct ib_device *ibdev,
 
 	memset(&guid_info_rec, 0, sizeof (struct ib_sa_guidinfo_rec));
 
-	guid_info_rec.lid = cpu_to_be16(attr.lid);
+	guid_info_rec.lid = cpu_to_be16((u16)attr.lid);
 	guid_info_rec.block_num = index;
 
 	memcpy(guid_info_rec.guid_info_list, rec_det->all_recs,
diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index 1672907..404ec4e 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -821,7 +821,7 @@ static int ib_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 	    in_mad->mad_hdr.method == IB_MGMT_METHOD_SET &&
 	    in_mad->mad_hdr.attr_id == IB_SMP_ATTR_PORT_INFO &&
 	    !ib_query_port(ibdev, port_num, &pattr))
-		prev_lid = pattr.lid;
+		prev_lid = (u16)pattr.lid;
 
 	err = mlx4_MAD_IFC(to_mdev(ibdev),
 			   (mad_flags & IB_MAD_IGNORE_MKEY ? MLX4_MAD_IFC_IGNORE_MKEY : 0) |
diff --git a/drivers/infiniband/hw/mthca/mthca_mad.c b/drivers/infiniband/hw/mthca/mthca_mad.c
index 9139405..b503d160 100644
--- a/drivers/infiniband/hw/mthca/mthca_mad.c
+++ b/drivers/infiniband/hw/mthca/mthca_mad.c
@@ -255,7 +255,7 @@ int mthca_process_mad(struct ib_device *ibdev,
 	    in_mad->mad_hdr.method == IB_MGMT_METHOD_SET &&
 	    in_mad->mad_hdr.attr_id == IB_SMP_ATTR_PORT_INFO &&
 	    !ib_query_port(ibdev, port_num, &pattr))
-		prev_lid = pattr.lid;
+		prev_lid = (u16)pattr.lid;
 
 	err = mthca_MAD_IFC(to_mdev(ibdev),
 			    mad_flags & IB_MAD_IGNORE_MKEY,
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index d3394b6..bff73b5 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -581,7 +581,7 @@ void ipoib_mcast_join_task(struct work_struct *work)
 			  port_attr.state);
 		return;
 	}
-	priv->local_lid = port_attr.lid;
+	priv->local_lid = (u16)port_attr.lid;
 	netif_addr_lock_bh(dev);
 
 	if (!test_bit(IPOIB_FLAG_DEV_ADDR_SET, &priv->flags)) {
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index c6d0c47..4dc66ba 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -515,7 +515,7 @@ static int srpt_refresh_port(struct srpt_port *sport)
 		goto err_query_port;
 
 	sport->sm_lid = (u16)port_attr.sm_lid;
-	sport->lid = port_attr.lid;
+	sport->lid = (u16)port_attr.lid;
 
 	ret = ib_query_gid(sport->sdev->device, sport->port, 0, &sport->gid,
 			   NULL);
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 294d3ed..3d80720 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -520,7 +520,7 @@ struct ib_port_attr {
 	u32			bad_pkey_cntr;
 	u32			qkey_viol_cntr;
 	u16			pkey_tbl_len;
-	u16			lid;
+	u32			lid;
 	u32			sm_lid;
 	u8			lmc;
 	u8			max_vl_num;
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 05/11] IB/core: Change wc.slid from 16 to 32 bits

[Dasaratharaman Chandramouli]

From: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

lid field in ib_wc is increased to 32 bits. This enables core
components to use the larger addresses if needed.
The user ABI is unchanged and return 16 bit values when queried.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/cm.c            | 4 ++--
 drivers/infiniband/core/mad_rmpp.c      | 4 ++--
 drivers/infiniband/core/user_mad.c      | 2 +-
 drivers/infiniband/core/uverbs_cmd.c    | 2 +-
 drivers/infiniband/hw/hfi1/mad.c        | 2 +-
 drivers/infiniband/hw/hfi1/rc.c         | 2 +-
 drivers/infiniband/hw/hfi1/ruc.c        | 2 +-
 drivers/infiniband/hw/hfi1/uc.c         | 2 +-
 drivers/infiniband/hw/mlx4/mad.c        | 6 +++---
 drivers/infiniband/hw/mlx4/mcg.c        | 2 +-
 drivers/infiniband/hw/mlx5/mad.c        | 2 +-
 drivers/infiniband/hw/mthca/mthca_cmd.c | 4 ++--
 drivers/infiniband/hw/mthca/mthca_mad.c | 2 +-
 drivers/infiniband/hw/qib/qib_rc.c      | 2 +-
 drivers/infiniband/hw/qib/qib_ruc.c     | 2 +-
 drivers/infiniband/hw/qib/qib_uc.c      | 2 +-
 drivers/infiniband/sw/rdmavt/cq.c       | 2 +-
 include/rdma/ib_verbs.h                 | 2 +-
 18 files changed, 23 insertions(+), 23 deletions(-)

diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index c995255..137c4c2 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -1576,7 +1576,7 @@ static void cm_process_routed_req(struct cm_req_msg *req_msg, struct ib_wc *wc)
 {
 	if (!cm_req_get_primary_subnet_local(req_msg)) {
 		if (req_msg->primary_local_lid == IB_LID_PERMISSIVE) {
-			req_msg->primary_local_lid = cpu_to_be16(wc->slid);
+			req_msg->primary_local_lid = cpu_to_be16((u16)wc->slid);
 			cm_req_set_primary_sl(req_msg, wc->sl);
 		}
 
@@ -1586,7 +1586,7 @@ static void cm_process_routed_req(struct cm_req_msg *req_msg, struct ib_wc *wc)
 
 	if (!cm_req_get_alt_subnet_local(req_msg)) {
 		if (req_msg->alt_local_lid == IB_LID_PERMISSIVE) {
-			req_msg->alt_local_lid = cpu_to_be16(wc->slid);
+			req_msg->alt_local_lid = cpu_to_be16((u16)wc->slid);
 			cm_req_set_alt_sl(req_msg, wc->sl);
 		}
 
diff --git a/drivers/infiniband/core/mad_rmpp.c b/drivers/infiniband/core/mad_rmpp.c
index c34dca3..8a076e1 100644
--- a/drivers/infiniband/core/mad_rmpp.c
+++ b/drivers/infiniband/core/mad_rmpp.c
@@ -316,7 +316,7 @@ static void recv_cleanup_handler(struct work_struct *work)
 	mad_hdr = &mad_recv_wc->recv_buf.mad->mad_hdr;
 	rmpp_recv->tid = mad_hdr->tid;
 	rmpp_recv->src_qp = mad_recv_wc->wc->src_qp;
-	rmpp_recv->slid = mad_recv_wc->wc->slid;
+	rmpp_recv->slid = (u16)mad_recv_wc->wc->slid;
 	rmpp_recv->mgmt_class = mad_hdr->mgmt_class;
 	rmpp_recv->class_version = mad_hdr->class_version;
 	rmpp_recv->method  = mad_hdr->method;
@@ -337,7 +337,7 @@ static void recv_cleanup_handler(struct work_struct *work)
 	list_for_each_entry(rmpp_recv, &agent->rmpp_list, list) {
 		if (rmpp_recv->tid == mad_hdr->tid &&
 		    rmpp_recv->src_qp == mad_recv_wc->wc->src_qp &&
-		    rmpp_recv->slid == mad_recv_wc->wc->slid &&
+		    rmpp_recv->slid == (u16)mad_recv_wc->wc->slid &&
 		    rmpp_recv->mgmt_class == mad_hdr->mgmt_class &&
 		    rmpp_recv->class_version == mad_hdr->class_version &&
 		    rmpp_recv->method == mad_hdr->method)
diff --git a/drivers/infiniband/core/user_mad.c b/drivers/infiniband/core/user_mad.c
index 415a318..2a0b928 100644
--- a/drivers/infiniband/core/user_mad.c
+++ b/drivers/infiniband/core/user_mad.c
@@ -229,7 +229,7 @@ static void recv_handler(struct ib_mad_agent *agent,
 	packet->mad.hdr.status	   = 0;
 	packet->mad.hdr.length	   = hdr_size(file) + mad_recv_wc->mad_len;
 	packet->mad.hdr.qpn	   = cpu_to_be32(mad_recv_wc->wc->src_qp);
-	packet->mad.hdr.lid	   = cpu_to_be16(mad_recv_wc->wc->slid);
+	packet->mad.hdr.lid	   = cpu_to_be16((u16)mad_recv_wc->wc->slid);
 	packet->mad.hdr.sl	   = mad_recv_wc->wc->sl;
 	packet->mad.hdr.path_bits  = mad_recv_wc->wc->dlid_path_bits;
 	packet->mad.hdr.pkey_index = mad_recv_wc->wc->pkey_index;
diff --git a/drivers/infiniband/core/uverbs_cmd.c b/drivers/infiniband/core/uverbs_cmd.c
index e135c08..a203cf2 100644
--- a/drivers/infiniband/core/uverbs_cmd.c
+++ b/drivers/infiniband/core/uverbs_cmd.c
@@ -1619,7 +1619,7 @@ static int copy_wc_to_user(void __user *dest, struct ib_wc *wc)
 	tmp.src_qp		= wc->src_qp;
 	tmp.wc_flags		= wc->wc_flags;
 	tmp.pkey_index		= wc->pkey_index;
-	tmp.slid		= wc->slid;
+	tmp.slid		= (u16)wc->slid;
 	tmp.sl			= wc->sl;
 	tmp.dlid_path_bits	= wc->dlid_path_bits;
 	tmp.port_num		= wc->port_num;
diff --git a/drivers/infiniband/hw/hfi1/mad.c b/drivers/infiniband/hw/hfi1/mad.c
index 9487c9b..c76e546 100644
--- a/drivers/infiniband/hw/hfi1/mad.c
+++ b/drivers/infiniband/hw/hfi1/mad.c
@@ -3974,7 +3974,7 @@ static int opa_local_smp_check(struct hfi1_ibport *ibp,
 			       const struct ib_wc *in_wc)
 {
 	struct hfi1_pportdata *ppd = ppd_from_ibp(ibp);
-	u16 slid = in_wc->slid;
+	u16 slid = (u16)in_wc->slid;
 	u16 pkey;
 
 	if (in_wc->pkey_index >= ARRAY_SIZE(ppd->pkeys))
diff --git a/drivers/infiniband/hw/hfi1/rc.c b/drivers/infiniband/hw/hfi1/rc.c
index caca6f5..40ae502 100644
--- a/drivers/infiniband/hw/hfi1/rc.c
+++ b/drivers/infiniband/hw/hfi1/rc.c
@@ -2306,7 +2306,7 @@ void hfi1_rc_rcv(struct hfi1_packet *packet)
 			wc.opcode = IB_WC_RECV;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = (u16)qp->remote_ah_attr.dlid;
+		wc.slid = qp->remote_ah_attr.dlid;
 		/*
 		 * It seems that IB mandates the presence of an SL in a
 		 * work completion only for the UD transport (see section
diff --git a/drivers/infiniband/hw/hfi1/ruc.c b/drivers/infiniband/hw/hfi1/ruc.c
index 2fe2b2f..777cfc8 100644
--- a/drivers/infiniband/hw/hfi1/ruc.c
+++ b/drivers/infiniband/hw/hfi1/ruc.c
@@ -591,7 +591,7 @@ static void ruc_loopback(struct rvt_qp *sqp)
 	wc.byte_len = wqe->length;
 	wc.qp = &qp->ibqp;
 	wc.src_qp = qp->remote_qpn;
-	wc.slid = (u16)qp->remote_ah_attr.dlid;
+	wc.slid = qp->remote_ah_attr.dlid;
 	wc.sl = qp->remote_ah_attr.sl;
 	wc.port_num = 1;
 	/* Signal completion event if the solicited bit is set. */
diff --git a/drivers/infiniband/hw/hfi1/uc.c b/drivers/infiniband/hw/hfi1/uc.c
index 0572dc7..5e6d1ba 100644
--- a/drivers/infiniband/hw/hfi1/uc.c
+++ b/drivers/infiniband/hw/hfi1/uc.c
@@ -451,7 +451,7 @@ void hfi1_uc_rcv(struct hfi1_packet *packet)
 		wc.status = IB_WC_SUCCESS;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = (u16)qp->remote_ah_attr.dlid;
+		wc.slid = qp->remote_ah_attr.dlid;
 		/*
 		 * It seems that IB mandates the presence of an SL in a
 		 * work completion only for the UD transport (see section
diff --git a/drivers/infiniband/hw/mlx4/mad.c b/drivers/infiniband/hw/mlx4/mad.c
index 404ec4e..8e50de0 100644
--- a/drivers/infiniband/hw/mlx4/mad.c
+++ b/drivers/infiniband/hw/mlx4/mad.c
@@ -167,7 +167,7 @@ int mlx4_MAD_IFC(struct mlx4_ib_dev *dev, int mad_ifc_flags,
 
 		op_modifier |= 0x4;
 
-		in_modifier |= in_wc->slid << 16;
+		in_modifier |= (u16)in_wc->slid << 16;
 	}
 
 	err = mlx4_cmd_box(dev->dev, inmailbox->dma, outmailbox->dma, in_modifier,
@@ -599,7 +599,7 @@ int mlx4_ib_send_to_slave(struct mlx4_ib_dev *dev, int slave, u8 port,
 		memcpy((char *)&tun_mad->hdr.slid_mac_47_32, &(wc->smac[4]), 2);
 	} else {
 		tun_mad->hdr.sl_vid = cpu_to_be16(((u16)(wc->sl)) << 12);
-		tun_mad->hdr.slid_mac_47_32 = cpu_to_be16(wc->slid);
+		tun_mad->hdr.slid_mac_47_32 = cpu_to_be16((u16)wc->slid);
 	}
 
 	ib_dma_sync_single_for_device(&dev->ib_dev,
@@ -787,7 +787,7 @@ static int ib_process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 		}
 	}
 
-	slid = in_wc ? in_wc->slid : be16_to_cpu(IB_LID_PERMISSIVE);
+	slid = in_wc ? (u16)in_wc->slid : be16_to_cpu(IB_LID_PERMISSIVE);
 
 	if (in_mad->mad_hdr.method == IB_MGMT_METHOD_TRAP && slid == 0) {
 		forward_trap(to_mdev(ibdev), port_num, in_mad);
diff --git a/drivers/infiniband/hw/mlx4/mcg.c b/drivers/infiniband/hw/mlx4/mcg.c
index d46a847..a21d37f 100644
--- a/drivers/infiniband/hw/mlx4/mcg.c
+++ b/drivers/infiniband/hw/mlx4/mcg.c
@@ -244,7 +244,7 @@ static int send_mad_to_slave(int slave, struct mlx4_ib_demux_ctx *ctx,
 	wc.sl = 0;
 	wc.dlid_path_bits = 0;
 	wc.port_num = ctx->port;
-	wc.slid = (u16)ah_attr.dlid;  /* opensm lid */
+	wc.slid = ah_attr.dlid;  /* opensm lid */
 	wc.src_qp = 1;
 	return mlx4_ib_send_to_slave(dev, slave, ctx->port, IB_QPT_GSI, &wc, NULL, mad);
 }
diff --git a/drivers/infiniband/hw/mlx5/mad.c b/drivers/infiniband/hw/mlx5/mad.c
index 39e5848..f0323b7 100644
--- a/drivers/infiniband/hw/mlx5/mad.c
+++ b/drivers/infiniband/hw/mlx5/mad.c
@@ -66,7 +66,7 @@ static int process_mad(struct ib_device *ibdev, int mad_flags, u8 port_num,
 	u16 slid;
 	int err;
 
-	slid = in_wc ? in_wc->slid : be16_to_cpu(IB_LID_PERMISSIVE);
+	slid = in_wc ? (u16)in_wc->slid : be16_to_cpu(IB_LID_PERMISSIVE);
 
 	if (in_mad->mad_hdr.method == IB_MGMT_METHOD_TRAP && slid == 0)
 		return IB_MAD_RESULT_SUCCESS | IB_MAD_RESULT_CONSUMED;
diff --git a/drivers/infiniband/hw/mthca/mthca_cmd.c b/drivers/infiniband/hw/mthca/mthca_cmd.c
index c7f49bb..d07f389 100644
--- a/drivers/infiniband/hw/mthca/mthca_cmd.c
+++ b/drivers/infiniband/hw/mthca/mthca_cmd.c
@@ -1913,7 +1913,7 @@ int mthca_MAD_IFC(struct mthca_dev *dev, int ignore_mkey, int ignore_bkey,
 			(in_wc->wc_flags & IB_WC_GRH ? 0x80 : 0);
 		MTHCA_PUT(inbox, val,               MAD_IFC_G_PATH_OFFSET);
 
-		MTHCA_PUT(inbox, in_wc->slid,       MAD_IFC_RLID_OFFSET);
+		MTHCA_PUT(inbox, (u16)in_wc->slid,       MAD_IFC_RLID_OFFSET);
 		MTHCA_PUT(inbox, in_wc->pkey_index, MAD_IFC_PKEY_OFFSET);
 
 		if (in_grh)
@@ -1921,7 +1921,7 @@ int mthca_MAD_IFC(struct mthca_dev *dev, int ignore_mkey, int ignore_bkey,
 
 		op_modifier |= 0x4;
 
-		in_modifier |= in_wc->slid << 16;
+		in_modifier |= (u16)in_wc->slid << 16;
 	}
 
 	err = mthca_cmd_box(dev, inmailbox->dma, outmailbox->dma,
diff --git a/drivers/infiniband/hw/mthca/mthca_mad.c b/drivers/infiniband/hw/mthca/mthca_mad.c
index b503d160..e9a7dd0 100644
--- a/drivers/infiniband/hw/mthca/mthca_mad.c
+++ b/drivers/infiniband/hw/mthca/mthca_mad.c
@@ -204,7 +204,7 @@ int mthca_process_mad(struct ib_device *ibdev,
 		      u16 *out_mad_pkey_index)
 {
 	int err;
-	u16 slid = in_wc ? in_wc->slid : be16_to_cpu(IB_LID_PERMISSIVE);
+	u16 slid = in_wc ? (u16)in_wc->slid : be16_to_cpu(IB_LID_PERMISSIVE);
 	u16 prev_lid = 0;
 	struct ib_port_attr pattr;
 	const struct ib_mad *in_mad = (const struct ib_mad *)in;
diff --git a/drivers/infiniband/hw/qib/qib_rc.c b/drivers/infiniband/hw/qib/qib_rc.c
index 91f0d08..9d0b2bc 100644
--- a/drivers/infiniband/hw/qib/qib_rc.c
+++ b/drivers/infiniband/hw/qib/qib_rc.c
@@ -2018,7 +2018,7 @@ void qib_rc_rcv(struct qib_ctxtdata *rcd, struct ib_header *hdr,
 			wc.opcode = IB_WC_RECV;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = (u16)qp->remote_ah_attr.dlid;
+		wc.slid = qp->remote_ah_attr.dlid;
 		wc.sl = qp->remote_ah_attr.sl;
 		/* zero fields that are N/A */
 		wc.vendor_err = 0;
diff --git a/drivers/infiniband/hw/qib/qib_ruc.c b/drivers/infiniband/hw/qib/qib_ruc.c
index 0b0620f..588b4ae 100644
--- a/drivers/infiniband/hw/qib/qib_ruc.c
+++ b/drivers/infiniband/hw/qib/qib_ruc.c
@@ -566,7 +566,7 @@ static void qib_ruc_loopback(struct rvt_qp *sqp)
 	wc.byte_len = wqe->length;
 	wc.qp = &qp->ibqp;
 	wc.src_qp = qp->remote_qpn;
-	wc.slid = (u16)qp->remote_ah_attr.dlid;
+	wc.slid = qp->remote_ah_attr.dlid;
 	wc.sl = qp->remote_ah_attr.sl;
 	wc.port_num = 1;
 	/* Signal completion event if the solicited bit is set. */
diff --git a/drivers/infiniband/hw/qib/qib_uc.c b/drivers/infiniband/hw/qib/qib_uc.c
index 7a10748..5b2d483 100644
--- a/drivers/infiniband/hw/qib/qib_uc.c
+++ b/drivers/infiniband/hw/qib/qib_uc.c
@@ -403,7 +403,7 @@ void qib_uc_rcv(struct qib_ibport *ibp, struct ib_header *hdr,
 		wc.status = IB_WC_SUCCESS;
 		wc.qp = &qp->ibqp;
 		wc.src_qp = qp->remote_qpn;
-		wc.slid = (u16)qp->remote_ah_attr.dlid;
+		wc.slid = qp->remote_ah_attr.dlid;
 		wc.sl = qp->remote_ah_attr.sl;
 		/* zero fields that are N/A */
 		wc.vendor_err = 0;
diff --git a/drivers/infiniband/sw/rdmavt/cq.c b/drivers/infiniband/sw/rdmavt/cq.c
index 6d9904a..a03b240 100644
--- a/drivers/infiniband/sw/rdmavt/cq.c
+++ b/drivers/infiniband/sw/rdmavt/cq.c
@@ -105,7 +105,7 @@ void rvt_cq_enter(struct rvt_cq *cq, struct ib_wc *entry, bool solicited)
 		wc->uqueue[head].src_qp = entry->src_qp;
 		wc->uqueue[head].wc_flags = entry->wc_flags;
 		wc->uqueue[head].pkey_index = entry->pkey_index;
-		wc->uqueue[head].slid = entry->slid;
+		wc->uqueue[head].slid = (u16)entry->slid;
 		wc->uqueue[head].sl = entry->sl;
 		wc->uqueue[head].dlid_path_bits = entry->dlid_path_bits;
 		wc->uqueue[head].port_num = entry->port_num;
diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
index 3d80720..b3f9130 100644
--- a/include/rdma/ib_verbs.h
+++ b/include/rdma/ib_verbs.h
@@ -896,7 +896,7 @@ struct ib_wc {
 	u32			src_qp;
 	int			wc_flags;
 	u16			pkey_index;
-	u16			slid;
+	u32			slid;
 	u8			sl;
 	u8			dlid_path_bits;
 	u8			port_num;	/* valid only for DR SMPs on switches */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 06/11] IB/mad: Ensure DR MADs are correctly specified when using OPA devices

[Dasaratharaman Chandramouli]

From: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>

Pure DR MADs do not need OPA GIDs to be specified in the GRH since
they do not rely on LID information.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/mad.c | 104 +++++++++++++++++++++++++++++++++++++-----
 include/rdma/opa_addr.h       |  17 +++++++
 2 files changed, 109 insertions(+), 12 deletions(-)

diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
index 40cbd6b..c0ee997 100644
--- a/drivers/infiniband/core/mad.c
+++ b/drivers/infiniband/core/mad.c
@@ -41,6 +41,7 @@
 #include <linux/slab.h>
 #include <linux/module.h>
 #include <rdma/ib_cache.h>
+#include <rdma/opa_addr.h>
 
 #include "mad_priv.h"
 #include "mad_rmpp.h"
@@ -731,6 +732,80 @@ static size_t mad_priv_dma_size(const struct ib_mad_private *mp)
 	return sizeof(struct ib_grh) + mp->mad_size;
 }
 
+static int verify_mad_ah(struct ib_mad_agent_private *mad_agent_priv,
+			 struct ib_mad_send_wr_private *mad_send_wr)
+{
+	struct ib_device *ib_dev = mad_agent_priv->qp_info->port_priv->device;
+	u8 port = mad_agent_priv->qp_info->port_priv->port_num;
+	struct ib_smp *smp = mad_send_wr->send_buf.mad;
+	struct opa_smp *opa_smp = (struct opa_smp *)smp;
+	u32 opa_drslid = be32_to_cpu(opa_smp->route.dr.dr_slid);
+	u32 opa_drdlid = be32_to_cpu(opa_smp->route.dr.dr_dlid);
+
+	bool dr_slid_is_permissive = (OPA_LID_PERMISSIVE ==
+				      opa_smp->route.dr.dr_slid) ? true : false;
+	bool dr_dlid_is_permissive = (OPA_LID_PERMISSIVE ==
+				      opa_smp->route.dr.dr_dlid) ? true : false;
+	bool drslid_is_ib_ucast = (opa_drslid <
+				   be16_to_cpu(IB_MULTICAST_LID_BASE)) ?
+					true : false;
+	bool drdlid_is_ib_ucast = (opa_drdlid <
+				   be16_to_cpu(IB_MULTICAST_LID_BASE)) ?
+					true : false;
+	bool drslid_is_ext = !drslid_is_ib_ucast && !dr_slid_is_permissive;
+	bool drdlid_is_ext = !drdlid_is_ib_ucast && !dr_dlid_is_permissive;
+	bool grh_present = false;
+	struct ib_ah_attr attr;
+	union ib_gid sgid;
+	int ret = 0;
+
+	ret = ib_query_ah(mad_send_wr->send_buf.ah, &attr);
+	if (ret)
+		return ret;
+	grh_present = (attr.ah_flags & IB_AH_GRH);
+	if (grh_present) {
+		ret = ib_query_gid(ib_dev, port, attr.grh.sgid_index,
+				   &sgid, NULL);
+		if (ret)
+			return ret;
+	}
+
+	if (smp->class_version == OPA_SMP_CLASS_VERSION) {
+		/*
+		 * Conditions when GRH info should not be specified
+		 * 1. both dr_slid and dr_dlid are permissve (Pure DR)
+		 * 2. both dr_slid and dr_dlid are less than 0xc000.
+		 *
+		 * Conditions when GRH info should be specified
+		 * 1. dr_dlid is not permissive and above 0xbfff
+		 * OR
+		 * 2. dr_slid is not permissive and above 0xbfff
+		 */
+		if (grh_present) {
+			if ((dr_slid_is_permissive &&
+			     dr_dlid_is_permissive) ||
+			     (drslid_is_ib_ucast && drdlid_is_ib_ucast))
+				if (ib_is_opa_gid(&attr.grh.dgid) &&
+				    ib_is_opa_gid(&sgid))
+					return -EINVAL;
+			if (drslid_is_ext && !ib_is_opa_gid(&sgid))
+				return -EINVAL;
+			if (drdlid_is_ext &&
+			    !ib_is_opa_gid(&attr.grh.dgid))
+				return -EINVAL;
+		} else { /* There is no GRH */
+			if (drslid_is_ext || drdlid_is_ext)
+				return -EINVAL;
+		}
+	} else {
+		if (grh_present)
+			if (ib_is_opa_gid(&attr.grh.dgid) &&
+			    ib_is_opa_gid(&sgid))
+				return -EINVAL;
+	}
+	return ret;
+}
+
 /*
  * Return 0 if SMP is to be sent
  * Return 1 if SMP was consumed locally (whether or not solicited)
@@ -754,8 +829,12 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
 	size_t mad_size = port_mad_size(mad_agent_priv->qp_info->port_priv);
 	u16 out_mad_pkey_index = 0;
 	u16 drslid;
-	bool opa = rdma_cap_opa_mad(mad_agent_priv->qp_info->port_priv->device,
-				    mad_agent_priv->qp_info->port_priv->port_num);
+	bool opa_mad =
+		rdma_cap_opa_mad(mad_agent_priv->qp_info->port_priv->device,
+				 mad_agent_priv->qp_info->port_priv->port_num);
+	bool opa_ah =
+		rdma_cap_opa_ah(mad_agent_priv->qp_info->port_priv->device,
+				mad_agent_priv->qp_info->port_priv->port_num);
 
 	if (rdma_cap_ib_switch(device) &&
 	    smp->mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE)
@@ -763,13 +842,21 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
 	else
 		port_num = mad_agent_priv->agent.port_num;
 
+	if (opa_mad && opa_ah) {
+		ret = verify_mad_ah(mad_agent_priv, mad_send_wr);
+		if (ret) {
+			dev_err(&device->dev,
+				"Error verifying MAD format\n");
+			goto out;
+		}
+	}
 	/*
 	 * Directed route handling starts if the initial LID routed part of
 	 * a request or the ending LID routed part of a response is empty.
 	 * If we are at the start of the LID routed part, don't update the
 	 * hop_ptr or hop_cnt.  See section 14.2.2, Vol 1 IB spec.
 	 */
-	if (opa && smp->class_version == OPA_SMP_CLASS_VERSION) {
+	if (opa_mad && smp->class_version == OPA_SMP_CLASS_VERSION) {
 		u32 opa_drslid;
 
 		if ((opa_get_smp_direction(opa_smp)
@@ -783,13 +870,6 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
 			goto out;
 		}
 		opa_drslid = be32_to_cpu(opa_smp->route.dr.dr_slid);
-		if (opa_drslid != be32_to_cpu(OPA_LID_PERMISSIVE) &&
-		    opa_drslid & 0xffff0000) {
-			ret = -EINVAL;
-			dev_err(&device->dev, "OPA Invalid dr_slid 0x%x\n",
-			       opa_drslid);
-			goto out;
-		}
 		drslid = (u16)(opa_drslid & 0x0000ffff);
 
 		/* Check to post send on QP or process locally */
@@ -834,7 +914,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
 		     send_wr->pkey_index,
 		     send_wr->port_num, &mad_wc);
 
-	if (opa && smp->base_version == OPA_MGMT_BASE_VERSION) {
+	if (opa_mad && smp->base_version == OPA_MGMT_BASE_VERSION) {
 		mad_wc.byte_len = mad_send_wr->send_buf.hdr_len
 					+ mad_send_wr->send_buf.data_len
 					+ sizeof(struct ib_grh);
@@ -891,7 +971,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
 	}
 
 	local->mad_send_wr = mad_send_wr;
-	if (opa) {
+	if (opa_mad) {
 		local->mad_send_wr->send_wr.pkey_index = out_mad_pkey_index;
 		local->return_wc_byte_len = mad_size;
 	}
diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
index 142b327..3e22937 100644
--- a/include/rdma/opa_addr.h
+++ b/include/rdma/opa_addr.h
@@ -33,6 +33,23 @@
 #if !defined(OPA_ADDR_H)
 #define OPA_ADDR_H
 
+#include <rdma/ib_verbs.h>
+
 #define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
 				 ? 0 : x)
+#define	OPA_SPECIAL_OUI		(0x00066AULL)
+
+/**
+ * ib_is_opa_gid: Returns true if the top 24 bits of the gid
+ * contains the OPA_STL_OUI identifier. This identifies that
+ * the provided gid is a special purpose GID meant to carry
+ * extended LID information.
+ *
+ * @gid: The Global identifier
+ */
+static inline bool ib_is_opa_gid(union ib_gid *gid)
+{
+	return ((be64_to_cpu(gid->global.interface_id) >> 40) ==
+		OPA_SPECIAL_OUI);
+}
 #endif /* OPA_ADDR_H */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 07/11] IB/mad: Change slid in RMPP recv from 16 to 32 bits

[Dasaratharaman Chandramouli]

OPA devices will contain larger lids in the wc.slid
which is now 32 bits. This change ensures RMPP handler
is able to retrieve the correct lid.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/mad_rmpp.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/core/mad_rmpp.c b/drivers/infiniband/core/mad_rmpp.c
index 8a076e1..057c1ec 100644
--- a/drivers/infiniband/core/mad_rmpp.c
+++ b/drivers/infiniband/core/mad_rmpp.c
@@ -64,7 +64,7 @@ struct mad_rmpp_recv {
 
 	__be64 tid;
 	u32 src_qp;
-	u16 slid;
+	u32 slid;
 	u8 mgmt_class;
 	u8 class_version;
 	u8 method;
@@ -316,7 +316,7 @@ static void recv_cleanup_handler(struct work_struct *work)
 	mad_hdr = &mad_recv_wc->recv_buf.mad->mad_hdr;
 	rmpp_recv->tid = mad_hdr->tid;
 	rmpp_recv->src_qp = mad_recv_wc->wc->src_qp;
-	rmpp_recv->slid = (u16)mad_recv_wc->wc->slid;
+	rmpp_recv->slid = mad_recv_wc->wc->slid;
 	rmpp_recv->mgmt_class = mad_hdr->mgmt_class;
 	rmpp_recv->class_version = mad_hdr->class_version;
 	rmpp_recv->method  = mad_hdr->method;
@@ -337,7 +337,7 @@ static void recv_cleanup_handler(struct work_struct *work)
 	list_for_each_entry(rmpp_recv, &agent->rmpp_list, list) {
 		if (rmpp_recv->tid == mad_hdr->tid &&
 		    rmpp_recv->src_qp == mad_recv_wc->wc->src_qp &&
-		    rmpp_recv->slid == (u16)mad_recv_wc->wc->slid &&
+		    rmpp_recv->slid == mad_recv_wc->wc->slid &&
 		    rmpp_recv->mgmt_class == mad_hdr->mgmt_class &&
 		    rmpp_recv->class_version == mad_hdr->class_version &&
 		    rmpp_recv->method == mad_hdr->method)
@@ -870,7 +870,7 @@ static int init_newwin(struct ib_mad_send_wr_private *mad_send_wr)
 		if (ib_query_ah(mad_send_wr->send_buf.ah, &ah_attr))
 			continue;
 
-		if (rmpp_recv->slid == (u16)ah_attr.dlid) {
+		if (rmpp_recv->slid == ah_attr.dlid) {
 			newwin = rmpp_recv->repwin;
 			break;
 		}
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 08/11] IB/SA: Program extended LID in SM Address handle

[Dasaratharaman Chandramouli]

Reuse port_attr.grh_required field to let IB SA know that extended LID
information needs to be set in the SM Address handle for OPA devices

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/core/sa_query.c | 8 +++++++-
 include/rdma/opa_addr.h            | 1 +
 2 files changed, 8 insertions(+), 1 deletion(-)

diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 81b742c..d59ee67 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -50,6 +50,7 @@
 #include <uapi/rdma/ib_user_sa.h>
 #include <rdma/ib_marshall.h>
 #include <rdma/ib_addr.h>
+#include <rdma/opa_addr.h>
 #include "sa.h"
 #include "core_priv.h"
 
@@ -964,7 +965,12 @@ static void update_sm_ah(struct work_struct *work)
 	if (port_attr.grh_required) {
 		ah_attr.ah_flags = IB_AH_GRH;
 		ah_attr.grh.dgid.global.subnet_prefix = cpu_to_be64(port_attr.subnet_prefix);
-		ah_attr.grh.dgid.global.interface_id = cpu_to_be64(IB_SA_WELL_KNOWN_GUID);
+		if (rdma_cap_opa_ah(port->agent->device, port->port_num))
+			ah_attr.grh.dgid.global.interface_id =
+				OPA_MAKE_ID(ah_attr.dlid);
+		else
+			ah_attr.grh.dgid.global.interface_id =
+				cpu_to_be64(IB_SA_WELL_KNOWN_GUID);
 	}
 
 	new_ah->ah = ib_create_ah(port->agent->qp->pd, &ah_attr);
diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
index 3e22937..5c713bc 100644
--- a/include/rdma/opa_addr.h
+++ b/include/rdma/opa_addr.h
@@ -38,6 +38,7 @@
 #define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
 				 ? 0 : x)
 #define	OPA_SPECIAL_OUI		(0x00066AULL)
+#define OPA_MAKE_ID(x)          (cpu_to_be64(OPA_SPECIAL_OUI << 40 | (x)))
 
 /**
  * ib_is_opa_gid: Returns true if the top 24 bits of the gid
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 09/11] IB/IPoIB: Retrieve 32 bit LIDs from path records when running on OPA devices

[Dasaratharaman Chandramouli]

Path record responses will contain the 32 bit LID information in the
SGID and DGID field of the responses. Modify IPoIB to use these extended
LIDs in datagram and connected mode communication.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/ulp/ipoib/ipoib.h           |  4 +++-
 drivers/infiniband/ulp/ipoib/ipoib_cm.c        | 11 +++++++++++
 drivers/infiniband/ulp/ipoib/ipoib_main.c      | 26 ++++++++++++++++++++++++++
 drivers/infiniband/ulp/ipoib/ipoib_multicast.c |  2 +-
 include/rdma/opa_addr.h                        | 12 ++++++++++++
 5 files changed, 53 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h
index 7b8d2d9..fad4560 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -352,7 +352,7 @@ struct ipoib_dev_priv {
 	u32		  qkey;
 
 	union ib_gid local_gid;
-	u16	     local_lid;
+	u32	     local_lid;
 
 	unsigned int admin_mtu;
 	unsigned int mcast_mtu;
@@ -421,6 +421,8 @@ struct ipoib_path {
 	struct rb_node	      rb_node;
 	struct list_head      list;
 	int  		      valid;
+	u32		      dlid;
+	u32		      slid;
 };
 
 struct ipoib_neigh {
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 4ad297d..c27df76 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -38,6 +38,7 @@
 #include <linux/slab.h>
 #include <linux/vmalloc.h>
 #include <linux/moduleparam.h>
+#include <rdma/opa_addr.h>
 
 #include "ipoib.h"
 
@@ -1356,6 +1357,16 @@ static void ipoib_cm_tx_start(struct work_struct *work)
 		}
 		memcpy(&pathrec, &p->path->pathrec, sizeof pathrec);
 
+		if (rdma_cap_opa_ah(priv->ca, priv->port)) {
+			if (p->path->dlid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
+				pathrec.dgid.global.interface_id =
+					OPA_MAKE_ID(p->path->dlid);
+
+			if (p->path->slid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
+				pathrec.sgid.global.interface_id =
+					OPA_MAKE_ID(p->path->slid);
+		}
+
 		spin_unlock_irqrestore(&priv->lock, flags);
 		netif_tx_unlock_bh(dev);
 
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 5636fc3..474c3bf 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -52,6 +52,7 @@
 #include <linux/inetdevice.h>
 #include <rdma/ib_cache.h>
 #include <linux/pci.h>
+#include <rdma/opa_addr.h>
 
 #define DRV_VERSION "1.0.0"
 
@@ -766,6 +767,31 @@ static void path_rec_completion(int status,
 	spin_lock_irqsave(&priv->lock, flags);
 
 	if (!IS_ERR_OR_NULL(ah)) {
+		/*
+		 * Extended LIDs might get programmed into GIDs in the
+		 * case of OPA devices. Since we have created the ah
+		 * above which would have made use of the lids, now is
+		 * a good time to change them back to regular GIDs after
+		 * saving the extended LIDs.
+		 */
+		if (rdma_cap_opa_ah(priv->ca, priv->port) &&
+		    ib_is_opa_gid(&pathrec->sgid)) {
+			path->slid = opa_get_lid_from_gid(&pathrec->sgid);
+			pathrec->sgid = path->pathrec.sgid;
+		} else {
+			path->slid = be16_to_cpu(pathrec->slid);
+		}
+
+		if (rdma_cap_opa_ah(priv->ca, priv->port) &&
+		    ib_is_opa_gid(&pathrec->dgid)) {
+			path->dlid = opa_get_lid_from_gid(&pathrec->dgid);
+			pathrec->dgid = path->pathrec.dgid;
+		} else {
+			path->dlid = be16_to_cpu(pathrec->dlid);
+		}
+		ipoib_dbg(priv, "PathRec SGID %pI6 DGID %pI6\n",
+			  pathrec->sgid.raw, pathrec->dgid.raw);
+
 		path->pathrec = *pathrec;
 
 		old_ah   = path->ah;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
index bff73b5..d3394b6 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_multicast.c
@@ -581,7 +581,7 @@ void ipoib_mcast_join_task(struct work_struct *work)
 			  port_attr.state);
 		return;
 	}
-	priv->local_lid = (u16)port_attr.lid;
+	priv->local_lid = port_attr.lid;
 	netif_addr_lock_bh(dev);
 
 	if (!test_bit(IPOIB_FLAG_DEV_ADDR_SET, &priv->flags)) {
diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
index 5c713bc..d0a37d0 100644
--- a/include/rdma/opa_addr.h
+++ b/include/rdma/opa_addr.h
@@ -53,4 +53,16 @@ static inline bool ib_is_opa_gid(union ib_gid *gid)
 	return ((be64_to_cpu(gid->global.interface_id) >> 40) ==
 		OPA_SPECIAL_OUI);
 }
+
+/**
+ * opa_get_lid_from_gid: Returns the last 32 bits of the gid.
+ * OPA devices use one of the gids in the gid table to also
+ * store the lid.
+ *
+ * @gid: The Global identifier
+ */
+static inline u32 opa_get_lid_from_gid(union ib_gid *gid)
+{
+	return be64_to_cpu(gid->global.interface_id) & 0xFFFFFFFF;
+}
 #endif /* OPA_ADDR_H */
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 10/11] IB/IPoIB: Modify ipoib_get_net_dev_by_params to lookup gid table

[Dasaratharaman Chandramouli]

ipoib_get_net_dev_by_params compares incoming gid with local_gid
which is gid at index 0 of the gid table. OPA devices using larger
LIDs may have a different GID format than whats setup in the local_gid
field. Do a search of the gid table in those cases.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/ulp/ipoib/ipoib_main.c | 37 +++++++++++++++++++++++++++++--
 1 file changed, 35 insertions(+), 2 deletions(-)

diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 474c3bf..3e135df 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -328,6 +328,40 @@ static struct net_device *ipoib_get_net_dev_match_addr(
 	return result;
 }
 
+/* retuns true if the incoming gid is assigned to the IPoIB
+ * netdev interface
+ *
+ * OPA devices may have the incoming GID in the OPA GID
+ * format which might not necessarily be assigned to the
+ * netdev interface. This necessitates searching the GID
+ * table to match this OPA GID.
+ */
+static bool ipoib_check_gid(struct ipoib_dev_priv *priv,
+			    const union ib_gid *gid)
+{
+	bool is_local_gid;
+	struct ib_port_attr attr;
+	union ib_gid port_gid;
+	int i;
+
+	if (!gid)
+		return true;
+
+	is_local_gid = !memcmp(gid, &priv->local_gid, sizeof(*gid));
+
+	if (!rdma_cap_opa_ah(priv->ca, priv->port) || is_local_gid)
+		return is_local_gid;
+
+	if (ib_query_port(priv->ca, priv->port, &attr))
+		return false;
+	for (i = 1; i < attr.gid_tbl_len; i++) {
+		if (ib_query_gid(priv->ca, priv->port, i, &port_gid, NULL))
+			return false;
+		if (!memcmp(gid, &port_gid, sizeof(*gid)))
+			return true;
+	}
+	return false;
+}
 /* returns the number of IPoIB netdevs on top a given ipoib device matching a
  * pkey_index and address, if one exists.
  *
@@ -344,8 +378,7 @@ static int ipoib_match_gid_pkey_addr(struct ipoib_dev_priv *priv,
 	struct net_device *net_dev = NULL;
 	int matches = 0;
 
-	if (priv->pkey_index == pkey_index &&
-	    (!gid || !memcmp(gid, &priv->local_gid, sizeof(*gid)))) {
+	if (priv->pkey_index == pkey_index && ipoib_check_gid(priv, gid)) {
 		if (!addr) {
 			net_dev = ipoib_get_master_net_dev(priv->dev);
 		} else {
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 11/11] IB/srpt: Increase lid and sm_lid to 32 bits

[Dasaratharaman Chandramouli]

srpt contains lid and sm_lid fields which are 16 bits in
length, increase them to 32 bits.

Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
 drivers/infiniband/ulp/srpt/ib_srpt.c | 4 ++--
 drivers/infiniband/ulp/srpt/ib_srpt.h | 4 ++--
 2 files changed, 4 insertions(+), 4 deletions(-)

diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 4dc66ba..0b1f69e 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -514,8 +514,8 @@ static int srpt_refresh_port(struct srpt_port *sport)
 	if (ret)
 		goto err_query_port;
 
-	sport->sm_lid = (u16)port_attr.sm_lid;
-	sport->lid = (u16)port_attr.lid;
+	sport->sm_lid = port_attr.sm_lid;
+	sport->lid = port_attr.lid;
 
 	ret = ib_query_gid(sport->sdev->device, sport->port, 0, &sport->gid,
 			   NULL);
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.h b/drivers/infiniband/ulp/srpt/ib_srpt.h
index 5818787..580bb75 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.h
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.h
@@ -322,8 +322,8 @@ struct srpt_port {
 	bool			enabled;
 	u8			port_guid[64];
 	u8			port;
-	u16			sm_lid;
-	u16			lid;
+	u32			sm_lid;
+	u32			lid;
 	union ib_gid		gid;
 	struct work_struct	work;
 	struct se_portal_group	port_tpg_1;
-- 
1.8.3.1

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 02/11] IB/core: Change port_attr.sm_lid from 16 to 32 bits

[Jason Gunthorpe]

On Tue, Nov 22, 2016 at 02:38:43PM -0500, Dasaratharaman Chandramouli wrote:
> +++ b/drivers/infiniband/core/sa_query.c
> @@ -958,7 +958,7 @@ static void update_sm_ah(struct work_struct *work)
>  		pr_err("Couldn't find index for default PKey\n");
>  
>  	memset(&ah_attr, 0, sizeof ah_attr);
> -	ah_attr.dlid     = port_attr.sm_lid;
> +	ah_attr.dlid     = (u16)port_attr.sm_lid;

Why are we dropping bits here?

> +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
> @@ -514,7 +514,7 @@ static int srpt_refresh_port(struct srpt_port *sport)
>  	if (ret)
>  		goto err_query_port;
>  
> -	sport->sm_lid = port_attr.sm_lid;
> +	sport->sm_lid = (u16)port_attr.sm_lid;
>  	sport->lid = port_attr.lid;

And here..

> +#if !defined(OPA_ADDR_H)
> +#define OPA_ADDR_H

I don't think we need a header file for this.

> +#define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
> +				 ? 0 : x)

static inline function please.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 02/11] IB/core: Change port_attr.sm_lid from 16 to 32 bits

[Chandramouli, Dasaratharaman]

On 11/22/2016 12:57 PM, Jason Gunthorpe wrote:
> On Tue, Nov 22, 2016 at 02:38:43PM -0500, Dasaratharaman Chandramouli wrote:
>> +++ b/drivers/infiniband/core/sa_query.c
>> @@ -958,7 +958,7 @@ static void update_sm_ah(struct work_struct *work)
>>  		pr_err("Couldn't find index for default PKey\n");
>>
>>  	memset(&ah_attr, 0, sizeof ah_attr);
>> -	ah_attr.dlid     = port_attr.sm_lid;
>> +	ah_attr.dlid     = (u16)port_attr.sm_lid;
>
> Why are we dropping bits here?

Patch 3 increases ah_attr.dlid to 32 bits and the typecast is dropped.
We added it in this patch just to avoid compiler warnings, if any.
>
>> +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
>> @@ -514,7 +514,7 @@ static int srpt_refresh_port(struct srpt_port *sport)
>>  	if (ret)
>>  		goto err_query_port;
>>
>> -	sport->sm_lid = port_attr.sm_lid;
>> +	sport->sm_lid = (u16)port_attr.sm_lid;
>>  	sport->lid = port_attr.lid;
>
> And here..
Patch 11 increases lid and sm_lid fields in struct srpt_port and the 
typecast is dropped.
We added it in this patch just to avoid compiler warnings, if any.
>
>> +#if !defined(OPA_ADDR_H)
>> +#define OPA_ADDR_H
>
> I don't think we need a header file for this.
There are other helpers specific to OPA addresses that were required in 
later patches which we thought can go in a separate file.
>
>> +#define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
>> +				 ? 0 : x)
>
> static inline function please.
Will change. Thanks.
>
> Jason
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 10/11] IB/IPoIB: Modify ipoib_get_net_dev_by_params to lookup gid table

[Jason Gunthorpe]

On Tue, Nov 22, 2016 at 02:38:51PM -0500, Dasaratharaman Chandramouli wrote:
> ipoib_get_net_dev_by_params compares incoming gid with local_gid
> which is gid at index 0 of the gid table. OPA devices using larger
> LIDs may have a different GID format than whats setup in the local_gid
> field. Do a search of the gid table in those cases.

Sorry, NAK, the incoming GID has to exactly match what is in IPoIB
LLADDR, you can't do this or it breaks how the virtualization stuff is
supposed to work with gid aliases.

I'm skeptical about the other ipoib patch as well since ipoib should
be dealing natively with 32 bit lids, I think you need to fix the path
record stuff too instead of opening coding that in ipoib.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 02/11] IB/core: Change port_attr.sm_lid from 16 to 32 bits

[Jason Gunthorpe]

On Tue, Nov 22, 2016 at 01:13:26PM -0800, Chandramouli, Dasaratharaman wrote:
> 
> 
> On 11/22/2016 12:57 PM, Jason Gunthorpe wrote:
> >On Tue, Nov 22, 2016 at 02:38:43PM -0500, Dasaratharaman Chandramouli wrote:
> >>+++ b/drivers/infiniband/core/sa_query.c
> >>@@ -958,7 +958,7 @@ static void update_sm_ah(struct work_struct *work)
> >> 		pr_err("Couldn't find index for default PKey\n");
> >>
> >> 	memset(&ah_attr, 0, sizeof ah_attr);
> >>-	ah_attr.dlid     = port_attr.sm_lid;
> >>+	ah_attr.dlid     = (u16)port_attr.sm_lid;
> >
> >Why are we dropping bits here?
> 
> Patch 3 increases ah_attr.dlid to 32 bits and the typecast is dropped.
> We added it in this patch just to avoid compiler warnings, if any.

It would be alot better to fix this series so adding typecasts and
then dropping them didn't happen so extensively. Just reodering some
patched would do the trick. That would make it alot less churny and
easy to read..

> >>-	sport->sm_lid = port_attr.sm_lid;
> >>+	sport->sm_lid = (u16)port_attr.sm_lid;
> >> 	sport->lid = port_attr.lid;
> >
> >And here..

> Patch 11 increases lid and sm_lid fields in struct srpt_port and the
> typecast is dropped.  We added it in this patch just to avoid
> compiler warnings, if any.

Eg if you do patch 11 first this hunk goes away.

I also don't really care for all the added u16 casts, the compiler
doesn't warn on implicit demotion without a higher warning level...

> >
> >>+#define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
> >>+				 ? 0 : x)
> >
> >static inline function please.
> Will change. Thanks.

All of them please.

And I think you should re-think how this is being used, the pattern
around OPA_TO_IB_UCAST_LID is copied too many times for my liking, and
isn't this UAPI compatability only?

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 02/11] IB/core: Change port_attr.sm_lid from 16 to 32 bits

[Chandramouli, Dasaratharaman]

On 11/22/2016 1:24 PM, Jason Gunthorpe wrote:
> On Tue, Nov 22, 2016 at 01:13:26PM -0800, Chandramouli, Dasaratharaman wrote:
>>
>>
>> On 11/22/2016 12:57 PM, Jason Gunthorpe wrote:
>>> On Tue, Nov 22, 2016 at 02:38:43PM -0500, Dasaratharaman Chandramouli wrote:
>>>> +++ b/drivers/infiniband/core/sa_query.c
>>>> @@ -958,7 +958,7 @@ static void update_sm_ah(struct work_struct *work)
>>>> 		pr_err("Couldn't find index for default PKey\n");
>>>>
>>>> 	memset(&ah_attr, 0, sizeof ah_attr);
>>>> -	ah_attr.dlid     = port_attr.sm_lid;
>>>> +	ah_attr.dlid     = (u16)port_attr.sm_lid;
>>>
>>> Why are we dropping bits here?
>>
>> Patch 3 increases ah_attr.dlid to 32 bits and the typecast is dropped.
>> We added it in this patch just to avoid compiler warnings, if any.
>
> It would be alot better to fix this series so adding typecasts and
> then dropping them didn't happen so extensively. Just reodering some
> patched would do the trick. That would make it alot less churny and
> easy to read..
Yes, will re-order them. Thanks
>
>>>> -	sport->sm_lid = port_attr.sm_lid;
>>>> +	sport->sm_lid = (u16)port_attr.sm_lid;
>>>> 	sport->lid = port_attr.lid;
>>>
>>> And here..
>
>> Patch 11 increases lid and sm_lid fields in struct srpt_port and the
>> typecast is dropped.  We added it in this patch just to avoid
>> compiler warnings, if any.
>
> Eg if you do patch 11 first this hunk goes away.
>
> I also don't really care for all the added u16 casts, the compiler
> doesn't warn on implicit demotion without a higher warning level...
>
It's that higher warning level that we were a little concerned about.
If you feel very strongly about not having these casts, they can be 
removed but i would prefer leaving them there to silence certain odd 
ball compiler configurations.
>>>
>>>> +#define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
>>>> +				 ? 0 : x)
>>>
>>> static inline function please.
>> Will change. Thanks.
>
> All of them please.
Will change.
>
> And I think you should re-think how this is being used, the pattern
> around OPA_TO_IB_UCAST_LID is copied too many times for my liking, and
> isn't this UAPI compatability only?
They are only used when there is a user-space query into the kernel for 
lid and sm_lid.
>
> Jason
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 06/11] IB/mad: Ensure DR MADs are correctly specified when using OPA devices

[Hal Rosenstock]

On 11/22/2016 2:38 PM, Dasaratharaman Chandramouli wrote:
> From: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> 
> Pure DR MADs do not need OPA GIDs to be specified in the GRH since
> they do not rely on LID information.
> 
> Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> ---
>  drivers/infiniband/core/mad.c | 104 +++++++++++++++++++++++++++++++++++++-----
>  include/rdma/opa_addr.h       |  17 +++++++
>  2 files changed, 109 insertions(+), 12 deletions(-)
> 
> diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
> index 40cbd6b..c0ee997 100644
> --- a/drivers/infiniband/core/mad.c
> +++ b/drivers/infiniband/core/mad.c
> @@ -41,6 +41,7 @@
>  #include <linux/slab.h>
>  #include <linux/module.h>
>  #include <rdma/ib_cache.h>
> +#include <rdma/opa_addr.h>
>  
>  #include "mad_priv.h"
>  #include "mad_rmpp.h"
> @@ -731,6 +732,80 @@ static size_t mad_priv_dma_size(const struct ib_mad_private *mp)
>  	return sizeof(struct ib_grh) + mp->mad_size;
>  }
>  
> +static int verify_mad_ah(struct ib_mad_agent_private *mad_agent_priv,

I think it would be better if this were named opa_verify_mad_ah to make
it clearer that this is an OPA only routine.

-- Hal

> +			 struct ib_mad_send_wr_private *mad_send_wr)
> +{
> +	struct ib_device *ib_dev = mad_agent_priv->qp_info->port_priv->device;
> +	u8 port = mad_agent_priv->qp_info->port_priv->port_num;
> +	struct ib_smp *smp = mad_send_wr->send_buf.mad;
> +	struct opa_smp *opa_smp = (struct opa_smp *)smp;
> +	u32 opa_drslid = be32_to_cpu(opa_smp->route.dr.dr_slid);
> +	u32 opa_drdlid = be32_to_cpu(opa_smp->route.dr.dr_dlid);
> +
> +	bool dr_slid_is_permissive = (OPA_LID_PERMISSIVE ==
> +				      opa_smp->route.dr.dr_slid) ? true : false;
> +	bool dr_dlid_is_permissive = (OPA_LID_PERMISSIVE ==
> +				      opa_smp->route.dr.dr_dlid) ? true : false;
> +	bool drslid_is_ib_ucast = (opa_drslid <
> +				   be16_to_cpu(IB_MULTICAST_LID_BASE)) ?
> +					true : false;
> +	bool drdlid_is_ib_ucast = (opa_drdlid <
> +				   be16_to_cpu(IB_MULTICAST_LID_BASE)) ?
> +					true : false;
> +	bool drslid_is_ext = !drslid_is_ib_ucast && !dr_slid_is_permissive;
> +	bool drdlid_is_ext = !drdlid_is_ib_ucast && !dr_dlid_is_permissive;
> +	bool grh_present = false;
> +	struct ib_ah_attr attr;
> +	union ib_gid sgid;
> +	int ret = 0;
> +
> +	ret = ib_query_ah(mad_send_wr->send_buf.ah, &attr);
> +	if (ret)
> +		return ret;
> +	grh_present = (attr.ah_flags & IB_AH_GRH);
> +	if (grh_present) {
> +		ret = ib_query_gid(ib_dev, port, attr.grh.sgid_index,
> +				   &sgid, NULL);
> +		if (ret)
> +			return ret;
> +	}
> +
> +	if (smp->class_version == OPA_SMP_CLASS_VERSION) {
> +		/*
> +		 * Conditions when GRH info should not be specified
> +		 * 1. both dr_slid and dr_dlid are permissve (Pure DR)
> +		 * 2. both dr_slid and dr_dlid are less than 0xc000.
> +		 *
> +		 * Conditions when GRH info should be specified
> +		 * 1. dr_dlid is not permissive and above 0xbfff
> +		 * OR
> +		 * 2. dr_slid is not permissive and above 0xbfff
> +		 */
> +		if (grh_present) {
> +			if ((dr_slid_is_permissive &&
> +			     dr_dlid_is_permissive) ||
> +			     (drslid_is_ib_ucast && drdlid_is_ib_ucast))
> +				if (ib_is_opa_gid(&attr.grh.dgid) &&
> +				    ib_is_opa_gid(&sgid))
> +					return -EINVAL;
> +			if (drslid_is_ext && !ib_is_opa_gid(&sgid))
> +				return -EINVAL;
> +			if (drdlid_is_ext &&
> +			    !ib_is_opa_gid(&attr.grh.dgid))
> +				return -EINVAL;
> +		} else { /* There is no GRH */
> +			if (drslid_is_ext || drdlid_is_ext)
> +				return -EINVAL;
> +		}
> +	} else {
> +		if (grh_present)
> +			if (ib_is_opa_gid(&attr.grh.dgid) &&
> +			    ib_is_opa_gid(&sgid))
> +				return -EINVAL;
> +	}
> +	return ret;
> +}
> +
>  /*
>   * Return 0 if SMP is to be sent
>   * Return 1 if SMP was consumed locally (whether or not solicited)
> @@ -754,8 +829,12 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>  	size_t mad_size = port_mad_size(mad_agent_priv->qp_info->port_priv);
>  	u16 out_mad_pkey_index = 0;
>  	u16 drslid;
> -	bool opa = rdma_cap_opa_mad(mad_agent_priv->qp_info->port_priv->device,
> -				    mad_agent_priv->qp_info->port_priv->port_num);
> +	bool opa_mad =
> +		rdma_cap_opa_mad(mad_agent_priv->qp_info->port_priv->device,
> +				 mad_agent_priv->qp_info->port_priv->port_num);
> +	bool opa_ah =
> +		rdma_cap_opa_ah(mad_agent_priv->qp_info->port_priv->device,
> +				mad_agent_priv->qp_info->port_priv->port_num);
>  
>  	if (rdma_cap_ib_switch(device) &&
>  	    smp->mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE)
> @@ -763,13 +842,21 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>  	else
>  		port_num = mad_agent_priv->agent.port_num;
>  
> +	if (opa_mad && opa_ah) {
> +		ret = verify_mad_ah(mad_agent_priv, mad_send_wr);
> +		if (ret) {
> +			dev_err(&device->dev,
> +				"Error verifying MAD format\n");
> +			goto out;
> +		}
> +	}
>  	/*
>  	 * Directed route handling starts if the initial LID routed part of
>  	 * a request or the ending LID routed part of a response is empty.
>  	 * If we are at the start of the LID routed part, don't update the
>  	 * hop_ptr or hop_cnt.  See section 14.2.2, Vol 1 IB spec.
>  	 */
> -	if (opa && smp->class_version == OPA_SMP_CLASS_VERSION) {
> +	if (opa_mad && smp->class_version == OPA_SMP_CLASS_VERSION) {
>  		u32 opa_drslid;
>  
>  		if ((opa_get_smp_direction(opa_smp)
> @@ -783,13 +870,6 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>  			goto out;
>  		}
>  		opa_drslid = be32_to_cpu(opa_smp->route.dr.dr_slid);
> -		if (opa_drslid != be32_to_cpu(OPA_LID_PERMISSIVE) &&
> -		    opa_drslid & 0xffff0000) {
> -			ret = -EINVAL;
> -			dev_err(&device->dev, "OPA Invalid dr_slid 0x%x\n",
> -			       opa_drslid);
> -			goto out;
> -		}
>  		drslid = (u16)(opa_drslid & 0x0000ffff);
>  
>  		/* Check to post send on QP or process locally */
> @@ -834,7 +914,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>  		     send_wr->pkey_index,
>  		     send_wr->port_num, &mad_wc);
>  
> -	if (opa && smp->base_version == OPA_MGMT_BASE_VERSION) {
> +	if (opa_mad && smp->base_version == OPA_MGMT_BASE_VERSION) {
>  		mad_wc.byte_len = mad_send_wr->send_buf.hdr_len
>  					+ mad_send_wr->send_buf.data_len
>  					+ sizeof(struct ib_grh);
> @@ -891,7 +971,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>  	}
>  
>  	local->mad_send_wr = mad_send_wr;
> -	if (opa) {
> +	if (opa_mad) {
>  		local->mad_send_wr->send_wr.pkey_index = out_mad_pkey_index;
>  		local->return_wc_byte_len = mad_size;
>  	}
> diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
> index 142b327..3e22937 100644
> --- a/include/rdma/opa_addr.h
> +++ b/include/rdma/opa_addr.h
> @@ -33,6 +33,23 @@
>  #if !defined(OPA_ADDR_H)
>  #define OPA_ADDR_H
>  
> +#include <rdma/ib_verbs.h>
> +
>  #define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
>  				 ? 0 : x)
> +#define	OPA_SPECIAL_OUI		(0x00066AULL)
> +
> +/**
> + * ib_is_opa_gid: Returns true if the top 24 bits of the gid
> + * contains the OPA_STL_OUI identifier. This identifies that
> + * the provided gid is a special purpose GID meant to carry
> + * extended LID information.
> + *
> + * @gid: The Global identifier
> + */
> +static inline bool ib_is_opa_gid(union ib_gid *gid)
> +{
> +	return ((be64_to_cpu(gid->global.interface_id) >> 40) ==
> +		OPA_SPECIAL_OUI);
> +}
>  #endif /* OPA_ADDR_H */
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 06/11] IB/mad: Ensure DR MADs are correctly specified when using OPA devices

[Chandramouli, Dasaratharaman]

On 11/23/2016 6:21 AM, Hal Rosenstock wrote:
> On 11/22/2016 2:38 PM, Dasaratharaman Chandramouli wrote:
>> From: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>>
>> Pure DR MADs do not need OPA GIDs to be specified in the GRH since
>> they do not rely on LID information.
>>
>> Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> ---
>>  drivers/infiniband/core/mad.c | 104 +++++++++++++++++++++++++++++++++++++-----
>>  include/rdma/opa_addr.h       |  17 +++++++
>>  2 files changed, 109 insertions(+), 12 deletions(-)
>>
>> diff --git a/drivers/infiniband/core/mad.c b/drivers/infiniband/core/mad.c
>> index 40cbd6b..c0ee997 100644
>> --- a/drivers/infiniband/core/mad.c
>> +++ b/drivers/infiniband/core/mad.c
>> @@ -41,6 +41,7 @@
>>  #include <linux/slab.h>
>>  #include <linux/module.h>
>>  #include <rdma/ib_cache.h>
>> +#include <rdma/opa_addr.h>
>>
>>  #include "mad_priv.h"
>>  #include "mad_rmpp.h"
>> @@ -731,6 +732,80 @@ static size_t mad_priv_dma_size(const struct ib_mad_private *mp)
>>  	return sizeof(struct ib_grh) + mp->mad_size;
>>  }
>>
>> +static int verify_mad_ah(struct ib_mad_agent_private *mad_agent_priv,
>
> I think it would be better if this were named opa_verify_mad_ah to make
> it clearer that this is an OPA only routine.
>
> -- Hal
Thanks. Will rename this to opa_verify_mad_ah

-Dasa
>
>> +			 struct ib_mad_send_wr_private *mad_send_wr)
>> +{
>> +	struct ib_device *ib_dev = mad_agent_priv->qp_info->port_priv->device;
>> +	u8 port = mad_agent_priv->qp_info->port_priv->port_num;
>> +	struct ib_smp *smp = mad_send_wr->send_buf.mad;
>> +	struct opa_smp *opa_smp = (struct opa_smp *)smp;
>> +	u32 opa_drslid = be32_to_cpu(opa_smp->route.dr.dr_slid);
>> +	u32 opa_drdlid = be32_to_cpu(opa_smp->route.dr.dr_dlid);
>> +
>> +	bool dr_slid_is_permissive = (OPA_LID_PERMISSIVE ==
>> +				      opa_smp->route.dr.dr_slid) ? true : false;
>> +	bool dr_dlid_is_permissive = (OPA_LID_PERMISSIVE ==
>> +				      opa_smp->route.dr.dr_dlid) ? true : false;
>> +	bool drslid_is_ib_ucast = (opa_drslid <
>> +				   be16_to_cpu(IB_MULTICAST_LID_BASE)) ?
>> +					true : false;
>> +	bool drdlid_is_ib_ucast = (opa_drdlid <
>> +				   be16_to_cpu(IB_MULTICAST_LID_BASE)) ?
>> +					true : false;
>> +	bool drslid_is_ext = !drslid_is_ib_ucast && !dr_slid_is_permissive;
>> +	bool drdlid_is_ext = !drdlid_is_ib_ucast && !dr_dlid_is_permissive;
>> +	bool grh_present = false;
>> +	struct ib_ah_attr attr;
>> +	union ib_gid sgid;
>> +	int ret = 0;
>> +
>> +	ret = ib_query_ah(mad_send_wr->send_buf.ah, &attr);
>> +	if (ret)
>> +		return ret;
>> +	grh_present = (attr.ah_flags & IB_AH_GRH);
>> +	if (grh_present) {
>> +		ret = ib_query_gid(ib_dev, port, attr.grh.sgid_index,
>> +				   &sgid, NULL);
>> +		if (ret)
>> +			return ret;
>> +	}
>> +
>> +	if (smp->class_version == OPA_SMP_CLASS_VERSION) {
>> +		/*
>> +		 * Conditions when GRH info should not be specified
>> +		 * 1. both dr_slid and dr_dlid are permissve (Pure DR)
>> +		 * 2. both dr_slid and dr_dlid are less than 0xc000.
>> +		 *
>> +		 * Conditions when GRH info should be specified
>> +		 * 1. dr_dlid is not permissive and above 0xbfff
>> +		 * OR
>> +		 * 2. dr_slid is not permissive and above 0xbfff
>> +		 */
>> +		if (grh_present) {
>> +			if ((dr_slid_is_permissive &&
>> +			     dr_dlid_is_permissive) ||
>> +			     (drslid_is_ib_ucast && drdlid_is_ib_ucast))
>> +				if (ib_is_opa_gid(&attr.grh.dgid) &&
>> +				    ib_is_opa_gid(&sgid))
>> +					return -EINVAL;
>> +			if (drslid_is_ext && !ib_is_opa_gid(&sgid))
>> +				return -EINVAL;
>> +			if (drdlid_is_ext &&
>> +			    !ib_is_opa_gid(&attr.grh.dgid))
>> +				return -EINVAL;
>> +		} else { /* There is no GRH */
>> +			if (drslid_is_ext || drdlid_is_ext)
>> +				return -EINVAL;
>> +		}
>> +	} else {
>> +		if (grh_present)
>> +			if (ib_is_opa_gid(&attr.grh.dgid) &&
>> +			    ib_is_opa_gid(&sgid))
>> +				return -EINVAL;
>> +	}
>> +	return ret;
>> +}
>> +
>>  /*
>>   * Return 0 if SMP is to be sent
>>   * Return 1 if SMP was consumed locally (whether or not solicited)
>> @@ -754,8 +829,12 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>>  	size_t mad_size = port_mad_size(mad_agent_priv->qp_info->port_priv);
>>  	u16 out_mad_pkey_index = 0;
>>  	u16 drslid;
>> -	bool opa = rdma_cap_opa_mad(mad_agent_priv->qp_info->port_priv->device,
>> -				    mad_agent_priv->qp_info->port_priv->port_num);
>> +	bool opa_mad =
>> +		rdma_cap_opa_mad(mad_agent_priv->qp_info->port_priv->device,
>> +				 mad_agent_priv->qp_info->port_priv->port_num);
>> +	bool opa_ah =
>> +		rdma_cap_opa_ah(mad_agent_priv->qp_info->port_priv->device,
>> +				mad_agent_priv->qp_info->port_priv->port_num);
>>
>>  	if (rdma_cap_ib_switch(device) &&
>>  	    smp->mgmt_class == IB_MGMT_CLASS_SUBN_DIRECTED_ROUTE)
>> @@ -763,13 +842,21 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>>  	else
>>  		port_num = mad_agent_priv->agent.port_num;
>>
>> +	if (opa_mad && opa_ah) {
>> +		ret = verify_mad_ah(mad_agent_priv, mad_send_wr);
>> +		if (ret) {
>> +			dev_err(&device->dev,
>> +				"Error verifying MAD format\n");
>> +			goto out;
>> +		}
>> +	}
>>  	/*
>>  	 * Directed route handling starts if the initial LID routed part of
>>  	 * a request or the ending LID routed part of a response is empty.
>>  	 * If we are at the start of the LID routed part, don't update the
>>  	 * hop_ptr or hop_cnt.  See section 14.2.2, Vol 1 IB spec.
>>  	 */
>> -	if (opa && smp->class_version == OPA_SMP_CLASS_VERSION) {
>> +	if (opa_mad && smp->class_version == OPA_SMP_CLASS_VERSION) {
>>  		u32 opa_drslid;
>>
>>  		if ((opa_get_smp_direction(opa_smp)
>> @@ -783,13 +870,6 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>>  			goto out;
>>  		}
>>  		opa_drslid = be32_to_cpu(opa_smp->route.dr.dr_slid);
>> -		if (opa_drslid != be32_to_cpu(OPA_LID_PERMISSIVE) &&
>> -		    opa_drslid & 0xffff0000) {
>> -			ret = -EINVAL;
>> -			dev_err(&device->dev, "OPA Invalid dr_slid 0x%x\n",
>> -			       opa_drslid);
>> -			goto out;
>> -		}
>>  		drslid = (u16)(opa_drslid & 0x0000ffff);
>>
>>  		/* Check to post send on QP or process locally */
>> @@ -834,7 +914,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>>  		     send_wr->pkey_index,
>>  		     send_wr->port_num, &mad_wc);
>>
>> -	if (opa && smp->base_version == OPA_MGMT_BASE_VERSION) {
>> +	if (opa_mad && smp->base_version == OPA_MGMT_BASE_VERSION) {
>>  		mad_wc.byte_len = mad_send_wr->send_buf.hdr_len
>>  					+ mad_send_wr->send_buf.data_len
>>  					+ sizeof(struct ib_grh);
>> @@ -891,7 +971,7 @@ static int handle_outgoing_dr_smp(struct ib_mad_agent_private *mad_agent_priv,
>>  	}
>>
>>  	local->mad_send_wr = mad_send_wr;
>> -	if (opa) {
>> +	if (opa_mad) {
>>  		local->mad_send_wr->send_wr.pkey_index = out_mad_pkey_index;
>>  		local->return_wc_byte_len = mad_size;
>>  	}
>> diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
>> index 142b327..3e22937 100644
>> --- a/include/rdma/opa_addr.h
>> +++ b/include/rdma/opa_addr.h
>> @@ -33,6 +33,23 @@
>>  #if !defined(OPA_ADDR_H)
>>  #define OPA_ADDR_H
>>
>> +#include <rdma/ib_verbs.h>
>> +
>>  #define OPA_TO_IB_UCAST_LID(x)	(((x) >= be16_to_cpu(IB_MULTICAST_LID_BASE)) \
>>  				 ? 0 : x)
>> +#define	OPA_SPECIAL_OUI		(0x00066AULL)
>> +
>> +/**
>> + * ib_is_opa_gid: Returns true if the top 24 bits of the gid
>> + * contains the OPA_STL_OUI identifier. This identifies that
>> + * the provided gid is a special purpose GID meant to carry
>> + * extended LID information.
>> + *
>> + * @gid: The Global identifier
>> + */
>> +static inline bool ib_is_opa_gid(union ib_gid *gid)
>> +{
>> +	return ((be64_to_cpu(gid->global.interface_id) >> 40) ==
>> +		OPA_SPECIAL_OUI);
>> +}
>>  #endif /* OPA_ADDR_H */
>>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1306 bytes --]

On Tue, Nov 22, 2016 at 02:38:44PM -0500, Dasaratharaman Chandramouli wrote:
> dlid field in ah_attr is increased to 32 bits. This
> enables core components to use the larger addresses if needed.
> The user ABI is unchanged and userspace applications can use
> 16 bit lids when creating and modifying address handles.
> 
> Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli@intel.com>
> Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
> ---

<...>

> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
> index 52216f6..294d3ed 100644
> --- a/include/rdma/ib_verbs.h
> +++ b/include/rdma/ib_verbs.h
> @@ -814,7 +814,7 @@ struct ib_mr_status {
>  
>  struct ib_ah_attr {
>  	struct ib_global_route	grh;
> -	u16			dlid;
> +	u32			dlid;
>  	u8			sl;
>  	u8			src_path_bits;
>  	u8			static_rate;

It looks like this structure can be reshuffled. Can you run pahole on it, please?

Thanks

> -- 
> 1.8.3.1
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

RE: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Hefty, Sean]

Example:

> -		if (rmpp_recv->slid == ah_attr.dlid) {
> +		if (rmpp_recv->slid == (u16)ah_attr.dlid) {

>  struct ib_ah_attr {
>  	struct ib_global_route	grh;
> -	u16			dlid;
> +	u32			dlid;

There are a bunch of places where the lid is simply cast back to a u16.  How is the code supposed to determine if a u16 cast is a safe operation or not?

I don’t like the overloading of the term 'lid' to sometimes mean a 16-bit value and other times a 32-bit value, with no way of distinguishing which one we have.  Maybe we need to call out a version (lidv2 = 32-bits) or carry the size.  I don't have a strong idea here.  But when looking at this patch, lid is re-defined to u32, then nearly everywhere cast back to u16, which looks questionable.

- Sean 
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Chandramouli, Dasaratharaman]

On 12/8/2016 1:12 PM, Hefty, Sean wrote:
> Example:
>
>> -		if (rmpp_recv->slid == ah_attr.dlid) {
>> +		if (rmpp_recv->slid == (u16)ah_attr.dlid) {
>
>>  struct ib_ah_attr {
>>  	struct ib_global_route	grh;
>> -	u16			dlid;
>> +	u32			dlid;
>
> There are a bunch of places where the lid is simply cast back to a u16.  How is the code supposed to determine if a u16 cast is a safe operation or not?

The idea is that all underlying drivers which use the dlid field from 
the address handle will use then after casting to u16. That way there is 
no change in the driver behavior.

The only exception is the hfi1 driver which will use the full 32 bit 
lid. Patches specific to hfi1 driver are in the works and will be posted 
once this patch series is accepted.

ULPs, and other core data structures like the ones in rpmm_recv have 
been modified to be 32 bits. The u16 cast you see in this patch was 
added to avoid compiler warnings. Patch 07/11 removes the cast when 
rmpp_recv->slid is increased to 32 bits. Like Jason pointed out before, 
i will re-order these patches which will avoid some of these temporary 
casts.
>
> I don’t like the overloading of the term 'lid' to sometimes mean a 16-bit value and other times a 32-bit value, with no way of distinguishing which one we have.  Maybe we need to call out a version (lidv2 = 32-bits) or carry the size.  I don't have a strong idea here.  But when looking at this patch, lid is re-defined to u32, then nearly everywhere cast back to u16, which looks questionable.
The intention was to not cause too many changes to the underlying 
drivers when the lid field was increased. Casting was a simpler approach.

In summary, once all these patches are through, the cast to u16 only be 
in the drivers and not in the core data structures.
>
> - Sean
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Chandramouli, Dasaratharaman]

On 12/8/2016 7:13 AM, Leon Romanovsky wrote:
> On Tue, Nov 22, 2016 at 02:38:44PM -0500, Dasaratharaman Chandramouli wrote:
>> dlid field in ah_attr is increased to 32 bits. This
>> enables core components to use the larger addresses if needed.
>> The user ABI is unchanged and userspace applications can use
>> 16 bit lids when creating and modifying address handles.
>>
>> Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> Signed-off-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
>> ---
>
> <...>
>
>> diff --git a/include/rdma/ib_verbs.h b/include/rdma/ib_verbs.h
>> index 52216f6..294d3ed 100644
>> --- a/include/rdma/ib_verbs.h
>> +++ b/include/rdma/ib_verbs.h
>> @@ -814,7 +814,7 @@ struct ib_mr_status {
>>
>>  struct ib_ah_attr {
>>  	struct ib_global_route	grh;
>> -	u16			dlid;
>> +	u32			dlid;
>>  	u8			sl;
>>  	u8			src_path_bits;
>>  	u8			static_rate;
>
> It looks like this structure can be reshuffled. Can you run pahole on it, please?
Will run before posting the next revision. Thanks.
>
> Thanks
>
>> --
>> 1.8.3.1
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
>> the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 10/11] IB/IPoIB: Modify ipoib_get_net_dev_by_params to lookup gid table

[Chandramouli, Dasaratharaman]

On 11/22/2016 1:20 PM, Jason Gunthorpe wrote:
> On Tue, Nov 22, 2016 at 02:38:51PM -0500, Dasaratharaman Chandramouli wrote:
>> ipoib_get_net_dev_by_params compares incoming gid with local_gid
>> which is gid at index 0 of the gid table. OPA devices using larger
>> LIDs may have a different GID format than whats setup in the local_gid
>> field. Do a search of the gid table in those cases.
>
> Sorry, NAK, the incoming GID has to exactly match what is in IPoIB
> LLADDR, you can't do this or it breaks how the virtualization stuff is
> supposed to work with gid aliases.
>
> I'm skeptical about the other ipoib patch as well since ipoib should
> be dealing natively with 32 bit lids, I think you need to fix the path
> record stuff too instead of opening coding that in ipoib.

Thanks for the clarification. Will attempt to keep the incoming GID 
consistent with the GID assigned to the IPoIB interface.
Regarding your comments on path records, we are looking into supporting 
32 bit LIDs in the path record response.
Will post the revised patches.

>
> Jason
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Jason Gunthorpe]

On Thu, Dec 08, 2016 at 09:12:54PM +0000, Hefty, Sean wrote:
> Example:
> 
> > -		if (rmpp_recv->slid == ah_attr.dlid) {
> > +		if (rmpp_recv->slid == (u16)ah_attr.dlid) {
> 
> >  struct ib_ah_attr {
> >  	struct ib_global_route	grh;
> > -	u16			dlid;
> > +	u32			dlid;
> 
> There are a bunch of places where the lid is simply cast back to a
> u16.  How is the code supposed to determine if a u16 cast is a safe
> operation or not?
> 
> I don?t like the overloading of the term 'lid' to sometimes mean a
> 16-bit value and other times a 32-bit value, with no way of
> distinguishing which one we have.  Maybe we need to call out a
> version (lidv2 = 32-bits) or carry the size.  I don't have a strong
> idea here.  But when looking at this patch, lid is re-defined to
> u32, then nearly everywhere cast back to u16, which looks
> questionable.

Yes, I also don't like this.

I would be much happier to see a proper tagged union introduced for
storing the unpacked AH and have that replace all of these hacky
places. That would help with rocee and iwarp too..

struct rdma_ah_data {
    enum rdma_ah_kind kind;
    union
    {
        struct ib_ah_attr ib;
        struct opa_ah_attr opa;
        struct rocev1_ah_attr rocev1;
        struct rocev2_ah_attr rocev2;
        struct iwarp_ah_attr iwarp;
    };
};

We really need to start tagging our data properly or we are going to
have a hard time maintaining this.

Jason
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Hefty, Sean]

> > I don’t like the overloading of the term 'lid' to sometimes mean a
> 16-bit value and other times a 32-bit value, with no way of
> distinguishing which one we have.  Maybe we need to call out a version
> (lidv2 = 32-bits) or carry the size.  I don't have a strong idea here.
> But when looking at this patch, lid is re-defined to u32, then nearly
> everywhere cast back to u16, which looks questionable.
> The intention was to not cause too many changes to the underlying
> drivers when the lid field was increased. Casting was a simpler
> approach.

The problem is this is how most of the stack has evolved, and the resulting architecture is a mess.  This is fundamentally changing the definition of what a LID is.
 
For example, why stop at 4 bytes?  Why not let the LID store a MAC address?

At some point we need to deal with the fact that we have devices that support completely different L2, L3, and/or L4 addresses.  And even where the addresses are the same size, they aren't from the same domain.

Does anyone know how the net stack handles this?  Are we looking at an array of bytes + length?
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: [PATCH v2 03/11] IB/core: Change ah_attr.dlid from 16 to 32 bits

[Hefty, Sean]

> struct rdma_ah_data {
>     enum rdma_ah_kind kind;
>     union
>     {
>         struct ib_ah_attr ib;
>         struct opa_ah_attr opa;
>         struct rocev1_ah_attr rocev1;
>         struct rocev2_ah_attr rocev2;
>         struct iwarp_ah_attr iwarp;
>     };
> };

This approach looks good to me.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html