* [PATCH rdma-next 0/6] Add support for OPA path records
@ 2017-04-17 20:56 Dasaratharaman Chandramouli
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
0 siblings, 1 reply; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
Traditional IB path records have 16 bit SLID and DLID
fields that cannot really be used by applications and ULPs working
with 32-bit LIDs. OPA path records add support for 32 bit
SLIDs and DLIDs. Support for OPA path records is specified
by bit 26 of the capmask2 field in the OPA class portinfo.
Note that these patches apply on the earlier patch series with subject
"[PATCH v2 rdma-next 00/18] Enhance ib_ah_attr to specify types"
This patches are organized as follows:
* Patch 1 performs checkpatch fixes that later patches
touch upon.
* Patch 2 renames ib_sa_path_rec to sa_path_rec
* Patch 3 introduces the 'rec_type' field that identifies
a path record type. This field replaces the existing
'gid_type' field.
* Patch 4 splits the existing sa_path_rec into a union of
sa_path_rec_ib and sa_path_rec_roce structures. Fields that
are common to both continue to be part of sa_path_rec.
* Patch 5 extends sa_path_rec to have path records of type opa
* Patch 6 enables SA component to query for OPA path records if
ULPs and other kernel components request for it
Dasaratharaman Chandramouli (6):
IB/CM: Add braces when using sizeof
IB/SA: Rename ib_sa_path_rec to sa_path_rec
IB/SA: Introduce path record specific types
IB/SA: Split struct sa_path_rec based on IB and ROCE specific fields
IB/SA: Add OPA path record type
IB/SA: Add support to query OPA path records
drivers/infiniband/core/cm.c | 103 ++++++----
drivers/infiniband/core/cma.c | 50 +++--
drivers/infiniband/core/sa_query.c | 320 ++++++++++++++++++++++++++----
drivers/infiniband/core/ucm.c | 6 +-
drivers/infiniband/core/ucma.c | 24 ++-
drivers/infiniband/core/uverbs_marshall.c | 52 +++--
drivers/infiniband/ulp/ipoib/ipoib.h | 2 +-
drivers/infiniband/ulp/ipoib/ipoib_cm.c | 6 +-
drivers/infiniband/ulp/ipoib/ipoib_fs.c | 6 +-
drivers/infiniband/ulp/ipoib/ipoib_main.c | 22 +-
drivers/infiniband/ulp/srp/ib_srp.c | 13 +-
drivers/infiniband/ulp/srp/ib_srp.h | 2 +-
drivers/infiniband/ulp/srpt/ib_srpt.c | 3 +-
include/rdma/ib_cm.h | 14 +-
include/rdma/ib_mad.h | 13 ++
include/rdma/ib_marshall.h | 4 +-
include/rdma/ib_sa.h | 287 +++++++++++++++++++++++++--
include/rdma/opa_addr.h | 79 ++++++++
include/rdma/rdma_cm.h | 2 +-
include/rdma/rdma_cm_ib.h | 2 +-
20 files changed, 849 insertions(+), 161 deletions(-)
create mode 100644 include/rdma/opa_addr.h
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* [PATCH rdma-next 1/6] IB/CM: Add braces when using sizeof
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
@ 2017-04-17 20:56 ` Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 2/6] IB/SA: Rename ib_sa_path_rec to sa_path_rec Dasaratharaman Chandramouli
` (4 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
This patch adds braces around parameters to sizeof
as called out by checkpatch
Reviewed-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/cm.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index d35c2de..53661f34 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -1404,7 +1404,7 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
struct ib_sa_path_rec *primary_path,
struct ib_sa_path_rec *alt_path)
{
- memset(primary_path, 0, sizeof *primary_path);
+ memset(primary_path, 0, sizeof(*primary_path));
primary_path->dgid = req_msg->primary_local_gid;
primary_path->sgid = req_msg->primary_remote_gid;
primary_path->dlid = req_msg->primary_local_lid;
@@ -1426,7 +1426,7 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
primary_path->service_id = req_msg->service_id;
if (req_msg->alt_local_lid) {
- memset(alt_path, 0, sizeof *alt_path);
+ memset(alt_path, 0, sizeof(*alt_path));
alt_path->dgid = req_msg->alt_local_gid;
alt_path->sgid = req_msg->alt_remote_gid;
alt_path->dlid = req_msg->alt_local_lid;
@@ -3712,7 +3712,7 @@ static void cm_recv_handler(struct ib_mad_agent *mad_agent,
atomic_long_inc(&port->counter_group[CM_RECV].
counter[attr_id - CM_ATTR_ID_OFFSET]);
- work = kmalloc(sizeof *work + sizeof(struct ib_sa_path_rec) * paths,
+ work = kmalloc(sizeof(*work) + sizeof(struct ib_sa_path_rec) * paths,
GFP_KERNEL);
if (!work) {
ib_free_recv_mad(mad_recv_wc);
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH rdma-next 2/6] IB/SA: Rename ib_sa_path_rec to sa_path_rec
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-17 20:56 ` [PATCH rdma-next 1/6] IB/CM: Add braces when using sizeof Dasaratharaman Chandramouli
@ 2017-04-17 20:56 ` Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 3/6] IB/SA: Introduce path record specific types Dasaratharaman Chandramouli
` (3 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
Rename ib_sa_path_rec to a more generic sa_path_rec.
This is part of extending ib_sa to also support OPA
path records in addition to the IB defined path records.
Reviewed-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/cm.c | 20 ++++++++++----------
drivers/infiniband/core/cma.c | 8 ++++----
drivers/infiniband/core/sa_query.c | 20 ++++++++++----------
drivers/infiniband/core/ucm.c | 6 +++---
drivers/infiniband/core/ucma.c | 2 +-
drivers/infiniband/core/uverbs_marshall.c | 4 ++--
drivers/infiniband/ulp/ipoib/ipoib.h | 2 +-
drivers/infiniband/ulp/ipoib/ipoib_cm.c | 6 +++---
drivers/infiniband/ulp/ipoib/ipoib_main.c | 2 +-
drivers/infiniband/ulp/srp/ib_srp.c | 2 +-
drivers/infiniband/ulp/srp/ib_srp.h | 2 +-
include/rdma/ib_cm.h | 14 +++++++-------
include/rdma/ib_marshall.h | 4 ++--
include/rdma/ib_sa.h | 14 +++++++-------
include/rdma/rdma_cm.h | 2 +-
include/rdma/rdma_cm_ib.h | 2 +-
16 files changed, 55 insertions(+), 55 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index 53661f34..fe83823 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -241,7 +241,7 @@ struct cm_work {
__be32 local_id; /* Established / timewait */
__be32 remote_id;
struct ib_cm_event cm_event;
- struct ib_sa_path_rec path[0];
+ struct sa_path_rec path[0];
};
struct cm_timewait_info {
@@ -440,7 +440,7 @@ static void cm_init_av_for_response(struct cm_port *port, struct ib_wc *wc,
grh, &av->ah_attr);
}
-static int cm_init_av_by_path(struct ib_sa_path_rec *path, struct cm_av *av,
+static int cm_init_av_by_path(struct sa_path_rec *path, struct cm_av *av,
struct cm_id_private *cm_id_priv)
{
struct cm_device *cm_dev;
@@ -1172,8 +1172,8 @@ static void cm_format_req(struct cm_req_msg *req_msg,
struct cm_id_private *cm_id_priv,
struct ib_cm_req_param *param)
{
- struct ib_sa_path_rec *pri_path = param->primary_path;
- struct ib_sa_path_rec *alt_path = param->alternate_path;
+ struct sa_path_rec *pri_path = param->primary_path;
+ struct sa_path_rec *alt_path = param->alternate_path;
cm_format_mad_hdr(&req_msg->hdr, CM_REQ_ATTR_ID,
cm_form_tid(cm_id_priv, CM_MSG_SEQUENCE_REQ));
@@ -1401,8 +1401,8 @@ static inline int cm_is_active_peer(__be64 local_ca_guid, __be64 remote_ca_guid,
}
static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
- struct ib_sa_path_rec *primary_path,
- struct ib_sa_path_rec *alt_path)
+ struct sa_path_rec *primary_path,
+ struct sa_path_rec *alt_path)
{
memset(primary_path, 0, sizeof(*primary_path));
primary_path->dgid = req_msg->primary_local_gid;
@@ -2815,7 +2815,7 @@ static int cm_mra_handler(struct cm_work *work)
static void cm_format_lap(struct cm_lap_msg *lap_msg,
struct cm_id_private *cm_id_priv,
- struct ib_sa_path_rec *alternate_path,
+ struct sa_path_rec *alternate_path,
const void *private_data,
u8 private_data_len)
{
@@ -2845,7 +2845,7 @@ static void cm_format_lap(struct cm_lap_msg *lap_msg,
}
int ib_send_cm_lap(struct ib_cm_id *cm_id,
- struct ib_sa_path_rec *alternate_path,
+ struct sa_path_rec *alternate_path,
const void *private_data,
u8 private_data_len)
{
@@ -2899,7 +2899,7 @@ int ib_send_cm_lap(struct ib_cm_id *cm_id,
EXPORT_SYMBOL(ib_send_cm_lap);
static void cm_format_path_from_lap(struct cm_id_private *cm_id_priv,
- struct ib_sa_path_rec *path,
+ struct sa_path_rec *path,
struct cm_lap_msg *lap_msg)
{
memset(path, 0, sizeof *path);
@@ -3712,7 +3712,7 @@ static void cm_recv_handler(struct ib_mad_agent *mad_agent,
atomic_long_inc(&port->counter_group[CM_RECV].
counter[attr_id - CM_ATTR_ID_OFFSET]);
- work = kmalloc(sizeof(*work) + sizeof(struct ib_sa_path_rec) * paths,
+ work = kmalloc(sizeof(*work) + sizeof(struct sa_path_rec) * paths,
GFP_KERNEL);
if (!work) {
ib_free_recv_mad(mad_recv_wc);
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 21244f7..1185d58 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1128,7 +1128,7 @@ static inline int cma_any_port(struct sockaddr *addr)
static void cma_save_ib_info(struct sockaddr *src_addr,
struct sockaddr *dst_addr,
struct rdma_cm_id *listen_id,
- struct ib_sa_path_rec *path)
+ struct sa_path_rec *path)
{
struct sockaddr_ib *listen_ib, *ib;
@@ -2298,7 +2298,7 @@ void rdma_set_service_type(struct rdma_cm_id *id, int tos)
}
EXPORT_SYMBOL(rdma_set_service_type);
-static void cma_query_handler(int status, struct ib_sa_path_rec *path_rec,
+static void cma_query_handler(int status, struct sa_path_rec *path_rec,
void *context)
{
struct cma_work *work = context;
@@ -2325,7 +2325,7 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms,
struct cma_work *work)
{
struct rdma_dev_addr *dev_addr = &id_priv->id.route.addr.dev_addr;
- struct ib_sa_path_rec path_rec;
+ struct sa_path_rec path_rec;
ib_sa_comp_mask comp_mask;
struct sockaddr_in6 *sin6;
struct sockaddr_ib *sib;
@@ -2450,7 +2450,7 @@ static int cma_resolve_ib_route(struct rdma_id_private *id_priv, int timeout_ms)
}
int rdma_set_ib_paths(struct rdma_cm_id *id,
- struct ib_sa_path_rec *path_rec, int num_paths)
+ struct sa_path_rec *path_rec, int num_paths)
{
struct rdma_id_private *id_priv;
int ret;
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index c04ac94..87ffc82 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -129,7 +129,7 @@ struct ib_sa_service_query {
};
struct ib_sa_path_query {
- void (*callback)(int, struct ib_sa_path_rec *, void *);
+ void (*callback)(int, struct sa_path_rec *, void *);
void *context;
struct ib_sa_query sa_query;
};
@@ -188,8 +188,8 @@ struct ib_sa_mcmember_query {
static u32 tid;
#define PATH_REC_FIELD(field) \
- .struct_offset_bytes = offsetof(struct ib_sa_path_rec, field), \
- .struct_size_bytes = sizeof ((struct ib_sa_path_rec *) 0)->field, \
+ .struct_offset_bytes = offsetof(struct sa_path_rec, field), \
+ .struct_size_bytes = sizeof((struct sa_path_rec *)0)->field, \
.field_name = "sa_path_rec:" #field
static const struct ib_field path_rec_table[] = {
@@ -618,7 +618,7 @@ static inline int ib_sa_query_cancelled(struct ib_sa_query *query)
static void ib_nl_set_path_rec_attrs(struct sk_buff *skb,
struct ib_sa_query *query)
{
- struct ib_sa_path_rec *sa_rec = query->mad_buf->context[1];
+ struct sa_path_rec *sa_rec = query->mad_buf->context[1];
struct ib_sa_mad *mad = query->mad_buf->mad;
ib_sa_comp_mask comp_mask = mad->sa_hdr.comp_mask;
u16 val16;
@@ -1099,7 +1099,7 @@ static u8 get_src_path_mask(struct ib_device *device, u8 port_num)
}
int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
- struct ib_sa_path_rec *rec,
+ struct sa_path_rec *rec,
struct rdma_ah_attr *ah_attr)
{
int ret;
@@ -1301,13 +1301,13 @@ static int send_mad(struct ib_sa_query *query, int timeout_ms, gfp_t gfp_mask)
return ret ? ret : id;
}
-void ib_sa_unpack_path(void *attribute, struct ib_sa_path_rec *rec)
+void ib_sa_unpack_path(void *attribute, struct sa_path_rec *rec)
{
ib_unpack(path_rec_table, ARRAY_SIZE(path_rec_table), attribute, rec);
}
EXPORT_SYMBOL(ib_sa_unpack_path);
-void ib_sa_pack_path(struct ib_sa_path_rec *rec, void *attribute)
+void ib_sa_pack_path(struct sa_path_rec *rec, void *attribute)
{
ib_pack(path_rec_table, ARRAY_SIZE(path_rec_table), rec, attribute);
}
@@ -1321,7 +1321,7 @@ static void ib_sa_path_rec_callback(struct ib_sa_query *sa_query,
container_of(sa_query, struct ib_sa_path_query, sa_query);
if (mad) {
- struct ib_sa_path_rec rec;
+ struct sa_path_rec rec;
ib_unpack(path_rec_table, ARRAY_SIZE(path_rec_table),
mad->data, &rec);
@@ -1366,11 +1366,11 @@ static void ib_sa_path_rec_release(struct ib_sa_query *sa_query)
*/
int ib_sa_path_rec_get(struct ib_sa_client *client,
struct ib_device *device, u8 port_num,
- struct ib_sa_path_rec *rec,
+ struct sa_path_rec *rec,
ib_sa_comp_mask comp_mask,
int timeout_ms, gfp_t gfp_mask,
void (*callback)(int status,
- struct ib_sa_path_rec *resp,
+ struct sa_path_rec *resp,
void *context),
void *context,
struct ib_sa_query **sa_query)
diff --git a/drivers/infiniband/core/ucm.c b/drivers/infiniband/core/ucm.c
index cc0d51f..80d0fca 100644
--- a/drivers/infiniband/core/ucm.c
+++ b/drivers/infiniband/core/ucm.c
@@ -702,10 +702,10 @@ static int ib_ucm_alloc_data(const void **dest, u64 src, u32 len)
return 0;
}
-static int ib_ucm_path_get(struct ib_sa_path_rec **path, u64 src)
+static int ib_ucm_path_get(struct sa_path_rec **path, u64 src)
{
struct ib_user_path_rec upath;
- struct ib_sa_path_rec *sa_path;
+ struct sa_path_rec *sa_path;
*path = NULL;
@@ -962,7 +962,7 @@ static ssize_t ib_ucm_send_lap(struct ib_ucm_file *file,
int in_len, int out_len)
{
struct ib_ucm_context *ctx;
- struct ib_sa_path_rec *path = NULL;
+ struct sa_path_rec *path = NULL;
struct ib_ucm_lap cmd;
const void *data = NULL;
int result;
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index e12f8fa..2beff90 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -1197,7 +1197,7 @@ static int ucma_set_option_id(struct ucma_context *ctx, int optname,
static int ucma_set_ib_path(struct ucma_context *ctx,
struct ib_path_rec_data *path_data, size_t optlen)
{
- struct ib_sa_path_rec sa_path;
+ struct sa_path_rec sa_path;
struct rdma_cm_event event;
int ret;
diff --git a/drivers/infiniband/core/uverbs_marshall.c b/drivers/infiniband/core/uverbs_marshall.c
index cb4ba16..eda6f30 100644
--- a/drivers/infiniband/core/uverbs_marshall.c
+++ b/drivers/infiniband/core/uverbs_marshall.c
@@ -97,7 +97,7 @@ void ib_copy_qp_attr_to_user(struct ib_uverbs_qp_attr *dst,
EXPORT_SYMBOL(ib_copy_qp_attr_to_user);
void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
- struct ib_sa_path_rec *src)
+ struct sa_path_rec *src)
{
memcpy(dst->dgid, src->dgid.raw, sizeof src->dgid);
memcpy(dst->sgid, src->sgid.raw, sizeof src->sgid);
@@ -122,7 +122,7 @@ void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
}
EXPORT_SYMBOL(ib_copy_path_rec_to_user);
-void ib_copy_path_rec_from_user(struct ib_sa_path_rec *dst,
+void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
struct ib_user_path_rec *src)
{
memcpy(dst->dgid.raw, src->dgid, sizeof dst->dgid);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib.h b/drivers/infiniband/ulp/ipoib/ipoib.h
index 866ecfa..8b657cb 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib.h
+++ b/drivers/infiniband/ulp/ipoib/ipoib.h
@@ -416,7 +416,7 @@ struct ipoib_ah {
struct ipoib_path {
struct net_device *dev;
- struct ib_sa_path_rec pathrec;
+ struct sa_path_rec pathrec;
struct ipoib_ah *ah;
struct sk_buff_head queue;
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_cm.c b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
index 0cdf2b7..a13d3d6 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_cm.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_cm.c
@@ -1068,7 +1068,7 @@ static struct ib_qp *ipoib_cm_create_tx_qp(struct net_device *dev, struct ipoib_
static int ipoib_cm_send_req(struct net_device *dev,
struct ib_cm_id *id, struct ib_qp *qp,
u32 qpn,
- struct ib_sa_path_rec *pathrec)
+ struct sa_path_rec *pathrec)
{
struct ipoib_dev_priv *priv = netdev_priv(dev);
struct ipoib_cm_data data = {};
@@ -1128,7 +1128,7 @@ static int ipoib_cm_modify_tx_init(struct net_device *dev,
}
static int ipoib_cm_tx_init(struct ipoib_cm_tx *p, u32 qpn,
- struct ib_sa_path_rec *pathrec)
+ struct sa_path_rec *pathrec)
{
struct ipoib_dev_priv *priv = netdev_priv(p->dev);
int ret;
@@ -1332,7 +1332,7 @@ static void ipoib_cm_tx_start(struct work_struct *work)
struct ipoib_path *path;
int ret;
- struct ib_sa_path_rec pathrec;
+ struct sa_path_rec pathrec;
u32 qpn;
netif_tx_lock_bh(dev);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 63c8092..5e7939f 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -716,7 +716,7 @@ void ipoib_flush_paths(struct net_device *dev)
}
static void path_rec_completion(int status,
- struct ib_sa_path_rec *pathrec,
+ struct sa_path_rec *pathrec,
void *path_ptr)
{
struct ipoib_path *path = path_ptr;
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index cee4626..ded9b9a 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -643,7 +643,7 @@ static void srp_free_ch_ib(struct srp_target_port *target,
}
static void srp_path_rec_completion(int status,
- struct ib_sa_path_rec *pathrec,
+ struct sa_path_rec *pathrec,
void *ch_ptr)
{
struct srp_rdma_ch *ch = ch_ptr;
diff --git a/drivers/infiniband/ulp/srp/ib_srp.h b/drivers/infiniband/ulp/srp/ib_srp.h
index 32ed40d..ab9077b 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.h
+++ b/drivers/infiniband/ulp/srp/ib_srp.h
@@ -152,7 +152,7 @@ struct srp_rdma_ch {
struct completion done;
int status;
- struct ib_sa_path_rec path;
+ struct sa_path_rec path;
struct ib_sa_query *path_query;
int path_query_id;
diff --git a/include/rdma/ib_cm.h b/include/rdma/ib_cm.h
index b49258b..7979cb0 100644
--- a/include/rdma/ib_cm.h
+++ b/include/rdma/ib_cm.h
@@ -117,8 +117,8 @@ struct ib_cm_req_event_param {
u8 port;
- struct ib_sa_path_rec *primary_path;
- struct ib_sa_path_rec *alternate_path;
+ struct sa_path_rec *primary_path;
+ struct sa_path_rec *alternate_path;
__be64 remote_ca_guid;
u32 remote_qkey;
@@ -197,7 +197,7 @@ struct ib_cm_mra_event_param {
};
struct ib_cm_lap_event_param {
- struct ib_sa_path_rec *alternate_path;
+ struct sa_path_rec *alternate_path;
};
enum ib_cm_apr_status {
@@ -363,8 +363,8 @@ struct ib_cm_id *ib_cm_insert_listen(struct ib_device *device,
__be64 service_id);
struct ib_cm_req_param {
- struct ib_sa_path_rec *primary_path;
- struct ib_sa_path_rec *alternate_path;
+ struct sa_path_rec *primary_path;
+ struct sa_path_rec *alternate_path;
__be64 service_id;
u32 qp_num;
enum ib_qp_type qp_type;
@@ -521,7 +521,7 @@ int ib_send_cm_mra(struct ib_cm_id *cm_id,
* @private_data_len: Size of the private data buffer, in bytes.
*/
int ib_send_cm_lap(struct ib_cm_id *cm_id,
- struct ib_sa_path_rec *alternate_path,
+ struct sa_path_rec *alternate_path,
const void *private_data,
u8 private_data_len);
@@ -565,7 +565,7 @@ int ib_send_cm_apr(struct ib_cm_id *cm_id,
u8 private_data_len);
struct ib_cm_sidr_req_param {
- struct ib_sa_path_rec *path;
+ struct sa_path_rec *path;
__be64 service_id;
int timeout_ms;
const void *private_data;
diff --git a/include/rdma/ib_marshall.h b/include/rdma/ib_marshall.h
index 42a0fc6..68cef3b 100644
--- a/include/rdma/ib_marshall.h
+++ b/include/rdma/ib_marshall.h
@@ -45,9 +45,9 @@ void ib_copy_ah_attr_to_user(struct ib_uverbs_ah_attr *dst,
struct rdma_ah_attr *src);
void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
- struct ib_sa_path_rec *src);
+ struct sa_path_rec *src);
-void ib_copy_path_rec_from_user(struct ib_sa_path_rec *dst,
+void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
struct ib_user_path_rec *src);
#endif /* IB_USER_MARSHALL_H */
diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h
index 3ff8e64..7efdfb1 100644
--- a/include/rdma/ib_sa.h
+++ b/include/rdma/ib_sa.h
@@ -148,7 +148,7 @@ enum ib_sa_mc_join_states {
#define IB_SA_PATH_REC_PACKET_LIFE_TIME IB_SA_COMP_MASK(21)
#define IB_SA_PATH_REC_PREFERENCE IB_SA_COMP_MASK(22)
-struct ib_sa_path_rec {
+struct sa_path_rec {
__be64 service_id;
union ib_gid dgid;
union ib_gid sgid;
@@ -179,7 +179,7 @@ struct ib_sa_path_rec {
enum ib_gid_type gid_type;
};
-static inline struct net_device *ib_get_ndev_from_path(struct ib_sa_path_rec *rec)
+static inline struct net_device *ib_get_ndev_from_path(struct sa_path_rec *rec)
{
return rec->net ? dev_get_by_index(rec->net, rec->ifindex) : NULL;
}
@@ -323,11 +323,11 @@ struct ib_sa_client {
int ib_sa_path_rec_get(struct ib_sa_client *client,
struct ib_device *device, u8 port_num,
- struct ib_sa_path_rec *rec,
+ struct sa_path_rec *rec,
ib_sa_comp_mask comp_mask,
int timeout_ms, gfp_t gfp_mask,
void (*callback)(int status,
- struct ib_sa_path_rec *resp,
+ struct sa_path_rec *resp,
void *context),
void *context,
struct ib_sa_query **query);
@@ -428,20 +428,20 @@ int ib_init_ah_from_mcmember(struct ib_device *device, u8 port_num,
* path record.
*/
int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
- struct ib_sa_path_rec *rec,
+ struct sa_path_rec *rec,
struct rdma_ah_attr *ah_attr);
/**
* ib_sa_pack_path - Conert a path record from struct ib_sa_path_rec
* to IB MAD wire format.
*/
-void ib_sa_pack_path(struct ib_sa_path_rec *rec, void *attribute);
+void ib_sa_pack_path(struct sa_path_rec *rec, void *attribute);
/**
* ib_sa_unpack_path - Convert a path record from MAD format to struct
* ib_sa_path_rec.
*/
-void ib_sa_unpack_path(void *attribute, struct ib_sa_path_rec *rec);
+void ib_sa_unpack_path(void *attribute, struct sa_path_rec *rec);
/* Support GuidInfoRecord */
int ib_sa_guid_info_rec_query(struct ib_sa_client *client,
diff --git a/include/rdma/rdma_cm.h b/include/rdma/rdma_cm.h
index c9ac1e1..3d2eed3 100644
--- a/include/rdma/rdma_cm.h
+++ b/include/rdma/rdma_cm.h
@@ -85,7 +85,7 @@ struct rdma_addr {
struct rdma_route {
struct rdma_addr addr;
- struct ib_sa_path_rec *path_rec;
+ struct sa_path_rec *path_rec;
int num_paths;
};
diff --git a/include/rdma/rdma_cm_ib.h b/include/rdma/rdma_cm_ib.h
index 2389c3b..6947a6b 100644
--- a/include/rdma/rdma_cm_ib.h
+++ b/include/rdma/rdma_cm_ib.h
@@ -46,7 +46,7 @@
* connection and replaces the call to rdma_resolve_route.
*/
int rdma_set_ib_paths(struct rdma_cm_id *id,
- struct ib_sa_path_rec *path_rec, int num_paths);
+ struct sa_path_rec *path_rec, int num_paths);
/* Global qkey for UDP QPs and multicast groups. */
#define RDMA_UDP_QKEY 0x01234567
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH rdma-next 3/6] IB/SA: Introduce path record specific types
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-17 20:56 ` [PATCH rdma-next 1/6] IB/CM: Add braces when using sizeof Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 2/6] IB/SA: Rename ib_sa_path_rec to sa_path_rec Dasaratharaman Chandramouli
@ 2017-04-17 20:56 ` Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 4/6] IB/SA: Split struct sa_path_rec based on IB and ROCE specific fields Dasaratharaman Chandramouli
` (2 subsequent siblings)
5 siblings, 0 replies; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
struct sa_path_rec has a gid_type field. This patch introduces a more
generic path record specific type 'rec_type' which is either IB, ROCE v1
or ROCE v2. The patch also provides conversion functions to get
a gid type from a path record type and vice versa
Reviewed-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/cm.c | 15 +++++++++++---
drivers/infiniband/core/cma.c | 16 +++++++++------
drivers/infiniband/core/sa_query.c | 14 ++++++++-----
drivers/infiniband/core/uverbs_marshall.c | 2 +-
include/rdma/ib_sa.h | 33 ++++++++++++++++++++++++++++++-
5 files changed, 64 insertions(+), 16 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index fe83823..39e306a 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -453,7 +453,8 @@ static int cm_init_av_by_path(struct sa_path_rec *path, struct cm_av *av,
read_lock_irqsave(&cm.device_lock, flags);
list_for_each_entry(cm_dev, &cm.device_list, list) {
if (!ib_find_cached_gid(cm_dev->ib_device, &path->sgid,
- path->gid_type, ndev, &p, NULL)) {
+ sa_conv_pathrec_to_gid_type(path),
+ ndev, &p, NULL)) {
port = cm_dev->port[p-1];
break;
}
@@ -1775,8 +1776,11 @@ static int cm_req_handler(struct cm_work *work)
work->path[0].ifindex = gid_attr.ndev->ifindex;
work->path[0].net = dev_net(gid_attr.ndev);
dev_put(gid_attr.ndev);
+ work->path[0].rec_type =
+ sa_conv_gid_to_pathrec_type(gid_attr.gid_type);
+ } else {
+ work->path[0].rec_type = SA_PATH_REC_TYPE_IB;
}
- work->path[0].gid_type = gid_attr.gid_type;
ret = cm_init_av_by_path(&work->path[0], &cm_id_priv->av,
cm_id_priv);
}
@@ -1789,8 +1793,13 @@ static int cm_req_handler(struct cm_work *work)
work->path[0].ifindex = gid_attr.ndev->ifindex;
work->path[0].net = dev_net(gid_attr.ndev);
dev_put(gid_attr.ndev);
+ work->path[0].rec_type =
+ sa_conv_gid_to_pathrec_type(gid_attr.gid_type);
+ } else {
+ work->path[0].rec_type = SA_PATH_REC_TYPE_IB;
}
- work->path[0].gid_type = gid_attr.gid_type;
+ if (req_msg->alt_local_lid)
+ work->path[1].rec_type = work->path[0].rec_type;
ib_send_cm_rej(cm_id, IB_CM_REJ_INVALID_GID,
&work->path[0].sgid, sizeof work->path[0].sgid,
NULL, 0);
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 1185d58..a6195d5 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -2529,6 +2529,7 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
struct cma_work *work;
int ret;
struct net_device *ndev = NULL;
+ enum ib_gid_type gid_type = IB_GID_TYPE_IB;
u8 default_roce_tos = id_priv->cma_dev->default_roce_tos[id_priv->id.port_num -
rdma_start_port(id_priv->cma_dev->device)];
u8 tos = id_priv->tos_set ? id_priv->tos : default_roce_tos;
@@ -2577,10 +2578,11 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
route->path_rec->ifindex = ndev->ifindex;
supported_gids = roce_gid_type_mask_support(id_priv->id.device,
id_priv->id.port_num);
- route->path_rec->gid_type =
- cma_route_gid_type(addr->dev_addr.network,
- supported_gids,
- id_priv->gid_type);
+ gid_type = cma_route_gid_type(addr->dev_addr.network,
+ supported_gids,
+ id_priv->gid_type);
+ route->path_rec->rec_type =
+ sa_conv_gid_to_pathrec_type(gid_type);
}
if (!ndev) {
ret = -ENODEV;
@@ -2595,8 +2597,10 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
&route->path_rec->dgid);
/* Use the hint from IP Stack to select GID Type */
- if (route->path_rec->gid_type < ib_network_to_gid_type(addr->dev_addr.network))
- route->path_rec->gid_type = ib_network_to_gid_type(addr->dev_addr.network);
+ if (gid_type < ib_network_to_gid_type(addr->dev_addr.network))
+ gid_type = ib_network_to_gid_type(addr->dev_addr.network);
+ route->path_rec->rec_type = sa_conv_gid_to_pathrec_type(gid_type);
+
if (((struct sockaddr *)&id_priv->id.route.addr.dst_addr)->sa_family != AF_IB)
/* TODO: get the hoplimit from the inet/inet6 device */
route->path_rec->hop_limit = addr->dev_addr.hoplimit;
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 87ffc82..050a8e6 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -1144,7 +1144,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
if ((dev_addr.network == RDMA_NETWORK_IPV4 ||
dev_addr.network == RDMA_NETWORK_IPV6) &&
- rec->gid_type != IB_GID_TYPE_ROCE_UDP_ENCAP)
+ rec->rec_type != SA_PATH_REC_TYPE_ROCE_V2)
return -EINVAL;
idev = device->get_netdev(device, port_num);
@@ -1175,9 +1175,10 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
}
if (rec->hop_limit > 0 || use_roce) {
- ret = ib_find_cached_gid_by_port(device, &rec->sgid,
- rec->gid_type, port_num, ndev,
- &gid_index);
+ enum ib_gid_type type = sa_conv_pathrec_to_gid_type(rec);
+
+ ret = ib_find_cached_gid_by_port(device, &rec->sgid, type,
+ port_num, ndev, &gid_index);
if (ret) {
if (ndev)
dev_put(ndev);
@@ -1327,7 +1328,7 @@ static void ib_sa_path_rec_callback(struct ib_sa_query *sa_query,
mad->data, &rec);
rec.net = NULL;
rec.ifindex = 0;
- rec.gid_type = IB_GID_TYPE_IB;
+ rec.rec_type = SA_PATH_REC_TYPE_IB;
eth_zero_addr(rec.dmac);
query->callback(status, &rec, query->context);
} else
@@ -1385,6 +1386,9 @@ int ib_sa_path_rec_get(struct ib_sa_client *client,
if (!sa_dev)
return -ENODEV;
+ if (rec->rec_type != SA_PATH_REC_TYPE_IB)
+ return -EINVAL;
+
port = &sa_dev->port[port_num - sa_dev->start_port];
agent = port->agent;
diff --git a/drivers/infiniband/core/uverbs_marshall.c b/drivers/infiniband/core/uverbs_marshall.c
index eda6f30..b4e9ce8 100644
--- a/drivers/infiniband/core/uverbs_marshall.c
+++ b/drivers/infiniband/core/uverbs_marshall.c
@@ -149,6 +149,6 @@ void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
memset(dst->dmac, 0, sizeof(dst->dmac));
dst->net = NULL;
dst->ifindex = 0;
- dst->gid_type = IB_GID_TYPE_IB;
+ dst->rec_type = SA_PATH_REC_TYPE_IB;
}
EXPORT_SYMBOL(ib_copy_path_rec_from_user);
diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h
index 7efdfb1..c009147 100644
--- a/include/rdma/ib_sa.h
+++ b/include/rdma/ib_sa.h
@@ -147,6 +147,11 @@ enum ib_sa_mc_join_states {
#define IB_SA_PATH_REC_PACKET_LIFE_TIME_SELECTOR IB_SA_COMP_MASK(20)
#define IB_SA_PATH_REC_PACKET_LIFE_TIME IB_SA_COMP_MASK(21)
#define IB_SA_PATH_REC_PREFERENCE IB_SA_COMP_MASK(22)
+enum sa_path_rec_type {
+ SA_PATH_REC_TYPE_IB,
+ SA_PATH_REC_TYPE_ROCE_V1,
+ SA_PATH_REC_TYPE_ROCE_V2
+};
struct sa_path_rec {
__be64 service_id;
@@ -176,7 +181,7 @@ struct sa_path_rec {
int ifindex;
/* ignored in IB */
struct net *net;
- enum ib_gid_type gid_type;
+ enum sa_path_rec_type rec_type;
};
static inline struct net_device *ib_get_ndev_from_path(struct sa_path_rec *rec)
@@ -184,6 +189,32 @@ static inline struct net_device *ib_get_ndev_from_path(struct sa_path_rec *rec)
return rec->net ? dev_get_by_index(rec->net, rec->ifindex) : NULL;
}
+static inline enum ib_gid_type
+ sa_conv_pathrec_to_gid_type(struct sa_path_rec *rec)
+{
+ switch (rec->rec_type) {
+ case SA_PATH_REC_TYPE_ROCE_V1:
+ return IB_GID_TYPE_ROCE;
+ case SA_PATH_REC_TYPE_ROCE_V2:
+ return IB_GID_TYPE_ROCE_UDP_ENCAP;
+ default:
+ return IB_GID_TYPE_IB;
+ }
+}
+
+static inline enum sa_path_rec_type
+ sa_conv_gid_to_pathrec_type(enum ib_gid_type type)
+{
+ switch (type) {
+ case IB_GID_TYPE_ROCE:
+ return SA_PATH_REC_TYPE_ROCE_V1;
+ case IB_GID_TYPE_ROCE_UDP_ENCAP:
+ return SA_PATH_REC_TYPE_ROCE_V2;
+ default:
+ return SA_PATH_REC_TYPE_IB;
+ }
+}
+
#define IB_SA_MCMEMBER_REC_MGID IB_SA_COMP_MASK( 0)
#define IB_SA_MCMEMBER_REC_PORT_GID IB_SA_COMP_MASK( 1)
#define IB_SA_MCMEMBER_REC_QKEY IB_SA_COMP_MASK( 2)
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH rdma-next 4/6] IB/SA: Split struct sa_path_rec based on IB and ROCE specific fields
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
` (2 preceding siblings ...)
2017-04-17 20:56 ` [PATCH rdma-next 3/6] IB/SA: Introduce path record specific types Dasaratharaman Chandramouli
@ 2017-04-17 20:56 ` Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 5/6] IB/SA: Add OPA path record type Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records Dasaratharaman Chandramouli
5 siblings, 0 replies; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
sa_path_rec now contains a union of sa_path_rec_ib and sa_path_rec_roce
based on the type of the path record. Note that fields applicable to
path record type ROCE v1 and ROCE v2 fall under sa_path_rec_roce.
Accessor functions are added to these fields so the caller doesn't have
to know the type.
Reviewed-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/cm.c | 62 +++++++------
drivers/infiniband/core/cma.c | 22 +++--
drivers/infiniband/core/sa_query.c | 32 ++++---
drivers/infiniband/core/uverbs_marshall.c | 20 ++--
drivers/infiniband/ulp/ipoib/ipoib_fs.c | 6 +-
drivers/infiniband/ulp/ipoib/ipoib_main.c | 11 ++-
drivers/infiniband/ulp/srp/ib_srp.c | 7 +-
include/rdma/ib_sa.h | 147 +++++++++++++++++++++++++++---
8 files changed, 224 insertions(+), 83 deletions(-)
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index 39e306a..ca742e8 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -1203,8 +1203,8 @@ static void cm_format_req(struct cm_req_msg *req_msg,
}
if (pri_path->hop_limit <= 1) {
- req_msg->primary_local_lid = pri_path->slid;
- req_msg->primary_remote_lid = pri_path->dlid;
+ req_msg->primary_local_lid = sa_path_get_slid(pri_path);
+ req_msg->primary_remote_lid = sa_path_get_dlid(pri_path);
} else {
/* Work-around until there's a way to obtain remote LID info */
req_msg->primary_local_lid = IB_LID_PERMISSIVE;
@@ -1224,8 +1224,8 @@ static void cm_format_req(struct cm_req_msg *req_msg,
if (alt_path) {
if (alt_path->hop_limit <= 1) {
- req_msg->alt_local_lid = alt_path->slid;
- req_msg->alt_remote_lid = alt_path->dlid;
+ req_msg->alt_local_lid = sa_path_get_slid(alt_path);
+ req_msg->alt_remote_lid = sa_path_get_dlid(alt_path);
} else {
req_msg->alt_local_lid = IB_LID_PERMISSIVE;
req_msg->alt_remote_lid = IB_LID_PERMISSIVE;
@@ -1405,11 +1405,10 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
struct sa_path_rec *primary_path,
struct sa_path_rec *alt_path)
{
- memset(primary_path, 0, sizeof(*primary_path));
primary_path->dgid = req_msg->primary_local_gid;
primary_path->sgid = req_msg->primary_remote_gid;
- primary_path->dlid = req_msg->primary_local_lid;
- primary_path->slid = req_msg->primary_remote_lid;
+ sa_path_set_dlid(primary_path, req_msg->primary_local_lid);
+ sa_path_set_slid(primary_path, req_msg->primary_remote_lid);
primary_path->flow_label = cm_req_get_primary_flow_label(req_msg);
primary_path->hop_limit = req_msg->primary_hop_limit;
primary_path->traffic_class = req_msg->primary_traffic_class;
@@ -1424,14 +1423,13 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
primary_path->packet_life_time =
cm_req_get_primary_local_ack_timeout(req_msg);
primary_path->packet_life_time -= (primary_path->packet_life_time > 0);
- primary_path->service_id = req_msg->service_id;
+ sa_path_set_service_id(primary_path, req_msg->service_id);
if (req_msg->alt_local_lid) {
- memset(alt_path, 0, sizeof(*alt_path));
alt_path->dgid = req_msg->alt_local_gid;
alt_path->sgid = req_msg->alt_remote_gid;
- alt_path->dlid = req_msg->alt_local_lid;
- alt_path->slid = req_msg->alt_remote_lid;
+ sa_path_set_dlid(alt_path, req_msg->alt_local_lid);
+ sa_path_set_slid(alt_path, req_msg->alt_remote_lid);
alt_path->flow_label = cm_req_get_alt_flow_label(req_msg);
alt_path->hop_limit = req_msg->alt_hop_limit;
alt_path->traffic_class = req_msg->alt_traffic_class;
@@ -1446,7 +1444,7 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
alt_path->packet_life_time =
cm_req_get_alt_local_ack_timeout(req_msg);
alt_path->packet_life_time -= (alt_path->packet_life_time > 0);
- alt_path->service_id = req_msg->service_id;
+ sa_path_set_service_id(alt_path, req_msg->service_id);
}
}
@@ -1760,27 +1758,34 @@ static int cm_req_handler(struct cm_work *work)
cm_id_priv->id.service_mask = ~cpu_to_be64(0);
cm_process_routed_req(req_msg, work->mad_recv_wc->wc);
- cm_format_paths_from_req(req_msg, &work->path[0], &work->path[1]);
- if (cm_id_priv->av.ah_attr.type == RDMA_AH_ATTR_TYPE_ROCE)
- memcpy(work->path[0].dmac, cm_id_priv->av.ah_attr.roce.dmac,
- ETH_ALEN);
+ memset(&work->path[0], 0, sizeof(work->path[0]));
+ memset(&work->path[1], 0, sizeof(work->path[1]));
grh = rdma_ah_read_grh(&cm_id_priv->av.ah_attr);
- work->path[0].hop_limit = grh->hop_limit;
ret = ib_get_cached_gid(work->port->cm_dev->ib_device,
work->port->port_num,
grh->sgid_index,
&gid, &gid_attr);
if (!ret) {
if (gid_attr.ndev) {
- work->path[0].ifindex = gid_attr.ndev->ifindex;
- work->path[0].net = dev_net(gid_attr.ndev);
- dev_put(gid_attr.ndev);
work->path[0].rec_type =
sa_conv_gid_to_pathrec_type(gid_attr.gid_type);
+ sa_path_set_ifindex(&work->path[0],
+ gid_attr.ndev->ifindex);
+ sa_path_set_ndev(&work->path[0],
+ dev_net(gid_attr.ndev));
+ dev_put(gid_attr.ndev);
} else {
work->path[0].rec_type = SA_PATH_REC_TYPE_IB;
}
+ if (req_msg->alt_local_lid)
+ work->path[1].rec_type = work->path[0].rec_type;
+ cm_format_paths_from_req(req_msg, &work->path[0],
+ &work->path[1]);
+ if (cm_id_priv->av.ah_attr.type == RDMA_AH_ATTR_TYPE_ROCE)
+ sa_path_set_dmac(&work->path[0],
+ cm_id_priv->av.ah_attr.roce.dmac);
+ work->path[0].hop_limit = grh->hop_limit;
ret = cm_init_av_by_path(&work->path[0], &cm_id_priv->av,
cm_id_priv);
}
@@ -1790,11 +1795,13 @@ static int cm_req_handler(struct cm_work *work)
&work->path[0].sgid,
&gid_attr);
if (!err && gid_attr.ndev) {
- work->path[0].ifindex = gid_attr.ndev->ifindex;
- work->path[0].net = dev_net(gid_attr.ndev);
- dev_put(gid_attr.ndev);
work->path[0].rec_type =
sa_conv_gid_to_pathrec_type(gid_attr.gid_type);
+ sa_path_set_ifindex(&work->path[0],
+ gid_attr.ndev->ifindex);
+ sa_path_set_ndev(&work->path[0],
+ dev_net(gid_attr.ndev));
+ dev_put(gid_attr.ndev);
} else {
work->path[0].rec_type = SA_PATH_REC_TYPE_IB;
}
@@ -2835,8 +2842,8 @@ static void cm_format_lap(struct cm_lap_msg *lap_msg,
cm_lap_set_remote_qpn(lap_msg, cm_id_priv->remote_qpn);
/* todo: need remote CM response timeout */
cm_lap_set_remote_resp_timeout(lap_msg, 0x1F);
- lap_msg->alt_local_lid = alternate_path->slid;
- lap_msg->alt_remote_lid = alternate_path->dlid;
+ lap_msg->alt_local_lid = sa_path_get_slid(alternate_path);
+ lap_msg->alt_remote_lid = sa_path_get_dlid(alternate_path);
lap_msg->alt_local_gid = alternate_path->sgid;
lap_msg->alt_remote_gid = alternate_path->dgid;
cm_lap_set_flow_label(lap_msg, alternate_path->flow_label);
@@ -2912,10 +2919,11 @@ static void cm_format_path_from_lap(struct cm_id_private *cm_id_priv,
struct cm_lap_msg *lap_msg)
{
memset(path, 0, sizeof *path);
+ path->rec_type = SA_PATH_REC_TYPE_IB;
path->dgid = lap_msg->alt_local_gid;
path->sgid = lap_msg->alt_remote_gid;
- path->dlid = lap_msg->alt_local_lid;
- path->slid = lap_msg->alt_remote_lid;
+ sa_path_set_dlid(path, lap_msg->alt_local_lid);
+ sa_path_set_slid(path, lap_msg->alt_remote_lid);
path->flow_label = cm_lap_get_flow_label(lap_msg);
path->hop_limit = lap_msg->alt_hop_limit;
path->traffic_class = cm_lap_get_traffic_class(lap_msg);
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index a6195d5..83ed2ac 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -1140,7 +1140,7 @@ static void cma_save_ib_info(struct sockaddr *src_addr,
ib->sib_pkey = path->pkey;
ib->sib_flowinfo = path->flow_label;
memcpy(&ib->sib_addr, &path->sgid, 16);
- ib->sib_sid = path->service_id;
+ ib->sib_sid = sa_path_get_service_id(path);
ib->sib_scope_id = 0;
} else {
ib->sib_pkey = listen_ib->sib_pkey;
@@ -1274,7 +1274,8 @@ static int cma_save_req_info(const struct ib_cm_event *ib_event,
memcpy(&req->local_gid, &req_param->primary_path->sgid,
sizeof(req->local_gid));
req->has_gid = true;
- req->service_id = req_param->primary_path->service_id;
+ req->service_id =
+ sa_path_get_service_id(req_param->primary_path);
req->pkey = be16_to_cpu(req_param->primary_path->pkey);
if (req->pkey != req_param->bth_pkey)
pr_warn_ratelimited("RDMA CMA: got different BTH P_Key (0x%x) and primary path P_Key (0x%x)\n"
@@ -1822,8 +1823,8 @@ static struct rdma_id_private *cma_new_conn_id(struct rdma_cm_id *listen_id,
struct rdma_cm_id *id;
struct rdma_route *rt;
const sa_family_t ss_family = listen_id->route.addr.src_addr.ss_family;
- const __be64 service_id =
- ib_event->param.req_rcvd.primary_path->service_id;
+ struct sa_path_rec *path = ib_event->param.req_rcvd.primary_path;
+ const __be64 service_id = sa_path_get_service_id(path);
int ret;
id = rdma_create_id(listen_id->route.addr.dev_addr.net,
@@ -1845,7 +1846,7 @@ static struct rdma_id_private *cma_new_conn_id(struct rdma_cm_id *listen_id,
if (!rt->path_rec)
goto err;
- rt->path_rec[0] = *ib_event->param.req_rcvd.primary_path;
+ rt->path_rec[0] = *path;
if (rt->num_paths == 2)
rt->path_rec[1] = *ib_event->param.req_rcvd.alternate_path;
@@ -2331,12 +2332,15 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms,
struct sockaddr_ib *sib;
memset(&path_rec, 0, sizeof path_rec);
+ path_rec.rec_type = SA_PATH_REC_TYPE_IB;
rdma_addr_get_sgid(dev_addr, &path_rec.sgid);
rdma_addr_get_dgid(dev_addr, &path_rec.dgid);
path_rec.pkey = cpu_to_be16(ib_addr_get_pkey(dev_addr));
path_rec.numb_path = 1;
path_rec.reversible = 1;
- path_rec.service_id = rdma_get_service_id(&id_priv->id, cma_dst_addr(id_priv));
+ sa_path_set_service_id(&path_rec,
+ rdma_get_service_id(&id_priv->id,
+ cma_dst_addr(id_priv)));
comp_mask = IB_SA_PATH_REC_DGID | IB_SA_PATH_REC_SGID |
IB_SA_PATH_REC_PKEY | IB_SA_PATH_REC_NUMB_PATH |
@@ -2574,8 +2578,6 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
}
}
- route->path_rec->net = &init_net;
- route->path_rec->ifindex = ndev->ifindex;
supported_gids = roce_gid_type_mask_support(id_priv->id.device,
id_priv->id.port_num);
gid_type = cma_route_gid_type(addr->dev_addr.network,
@@ -2583,13 +2585,15 @@ static int cma_resolve_iboe_route(struct rdma_id_private *id_priv)
id_priv->gid_type);
route->path_rec->rec_type =
sa_conv_gid_to_pathrec_type(gid_type);
+ sa_path_set_ndev(route->path_rec, &init_net);
+ sa_path_set_ifindex(route->path_rec, ndev->ifindex);
}
if (!ndev) {
ret = -ENODEV;
goto err2;
}
- memcpy(route->path_rec->dmac, addr->dev_addr.dst_dev_addr, ETH_ALEN);
+ sa_path_set_dmac(route->path_rec, addr->dev_addr.dst_dev_addr);
rdma_ip2gid((struct sockaddr *)&id_priv->id.route.addr.src_addr,
&route->path_rec->sgid);
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 050a8e6..12614c9 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -193,7 +193,7 @@ struct ib_sa_mcmember_query {
.field_name = "sa_path_rec:" #field
static const struct ib_field path_rec_table[] = {
- { PATH_REC_FIELD(service_id),
+ { PATH_REC_FIELD(ib.service_id),
.offset_words = 0,
.offset_bits = 0,
.size_bits = 64 },
@@ -205,15 +205,15 @@ struct ib_sa_mcmember_query {
.offset_words = 6,
.offset_bits = 0,
.size_bits = 128 },
- { PATH_REC_FIELD(dlid),
+ { PATH_REC_FIELD(ib.dlid),
.offset_words = 10,
.offset_bits = 0,
.size_bits = 16 },
- { PATH_REC_FIELD(slid),
+ { PATH_REC_FIELD(ib.slid),
.offset_words = 10,
.offset_bits = 16,
.size_bits = 16 },
- { PATH_REC_FIELD(raw_traffic),
+ { PATH_REC_FIELD(ib.raw_traffic),
.offset_words = 11,
.offset_bits = 0,
.size_bits = 1 },
@@ -643,7 +643,7 @@ static void ib_nl_set_path_rec_attrs(struct sk_buff *skb,
/* Now build the attributes */
if (comp_mask & IB_SA_PATH_REC_SERVICE_ID) {
- val64 = be64_to_cpu(sa_rec->service_id);
+ val64 = be64_to_cpu(sa_path_get_service_id(sa_rec));
nla_put(skb, RDMA_NLA_F_MANDATORY | LS_NLA_TYPE_SERVICE_ID,
sizeof(val64), &val64);
}
@@ -1110,9 +1110,9 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
memset(ah_attr, 0, sizeof *ah_attr);
ah_attr->type = rdma_ah_find_type(device, port_num);
- rdma_ah_set_dlid(ah_attr, be16_to_cpu(rec->dlid));
+ rdma_ah_set_dlid(ah_attr, be16_to_cpu(sa_path_get_dlid(rec)));
rdma_ah_set_sl(ah_attr, rec->sl);
- rdma_ah_set_path_bits(ah_attr, be16_to_cpu(rec->slid) &
+ rdma_ah_set_path_bits(ah_attr, be16_to_cpu(sa_path_get_slid(rec)) &
get_src_path_mask(device, port_num));
rdma_ah_set_port_num(ah_attr, port_num);
rdma_ah_set_static_rate(ah_attr, rec->rate);
@@ -1121,9 +1121,13 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
if (use_roce) {
struct net_device *idev;
struct net_device *resolved_dev;
- struct rdma_dev_addr dev_addr = {.bound_dev_if = rec->ifindex,
- .net = rec->net ? rec->net :
- &init_net};
+ struct rdma_dev_addr dev_addr = {
+ .bound_dev_if = ((sa_path_get_ifindex(rec) >= 0) ?
+ sa_path_get_ifindex(rec) : 0),
+ .net = sa_path_get_ndev(rec) ?
+ sa_path_get_ndev(rec) :
+ &init_net
+ };
union {
struct sockaddr _sockaddr;
struct sockaddr_in _sockaddr_in;
@@ -1194,7 +1198,7 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
}
if (use_roce)
- memcpy(ah_attr->roce.dmac, rec->dmac, ETH_ALEN);
+ memcpy(ah_attr->roce.dmac, sa_path_get_dmac(rec), ETH_ALEN);
return 0;
}
@@ -1326,10 +1330,10 @@ static void ib_sa_path_rec_callback(struct ib_sa_query *sa_query,
ib_unpack(path_rec_table, ARRAY_SIZE(path_rec_table),
mad->data, &rec);
- rec.net = NULL;
- rec.ifindex = 0;
rec.rec_type = SA_PATH_REC_TYPE_IB;
- eth_zero_addr(rec.dmac);
+ sa_path_set_ndev(&rec, NULL);
+ sa_path_set_ifindex(&rec, 0);
+ sa_path_set_dmac_zero(&rec);
query->callback(status, &rec, query->context);
} else
query->callback(status, NULL, query->context);
diff --git a/drivers/infiniband/core/uverbs_marshall.c b/drivers/infiniband/core/uverbs_marshall.c
index b4e9ce8..50575b6 100644
--- a/drivers/infiniband/core/uverbs_marshall.c
+++ b/drivers/infiniband/core/uverbs_marshall.c
@@ -102,9 +102,9 @@ void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
memcpy(dst->dgid, src->dgid.raw, sizeof src->dgid);
memcpy(dst->sgid, src->sgid.raw, sizeof src->sgid);
- dst->dlid = src->dlid;
- dst->slid = src->slid;
- dst->raw_traffic = src->raw_traffic;
+ dst->dlid = sa_path_get_dlid(src);
+ dst->slid = sa_path_get_slid(src);
+ dst->raw_traffic = sa_path_get_raw_traffic(src);
dst->flow_label = src->flow_label;
dst->hop_limit = src->hop_limit;
dst->traffic_class = src->traffic_class;
@@ -128,9 +128,10 @@ void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
memcpy(dst->dgid.raw, src->dgid, sizeof dst->dgid);
memcpy(dst->sgid.raw, src->sgid, sizeof dst->sgid);
- dst->dlid = src->dlid;
- dst->slid = src->slid;
- dst->raw_traffic = src->raw_traffic;
+ dst->rec_type = SA_PATH_REC_TYPE_IB;
+ sa_path_set_dlid(dst, src->dlid);
+ sa_path_set_slid(dst, src->slid);
+ sa_path_set_raw_traffic(dst, src->raw_traffic);
dst->flow_label = src->flow_label;
dst->hop_limit = src->hop_limit;
dst->traffic_class = src->traffic_class;
@@ -146,9 +147,8 @@ void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
dst->preference = src->preference;
dst->packet_life_time_selector = src->packet_life_time_selector;
- memset(dst->dmac, 0, sizeof(dst->dmac));
- dst->net = NULL;
- dst->ifindex = 0;
- dst->rec_type = SA_PATH_REC_TYPE_IB;
+ sa_path_set_dmac_zero(dst);
+ sa_path_set_ndev(dst, NULL);
+ sa_path_set_ifindex(dst, 0);
}
EXPORT_SYMBOL(ib_copy_path_rec_from_user);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_fs.c b/drivers/infiniband/ulp/ipoib/ipoib_fs.c
index 09396bd..aa9083f 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_fs.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_fs.c
@@ -210,16 +210,16 @@ static int ipoib_path_seq_show(struct seq_file *file, void *iter_ptr)
seq_printf(file,
"GID: %s\n"
" complete: %6s\n",
- gid_buf, path.pathrec.dlid ? "yes" : "no");
+ gid_buf, sa_path_get_dlid(&path.pathrec) ? "yes" : "no");
- if (path.pathrec.dlid) {
+ if (sa_path_get_dlid(&path.pathrec)) {
rate = ib_rate_to_mbps(path.pathrec.rate);
seq_printf(file,
" DLID: 0x%04x\n"
" SL: %12d\n"
" rate: %8d.%d Gb/sec\n",
- be16_to_cpu(path.pathrec.dlid),
+ be16_to_cpu(sa_path_get_dlid(&path.pathrec)),
path.pathrec.sl,
rate / 1000, rate % 1000);
}
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 5e7939f..cc88d85 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -669,7 +669,7 @@ void ipoib_mark_paths_invalid(struct net_device *dev)
list_for_each_entry_safe(path, tp, &priv->path_list, list) {
ipoib_dbg(priv, "mark path LID 0x%04x GID %pI6 invalid\n",
- be16_to_cpu(path->pathrec.dlid),
+ be16_to_cpu(sa_path_get_dlid(&path->pathrec)),
path->pathrec.dgid.raw);
path->valid = 0;
}
@@ -731,7 +731,8 @@ static void path_rec_completion(int status,
if (!status)
ipoib_dbg(priv, "PathRec LID 0x%04x for GID %pI6\n",
- be16_to_cpu(pathrec->dlid), pathrec->dgid.raw);
+ be16_to_cpu(sa_path_get_dlid(pathrec)),
+ pathrec->dgid.raw);
else
ipoib_dbg(priv, "PathRec status %d for GID %pI6\n",
status, path->pathrec.dgid.raw);
@@ -754,7 +755,8 @@ static void path_rec_completion(int status,
path->ah = ah;
ipoib_dbg(priv, "created address handle %p for LID 0x%04x, SL %d\n",
- ah, be16_to_cpu(pathrec->dlid), pathrec->sl);
+ ah, be16_to_cpu(sa_path_get_dlid(pathrec)),
+ pathrec->sl);
while ((skb = __skb_dequeue(&path->queue)))
__skb_queue_tail(&skqueue, skb);
@@ -830,6 +832,7 @@ static struct ipoib_path *path_rec_create(struct net_device *dev, void *gid)
INIT_LIST_HEAD(&path->neigh_list);
+ path->pathrec.rec_type = SA_PATH_REC_TYPE_IB;
memcpy(path->pathrec.dgid.raw, gid, sizeof (union ib_gid));
path->pathrec.sgid = priv->local_gid;
path->pathrec.pkey = cpu_to_be16(priv->pkey);
@@ -995,7 +998,7 @@ static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev,
if (path->ah) {
ipoib_dbg(priv, "Send unicast ARP to %04x\n",
- be16_to_cpu(path->pathrec.dlid));
+ be16_to_cpu(sa_path_get_dlid(&path->pathrec)));
spin_unlock_irqrestore(&priv->lock, flags);
ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr));
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index ded9b9a..89341dd 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -312,10 +312,11 @@ static int srp_new_cm_id(struct srp_rdma_ch *ch)
if (ch->cm_id)
ib_destroy_cm_id(ch->cm_id);
ch->cm_id = new_cm_id;
+ ch->path.rec_type = SA_PATH_REC_TYPE_IB;
ch->path.sgid = target->sgid;
ch->path.dgid = target->orig_dgid;
ch->path.pkey = target->pkey;
- ch->path.service_id = target->service_id;
+ sa_path_set_service_id(&ch->path, target->service_id);
return 0;
}
@@ -2399,12 +2400,12 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
switch (event->param.rej_rcvd.reason) {
case IB_CM_REJ_PORT_CM_REDIRECT:
cpi = event->param.rej_rcvd.ari;
- ch->path.dlid = cpi->redirect_lid;
+ sa_path_set_dlid(&ch->path, cpi->redirect_lid);
ch->path.pkey = cpi->redirect_pkey;
cm_id->remote_cm_qpn = be32_to_cpu(cpi->redirect_qp) & 0x00ffffff;
memcpy(ch->path.dgid.raw, cpi->redirect_gid, 16);
- ch->status = ch->path.dlid ?
+ ch->status = sa_path_get_dlid(&ch->path) ?
SRP_DLID_REDIRECT : SRP_PORT_REDIRECT;
break;
diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h
index c009147..c72c949 100644
--- a/include/rdma/ib_sa.h
+++ b/include/rdma/ib_sa.h
@@ -43,6 +43,7 @@
#include <rdma/ib_verbs.h>
#include <rdma/ib_mad.h>
+#include <rdma/ib_addr.h>
enum {
IB_SA_CLASS_VERSION = 2, /* IB spec version 1.1/1.2 */
@@ -147,19 +148,32 @@ enum ib_sa_mc_join_states {
#define IB_SA_PATH_REC_PACKET_LIFE_TIME_SELECTOR IB_SA_COMP_MASK(20)
#define IB_SA_PATH_REC_PACKET_LIFE_TIME IB_SA_COMP_MASK(21)
#define IB_SA_PATH_REC_PREFERENCE IB_SA_COMP_MASK(22)
+
enum sa_path_rec_type {
SA_PATH_REC_TYPE_IB,
SA_PATH_REC_TYPE_ROCE_V1,
SA_PATH_REC_TYPE_ROCE_V2
};
-struct sa_path_rec {
+struct sa_path_rec_ib {
__be64 service_id;
- union ib_gid dgid;
- union ib_gid sgid;
__be16 dlid;
__be16 slid;
u8 raw_traffic;
+};
+
+struct sa_path_rec_roce {
+ u8 dmac[ETH_ALEN];
+ /* ignored in IB */
+ int ifindex;
+ /* ignored in IB */
+ struct net *net;
+
+};
+
+struct sa_path_rec {
+ union ib_gid dgid;
+ union ib_gid sgid;
/* reserved */
__be32 flow_label;
u8 hop_limit;
@@ -176,19 +190,13 @@ struct sa_path_rec {
u8 packet_life_time_selector;
u8 packet_life_time;
u8 preference;
- u8 dmac[ETH_ALEN];
- /* ignored in IB */
- int ifindex;
- /* ignored in IB */
- struct net *net;
+ union {
+ struct sa_path_rec_ib ib;
+ struct sa_path_rec_roce roce;
+ };
enum sa_path_rec_type rec_type;
};
-static inline struct net_device *ib_get_ndev_from_path(struct sa_path_rec *rec)
-{
- return rec->net ? dev_get_by_index(rec->net, rec->ifindex) : NULL;
-}
-
static inline enum ib_gid_type
sa_conv_pathrec_to_gid_type(struct sa_path_rec *rec)
{
@@ -490,4 +498,117 @@ bool ib_sa_sendonly_fullmem_support(struct ib_sa_client *client,
struct ib_device *device,
u8 port_num);
+static inline bool sa_path_is_roce(struct sa_path_rec *rec)
+{
+ return ((rec->rec_type == SA_PATH_REC_TYPE_ROCE_V1) ||
+ (rec->rec_type == SA_PATH_REC_TYPE_ROCE_V2));
+}
+
+static inline void sa_path_set_service_id(struct sa_path_rec *rec,
+ __be64 service_id)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ rec->ib.service_id = service_id;
+}
+
+static inline void sa_path_set_slid(struct sa_path_rec *rec, __be16 slid)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ rec->ib.slid = slid;
+}
+
+static inline void sa_path_set_dlid(struct sa_path_rec *rec, __be16 dlid)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ rec->ib.dlid = dlid;
+}
+
+static inline void sa_path_set_raw_traffic(struct sa_path_rec *rec,
+ u8 raw_traffic)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ rec->ib.raw_traffic = raw_traffic;
+}
+
+static inline __be64 sa_path_get_service_id(struct sa_path_rec *rec)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ return rec->ib.service_id;
+ return 0;
+}
+
+static inline __be16 sa_path_get_slid(struct sa_path_rec *rec)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ return rec->ib.slid;
+ return 0;
+}
+
+static inline __be16 sa_path_get_dlid(struct sa_path_rec *rec)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ return rec->ib.dlid;
+ return 0;
+}
+
+static inline u8 sa_path_get_raw_traffic(struct sa_path_rec *rec)
+{
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB)
+ return rec->ib.raw_traffic;
+ return 0;
+}
+
+static inline void sa_path_set_dmac(struct sa_path_rec *rec, u8 *dmac)
+{
+ if (sa_path_is_roce(rec))
+ memcpy(rec->roce.dmac, dmac, ETH_ALEN);
+}
+
+static inline void sa_path_set_dmac_zero(struct sa_path_rec *rec)
+{
+ if (sa_path_is_roce(rec))
+ eth_zero_addr(rec->roce.dmac);
+}
+
+static inline void sa_path_set_ifindex(struct sa_path_rec *rec, int ifindex)
+{
+ if (sa_path_is_roce(rec))
+ rec->roce.ifindex = ifindex;
+}
+
+static inline void sa_path_set_ndev(struct sa_path_rec *rec, struct net *net)
+{
+ if (sa_path_is_roce(rec))
+ rec->roce.net = net;
+}
+
+static inline u8 *sa_path_get_dmac(struct sa_path_rec *rec)
+{
+ if (sa_path_is_roce(rec))
+ return rec->roce.dmac;
+ return NULL;
+}
+
+static inline int sa_path_get_ifindex(struct sa_path_rec *rec)
+{
+ if (sa_path_is_roce(rec))
+ return rec->roce.ifindex;
+ return 0;
+}
+
+static inline struct net *sa_path_get_ndev(struct sa_path_rec *rec)
+{
+ if (sa_path_is_roce(rec))
+ return rec->roce.net;
+ return NULL;
+}
+
+static inline struct net_device *ib_get_ndev_from_path(struct sa_path_rec *rec)
+{
+ return sa_path_get_ndev(rec) ?
+ dev_get_by_index(sa_path_get_ndev(rec),
+ sa_path_get_ifindex(rec))
+ : NULL;
+}
+
#endif /* IB_SA_H */
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH rdma-next 5/6] IB/SA: Add OPA path record type
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
` (3 preceding siblings ...)
2017-04-17 20:56 ` [PATCH rdma-next 4/6] IB/SA: Split struct sa_path_rec based on IB and ROCE specific fields Dasaratharaman Chandramouli
@ 2017-04-17 20:56 ` Dasaratharaman Chandramouli
[not found] ` <1492462590-64866-6-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-17 20:56 ` [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records Dasaratharaman Chandramouli
5 siblings, 1 reply; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
Add opa_sa_path_rec to sa_path_rec data structure.
The 'type' field in sa_path_rec identifies the
type of the path record.
Reviewed-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/cm.c | 34 ++++++---
drivers/infiniband/core/sa_query.c | 4 +-
drivers/infiniband/core/ucma.c | 22 +++++-
drivers/infiniband/core/uverbs_marshall.c | 40 ++++++++--
drivers/infiniband/ulp/ipoib/ipoib_fs.c | 2 +-
drivers/infiniband/ulp/ipoib/ipoib_main.c | 14 ++--
drivers/infiniband/ulp/srp/ib_srp.c | 2 +-
drivers/infiniband/ulp/srpt/ib_srpt.c | 3 +-
include/rdma/ib_sa.h | 121 +++++++++++++++++++++++++++---
include/rdma/opa_addr.h | 79 +++++++++++++++++++
10 files changed, 278 insertions(+), 43 deletions(-)
create mode 100644 include/rdma/opa_addr.h
diff --git a/drivers/infiniband/core/cm.c b/drivers/infiniband/core/cm.c
index ca742e8..1844770 100644
--- a/drivers/infiniband/core/cm.c
+++ b/drivers/infiniband/core/cm.c
@@ -1203,8 +1203,10 @@ static void cm_format_req(struct cm_req_msg *req_msg,
}
if (pri_path->hop_limit <= 1) {
- req_msg->primary_local_lid = sa_path_get_slid(pri_path);
- req_msg->primary_remote_lid = sa_path_get_dlid(pri_path);
+ req_msg->primary_local_lid =
+ htons(ntohl(sa_path_get_slid(pri_path)));
+ req_msg->primary_remote_lid =
+ htons(ntohl(sa_path_get_dlid(pri_path)));
} else {
/* Work-around until there's a way to obtain remote LID info */
req_msg->primary_local_lid = IB_LID_PERMISSIVE;
@@ -1224,8 +1226,10 @@ static void cm_format_req(struct cm_req_msg *req_msg,
if (alt_path) {
if (alt_path->hop_limit <= 1) {
- req_msg->alt_local_lid = sa_path_get_slid(alt_path);
- req_msg->alt_remote_lid = sa_path_get_dlid(alt_path);
+ req_msg->alt_local_lid =
+ htons(ntohl(sa_path_get_slid(alt_path)));
+ req_msg->alt_remote_lid =
+ htons(ntohl(sa_path_get_dlid(alt_path)));
} else {
req_msg->alt_local_lid = IB_LID_PERMISSIVE;
req_msg->alt_remote_lid = IB_LID_PERMISSIVE;
@@ -1407,8 +1411,10 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
{
primary_path->dgid = req_msg->primary_local_gid;
primary_path->sgid = req_msg->primary_remote_gid;
- sa_path_set_dlid(primary_path, req_msg->primary_local_lid);
- sa_path_set_slid(primary_path, req_msg->primary_remote_lid);
+ sa_path_set_dlid(primary_path,
+ htonl(ntohs(req_msg->primary_local_lid)));
+ sa_path_set_slid(primary_path,
+ htonl(ntohs(req_msg->primary_remote_lid)));
primary_path->flow_label = cm_req_get_primary_flow_label(req_msg);
primary_path->hop_limit = req_msg->primary_hop_limit;
primary_path->traffic_class = req_msg->primary_traffic_class;
@@ -1428,8 +1434,10 @@ static void cm_format_paths_from_req(struct cm_req_msg *req_msg,
if (req_msg->alt_local_lid) {
alt_path->dgid = req_msg->alt_local_gid;
alt_path->sgid = req_msg->alt_remote_gid;
- sa_path_set_dlid(alt_path, req_msg->alt_local_lid);
- sa_path_set_slid(alt_path, req_msg->alt_remote_lid);
+ sa_path_set_dlid(alt_path,
+ htonl(ntohs(req_msg->alt_local_lid)));
+ sa_path_set_slid(alt_path,
+ htonl(ntohs(req_msg->alt_remote_lid)));
alt_path->flow_label = cm_req_get_alt_flow_label(req_msg);
alt_path->hop_limit = req_msg->alt_hop_limit;
alt_path->traffic_class = req_msg->alt_traffic_class;
@@ -2842,8 +2850,10 @@ static void cm_format_lap(struct cm_lap_msg *lap_msg,
cm_lap_set_remote_qpn(lap_msg, cm_id_priv->remote_qpn);
/* todo: need remote CM response timeout */
cm_lap_set_remote_resp_timeout(lap_msg, 0x1F);
- lap_msg->alt_local_lid = sa_path_get_slid(alternate_path);
- lap_msg->alt_remote_lid = sa_path_get_dlid(alternate_path);
+ lap_msg->alt_local_lid =
+ htons(ntohl(sa_path_get_slid(alternate_path)));
+ lap_msg->alt_remote_lid =
+ htons(ntohl(sa_path_get_dlid(alternate_path)));
lap_msg->alt_local_gid = alternate_path->sgid;
lap_msg->alt_remote_gid = alternate_path->dgid;
cm_lap_set_flow_label(lap_msg, alternate_path->flow_label);
@@ -2922,8 +2932,8 @@ static void cm_format_path_from_lap(struct cm_id_private *cm_id_priv,
path->rec_type = SA_PATH_REC_TYPE_IB;
path->dgid = lap_msg->alt_local_gid;
path->sgid = lap_msg->alt_remote_gid;
- sa_path_set_dlid(path, lap_msg->alt_local_lid);
- sa_path_set_slid(path, lap_msg->alt_remote_lid);
+ sa_path_set_dlid(path, htonl(ntohs(lap_msg->alt_local_lid)));
+ sa_path_set_slid(path, htonl(ntohs(lap_msg->alt_remote_lid)));
path->flow_label = cm_lap_get_flow_label(lap_msg);
path->hop_limit = lap_msg->alt_hop_limit;
path->traffic_class = cm_lap_get_traffic_class(lap_msg);
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 12614c9..745906e 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -1110,9 +1110,9 @@ int ib_init_ah_from_path(struct ib_device *device, u8 port_num,
memset(ah_attr, 0, sizeof *ah_attr);
ah_attr->type = rdma_ah_find_type(device, port_num);
- rdma_ah_set_dlid(ah_attr, be16_to_cpu(sa_path_get_dlid(rec)));
+ rdma_ah_set_dlid(ah_attr, be32_to_cpu(sa_path_get_dlid(rec)));
rdma_ah_set_sl(ah_attr, rec->sl);
- rdma_ah_set_path_bits(ah_attr, be16_to_cpu(sa_path_get_slid(rec)) &
+ rdma_ah_set_path_bits(ah_attr, be32_to_cpu(sa_path_get_slid(rec)) &
get_src_path_mask(device, port_num));
rdma_ah_set_port_num(ah_attr, port_num);
rdma_ah_set_static_rate(ah_attr, rec->rate);
diff --git a/drivers/infiniband/core/ucma.c b/drivers/infiniband/core/ucma.c
index 2beff90..276f0ef 100644
--- a/drivers/infiniband/core/ucma.c
+++ b/drivers/infiniband/core/ucma.c
@@ -898,11 +898,18 @@ static ssize_t ucma_query_path(struct ucma_context *ctx,
for (i = 0, out_len -= sizeof(*resp);
i < resp->num_paths && out_len > sizeof(struct ib_path_rec_data);
i++, out_len -= sizeof(struct ib_path_rec_data)) {
+ struct sa_path_rec *rec = &ctx->cm_id->route.path_rec[i];
resp->path_data[i].flags = IB_PATH_GMP | IB_PATH_PRIMARY |
IB_PATH_BIDIRECTIONAL;
- ib_sa_pack_path(&ctx->cm_id->route.path_rec[i],
- &resp->path_data[i].path_rec);
+ if (rec->rec_type == SA_PATH_REC_TYPE_IB) {
+ ib_sa_pack_path(rec, &resp->path_data[i].path_rec);
+ } else {
+ struct sa_path_rec ib;
+
+ sa_convert_path_opa_to_ib(&ib, rec);
+ ib_sa_pack_path(&ib, &resp->path_data[i].path_rec);
+ }
}
if (copy_to_user(response, resp,
@@ -1215,8 +1222,17 @@ static int ucma_set_ib_path(struct ucma_context *ctx,
memset(&sa_path, 0, sizeof(sa_path));
+ sa_path.rec_type = SA_PATH_REC_TYPE_IB;
ib_sa_unpack_path(path_data->path_rec, &sa_path);
- ret = rdma_set_ib_paths(ctx->cm_id, &sa_path, 1);
+
+ if (rdma_cap_opa_ah(ctx->cm_id->device, ctx->cm_id->port_num)) {
+ struct sa_path_rec opa;
+
+ sa_convert_path_ib_to_opa(&opa, &sa_path);
+ ret = rdma_set_ib_paths(ctx->cm_id, &opa, 1);
+ } else {
+ ret = rdma_set_ib_paths(ctx->cm_id, &sa_path, 1);
+ }
if (ret)
return ret;
diff --git a/drivers/infiniband/core/uverbs_marshall.c b/drivers/infiniband/core/uverbs_marshall.c
index 50575b6..8b9587f 100644
--- a/drivers/infiniband/core/uverbs_marshall.c
+++ b/drivers/infiniband/core/uverbs_marshall.c
@@ -96,14 +96,14 @@ void ib_copy_qp_attr_to_user(struct ib_uverbs_qp_attr *dst,
}
EXPORT_SYMBOL(ib_copy_qp_attr_to_user);
-void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
- struct sa_path_rec *src)
+void __ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
+ struct sa_path_rec *src)
{
memcpy(dst->dgid, src->dgid.raw, sizeof src->dgid);
memcpy(dst->sgid, src->sgid.raw, sizeof src->sgid);
- dst->dlid = sa_path_get_dlid(src);
- dst->slid = sa_path_get_slid(src);
+ dst->dlid = htons(ntohl(sa_path_get_dlid(src)));
+ dst->slid = htons(ntohl(sa_path_get_slid(src)));
dst->raw_traffic = sa_path_get_raw_traffic(src);
dst->flow_label = src->flow_label;
dst->hop_limit = src->hop_limit;
@@ -120,17 +120,42 @@ void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
dst->preference = src->preference;
dst->packet_life_time_selector = src->packet_life_time_selector;
}
+
+void ib_copy_path_rec_to_user(struct ib_user_path_rec *dst,
+ struct sa_path_rec *src)
+{
+ struct sa_path_rec rec;
+
+ if (src->rec_type == SA_PATH_REC_TYPE_OPA) {
+ sa_convert_path_opa_to_ib(&rec, src);
+ __ib_copy_path_rec_to_user(dst, &rec);
+ return;
+ }
+ __ib_copy_path_rec_to_user(dst, src);
+}
EXPORT_SYMBOL(ib_copy_path_rec_to_user);
void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
struct ib_user_path_rec *src)
{
+ __be32 slid, dlid;
+
+ memset(dst, 0, sizeof(*dst));
+ if ((ib_is_opa_gid((union ib_gid *)src->sgid)) ||
+ (ib_is_opa_gid((union ib_gid *)src->dgid))) {
+ dst->rec_type = SA_PATH_REC_TYPE_OPA;
+ slid = htonl(opa_get_lid_from_gid((union ib_gid *)src->sgid));
+ dlid = htonl(opa_get_lid_from_gid((union ib_gid *)src->dgid));
+ } else {
+ dst->rec_type = SA_PATH_REC_TYPE_IB;
+ slid = htonl(ntohs(src->slid));
+ dlid = htonl(ntohs(src->dlid));
+ }
memcpy(dst->dgid.raw, src->dgid, sizeof dst->dgid);
memcpy(dst->sgid.raw, src->sgid, sizeof dst->sgid);
- dst->rec_type = SA_PATH_REC_TYPE_IB;
- sa_path_set_dlid(dst, src->dlid);
- sa_path_set_slid(dst, src->slid);
+ sa_path_set_dlid(dst, dlid);
+ sa_path_set_slid(dst, slid);
sa_path_set_raw_traffic(dst, src->raw_traffic);
dst->flow_label = src->flow_label;
dst->hop_limit = src->hop_limit;
@@ -147,6 +172,7 @@ void ib_copy_path_rec_from_user(struct sa_path_rec *dst,
dst->preference = src->preference;
dst->packet_life_time_selector = src->packet_life_time_selector;
+ /* TODO: No need to set this */
sa_path_set_dmac_zero(dst);
sa_path_set_ndev(dst, NULL);
sa_path_set_ifindex(dst, 0);
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_fs.c b/drivers/infiniband/ulp/ipoib/ipoib_fs.c
index aa9083f..f7f2590 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_fs.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_fs.c
@@ -219,7 +219,7 @@ static int ipoib_path_seq_show(struct seq_file *file, void *iter_ptr)
" DLID: 0x%04x\n"
" SL: %12d\n"
" rate: %8d.%d Gb/sec\n",
- be16_to_cpu(sa_path_get_dlid(&path.pathrec)),
+ be32_to_cpu(sa_path_get_dlid(&path.pathrec)),
path.pathrec.sl,
rate / 1000, rate % 1000);
}
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index cc88d85..4d3f146 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -668,9 +668,9 @@ void ipoib_mark_paths_invalid(struct net_device *dev)
spin_lock_irq(&priv->lock);
list_for_each_entry_safe(path, tp, &priv->path_list, list) {
- ipoib_dbg(priv, "mark path LID 0x%04x GID %pI6 invalid\n",
- be16_to_cpu(sa_path_get_dlid(&path->pathrec)),
- path->pathrec.dgid.raw);
+ ipoib_dbg(priv, "mark path LID 0x%08x GID %pI6 invalid\n",
+ be32_to_cpu(sa_path_get_dlid(&path->pathrec)),
+ path->pathrec.dgid.raw);
path->valid = 0;
}
@@ -731,7 +731,7 @@ static void path_rec_completion(int status,
if (!status)
ipoib_dbg(priv, "PathRec LID 0x%04x for GID %pI6\n",
- be16_to_cpu(sa_path_get_dlid(pathrec)),
+ be32_to_cpu(sa_path_get_dlid(pathrec)),
pathrec->dgid.raw);
else
ipoib_dbg(priv, "PathRec status %d for GID %pI6\n",
@@ -755,7 +755,7 @@ static void path_rec_completion(int status,
path->ah = ah;
ipoib_dbg(priv, "created address handle %p for LID 0x%04x, SL %d\n",
- ah, be16_to_cpu(sa_path_get_dlid(pathrec)),
+ ah, be32_to_cpu(sa_path_get_dlid(pathrec)),
pathrec->sl);
while ((skb = __skb_dequeue(&path->queue)))
@@ -997,8 +997,8 @@ static void unicast_arp_send(struct sk_buff *skb, struct net_device *dev,
}
if (path->ah) {
- ipoib_dbg(priv, "Send unicast ARP to %04x\n",
- be16_to_cpu(sa_path_get_dlid(&path->pathrec)));
+ ipoib_dbg(priv, "Send unicast ARP to %08x\n",
+ be32_to_cpu(sa_path_get_dlid(&path->pathrec)));
spin_unlock_irqrestore(&priv->lock, flags);
ipoib_send(dev, skb, path->ah, IPOIB_QPN(phdr->hwaddr));
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 89341dd..07877a8 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -2400,7 +2400,7 @@ static void srp_cm_rej_handler(struct ib_cm_id *cm_id,
switch (event->param.rej_rcvd.reason) {
case IB_CM_REJ_PORT_CM_REDIRECT:
cpi = event->param.rej_rcvd.ari;
- sa_path_set_dlid(&ch->path, cpi->redirect_lid);
+ sa_path_set_dlid(&ch->path, htonl(ntohs(cpi->redirect_lid)));
ch->path.pkey = cpi->redirect_pkey;
cm_id->remote_cm_qpn = be32_to_cpu(cpi->redirect_qp) & 0x00ffffff;
memcpy(ch->path.dgid.raw, cpi->redirect_gid, 16);
diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
index 7d6c199..38df02f 100644
--- a/drivers/infiniband/ulp/srpt/ib_srpt.c
+++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
@@ -1846,6 +1846,7 @@ static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
struct srp_login_rej *rej;
struct ib_cm_rep_param *rep_param;
struct srpt_rdma_ch *ch, *tmp_ch;
+ union ib_gid *sgid = ¶m->primary_path->sgid;
__be16 *guid;
u32 it_iu_len;
int i, ret = 0;
@@ -1992,7 +1993,7 @@ static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
goto destroy_ib;
}
- guid = (__be16 *)¶m->primary_path->sgid.global.interface_id;
+ guid = (__be16 *)&sgid->global.interface_id;
snprintf(ch->ini_guid, sizeof(ch->ini_guid), "%04x:%04x:%04x:%04x",
be16_to_cpu(guid[0]), be16_to_cpu(guid[1]),
be16_to_cpu(guid[2]), be16_to_cpu(guid[3]));
diff --git a/include/rdma/ib_sa.h b/include/rdma/ib_sa.h
index c72c949..0551cb4 100644
--- a/include/rdma/ib_sa.h
+++ b/include/rdma/ib_sa.h
@@ -44,6 +44,7 @@
#include <rdma/ib_verbs.h>
#include <rdma/ib_mad.h>
#include <rdma/ib_addr.h>
+#include <rdma/opa_addr.h>
enum {
IB_SA_CLASS_VERSION = 2, /* IB spec version 1.1/1.2 */
@@ -152,7 +153,8 @@ enum ib_sa_mc_join_states {
enum sa_path_rec_type {
SA_PATH_REC_TYPE_IB,
SA_PATH_REC_TYPE_ROCE_V1,
- SA_PATH_REC_TYPE_ROCE_V2
+ SA_PATH_REC_TYPE_ROCE_V2,
+ SA_PATH_REC_TYPE_OPA
};
struct sa_path_rec_ib {
@@ -171,6 +173,19 @@ struct sa_path_rec_roce {
};
+struct sa_path_rec_opa {
+ __be64 service_id;
+ __be32 dlid;
+ __be32 slid;
+ u8 raw_traffic;
+ u8 l2_8B;
+ u8 l2_10B;
+ u8 l2_9B;
+ u8 l2_16B;
+ u8 qos_type;
+ u8 qos_priority;
+};
+
struct sa_path_rec {
union ib_gid dgid;
union ib_gid sgid;
@@ -193,6 +208,7 @@ struct sa_path_rec {
union {
struct sa_path_rec_ib ib;
struct sa_path_rec_roce roce;
+ struct sa_path_rec_opa opa;
};
enum sa_path_rec_type rec_type;
};
@@ -223,6 +239,77 @@ struct sa_path_rec {
}
}
+static inline void path_conv_opa_to_ib(struct sa_path_rec *ib,
+ struct sa_path_rec *opa)
+{
+ if ((be32_to_cpu(opa->opa.dlid) >=
+ be16_to_cpu(IB_MULTICAST_LID_BASE)) ||
+ (be32_to_cpu(opa->opa.slid) >=
+ be16_to_cpu(IB_MULTICAST_LID_BASE))) {
+ /* Create OPA GID and zero out the LID */
+ ib->dgid.global.interface_id
+ = OPA_MAKE_ID(opa->opa.dlid);
+ ib->dgid.global.subnet_prefix
+ = opa->dgid.global.subnet_prefix;
+ ib->sgid.global.interface_id
+ = OPA_MAKE_ID(opa->opa.slid);
+ ib->dgid.global.subnet_prefix
+ = opa->dgid.global.subnet_prefix;
+ ib->ib.dlid = 0;
+
+ ib->ib.slid = 0;
+ } else {
+ ib->ib.dlid = opa->opa.dlid;
+ ib->ib.slid = opa->opa.slid;
+ }
+ ib->ib.service_id = opa->opa.service_id;
+ ib->ib.raw_traffic = opa->opa.raw_traffic;
+}
+
+static inline void path_conv_ib_to_opa(struct sa_path_rec *opa,
+ struct sa_path_rec *ib)
+{
+ __be32 slid, dlid;
+
+ if ((ib_is_opa_gid(&ib->sgid)) ||
+ (ib_is_opa_gid(&ib->dgid))) {
+ slid = htonl(opa_get_lid_from_gid(&ib->sgid));
+ dlid = htonl(opa_get_lid_from_gid(&ib->dgid));
+ } else {
+ slid = htonl(ntohs(ib->ib.slid));
+ dlid = htonl(ntohs(ib->ib.dlid));
+ }
+ opa->opa.slid = slid;
+ opa->opa.dlid = dlid;
+ opa->opa.service_id = ib->ib.service_id;
+ opa->opa.raw_traffic = ib->ib.raw_traffic;
+}
+
+/* Convert from OPA to IB path record */
+static inline void sa_convert_path_opa_to_ib(struct sa_path_rec *dest,
+ struct sa_path_rec *src)
+{
+ if (src->rec_type != SA_PATH_REC_TYPE_OPA)
+ return;
+
+ *dest = *src;
+ dest->rec_type = SA_PATH_REC_TYPE_IB;
+ path_conv_opa_to_ib(dest, src);
+}
+
+/* Convert from IB to OPA path record */
+static inline void sa_convert_path_ib_to_opa(struct sa_path_rec *dest,
+ struct sa_path_rec *src)
+{
+ if (src->rec_type != SA_PATH_REC_TYPE_IB)
+ return;
+
+ /* Do a structure copy and overwrite the relevant fields */
+ *dest = *src;
+ dest->rec_type = SA_PATH_REC_TYPE_OPA;
+ path_conv_ib_to_opa(dest, src);
+}
+
#define IB_SA_MCMEMBER_REC_MGID IB_SA_COMP_MASK( 0)
#define IB_SA_MCMEMBER_REC_PORT_GID IB_SA_COMP_MASK( 1)
#define IB_SA_MCMEMBER_REC_QKEY IB_SA_COMP_MASK( 2)
@@ -509,18 +596,24 @@ static inline void sa_path_set_service_id(struct sa_path_rec *rec,
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
rec->ib.service_id = service_id;
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ rec->opa.service_id = service_id;
}
-static inline void sa_path_set_slid(struct sa_path_rec *rec, __be16 slid)
+static inline void sa_path_set_slid(struct sa_path_rec *rec, __be32 slid)
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
- rec->ib.slid = slid;
+ rec->ib.slid = htons(ntohl(slid));
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ rec->opa.slid = slid;
}
-static inline void sa_path_set_dlid(struct sa_path_rec *rec, __be16 dlid)
+static inline void sa_path_set_dlid(struct sa_path_rec *rec, __be32 dlid)
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
- rec->ib.dlid = dlid;
+ rec->ib.dlid = htons(ntohl(dlid));
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ rec->opa.dlid = dlid;
}
static inline void sa_path_set_raw_traffic(struct sa_path_rec *rec,
@@ -528,26 +621,34 @@ static inline void sa_path_set_raw_traffic(struct sa_path_rec *rec,
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
rec->ib.raw_traffic = raw_traffic;
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ rec->opa.raw_traffic = raw_traffic;
}
static inline __be64 sa_path_get_service_id(struct sa_path_rec *rec)
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
return rec->ib.service_id;
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ return rec->opa.service_id;
return 0;
}
-static inline __be16 sa_path_get_slid(struct sa_path_rec *rec)
+static inline __be32 sa_path_get_slid(struct sa_path_rec *rec)
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
- return rec->ib.slid;
+ return htonl(ntohs(rec->ib.slid));
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ return rec->opa.dlid;
return 0;
}
-static inline __be16 sa_path_get_dlid(struct sa_path_rec *rec)
+static inline __be32 sa_path_get_dlid(struct sa_path_rec *rec)
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
- return rec->ib.dlid;
+ return htonl(ntohs(rec->ib.dlid));
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ return rec->opa.dlid;
return 0;
}
@@ -555,6 +656,8 @@ static inline u8 sa_path_get_raw_traffic(struct sa_path_rec *rec)
{
if (rec->rec_type == SA_PATH_REC_TYPE_IB)
return rec->ib.raw_traffic;
+ else if (rec->rec_type == SA_PATH_REC_TYPE_OPA)
+ return rec->opa.raw_traffic;
return 0;
}
diff --git a/include/rdma/opa_addr.h b/include/rdma/opa_addr.h
new file mode 100644
index 0000000..eace28f
--- /dev/null
+++ b/include/rdma/opa_addr.h
@@ -0,0 +1,79 @@
+/*
+ * Copyright(c) 2017 Intel Corporation.
+ *
+ * This file is provided under a dual BSD/GPLv2 license. When using or
+ * redistributing this file, you may do so under either license.
+ *
+ * GPL LICENSE SUMMARY
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of version 2 of the GNU General Public License as
+ * published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope that it will be useful, but
+ * WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
+ * General Public License for more details.
+ *
+ * BSD LICENSE
+ *
+ * Redistribution and use in source and binary forms, with or without
+ * modification, are permitted provided that the following conditions
+ * are met:
+ *
+ * - Redistributions of source code must retain the above copyright
+ * notice, this list of conditions and the following disclaimer.
+ * - Redistributions in binary form must reproduce the above copyright
+ * notice, this list of conditions and the following disclaimer in
+ * the documentation and/or other materials provided with the
+ * distribution.
+ * - Neither the name of Intel Corporation nor the names of its
+ * contributors may be used to endorse or promote products derived
+ * from this software without specific prior written permission.
+ *
+ * THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS
+ * "AS IS" AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT
+ * LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR
+ * A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT
+ * OWNER OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL,
+ * SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT
+ * LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE,
+ * DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY
+ * THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
+ * (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE
+ * OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
+ *
+ */
+
+#ifndef OPA_ADDR_H
+#define OPA_ADDR_H
+
+#define OPA_SPECIAL_OUI (0x00066AULL)
+#define OPA_MAKE_ID(x) (cpu_to_be64(OPA_SPECIAL_OUI << 40 | (x)))
+
+/**
+ * ib_is_opa_gid: Returns true if the top 24 bits of the gid
+ * contains the OPA_STL_OUI identifier. This identifies that
+ * the provided gid is a special purpose GID meant to carry
+ * extended LID information.
+ *
+ * @gid: The Global identifier
+ */
+static inline bool ib_is_opa_gid(union ib_gid *gid)
+{
+ return ((be64_to_cpu(gid->global.interface_id) >> 40) ==
+ OPA_SPECIAL_OUI);
+}
+
+/**
+ * opa_get_lid_from_gid: Returns the last 32 bits of the gid.
+ * OPA devices use one of the gids in the gid table to also
+ * store the lid.
+ *
+ * @gid: The Global identifier
+ */
+static inline u32 opa_get_lid_from_gid(union ib_gid *gid)
+{
+ return be64_to_cpu(gid->global.interface_id) & 0xFFFFFFFF;
+}
+#endif /* OPA_ADDR_H */
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 13+ messages in thread
* [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
` (4 preceding siblings ...)
2017-04-17 20:56 ` [PATCH rdma-next 5/6] IB/SA: Add OPA path record type Dasaratharaman Chandramouli
@ 2017-04-17 20:56 ` Dasaratharaman Chandramouli
[not found] ` <1492462590-64866-7-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
5 siblings, 1 reply; 13+ messages in thread
From: Dasaratharaman Chandramouli @ 2017-04-17 20:56 UTC (permalink / raw)
To: Doug Ledford; +Cc: linux-rdma
When the bit 26 of capmask2 field in OPA classport info
query is set, SA will query for OPA path records instead
of querying for IB path records. Note that OPA
path records can only be queried by kernel ULPs.
Userspace clients continue to query IB path records.
Reviewed-by: Don Hiatt <don.hiatt-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Reviewed-by: Ira Weiny <ira.weiny-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Signed-off-by: Dasaratharaman Chandramouli <dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
---
drivers/infiniband/core/cma.c | 6 +-
drivers/infiniband/core/sa_query.c | 264 ++++++++++++++++++++++++++++--
drivers/infiniband/ulp/ipoib/ipoib_main.c | 5 +-
drivers/infiniband/ulp/srp/ib_srp.c | 6 +-
include/rdma/ib_mad.h | 13 ++
5 files changed, 279 insertions(+), 15 deletions(-)
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 83ed2ac..fc595b99 100644
--- a/drivers/infiniband/core/cma.c
+++ b/drivers/infiniband/core/cma.c
@@ -2332,7 +2332,11 @@ static int cma_query_ib_route(struct rdma_id_private *id_priv, int timeout_ms,
struct sockaddr_ib *sib;
memset(&path_rec, 0, sizeof path_rec);
- path_rec.rec_type = SA_PATH_REC_TYPE_IB;
+
+ if (rdma_cap_opa_ah(id_priv->id.device, id_priv->id.port_num))
+ path_rec.rec_type = SA_PATH_REC_TYPE_OPA;
+ else
+ path_rec.rec_type = SA_PATH_REC_TYPE_IB;
rdma_addr_get_sgid(dev_addr, &path_rec.sgid);
rdma_addr_get_dgid(dev_addr, &path_rec.dgid);
path_rec.pkey = cpu_to_be16(ib_addr_get_pkey(dev_addr));
diff --git a/drivers/infiniband/core/sa_query.c b/drivers/infiniband/core/sa_query.c
index 745906e..471fe9c 100644
--- a/drivers/infiniband/core/sa_query.c
+++ b/drivers/infiniband/core/sa_query.c
@@ -132,6 +132,7 @@ struct ib_sa_path_query {
void (*callback)(int, struct sa_path_rec *, void *);
void *context;
struct ib_sa_query sa_query;
+ struct sa_path_rec *conv_pr;
};
struct ib_sa_guidinfo_query {
@@ -287,6 +288,136 @@ struct ib_sa_mcmember_query {
.size_bits = 48 },
};
+#define OPA_PATH_REC_FIELD(field) \
+ .struct_offset_bytes = \
+ offsetof(struct sa_path_rec, field), \
+ .struct_size_bytes = \
+ sizeof((struct sa_path_rec *)0)->field, \
+ .field_name = "sa_path_rec:" #field
+
+static const struct ib_field opa_path_rec_table[] = {
+ { OPA_PATH_REC_FIELD(opa.service_id),
+ .offset_words = 0,
+ .offset_bits = 0,
+ .size_bits = 64 },
+ { OPA_PATH_REC_FIELD(dgid),
+ .offset_words = 2,
+ .offset_bits = 0,
+ .size_bits = 128 },
+ { OPA_PATH_REC_FIELD(sgid),
+ .offset_words = 6,
+ .offset_bits = 0,
+ .size_bits = 128 },
+ { OPA_PATH_REC_FIELD(opa.dlid),
+ .offset_words = 10,
+ .offset_bits = 0,
+ .size_bits = 32 },
+ { OPA_PATH_REC_FIELD(opa.slid),
+ .offset_words = 11,
+ .offset_bits = 0,
+ .size_bits = 32 },
+ { OPA_PATH_REC_FIELD(opa.raw_traffic),
+ .offset_words = 12,
+ .offset_bits = 0,
+ .size_bits = 1 },
+ { RESERVED,
+ .offset_words = 12,
+ .offset_bits = 1,
+ .size_bits = 3 },
+ { OPA_PATH_REC_FIELD(flow_label),
+ .offset_words = 12,
+ .offset_bits = 4,
+ .size_bits = 20 },
+ { OPA_PATH_REC_FIELD(hop_limit),
+ .offset_words = 12,
+ .offset_bits = 24,
+ .size_bits = 8 },
+ { OPA_PATH_REC_FIELD(traffic_class),
+ .offset_words = 13,
+ .offset_bits = 0,
+ .size_bits = 8 },
+ { OPA_PATH_REC_FIELD(reversible),
+ .offset_words = 13,
+ .offset_bits = 8,
+ .size_bits = 1 },
+ { OPA_PATH_REC_FIELD(numb_path),
+ .offset_words = 13,
+ .offset_bits = 9,
+ .size_bits = 7 },
+ { OPA_PATH_REC_FIELD(pkey),
+ .offset_words = 13,
+ .offset_bits = 16,
+ .size_bits = 16 },
+ { OPA_PATH_REC_FIELD(opa.l2_8B),
+ .offset_words = 14,
+ .offset_bits = 0,
+ .size_bits = 1 },
+ { OPA_PATH_REC_FIELD(opa.l2_10B),
+ .offset_words = 14,
+ .offset_bits = 1,
+ .size_bits = 1 },
+ { OPA_PATH_REC_FIELD(opa.l2_9B),
+ .offset_words = 14,
+ .offset_bits = 2,
+ .size_bits = 1 },
+ { OPA_PATH_REC_FIELD(opa.l2_16B),
+ .offset_words = 14,
+ .offset_bits = 3,
+ .size_bits = 1 },
+ { RESERVED,
+ .offset_words = 14,
+ .offset_bits = 4,
+ .size_bits = 2 },
+ { OPA_PATH_REC_FIELD(opa.qos_type),
+ .offset_words = 14,
+ .offset_bits = 6,
+ .size_bits = 2 },
+ { OPA_PATH_REC_FIELD(opa.qos_priority),
+ .offset_words = 14,
+ .offset_bits = 8,
+ .size_bits = 8 },
+ { RESERVED,
+ .offset_words = 14,
+ .offset_bits = 16,
+ .size_bits = 3 },
+ { OPA_PATH_REC_FIELD(sl),
+ .offset_words = 14,
+ .offset_bits = 19,
+ .size_bits = 5 },
+ { RESERVED,
+ .offset_words = 14,
+ .offset_bits = 24,
+ .size_bits = 8 },
+ { OPA_PATH_REC_FIELD(mtu_selector),
+ .offset_words = 15,
+ .offset_bits = 0,
+ .size_bits = 2 },
+ { OPA_PATH_REC_FIELD(mtu),
+ .offset_words = 15,
+ .offset_bits = 2,
+ .size_bits = 6 },
+ { OPA_PATH_REC_FIELD(rate_selector),
+ .offset_words = 15,
+ .offset_bits = 8,
+ .size_bits = 2 },
+ { OPA_PATH_REC_FIELD(rate),
+ .offset_words = 15,
+ .offset_bits = 10,
+ .size_bits = 6 },
+ { OPA_PATH_REC_FIELD(packet_life_time_selector),
+ .offset_words = 15,
+ .offset_bits = 16,
+ .size_bits = 2 },
+ { OPA_PATH_REC_FIELD(packet_life_time),
+ .offset_words = 15,
+ .offset_bits = 18,
+ .size_bits = 6 },
+ { OPA_PATH_REC_FIELD(preference),
+ .offset_words = 15,
+ .offset_bits = 24,
+ .size_bits = 8 },
+};
+
#define MCMEMBER_REC_FIELD(field) \
.struct_offset_bytes = offsetof(struct ib_sa_mcmember_rec, field), \
.struct_size_bytes = sizeof ((struct ib_sa_mcmember_rec *) 0)->field, \
@@ -1283,7 +1414,8 @@ static int send_mad(struct ib_sa_query *query, int timeout_ms, gfp_t gfp_mask)
query->mad_buf->context[0] = query;
query->id = id;
- if (query->flags & IB_SA_ENABLE_LOCAL_SERVICE) {
+ if ((query->flags & IB_SA_ENABLE_LOCAL_SERVICE) &&
+ (!(query->flags & IB_SA_QUERY_OPA))) {
if (!ibnl_chk_listeners(RDMA_NL_GROUP_LS)) {
if (!ib_nl_make_request(query, gfp_mask))
return id;
@@ -1318,6 +1450,62 @@ void ib_sa_pack_path(struct sa_path_rec *rec, void *attribute)
}
EXPORT_SYMBOL(ib_sa_pack_path);
+static bool ib_sa_opa_pathrecord_support(struct ib_sa_client *client,
+ struct ib_device *device,
+ u8 port_num)
+{
+ struct ib_sa_device *sa_dev = ib_get_client_data(device, &sa_client);
+ struct ib_sa_port *port = &sa_dev->port[port_num - sa_dev->start_port];
+ unsigned long flags;
+ bool ret = false;
+
+ if (!sa_dev)
+ return ret;
+
+ spin_lock_irqsave(&port->classport_lock, flags);
+ if (!port->classport_info.valid)
+ goto ret;
+
+ if (port->classport_info.data.type == RDMA_CLASS_PORT_INFO_OPA)
+ ret = opa_get_cpi_capmask2(&port->classport_info.data.opa) &
+ OPA_CLASS_PORT_INFO_PR_SUPPORT;
+ret:
+ spin_unlock_irqrestore(&port->classport_lock, flags);
+ return ret;
+}
+
+enum opa_pr_supported {
+ PR_NOT_SUPPORTED,
+ PR_OPA_SUPPORTED,
+ PR_IB_SUPPORTED
+};
+
+/**
+ * Check if current PR query can be an OPA query.
+ * Retuns PR_NOT_SUPPORTED if a path record query is not
+ * possible, PR_OPA_SUPPORTED if an OPA path record query
+ * is possible and PR_IB_SUPPORTED if an IB path record
+ * query is possible.
+ */
+static int opa_pr_query_possible(struct ib_sa_client *client,
+ struct ib_device *device,
+ u8 port_num,
+ struct sa_path_rec *rec)
+{
+ struct ib_port_attr port_attr;
+
+ if (ib_query_port(device, port_num, &port_attr))
+ return PR_NOT_SUPPORTED;
+
+ if (ib_sa_opa_pathrecord_support(client, device, port_num))
+ return PR_OPA_SUPPORTED;
+
+ if (port_attr.lid >= be16_to_cpu(IB_MULTICAST_LID_BASE))
+ return PR_NOT_SUPPORTED;
+ else
+ return PR_IB_SUPPORTED;
+}
+
static void ib_sa_path_rec_callback(struct ib_sa_query *sa_query,
int status,
struct ib_sa_mad *mad)
@@ -1328,20 +1516,42 @@ static void ib_sa_path_rec_callback(struct ib_sa_query *sa_query,
if (mad) {
struct sa_path_rec rec;
- ib_unpack(path_rec_table, ARRAY_SIZE(path_rec_table),
- mad->data, &rec);
- rec.rec_type = SA_PATH_REC_TYPE_IB;
- sa_path_set_ndev(&rec, NULL);
- sa_path_set_ifindex(&rec, 0);
- sa_path_set_dmac_zero(&rec);
- query->callback(status, &rec, query->context);
+ if (sa_query->flags & IB_SA_QUERY_OPA) {
+ ib_unpack(opa_path_rec_table,
+ ARRAY_SIZE(opa_path_rec_table),
+ mad->data, &rec);
+ rec.rec_type = SA_PATH_REC_TYPE_OPA;
+ query->callback(status, &rec, query->context);
+ } else {
+ ib_unpack(path_rec_table,
+ ARRAY_SIZE(path_rec_table),
+ mad->data, &rec);
+ rec.rec_type = SA_PATH_REC_TYPE_IB;
+ sa_path_set_ndev(&rec, NULL);
+ sa_path_set_ifindex(&rec, 0);
+ sa_path_set_dmac_zero(&rec);
+
+ if (query->conv_pr) {
+ struct sa_path_rec opa;
+
+ memset(&opa, 0, sizeof(struct sa_path_rec));
+ sa_convert_path_ib_to_opa(&opa, &rec);
+ query->callback(status, &opa, query->context);
+ } else {
+ query->callback(status, &rec, query->context);
+ }
+ }
} else
query->callback(status, NULL, query->context);
}
static void ib_sa_path_rec_release(struct ib_sa_query *sa_query)
{
- kfree(container_of(sa_query, struct ib_sa_path_query, sa_query));
+ struct ib_sa_path_query *query =
+ container_of(sa_query, struct ib_sa_path_query, sa_query);
+
+ kfree(query->conv_pr);
+ kfree(query);
}
/**
@@ -1385,12 +1595,14 @@ int ib_sa_path_rec_get(struct ib_sa_client *client,
struct ib_sa_port *port;
struct ib_mad_agent *agent;
struct ib_sa_mad *mad;
+ enum opa_pr_supported status;
int ret;
if (!sa_dev)
return -ENODEV;
- if (rec->rec_type != SA_PATH_REC_TYPE_IB)
+ if ((rec->rec_type != SA_PATH_REC_TYPE_IB) &&
+ (rec->rec_type != SA_PATH_REC_TYPE_OPA))
return -EINVAL;
port = &sa_dev->port[port_num - sa_dev->start_port];
@@ -1401,6 +1613,23 @@ int ib_sa_path_rec_get(struct ib_sa_client *client,
return -ENOMEM;
query->sa_query.port = port;
+ if (rec->rec_type == SA_PATH_REC_TYPE_OPA) {
+ status = opa_pr_query_possible(client, device, port_num, rec);
+ if (status == PR_NOT_SUPPORTED) {
+ return -EINVAL;
+ goto err1;
+ } else if (status == PR_OPA_SUPPORTED) {
+ query->sa_query.flags |= IB_SA_QUERY_OPA;
+ } else {
+ query->conv_pr =
+ kmalloc(sizeof(*query->conv_pr), gfp_mask);
+ if (!query->conv_pr) {
+ ret = -ENOMEM;
+ goto err1;
+ }
+ }
+ }
+
ret = alloc_mad(&query->sa_query, gfp_mask);
if (ret)
goto err1;
@@ -1419,12 +1648,23 @@ int ib_sa_path_rec_get(struct ib_sa_client *client,
mad->mad_hdr.attr_id = cpu_to_be16(IB_SA_ATTR_PATH_REC);
mad->sa_hdr.comp_mask = comp_mask;
- ib_pack(path_rec_table, ARRAY_SIZE(path_rec_table), rec, mad->data);
+ if (query->sa_query.flags & IB_SA_QUERY_OPA) {
+ ib_pack(opa_path_rec_table, ARRAY_SIZE(opa_path_rec_table),
+ rec, mad->data);
+ } else if (query->conv_pr) {
+ sa_convert_path_opa_to_ib(query->conv_pr, rec);
+ ib_pack(path_rec_table, ARRAY_SIZE(path_rec_table),
+ query->conv_pr, mad->data);
+ } else {
+ ib_pack(path_rec_table, ARRAY_SIZE(path_rec_table),
+ rec, mad->data);
+ }
*sa_query = &query->sa_query;
query->sa_query.flags |= IB_SA_ENABLE_LOCAL_SERVICE;
- query->sa_query.mad_buf->context[1] = rec;
+ query->sa_query.mad_buf->context[1] = (query->conv_pr) ?
+ query->conv_pr : rec;
ret = send_mad(&query->sa_query, timeout_ms, gfp_mask);
if (ret < 0)
diff --git a/drivers/infiniband/ulp/ipoib/ipoib_main.c b/drivers/infiniband/ulp/ipoib/ipoib_main.c
index 4d3f146..7dfe424 100644
--- a/drivers/infiniband/ulp/ipoib/ipoib_main.c
+++ b/drivers/infiniband/ulp/ipoib/ipoib_main.c
@@ -832,7 +832,10 @@ static struct ipoib_path *path_rec_create(struct net_device *dev, void *gid)
INIT_LIST_HEAD(&path->neigh_list);
- path->pathrec.rec_type = SA_PATH_REC_TYPE_IB;
+ if (rdma_cap_opa_ah(priv->ca, priv->port))
+ path->pathrec.rec_type = SA_PATH_REC_TYPE_OPA;
+ else
+ path->pathrec.rec_type = SA_PATH_REC_TYPE_IB;
memcpy(path->pathrec.dgid.raw, gid, sizeof (union ib_gid));
path->pathrec.sgid = priv->local_gid;
path->pathrec.pkey = cpu_to_be16(priv->pkey);
diff --git a/drivers/infiniband/ulp/srp/ib_srp.c b/drivers/infiniband/ulp/srp/ib_srp.c
index 07877a8..def723a 100644
--- a/drivers/infiniband/ulp/srp/ib_srp.c
+++ b/drivers/infiniband/ulp/srp/ib_srp.c
@@ -312,7 +312,11 @@ static int srp_new_cm_id(struct srp_rdma_ch *ch)
if (ch->cm_id)
ib_destroy_cm_id(ch->cm_id);
ch->cm_id = new_cm_id;
- ch->path.rec_type = SA_PATH_REC_TYPE_IB;
+ if (rdma_cap_opa_ah(target->srp_host->srp_dev->dev,
+ target->srp_host->port))
+ ch->path.rec_type = SA_PATH_REC_TYPE_OPA;
+ else
+ ch->path.rec_type = SA_PATH_REC_TYPE_IB;
ch->path.sgid = target->sgid;
ch->path.dgid = target->orig_dgid;
ch->path.pkey = target->pkey;
diff --git a/include/rdma/ib_mad.h b/include/rdma/ib_mad.h
index 8e75f5d..2632516 100644
--- a/include/rdma/ib_mad.h
+++ b/include/rdma/ib_mad.h
@@ -262,6 +262,8 @@ struct ib_class_port_info {
__be32 trap_qkey;
};
+#define OPA_CLASS_PORT_INFO_PR_SUPPORT BIT(26)
+
struct opa_class_port_info {
u8 base_version;
u8 class_version;
@@ -340,6 +342,17 @@ static inline void ib_set_cpi_capmask2(struct ib_class_port_info *cpi,
IB_CLASS_PORT_INFO_RESP_TIME_FIELD_SIZE);
}
+/**
+ * opa_get_cpi_capmask2 - Returns the capmask2 value from
+ * cap_mask2_resp_time in ib_class_port_info.
+ * @cpi: A struct opa_class_port_info mad.
+ */
+static inline u32 opa_get_cpi_capmask2(struct opa_class_port_info *cpi)
+{
+ return (be32_to_cpu(cpi->cap_mask2_resp_time) >>
+ IB_CLASS_PORT_INFO_RESP_TIME_FIELD_SIZE);
+}
+
struct ib_mad_notice_attr {
u8 generic_type;
u8 prod_type_msb;
--
1.8.3.1
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply related [flat|nested] 13+ messages in thread
* Re: [PATCH rdma-next 5/6] IB/SA: Add OPA path record type
[not found] ` <1492462590-64866-6-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
@ 2017-04-19 14:50 ` Hal Rosenstock
[not found] ` <632589f6-b39b-f8e0-34b5-680170b5c44a-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
0 siblings, 1 reply; 13+ messages in thread
From: Hal Rosenstock @ 2017-04-19 14:50 UTC (permalink / raw)
To: Dasaratharaman Chandramouli, Doug Ledford; +Cc: linux-rdma
On 4/17/2017 4:56 PM, Dasaratharaman Chandramouli wrote:
> diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
> index 7d6c199..38df02f 100644
> --- a/drivers/infiniband/ulp/srpt/ib_srpt.c
> +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
> @@ -1846,6 +1846,7 @@ static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
> struct srp_login_rej *rej;
> struct ib_cm_rep_param *rep_param;
> struct srpt_rdma_ch *ch, *tmp_ch;
> + union ib_gid *sgid = ¶m->primary_path->sgid;
> __be16 *guid;
> u32 it_iu_len;
> int i, ret = 0;
> @@ -1992,7 +1993,7 @@ static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
> goto destroy_ib;
> }
>
> - guid = (__be16 *)¶m->primary_path->sgid.global.interface_id;
> + guid = (__be16 *)&sgid->global.interface_id;
Nit: This seems like an unrelated change.
-- Hal
> snprintf(ch->ini_guid, sizeof(ch->ini_guid), "%04x:%04x:%04x:%04x",
> be16_to_cpu(guid[0]), be16_to_cpu(guid[1]),
> be16_to_cpu(guid[2]), be16_to_cpu(guid[3]));
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records
[not found] ` <1492462590-64866-7-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
@ 2017-04-19 14:50 ` Hal Rosenstock
2017-04-19 14:52 ` Hal Rosenstock
1 sibling, 0 replies; 13+ messages in thread
From: Hal Rosenstock @ 2017-04-19 14:50 UTC (permalink / raw)
To: Dasaratharaman Chandramouli, Doug Ledford; +Cc: linux-rdma
On 4/17/2017 4:56 PM, Dasaratharaman Chandramouli wrote:
> When the bit 26 of capmask2 field in OPA classport info
> query is set, SA will query for OPA path records instead
> of querying for IB path records.
If/when additional SA query types are supported for OPA, will each
additional SA query type have separate CapMask2 bit or will they be
combined into 1 ? For example, would SA MCMemberRecord be different for
OPA (perhaps 32 bit MLID) and if so, would support be indicated by
different capability bit or same one ? Is that the only other one ?
Currently, IB SA is using up through bit 13 and new features continue to
be added (at least one more coming soon). There is some chance for
feature collision if OPA would care about supporting such new IB feature
if and when OPA and IB bitmasks meet with IB incrementing from bit 13
and OPA decrementing from bit 26. I wouldn't expect that to happen for a
while but best
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records
[not found] ` <1492462590-64866-7-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-19 14:50 ` Hal Rosenstock
@ 2017-04-19 14:52 ` Hal Rosenstock
[not found] ` <7304d2b8-87f4-e7ec-ebe4-ccaf34c09976-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
1 sibling, 1 reply; 13+ messages in thread
From: Hal Rosenstock @ 2017-04-19 14:52 UTC (permalink / raw)
To: Dasaratharaman Chandramouli, Doug Ledford; +Cc: linux-rdma
On 4/17/2017 4:56 PM, Dasaratharaman Chandramouli wrote:
> When the bit 26 of capmask2 field in OPA classport info
> query is set, SA will query for OPA path records instead
> of querying for IB path records.
If/when additional SA query types are supported for OPA, will each
additional SA query type have separate CapMask2 bit or will they be
combined into 1 ? For example, would SA MCMemberRecord be different for
OPA (perhaps 32 bit MLID) and if so, would support be indicated by
different capability bit or same one ? Is that the only other one ?
Currently, IB SA is using up through bit 13 and new features continue to
be added (at least one more coming soon). There is some chance for
feature collision if OPA would care about supporting such new IB feature
if and when OPA and IB bitmasks meet with IB incrementing from bit 13
and OPA decrementing from bit 26. I wouldn't expect that to happen for a
while but it would be best to anticipate this if it is of any concern to
OPA.
-- Hal
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records
[not found] ` <7304d2b8-87f4-e7ec-ebe4-ccaf34c09976-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2017-04-19 16:54 ` Chandramouli, Dasaratharaman
[not found] ` <c2ccd3ff-ed35-de86-73da-34a5d8d17551-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
0 siblings, 1 reply; 13+ messages in thread
From: Chandramouli, Dasaratharaman @ 2017-04-19 16:54 UTC (permalink / raw)
To: Hal Rosenstock, Doug Ledford; +Cc: linux-rdma
Hal,
On 4/19/2017 7:52 AM, Hal Rosenstock wrote:
> On 4/17/2017 4:56 PM, Dasaratharaman Chandramouli wrote:
>> When the bit 26 of capmask2 field in OPA classport info
>> query is set, SA will query for OPA path records instead
>> of querying for IB path records.
>
> If/when additional SA query types are supported for OPA, will each
> additional SA query type have separate CapMask2 bit or will they be
> combined into 1 ? For example, would SA MCMemberRecord be different for
> OPA (perhaps 32 bit MLID) and if so, would support be indicated by
> different capability bit or same one ? Is that the only other one ?
There are no plans to provide a separate capmask2 bit for each of the
other queries.
>
> Currently, IB SA is using up through bit 13 and new features continue to
> be added (at least one more coming soon). There is some chance for
> feature collision if OPA would care about supporting such new IB feature
> if and when OPA and IB bitmasks meet with IB incrementing from bit 13
> and OPA decrementing from bit 26. I wouldn't expect that to happen for a
> while but it would be best to anticipate this if it is of any concern to
> OPA.
I see your concern. The capmask2 bits for OPA is not intended to be in
sync with IB. If, for instance, we have a new IB feature that is exposed
by Bit n of capmask, two things are possible. 1) OPA might not support
the same feature. If the class port info query that is 'cached' by the
SA is of type OPA, it implicitly means that the feature is not
supported. This patch series adds a function
"ib_sa_sendonly_fullmem_support" that is a good example of that case.
When a caller calls this function and if the class port info is of type
OPA, we just return 'false' implying that OPA doesn't possess
sendonly_fullmember suppprt. 2) OPA might support the feature. In this
case, OPA class port info will expose the feature in the capmask2 field
bit not necessarily in the same bit n. SA should be able to handle these
differences.
> -- Hal
>
Thanks,
Dasa
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH rdma-next 5/6] IB/SA: Add OPA path record type
[not found] ` <632589f6-b39b-f8e0-34b5-680170b5c44a-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
@ 2017-04-19 16:55 ` Chandramouli, Dasaratharaman
0 siblings, 0 replies; 13+ messages in thread
From: Chandramouli, Dasaratharaman @ 2017-04-19 16:55 UTC (permalink / raw)
To: Hal Rosenstock, Doug Ledford; +Cc: linux-rdma
On 4/19/2017 7:50 AM, Hal Rosenstock wrote:
> On 4/17/2017 4:56 PM, Dasaratharaman Chandramouli wrote:
>> diff --git a/drivers/infiniband/ulp/srpt/ib_srpt.c b/drivers/infiniband/ulp/srpt/ib_srpt.c
>> index 7d6c199..38df02f 100644
>> --- a/drivers/infiniband/ulp/srpt/ib_srpt.c
>> +++ b/drivers/infiniband/ulp/srpt/ib_srpt.c
>> @@ -1846,6 +1846,7 @@ static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
>> struct srp_login_rej *rej;
>> struct ib_cm_rep_param *rep_param;
>> struct srpt_rdma_ch *ch, *tmp_ch;
>> + union ib_gid *sgid = ¶m->primary_path->sgid;
>> __be16 *guid;
>> u32 it_iu_len;
>> int i, ret = 0;
>> @@ -1992,7 +1993,7 @@ static int srpt_cm_req_recv(struct ib_cm_id *cm_id,
>> goto destroy_ib;
>> }
>>
>> - guid = (__be16 *)¶m->primary_path->sgid.global.interface_id;
>> + guid = (__be16 *)&sgid->global.interface_id;
>
> Nit: This seems like an unrelated change.
>
> -- Hal
Will fix.
Thanks.
>
>> snprintf(ch->ini_guid, sizeof(ch->ini_guid), "%04x:%04x:%04x:%04x",
>> be16_to_cpu(guid[0]), be16_to_cpu(guid[1]),
>> be16_to_cpu(guid[2]), be16_to_cpu(guid[3]));
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
* Re: [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records
[not found] ` <c2ccd3ff-ed35-de86-73da-34a5d8d17551-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
@ 2017-04-19 17:59 ` Hal Rosenstock
0 siblings, 0 replies; 13+ messages in thread
From: Hal Rosenstock @ 2017-04-19 17:59 UTC (permalink / raw)
To: Chandramouli, Dasaratharaman, Doug Ledford; +Cc: linux-rdma
Dasa,
On 4/19/2017 12:54 PM, Chandramouli, Dasaratharaman wrote:
>
> Hal,
>
> On 4/19/2017 7:52 AM, Hal Rosenstock wrote:
>> On 4/17/2017 4:56 PM, Dasaratharaman Chandramouli wrote:
>>> When the bit 26 of capmask2 field in OPA classport info
>>> query is set, SA will query for OPA path records instead
>>> of querying for IB path records.
>>
>> If/when additional SA query types are supported for OPA, will each
>> additional SA query type have separate CapMask2 bit or will they be
>> combined into 1 ? For example, would SA MCMemberRecord be different for
>> OPA (perhaps 32 bit MLID) and if so, would support be indicated by
>> different capability bit or same one ? Is that the only other one ?
>
> There are no plans to provide a separate capmask2 bit for each of the
> other queries.
Glad to hear this.
>>
>> Currently, IB SA is using up through bit 13 and new features continue to
>> be added (at least one more coming soon). There is some chance for
>> feature collision if OPA would care about supporting such new IB feature
>> if and when OPA and IB bitmasks meet with IB incrementing from bit 13
>> and OPA decrementing from bit 26. I wouldn't expect that to happen for a
>> while but it would be best to anticipate this if it is of any concern to
>> OPA.
>
> I see your concern. The capmask2 bits for OPA is not intended to be in
> sync with IB. If, for instance, we have a new IB feature that is exposed
> by Bit n of capmask, two things are possible. 1) OPA might not support
> the same feature. If the class port info query that is 'cached' by the
> SA is of type OPA, it implicitly means that the feature is not
> supported. This patch series adds a function
> "ib_sa_sendonly_fullmem_support" that is a good example of that case.
> When a caller calls this function and if the class port info is of type
> OPA, we just return 'false' implying that OPA doesn't possess
> sendonly_fullmember suppprt. 2) OPA might support the feature. In this
> case, OPA class port info will expose the feature in the capmask2 field
> bit not necessarily in the same bit n. SA should be able to handle these
> differences.
Yes, those are the choices available.
-- Hal
>> -- Hal
>>
>
> Thanks,
> Dasa
>
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 13+ messages in thread
end of thread, other threads:[~2017-04-19 17:59 UTC | newest]
Thread overview: 13+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-17 20:56 [PATCH rdma-next 0/6] Add support for OPA path records Dasaratharaman Chandramouli
[not found] ` <1492462590-64866-1-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-17 20:56 ` [PATCH rdma-next 1/6] IB/CM: Add braces when using sizeof Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 2/6] IB/SA: Rename ib_sa_path_rec to sa_path_rec Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 3/6] IB/SA: Introduce path record specific types Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 4/6] IB/SA: Split struct sa_path_rec based on IB and ROCE specific fields Dasaratharaman Chandramouli
2017-04-17 20:56 ` [PATCH rdma-next 5/6] IB/SA: Add OPA path record type Dasaratharaman Chandramouli
[not found] ` <1492462590-64866-6-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-19 14:50 ` Hal Rosenstock
[not found] ` <632589f6-b39b-f8e0-34b5-680170b5c44a-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2017-04-19 16:55 ` Chandramouli, Dasaratharaman
2017-04-17 20:56 ` [PATCH rdma-next 6/6] IB/SA: Add support to query OPA path records Dasaratharaman Chandramouli
[not found] ` <1492462590-64866-7-git-send-email-dasaratharaman.chandramouli-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-19 14:50 ` Hal Rosenstock
2017-04-19 14:52 ` Hal Rosenstock
[not found] ` <7304d2b8-87f4-e7ec-ebe4-ccaf34c09976-LDSdmyG8hGV8YrgS2mwiifqBs+8SCbDb@public.gmane.org>
2017-04-19 16:54 ` Chandramouli, Dasaratharaman
[not found] ` <c2ccd3ff-ed35-de86-73da-34a5d8d17551-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
2017-04-19 17:59 ` Hal Rosenstock
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.