* [PATCH RFC iproute-next 1/5] rdma: update rdma_netlink.h
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
@ 2018-02-14 21:05 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information Steve Wise
` (3 subsequent siblings)
4 siblings, 0 replies; 16+ messages in thread
From: Steve Wise @ 2018-02-14 21:05 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
From: Steve Wise <swise@opengridcomputing.com>
Pull in the latest rdma_netlink.h which has support for
the rdma nldev resource tracking objects being added
with this patch series.
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
include/uapi/rdma/rdma_netlink.h | 62 ++++++++++++++++++++++++++++++++++++++--
1 file changed, 59 insertions(+), 3 deletions(-)
diff --git a/include/uapi/rdma/rdma_netlink.h b/include/uapi/rdma/rdma_netlink.h
index dbac3b8..7052a7c 100644
--- a/include/uapi/rdma/rdma_netlink.h
+++ b/include/uapi/rdma/rdma_netlink.h
@@ -1,6 +1,6 @@
/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
-#ifndef _RDMA_NETLINK_H
-#define _RDMA_NETLINK_H
+#ifndef _UAPI_RDMA_NETLINK_H
+#define _UAPI_RDMA_NETLINK_H
#include <linux/types.h>
@@ -238,6 +238,14 @@ enum rdma_nldev_command {
RDMA_NLDEV_CMD_RES_QP_GET, /* can dump */
+ RDMA_NLDEV_CMD_RES_CM_ID_GET, /* can dump */
+
+ RDMA_NLDEV_CMD_RES_CQ_GET, /* can dump */
+
+ RDMA_NLDEV_CMD_RES_MR_GET, /* can dump */
+
+ RDMA_NLDEV_CMD_RES_PD_GET, /* can dump */
+
RDMA_NLDEV_NUM_OPS
};
@@ -350,6 +358,54 @@ enum rdma_nldev_attr {
*/
RDMA_NLDEV_ATTR_RES_KERN_NAME, /* string */
+ RDMA_NLDEV_ATTR_RES_CM_ID, /* nested table */
+ RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY, /* nested table */
+ /*
+ * rdma_cm_id port space.
+ */
+ RDMA_NLDEV_ATTR_RES_PS, /* u32 */
+ /*
+ * Source and destination IP address and port attributes.
+ */
+ RDMA_NLDEV_ATTR_RES_IPV4_SADDR, /* u8[4] */
+ RDMA_NLDEV_ATTR_RES_IPV4_DADDR, /* u8[4] */
+ RDMA_NLDEV_ATTR_RES_IPV6_SADDR, /* u8[16] */
+ RDMA_NLDEV_ATTR_RES_IPV6_DADDR, /* u8[16] */
+ RDMA_NLDEV_ATTR_RES_IP_SPORT, /* u16 */
+ RDMA_NLDEV_ATTR_RES_IP_DPORT, /* u16 */
+ /*
+ * ARPHRD_INFINIBAND, ARPHRD_ETHER, ...
+ */
+ RDMA_NLDEV_ATTR_RES_DEV_TYPE, /* u8 */
+ /*
+ * enum enum rdma_transport_type (IB, IWARP, ...)
+ */
+ RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE, /* u8 */
+ /*
+ * enum rdma_network_type (IB, IPv4, IPv6,...)
+ */
+ RDMA_NLDEV_ATTR_RES_NETWORK_TYPE, /* u8 */
+
+ RDMA_NLDEV_ATTR_RES_CQ, /* nested table */
+ RDMA_NLDEV_ATTR_RES_CQ_ENTRY, /* nested table */
+ RDMA_NLDEV_ATTR_RES_CQE, /* u32 */
+ RDMA_NLDEV_ATTR_RES_USECNT, /* u64 */
+ RDMA_NLDEV_ATTR_RES_POLL_CTX, /* u8 */
+
+ RDMA_NLDEV_ATTR_RES_MR, /* nested table */
+ RDMA_NLDEV_ATTR_RES_MR_ENTRY, /* nested table */
+ RDMA_NLDEV_ATTR_RES_RKEY, /* u32 */
+ RDMA_NLDEV_ATTR_RES_LKEY, /* u32 */
+ RDMA_NLDEV_ATTR_RES_IOVA, /* u64 */
+ RDMA_NLDEV_ATTR_RES_MRLEN, /* u64 */
+ RDMA_NLDEV_ATTR_RES_PGSIZE, /* u32 */
+
+ RDMA_NLDEV_ATTR_RES_PD, /* nested table */
+ RDMA_NLDEV_ATTR_RES_PD_ENTRY, /* nested table */
+ RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY, /* u32 */
+ RDMA_NLDEV_ATTR_RES_PD_FLAGS, /* u32 */
+ RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY, /* u32 */
+
RDMA_NLDEV_ATTR_MAX
};
-#endif /* _RDMA_NETLINK_H */
+#endif /* _UAPI_RDMA_NETLINK_H */
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH RFC iproute-next 4/5] rdma: Add MR resource tracking information
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
` (3 preceding siblings ...)
2018-02-14 21:07 ` [PATCH RFC iproute-next 3/5] rdma: Add CQ " Steve Wise
@ 2018-02-14 21:07 ` Steve Wise
2018-02-20 14:12 ` Leon Romanovsky
4 siblings, 1 reply; 16+ messages in thread
From: Steve Wise @ 2018-02-14 21:07 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
Sample output:
# rdma resource show mr
link cxgb4_0/- rkey 0x5e0e lkey 0x5e0e iova 0x7f9b60001c80 mrlen 64 pgsize 4096 pid 30503 comm rping
link cxgb4_0/- rkey 0x510d lkey 0x510d iova 0x17c7ee0 mrlen 16 pgsize 4096 pid 30503 comm rping
link cxgb4_0/- rkey 0x4a0c lkey 0x4a0c iova 0x17c7e38 mrlen 16 pgsize 4096 pid 30503 comm rping
link cxgb4_0/- rkey 0x480b lkey 0x480b iova 0x224b3f0 mrlen 64 pgsize 4096 pid 30498 comm rping
link cxgb4_0/- rkey 0x460a lkey 0x460a iova 0x224b350 mrlen 64 pgsize 4096 pid 30498 comm rping
link cxgb4_0/- rkey 0x4509 lkey 0x4509 iova 0x2245890 mrlen 16 pgsize 4096 pid 30498 comm rping
link cxgb4_0/- rkey 0x4208 lkey 0x4208 iova 0x22457e8 mrlen 16 pgsize 4096 pid 30498 comm rping
link mlx4_0/- rkey 0x38010700 lkey 0x38010700 iova 0x7f9b5c002f90 mrlen 64 pgsize 4096 pid 30494 comm rping
link mlx4_0/- rkey 0x38010600 lkey 0x38010600 iova 0x17c6c80 mrlen 16 pgsize 4096 pid 30494 comm rping
link mlx4_0/- rkey 0x38010500 lkey 0x38010500 iova 0x17c6bd8 mrlen 16 pgsize 4096 pid 30494 comm rping
link mlx4_0/- rkey 0x38010400 lkey 0x38010400 iova 0x1b68430 mrlen 64 pgsize 4096 pid 30489 comm rping
link mlx4_0/- rkey 0x38010300 lkey 0x38010300 iova 0x1b683a0 mrlen 64 pgsize 4096 pid 30489 comm rping
link mlx4_0/- rkey 0x38010200 lkey 0x38010200 iova 0x1b62890 mrlen 16 pgsize 4096 pid 30489 comm rping
link mlx4_0/- rkey 0x38010100 lkey 0x38010100 iova 0x1b627e8 mrlen 16 pgsize 4096 pid 30489 comm rping
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
include/json_writer.h | 2 +
lib/json_writer.c | 11 +++++
rdma/res.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++
rdma/utils.c | 7 +++
4 files changed, 146 insertions(+)
diff --git a/include/json_writer.h b/include/json_writer.h
index 1516aaf..34f2ccc 100644
--- a/include/json_writer.h
+++ b/include/json_writer.h
@@ -39,6 +39,7 @@ void jsonw_bool(json_writer_t *self, bool value);
void jsonw_float(json_writer_t *self, double number);
void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num);
void jsonw_uint(json_writer_t *self, uint64_t number);
+void jsonw_xint(json_writer_t *self, uint64_t number);
void jsonw_hu(json_writer_t *self, unsigned short number);
void jsonw_int(json_writer_t *self, int64_t number);
void jsonw_null(json_writer_t *self);
@@ -49,6 +50,7 @@ void jsonw_string_field(json_writer_t *self, const char *prop, const char *val);
void jsonw_bool_field(json_writer_t *self, const char *prop, bool value);
void jsonw_float_field(json_writer_t *self, const char *prop, double num);
void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num);
+void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num);
void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num);
void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num);
void jsonw_null_field(json_writer_t *self, const char *prop);
diff --git a/lib/json_writer.c b/lib/json_writer.c
index f3eeaf7..6d73a1b 100644
--- a/lib/json_writer.c
+++ b/lib/json_writer.c
@@ -224,6 +224,11 @@ void jsonw_uint(json_writer_t *self, uint64_t num)
jsonw_printf(self, "%"PRIu64, num);
}
+void jsonw_xint(json_writer_t *self, uint64_t num)
+{
+ jsonw_printf(self, "%"PRIx64, num);
+}
+
void jsonw_lluint(json_writer_t *self, unsigned long long int num)
{
jsonw_printf(self, "%llu", num);
@@ -268,6 +273,12 @@ void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num)
jsonw_uint(self, num);
}
+void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num)
+{
+ jsonw_name(self, prop);
+ jsonw_xint(self, num);
+}
+
void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num)
{
jsonw_name(self, prop);
diff --git a/rdma/res.c b/rdma/res.c
index 27c1efd..2b67d25 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -819,6 +819,119 @@ static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_key(struct rd *rd, const char *name, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_xint_field(rd->jw, name, val);
+ else
+ pr_out("%s 0x%x ", name, val);
+}
+
+static void print_iova(struct rd *rd, uint64_t val)
+{
+ if (rd->json_output)
+ jsonw_xint_field(rd->jw, "iova", val);
+ else
+ pr_out("iova 0x%" PRIx64 " ", val);
+}
+
+static void print_mrlen(struct rd *rd, uint64_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "mrlen", val);
+ else
+ pr_out("mrlen %" PRIu64 " ", val);
+}
+
+static void print_pgsize(struct rd *rd, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "pgsize", val);
+ else
+ pr_out("pgsize %u ", val);
+}
+
+static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ uint32_t idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_MR])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_MR];
+
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ uint32_t rkey, lkey, pgsize;
+ uint64_t iova, mrlen;
+ char *comm = NULL;
+ uint32_t pid = 0;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return MNL_CB_ERROR;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_RKEY] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_LKEY] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_IOVA] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_MRLEN] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_PGSIZE] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ rkey = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_RKEY]);
+ lkey = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LKEY]);
+ iova = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_IOVA]);
+ mrlen = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_MRLEN]);
+ pgsize = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PGSIZE]);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, 0, nla_line);
+ print_key(rd, "rkey", rkey);
+ print_key(rd, "lkey", lkey);
+ print_iova(rd, iova);
+ print_mrlen(rd, mrlen);
+ print_pgsize(rd, pgsize);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -880,6 +993,18 @@ filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
+static const struct
+filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
+ .is_number = false },
+ { .name = "rkey",
+ .is_number = true },
+ { .name = "lkey",
+ .is_number = true },
+ { .name = "pid",
+ .is_number = true }};
+
+RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
@@ -887,6 +1012,7 @@ static int res_show(struct rd *rd)
{ "qp", res_qp },
{ "cm_id", res_cm_id },
{ "cq", res_cq },
+ { "mr", res_mr },
{ 0 }
};
diff --git a/rdma/utils.c b/rdma/utils.c
index 11b34fe..34b195a 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -392,6 +392,13 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
[RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
[RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
[RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
+ [RDMA_NLDEV_ATTR_RES_MR] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_MR_ENTRY] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_RKEY] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_LKEY] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_IOVA] = MNL_TYPE_U64,
+ [RDMA_NLDEV_ATTR_RES_MRLEN] = MNL_TYPE_U64,
+ [RDMA_NLDEV_ATTR_RES_PGSIZE] = MNL_TYPE_U32,
};
int rd_attr_cb(const struct nlattr *attr, void *data)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
2018-02-14 21:05 ` [PATCH RFC iproute-next 1/5] rdma: update rdma_netlink.h Steve Wise
@ 2018-02-14 21:07 ` Steve Wise
2018-02-20 12:57 ` Leon Romanovsky
2018-02-14 21:07 ` [PATCH RFC iproute-next 5/5] rdma: Add PD " Steve Wise
` (2 subsequent siblings)
4 siblings, 1 reply; 16+ messages in thread
From: Steve Wise @ 2018-02-14 21:07 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
Sample output:
# rdma resource
2: cxgb4_0: pd 5 cq 2 qp 2 cm_id 3 mr 7
3: mlx4_0: pd 7 cq 3 qp 3 cm_id 3 mr 7
# rdma resource show cm_id
[root@stevo1 iproute2]# /root/stevo/iproute2/rdma/rdma resource show cm_id
link cxgb4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type --- transport-type IWARP pid 30485 comm rping src-addr 0.0.0.0 src-port 7174 dst-addr 0.0.0.0 dst-port 0
link cxgb4_0/2 lqpn 1048 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid 30503 comm rping src-addr 172.16.2.1 src-port 7174 dst-addr 172.16.2.1 dst-port 38246
link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid 30498 comm rping src-addr 172.16.2.1 src-port 38246 dst-addr 172.16.2.1 dst-port 7174
link mlx4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type --- transport-type IB pid 30485 comm rping src-addr 0.0.0.0 src-port 7174 dst-addr 0.0.0.0 dst-port 0
link mlx4_0/1 lqpn 539 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IB pid 30494 comm rping src-addr 172.16.99.1 src-port 7174 dst-addr 172.16.99.1 dst-port 43670
link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port 43670 dst-addr 172.16.99.1 dst-port 7174
# rdma resource show cm_id dst-port 7174
link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid 30498 comm rping src-addr 172.16.2.1 src-port 38246 dst-addr 172.16.2.1 dst-port 7174
link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port 43670 dst-addr 172.16.99.1 dst-port 7174
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
rdma/rdma.h | 1 +
rdma/res.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
rdma/utils.c | 12 +++
3 files changed, 321 insertions(+), 4 deletions(-)
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 5809f70..1ef0942 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -18,6 +18,7 @@
#include <libmnl/libmnl.h>
#include <rdma/rdma_netlink.h>
#include <time.h>
+#include <net/if_arp.h>
#include "list.h"
#include "utils.h"
diff --git a/rdma/res.c b/rdma/res.c
index 2a63e71..beae7dc 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -16,9 +16,11 @@ static int res_help(struct rd *rd)
{
pr_out("Usage: %s resource\n", rd->filename);
pr_out(" resource show [DEV]\n");
- pr_out(" resource show [qp]\n");
+ pr_out(" resource show [qp|cm_id]\n");
pr_out(" resource show qp link [DEV/PORT]\n");
pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
+ pr_out(" resource show cm_id link [DEV/PORT]\n");
+ pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
return 0;
}
@@ -431,6 +433,278 @@ static int res_qp_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_qp_type(struct rd *rd, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_string_field(rd->jw, "qp-type",
+ qp_types_to_str(val));
+ else
+ pr_out("qp-type %s ", qp_types_to_str(val));
+}
+
+static const char *cm_id_state_to_str(uint8_t idx)
+{
+ static const char * const cm_id_states_str[] = { "IDLE", "ADDR_QUERY",
+ "ADDR_RESOLVED", "ROUTE_QUERY", "ROUTE_RESOLVED",
+ "CONNECT", "DISCONNECT",
+ "ADDR_BOUND", "LISTEN", "DEVICE_REMOVAL", "DESTROYING" };
+
+ if (idx < ARRAY_SIZE(cm_id_states_str))
+ return cm_id_states_str[idx];
+ return "UNKNOWN";
+}
+
+enum rdma_port_space {
+ RDMA_PS_SDP = 0x0001,
+ RDMA_PS_IPOIB = 0x0002,
+ RDMA_PS_IB = 0x013F,
+ RDMA_PS_TCP = 0x0106,
+ RDMA_PS_UDP = 0x0111,
+};
+
+static const char *cm_id_ps_to_str(uint32_t ps)
+{
+ switch (ps) {
+ case RDMA_PS_SDP:
+ return "SDP";
+ case RDMA_PS_IPOIB:
+ return "IPoIB";
+ case RDMA_PS_IB:
+ return "IPoIB";
+ case RDMA_PS_TCP:
+ return "TCP";
+ case RDMA_PS_UDP:
+ return "UDP";
+ default:
+ return "---";
+ }
+}
+
+static const char *cm_id_dev_type_to_str(uint8_t dev_type)
+{
+ switch (dev_type) {
+ case ARPHRD_INFINIBAND:
+ return "IB";
+ case ARPHRD_ETHER:
+ return "ETH";
+ default:
+ return "---";
+ }
+}
+
+static const char *cm_id_transport_type_to_str(uint8_t transport_type)
+{
+ static const char * const transport_type_str[] = { "IB", "IWARP", "USNIC", "USNIC/UDP" };
+
+ if (transport_type < ARRAY_SIZE(transport_type_str))
+ return transport_type_str[transport_type];
+ return "---";
+}
+
+static void print_cm_id_state(struct rd *rd, uint8_t state)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "state", cm_id_state_to_str(state));
+ return;
+ }
+ pr_out("state %s ", cm_id_state_to_str(state));
+}
+
+static void print_ps(struct rd *rd, uint32_t ps)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "ps", cm_id_ps_to_str(ps));
+ return;
+ }
+ pr_out("ps %s ", cm_id_ps_to_str(ps));
+}
+
+static void print_dev_type(struct rd *rd, uint8_t dev_type)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "dev-type", cm_id_dev_type_to_str(dev_type));
+ return;
+ }
+ pr_out("dev-type %s ", cm_id_dev_type_to_str(dev_type));
+}
+
+static void print_transport_type(struct rd *rd, uint8_t transport_type)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "transport-type", cm_id_transport_type_to_str(transport_type));
+ return;
+ }
+ pr_out("transport-type %s ", cm_id_transport_type_to_str(transport_type));
+}
+
+static void print_ipaddr(struct rd *rd, const char *key, char *addrstr)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, key, addrstr);
+ return;
+ }
+ pr_out("%s %s ", key, addrstr);
+}
+
+static void print_ipport(struct rd *rd, const char *key, uint16_t ipport)
+{
+ if (rd->json_output) {
+ jsonw_uint_field(rd->jw, key, ipport);
+ return;
+ }
+ pr_out("%s %u ", key, ipport);
+}
+
+static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ int idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_CM_ID])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_CM_ID];
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ uint8_t dev_type, transport_type;
+ char src_addr_str[INET6_ADDRSTRLEN];
+ char dst_addr_str[INET6_ADDRSTRLEN];
+ uint8_t *src_addr, *dst_addr;
+ uint16_t src_port, dst_port;
+ uint32_t port = 0, pid = 0;
+ uint8_t type, state;
+ uint32_t lqpn = 0, ps;
+ char *comm = NULL;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return -EINVAL;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_TYPE] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_STATE] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]) ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR]) ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_IP_SPORT] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_IP_DPORT] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_PORT_INDEX])
+ port = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_PORT_INDEX]);
+
+ if (port && port != rd->port_idx)
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN])
+ lqpn = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LQPN]);
+ if (rd_check_is_filtered(rd, "lqpn", lqpn))
+ continue;
+
+ ps = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PS]);
+ if (rd_check_is_string_filtered(rd, "ps", cm_id_ps_to_str(ps)))
+ continue;
+
+ type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TYPE]);
+ if (rd_check_is_string_filtered(rd, "qp-type", qp_types_to_str(type)))
+ continue;
+
+ state = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_STATE]);
+ if (rd_check_is_string_filtered(rd, "state", cm_id_state_to_str(state)))
+ continue;
+
+ dev_type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_DEV_TYPE]);
+ if (rd_check_is_string_filtered(rd, "dev-type", cm_id_dev_type_to_str(dev_type)))
+ continue;
+
+ transport_type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE]);
+ if (rd_check_is_string_filtered(rd, "transport-type", cm_id_transport_type_to_str(transport_type)))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+ if (rd_check_is_filtered(rd, "pid", pid))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) {
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+ }
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR]) {
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR])
+ return -EINVAL;
+ src_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR]);
+ if (!inet_ntop(AF_INET, src_addr, src_addr_str, INET6_ADDRSTRLEN))
+ return -EINVAL;
+ dst_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR]);
+ if (!inet_ntop(AF_INET, dst_addr, dst_addr_str, INET6_ADDRSTRLEN))
+ return -EINVAL;
+ } else {
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR])
+ return -EINVAL;
+ src_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]);
+ if (!inet_ntop(AF_INET6, src_addr, src_addr_str, INET6_ADDRSTRLEN))
+ return -EINVAL;
+ dst_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR]);
+ if (!inet_ntop(AF_INET6, dst_addr, dst_addr_str, INET6_ADDRSTRLEN))
+ return -EINVAL;
+ }
+ if (rd_check_is_string_filtered(rd, "src-addr", src_addr_str))
+ continue;
+ if (rd_check_is_string_filtered(rd, "dst-addr", dst_addr_str))
+ continue;
+
+ src_port = mnl_attr_get_u16(nla_line[RDMA_NLDEV_ATTR_RES_IP_SPORT]);
+ dst_port = mnl_attr_get_u16(nla_line[RDMA_NLDEV_ATTR_RES_IP_DPORT]);
+ if (rd_check_is_filtered(rd, "src-port", src_port))
+ continue;
+ if (rd_check_is_filtered(rd, "dst-port", dst_port))
+ continue;
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, port, nla_line);
+ print_lqpn(rd, lqpn);
+ print_qp_type(rd, type);
+ print_cm_id_state(rd, state);
+ print_ps(rd, ps);
+ print_dev_type(rd, dev_type);
+ print_transport_type(rd, transport_type);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+ print_ipaddr(rd, "src-addr", src_addr_str);
+ print_ipport(rd, "src-port", src_port);
+ print_ipaddr(rd, "dst-addr", dst_addr_str);
+ print_ipport(rd, "dst-port", dst_port);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -438,9 +712,9 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
.is_number = false },
{ .name = "lqpn",
.is_number = true },
- { .name = "rqpn",
- .is_number = true },
- { .name = "pid",
+ { .name = "type",
+ .is_number = false },
+ { .name = "cm_id_state",
.is_number = true },
{ .name = "sq-psn",
.is_number = true },
@@ -455,11 +729,41 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
RES_FUNC(res_qp, RDMA_NLDEV_CMD_RES_QP_GET, qp_valid_filters, false);
+static const struct
+filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
+ .is_number = false },
+ { .name = "lqpn",
+ .is_number = true },
+ { .name = "qp-type",
+ .is_number = false },
+ { .name = "state",
+ .is_number = false },
+ { .name = "ps",
+ .is_number = false },
+ { .name = "dev-type",
+ .is_number = false },
+ { .name = "transport-type",
+ .is_number = false },
+ { .name = "pid",
+ .is_number = true },
+ { .name = "src-addr",
+ .is_number = false },
+ { .name = "src-port",
+ .is_number = true },
+ { .name = "dst-addr",
+ .is_number = false },
+ { .name = "dst-port",
+ .is_number = true }};
+
+RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters,
+ false);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
{ NULL, res_no_args },
{ "qp", res_qp },
+ { "cm_id", res_cm_id },
{ 0 }
};
diff --git a/rdma/utils.c b/rdma/utils.c
index f946016..906ca73 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -375,6 +375,18 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
[RDMA_NLDEV_ATTR_RES_STATE] = MNL_TYPE_U8,
[RDMA_NLDEV_ATTR_RES_PID] = MNL_TYPE_U32,
[RDMA_NLDEV_ATTR_RES_KERN_NAME] = MNL_TYPE_NUL_STRING,
+ [RDMA_NLDEV_ATTR_RES_CM_ID] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_IPV4_SADDR] = MNL_TYPE_UNSPEC,
+ [RDMA_NLDEV_ATTR_RES_IPV4_DADDR] = MNL_TYPE_UNSPEC,
+ [RDMA_NLDEV_ATTR_RES_IPV6_SADDR] = MNL_TYPE_UNSPEC,
+ [RDMA_NLDEV_ATTR_RES_IPV6_DADDR] = MNL_TYPE_UNSPEC,
+ [RDMA_NLDEV_ATTR_RES_IP_SPORT] = MNL_TYPE_U16,
+ [RDMA_NLDEV_ATTR_RES_IP_DPORT] = MNL_TYPE_U16,
+ [RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
+ [RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
+ [RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
};
int rd_attr_cb(const struct nlattr *attr, void *data)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH RFC iproute-next 5/5] rdma: Add PD resource tracking information
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
2018-02-14 21:05 ` [PATCH RFC iproute-next 1/5] rdma: update rdma_netlink.h Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information Steve Wise
@ 2018-02-14 21:07 ` Steve Wise
2018-02-23 14:22 ` Leon Romanovsky
2018-02-14 21:07 ` [PATCH RFC iproute-next 3/5] rdma: Add CQ " Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 4/5] rdma: Add MR " Steve Wise
4 siblings, 1 reply; 16+ messages in thread
From: Steve Wise @ 2018-02-14 21:07 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
Sample output:
# rdma resource show pd
link cxgb4_0/- local_dma_lkey 0x0 usecnt 4 flags 0x0 pid 30503 comm rping
link cxgb4_0/- local_dma_lkey 0x0 usecnt 0 flags 0x0 pid 30488 comm rping
link cxgb4_0/- local_dma_lkey 0x0 usecnt 5 flags 0x0 pid 30498 comm rping
link cxgb4_0/- local_dma_lkey 0x0 usecnt 0 flags 0x0 pid 30501 comm rping
link cxgb4_0/- local_dma_lkey 0x0 usecnt 0 flags 0x0 pid 0 comm [ib_srp]
link mlx4_0/- local_dma_lkey 0x0 usecnt 4 flags 0x0 pid 30494 comm rping
link mlx4_0/- local_dma_lkey 0x0 usecnt 0 flags 0x0 pid 30488 comm rping
link mlx4_0/- local_dma_lkey 0x0 usecnt 5 flags 0x0 pid 30489 comm rping
link mlx4_0/- local_dma_lkey 0x0 usecnt 0 flags 0x0 pid 30492 comm rping
link mlx4_0/- local_dma_lkey 0x8000 usecnt 0 flags 0x0 pid 0 comm [ib_srpt]
link mlx4_0/- local_dma_lkey 0x8000 usecnt 0 flags 0x0 pid 0 comm [ib_srp]
link mlx4_0/- local_dma_lkey 0x8000 usecnt 1 flags 0x0 pid 0 comm [ib_core]
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
rdma/res.c | 96 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
1 file changed, 96 insertions(+)
diff --git a/rdma/res.c b/rdma/res.c
index 2b67d25..5e6bc42 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -932,6 +932,93 @@ static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_flags(struct rd *rd, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_xint_field(rd->jw, "flags", val);
+ else
+ pr_out("flags 0x%x ", val);
+}
+
+static int res_pd_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ uint32_t idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_PD])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_PD];
+
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ uint32_t local_dma_lkey, unsafe_global_rkey = 0;
+ uint32_t flags, pid = 0;
+ char *comm = NULL;
+ uint64_t usecnt;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return MNL_CB_ERROR;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_PD_FLAGS] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ local_dma_lkey = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LOCAL_DMA_LKEY]);
+ usecnt = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
+ flags = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PD_FLAGS]);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY])
+ unsafe_global_rkey = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY]);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, 0, nla_line);
+ print_key(rd, "local_dma_lkey", local_dma_lkey);
+ print_usecnt(rd, usecnt);
+ print_flags(rd, flags);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_UNSAFE_GLOBAL_RKEY])
+ print_key(rd, "unsafe_global_rkey", unsafe_global_rkey);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -1005,6 +1092,14 @@ filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true);
+static const struct
+filters pd_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
+ .is_number = false },
+ { .name = "pid",
+ .is_number = true }};
+
+RES_FUNC(res_pd, RDMA_NLDEV_CMD_RES_PD_GET, pd_valid_filters, true);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
@@ -1013,6 +1108,7 @@ static int res_show(struct rd *rd)
{ "cm_id", res_cm_id },
{ "cq", res_cq },
{ "mr", res_mr },
+ { "pd", res_pd },
{ 0 }
};
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH RFC iproute-next 3/5] rdma: Add CQ resource tracking information
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
` (2 preceding siblings ...)
2018-02-14 21:07 ` [PATCH RFC iproute-next 5/5] rdma: Add PD " Steve Wise
@ 2018-02-14 21:07 ` Steve Wise
2018-02-20 13:09 ` Leon Romanovsky
2018-02-14 21:07 ` [PATCH RFC iproute-next 4/5] rdma: Add MR " Steve Wise
4 siblings, 1 reply; 16+ messages in thread
From: Steve Wise @ 2018-02-14 21:07 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
Sample output:
# rdma resource show cq
link cxgb4_0/- cqe 46 usecnt 2 pid 30503 comm rping
link cxgb4_0/- cqe 46 usecnt 2 pid 30498 comm rping
link mlx4_0/- cqe 63 usecnt 2 pid 30494 comm rping
link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
link mlx4_0/- cqe 1023 usecnt 2 poll_ctx WORKQUEUE pid 0 comm [ib_core]
# rdma resource show cq pid 30489
link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
Signed-off-by: Steve Wise <swise@opengridcomputing.com>
---
rdma/res.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
rdma/utils.c | 5 +++
2 files changed, 128 insertions(+)
diff --git a/rdma/res.c b/rdma/res.c
index beae7dc..27c1efd 100644
--- a/rdma/res.c
+++ b/rdma/res.c
@@ -21,6 +21,8 @@ static int res_help(struct rd *rd)
pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
pr_out(" resource show cm_id link [DEV/PORT]\n");
pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
+ pr_out(" resource show cq link [DEV/PORT]\n");
+ pr_out(" resource show cq link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
return 0;
}
@@ -705,6 +707,118 @@ static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
return MNL_CB_OK;
}
+static void print_cqe(struct rd *rd, uint32_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "cqe", val);
+ else
+ pr_out("cqe %u ", val);
+}
+
+static void print_usecnt(struct rd *rd, uint64_t val)
+{
+ if (rd->json_output)
+ jsonw_uint_field(rd->jw, "usecnt", val);
+ else
+ pr_out("usecnt %" PRIu64 " ", val);
+}
+
+static const char *poll_ctx_to_str(uint8_t idx)
+{
+ static const char * const cm_id_states_str[] = { "DIRECT", "SOFTIRQ",
+ "WORKQUEUE"};
+
+ if (idx < ARRAY_SIZE(cm_id_states_str))
+ return cm_id_states_str[idx];
+ return "UNKNOWN";
+}
+
+static void print_poll_ctx(struct rd *rd, uint8_t poll_ctx)
+{
+ if (rd->json_output) {
+ jsonw_string_field(rd->jw, "poll_ctx", poll_ctx_to_str(poll_ctx));
+ return;
+ }
+ pr_out("poll_ctx %s ", poll_ctx_to_str(poll_ctx));
+}
+
+static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
+{
+ struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
+ struct nlattr *nla_table, *nla_entry;
+ struct rd *rd = data;
+ const char *name;
+ uint32_t idx;
+
+ mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
+ if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
+ !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
+ !tb[RDMA_NLDEV_ATTR_RES_CQ])
+ return MNL_CB_ERROR;
+
+ name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
+ idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
+ nla_table = tb[RDMA_NLDEV_ATTR_RES_CQ];
+
+ mnl_attr_for_each_nested(nla_entry, nla_table) {
+ struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
+ char *comm = NULL;
+ uint32_t pid = 0;
+ uint8_t poll_ctx = 0;
+ uint64_t usecnt;
+ uint32_t cqe;
+ int err;
+
+ err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
+ if (err != MNL_CB_OK)
+ return MNL_CB_ERROR;
+
+ if (!nla_line[RDMA_NLDEV_ATTR_RES_CQE] ||
+ !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
+ (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
+ !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
+ return MNL_CB_ERROR;
+ }
+
+ cqe = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_CQE]);
+ usecnt = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
+ poll_ctx = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
+ pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
+ comm = get_task_name(pid);
+ }
+
+ if (rd_check_is_filtered(rd, "pid", pid))
+ continue;
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
+ /* discard const from mnl_attr_get_str */
+ comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
+
+ if (rd->json_output)
+ jsonw_start_array(rd->jw);
+
+ print_link(rd, idx, name, 0, nla_line);
+ print_cqe(rd, cqe);
+ print_usecnt(rd, usecnt);
+ if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
+ print_poll_ctx(rd, poll_ctx);
+ print_pid(rd, pid);
+ print_comm(rd, comm, nla_line);
+
+ if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
+ free(comm);
+
+ if (rd->json_output)
+ jsonw_end_array(rd->jw);
+ else
+ pr_out("\n");
+ }
+ return MNL_CB_OK;
+}
+
RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
static const struct
@@ -758,12 +872,21 @@ filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters,
false);
+static const struct
+filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
+ .is_number = false },
+ { .name = "pid",
+ .is_number = true }};
+
+RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
+
static int res_show(struct rd *rd)
{
const struct rd_cmd cmds[] = {
{ NULL, res_no_args },
{ "qp", res_qp },
{ "cm_id", res_cm_id },
+ { "cq", res_cq },
{ 0 }
};
diff --git a/rdma/utils.c b/rdma/utils.c
index 906ca73..11b34fe 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -387,6 +387,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
[RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
[RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
[RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
+ [RDMA_NLDEV_ATTR_RES_CQ] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_CQ_ENTRY] = MNL_TYPE_NESTED,
+ [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
+ [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
+ [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
};
int rd_attr_cb(const struct nlattr *attr, void *data)
--
1.8.3.1
^ permalink raw reply related [flat|nested] 16+ messages in thread
* [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking
@ 2018-02-19 20:10 Steve Wise
2018-02-14 21:05 ` [PATCH RFC iproute-next 1/5] rdma: update rdma_netlink.h Steve Wise
` (4 more replies)
0 siblings, 5 replies; 16+ messages in thread
From: Steve Wise @ 2018-02-19 20:10 UTC (permalink / raw)
To: dsahern; +Cc: leon, stephen, netdev, linux-rdma
Hello,
This series enhances the iproute2 rdma tool to include dumping of
connection manager id (cm_id), completion queue (cq), memory region (mr),
and protection domain (pd) rdma resources. It is the user-space part of
the resource tracking series under review now on the linux-rdma list [1].
Thanks,
Steve.
[1]https://www.spinics.net/lists/linux-rdma/msg60912.html
Steve Wise (5):
rdma: update rdma_netlink.h
rdma: Add CM_ID resource tracking information
rdma: Add CQ resource tracking information
rdma: Add MR resource tracking information
rdma: Add PD resource tracking information
include/json_writer.h | 2 +
include/uapi/rdma/rdma_netlink.h | 62 +++-
lib/json_writer.c | 11 +
rdma/rdma.h | 1 +
rdma/res.c | 657 ++++++++++++++++++++++++++++++++++++++-
rdma/utils.c | 24 ++
6 files changed, 750 insertions(+), 7 deletions(-)
--
1.8.3.1
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information
2018-02-14 21:07 ` [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information Steve Wise
@ 2018-02-20 12:57 ` Leon Romanovsky
2018-02-20 15:15 ` Parav Pandit
2018-02-26 15:05 ` Steve Wise
0 siblings, 2 replies; 16+ messages in thread
From: Leon Romanovsky @ 2018-02-20 12:57 UTC (permalink / raw)
To: Steve Wise; +Cc: dsahern, stephen, netdev, linux-rdma
[-- Attachment #1: Type: text/plain, Size: 15532 bytes --]
On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> Sample output:
>
> # rdma resource
> 2: cxgb4_0: pd 5 cq 2 qp 2 cm_id 3 mr 7
> 3: mlx4_0: pd 7 cq 3 qp 3 cm_id 3 mr 7
>
> # rdma resource show cm_id
> [root@stevo1 iproute2]# /root/stevo/iproute2/rdma/rdma resource show cm_id
> link cxgb4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type --- transport-type IWARP pid 30485 comm rping src-addr 0.0.0.0 src-port 7174 dst-addr 0.0.0.0 dst-port 0
> link cxgb4_0/2 lqpn 1048 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid 30503 comm rping src-addr 172.16.2.1 src-port 7174 dst-addr 172.16.2.1 dst-port 38246
> link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid 30498 comm rping src-addr 172.16.2.1 src-port 38246 dst-addr 172.16.2.1 dst-port 7174
> link mlx4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type --- transport-type IB pid 30485 comm rping src-addr 0.0.0.0 src-port 7174 dst-addr 0.0.0.0 dst-port 0
> link mlx4_0/1 lqpn 539 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IB pid 30494 comm rping src-addr 172.16.99.1 src-port 7174 dst-addr 172.16.99.1 dst-port 43670
> link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port 43670 dst-addr 172.16.99.1 dst-port 7174
>
> # rdma resource show cm_id dst-port 7174
> link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid 30498 comm rping src-addr 172.16.2.1 src-port 38246 dst-addr 172.16.2.1 dst-port 7174
> link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port 43670 dst-addr 172.16.99.1 dst-port 7174
>
> Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> ---
> rdma/rdma.h | 1 +
> rdma/res.c | 312 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> rdma/utils.c | 12 +++
> 3 files changed, 321 insertions(+), 4 deletions(-)
Thanks, for doing it.
>
> diff --git a/rdma/rdma.h b/rdma/rdma.h
> index 5809f70..1ef0942 100644
> --- a/rdma/rdma.h
> +++ b/rdma/rdma.h
> @@ -18,6 +18,7 @@
> #include <libmnl/libmnl.h>
> #include <rdma/rdma_netlink.h>
> #include <time.h>
> +#include <net/if_arp.h>
>
> #include "list.h"
> #include "utils.h"
> diff --git a/rdma/res.c b/rdma/res.c
> index 2a63e71..beae7dc 100644
> --- a/rdma/res.c
> +++ b/rdma/res.c
> @@ -16,9 +16,11 @@ static int res_help(struct rd *rd)
> {
> pr_out("Usage: %s resource\n", rd->filename);
> pr_out(" resource show [DEV]\n");
> - pr_out(" resource show [qp]\n");
> + pr_out(" resource show [qp|cm_id]\n");
> pr_out(" resource show qp link [DEV/PORT]\n");
> pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> + pr_out(" resource show cm_id link [DEV/PORT]\n");
> + pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> return 0;
> }
>
> @@ -431,6 +433,278 @@ static int res_qp_parse_cb(const struct nlmsghdr *nlh, void *data)
> return MNL_CB_OK;
> }
>
> +static void print_qp_type(struct rd *rd, uint32_t val)
> +{
> + if (rd->json_output)
> + jsonw_string_field(rd->jw, "qp-type",
> + qp_types_to_str(val));
> + else
> + pr_out("qp-type %s ", qp_types_to_str(val));
> +}
> +
> +static const char *cm_id_state_to_str(uint8_t idx)
> +{
> + static const char * const cm_id_states_str[] = { "IDLE", "ADDR_QUERY",
> + "ADDR_RESOLVED", "ROUTE_QUERY", "ROUTE_RESOLVED",
> + "CONNECT", "DISCONNECT",
> + "ADDR_BOUND", "LISTEN", "DEVICE_REMOVAL", "DESTROYING" };
> +
> + if (idx < ARRAY_SIZE(cm_id_states_str))
> + return cm_id_states_str[idx];
> + return "UNKNOWN";
> +}
> +
> +enum rdma_port_space {
> + RDMA_PS_SDP = 0x0001,
Do we still support this PS? It is not set in the kernel and Parav
posted internal patch to remove it.
> + RDMA_PS_IPOIB = 0x0002,
> + RDMA_PS_IB = 0x013F,
> + RDMA_PS_TCP = 0x0106,
> + RDMA_PS_UDP = 0x0111,
> +};
> +
> +static const char *cm_id_ps_to_str(uint32_t ps)
> +{
> + switch (ps) {
> + case RDMA_PS_SDP:
> + return "SDP";
The same question
> + case RDMA_PS_IPOIB:
> + return "IPoIB";
> + case RDMA_PS_IB:
> + return "IPoIB";
> + case RDMA_PS_TCP:
> + return "TCP";
> + case RDMA_PS_UDP:
> + return "UDP";
> + default:
> + return "---";
> + }
> +}
> +
> +static const char *cm_id_dev_type_to_str(uint8_t dev_type)
> +{
> + switch (dev_type) {
> + case ARPHRD_INFINIBAND:
> + return "IB";
> + case ARPHRD_ETHER:
> + return "ETH";
> + default:
> + return "---";
> + }
> +}
> +
> +static const char *cm_id_transport_type_to_str(uint8_t transport_type)
> +{
> + static const char * const transport_type_str[] = { "IB", "IWARP", "USNIC", "USNIC/UDP" };
> +
I know that it is part of CM_ID, but wonder if node_type of device is
not enough. The same question goes for device type, isn't it part of
"rdma dev .." output?
> + if (transport_type < ARRAY_SIZE(transport_type_str))
> + return transport_type_str[transport_type];
> + return "---";
> +}
> +
> +static void print_cm_id_state(struct rd *rd, uint8_t state)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, "state", cm_id_state_to_str(state));
> + return;
> + }
> + pr_out("state %s ", cm_id_state_to_str(state));
> +}
> +
> +static void print_ps(struct rd *rd, uint32_t ps)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, "ps", cm_id_ps_to_str(ps));
> + return;
> + }
> + pr_out("ps %s ", cm_id_ps_to_str(ps));
> +}
> +
> +static void print_dev_type(struct rd *rd, uint8_t dev_type)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, "dev-type", cm_id_dev_type_to_str(dev_type));
> + return;
> + }
> + pr_out("dev-type %s ", cm_id_dev_type_to_str(dev_type));
> +}
> +
> +static void print_transport_type(struct rd *rd, uint8_t transport_type)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, "transport-type", cm_id_transport_type_to_str(transport_type));
> + return;
> + }
> + pr_out("transport-type %s ", cm_id_transport_type_to_str(transport_type));
> +}
> +
> +static void print_ipaddr(struct rd *rd, const char *key, char *addrstr)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, key, addrstr);
> + return;
> + }
> + pr_out("%s %s ", key, addrstr);
> +}
> +
> +static void print_ipport(struct rd *rd, const char *key, uint16_t ipport)
> +{
> + if (rd->json_output) {
> + jsonw_uint_field(rd->jw, key, ipport);
> + return;
> + }
> + pr_out("%s %u ", key, ipport);
> +}
> +
> +static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
> +{
> + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> + struct nlattr *nla_table, *nla_entry;
> + struct rd *rd = data;
> + const char *name;
> + int idx;
> +
> + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> + !tb[RDMA_NLDEV_ATTR_RES_CM_ID])
> + return MNL_CB_ERROR;
> +
> + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> + nla_table = tb[RDMA_NLDEV_ATTR_RES_CM_ID];
> + mnl_attr_for_each_nested(nla_entry, nla_table) {
> + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> + uint8_t dev_type, transport_type;
> + char src_addr_str[INET6_ADDRSTRLEN];
> + char dst_addr_str[INET6_ADDRSTRLEN];
> + uint8_t *src_addr, *dst_addr;
> + uint16_t src_port, dst_port;
> + uint32_t port = 0, pid = 0;
> + uint8_t type, state;
> + uint32_t lqpn = 0, ps;
> + char *comm = NULL;
> + int err;
> +
> + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
> + if (err != MNL_CB_OK)
> + return -EINVAL;
> +
> + if (!nla_line[RDMA_NLDEV_ATTR_RES_TYPE] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_STATE] ||
> + (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR] &&
> + !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]) ||
> + (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR] &&
> + !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR]) ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_IP_SPORT] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_IP_DPORT] ||
> + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> + return MNL_CB_ERROR;
It is unreadable, any chances to use intermediate variables with
descriptive names?
> + }
> +
> + if (nla_line[RDMA_NLDEV_ATTR_PORT_INDEX])
> + port = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_PORT_INDEX]);
> +
> + if (port && port != rd->port_idx)
> + continue;
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN])
> + lqpn = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LQPN]);
> + if (rd_check_is_filtered(rd, "lqpn", lqpn))
> + continue;
> +
> + ps = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PS]);
> + if (rd_check_is_string_filtered(rd, "ps", cm_id_ps_to_str(ps)))
> + continue;
> +
> + type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TYPE]);
> + if (rd_check_is_string_filtered(rd, "qp-type", qp_types_to_str(type)))
> + continue;
> +
> + state = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_STATE]);
> + if (rd_check_is_string_filtered(rd, "state", cm_id_state_to_str(state)))
> + continue;
> +
> + dev_type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_DEV_TYPE]);
> + if (rd_check_is_string_filtered(rd, "dev-type", cm_id_dev_type_to_str(dev_type)))
> + continue;
> +
> + transport_type = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE]);
> + if (rd_check_is_string_filtered(rd, "transport-type", cm_id_transport_type_to_str(transport_type)))
> + continue;
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> + pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> + comm = get_task_name(pid);
> + }
> + if (rd_check_is_filtered(rd, "pid", pid))
free(comm) here
> + continue;
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) {
> + /* discard const from mnl_attr_get_str */
> + comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> + }
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR]) {
> + if (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR])
> + return -EINVAL;
> + src_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR]);
> + if (!inet_ntop(AF_INET, src_addr, src_addr_str, INET6_ADDRSTRLEN))
> + return -EINVAL;
> + dst_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR]);
> + if (!inet_ntop(AF_INET, dst_addr, dst_addr_str, INET6_ADDRSTRLEN))
> + return -EINVAL;
> + } else {
> + if (!nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR])
> + return -EINVAL;
> + src_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]);
> + if (!inet_ntop(AF_INET6, src_addr, src_addr_str, INET6_ADDRSTRLEN))
> + return -EINVAL;
> + dst_addr = mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR]);
> + if (!inet_ntop(AF_INET6, dst_addr, dst_addr_str, INET6_ADDRSTRLEN))
> + return -EINVAL;
> + }
> + if (rd_check_is_string_filtered(rd, "src-addr", src_addr_str))
> + continue;
> + if (rd_check_is_string_filtered(rd, "dst-addr", dst_addr_str))
> + continue;
> +
> + src_port = mnl_attr_get_u16(nla_line[RDMA_NLDEV_ATTR_RES_IP_SPORT]);
> + dst_port = mnl_attr_get_u16(nla_line[RDMA_NLDEV_ATTR_RES_IP_DPORT]);
> + if (rd_check_is_filtered(rd, "src-port", src_port))
> + continue;
> + if (rd_check_is_filtered(rd, "dst-port", dst_port))
> + continue;
> +
The same memory leaks as above, I put get_task_name() to the end of
QP parsing code with purpose to avoid dealing with free() calls.
> + if (rd->json_output)
> + jsonw_start_array(rd->jw);
> +
> + print_link(rd, idx, name, port, nla_line);
> + print_lqpn(rd, lqpn);
> + print_qp_type(rd, type);
> + print_cm_id_state(rd, state);
> + print_ps(rd, ps);
> + print_dev_type(rd, dev_type);
> + print_transport_type(rd, transport_type);
> + print_pid(rd, pid);
> + print_comm(rd, comm, nla_line);
> + print_ipaddr(rd, "src-addr", src_addr_str);
> + print_ipport(rd, "src-port", src_port);
> + print_ipaddr(rd, "dst-addr", dst_addr_str);
> + print_ipport(rd, "dst-port", dst_port);
Does "ip tool" have standard Re presentation for addr<->port tupples?
What about the following format src 1.1.1.1:1234 dst 2.2.2.2:6789?
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> + free(comm);
> +
> + if (rd->json_output)
> + jsonw_end_array(rd->jw);
> + else
> + pr_out("\n");
> + }
> + return MNL_CB_OK;
> +}
> +
> RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
>
> static const struct
> @@ -438,9 +712,9 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> .is_number = false },
> { .name = "lqpn",
> .is_number = true },
> - { .name = "rqpn",
> - .is_number = true },
> - { .name = "pid",
> + { .name = "type",
> + .is_number = false },
> + { .name = "cm_id_state",
> .is_number = true },
> { .name = "sq-psn",
> .is_number = true },
Why did you change qp_valid_filters?
> @@ -455,11 +729,41 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
>
> RES_FUNC(res_qp, RDMA_NLDEV_CMD_RES_QP_GET, qp_valid_filters, false);
>
> +static const struct
> +filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> + .is_number = false },
> + { .name = "lqpn",
> + .is_number = true },
> + { .name = "qp-type",
> + .is_number = false },
> + { .name = "state",
> + .is_number = false },
> + { .name = "ps",
> + .is_number = false },
> + { .name = "dev-type",
> + .is_number = false },
> + { .name = "transport-type",
> + .is_number = false },
> + { .name = "pid",
> + .is_number = true },
> + { .name = "src-addr",
> + .is_number = false },
> + { .name = "src-port",
> + .is_number = true },
> + { .name = "dst-addr",
> + .is_number = false },
> + { .name = "dst-port",
> + .is_number = true }};
> +
> +RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters,
> + false);
> +
> static int res_show(struct rd *rd)
> {
> const struct rd_cmd cmds[] = {
> { NULL, res_no_args },
> { "qp", res_qp },
> + { "cm_id", res_cm_id },
> { 0 }
> };
>
> diff --git a/rdma/utils.c b/rdma/utils.c
> index f946016..906ca73 100644
> --- a/rdma/utils.c
> +++ b/rdma/utils.c
> @@ -375,6 +375,18 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> [RDMA_NLDEV_ATTR_RES_STATE] = MNL_TYPE_U8,
> [RDMA_NLDEV_ATTR_RES_PID] = MNL_TYPE_U32,
> [RDMA_NLDEV_ATTR_RES_KERN_NAME] = MNL_TYPE_NUL_STRING,
> + [RDMA_NLDEV_ATTR_RES_CM_ID] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32,
> + [RDMA_NLDEV_ATTR_RES_IPV4_SADDR] = MNL_TYPE_UNSPEC,
> + [RDMA_NLDEV_ATTR_RES_IPV4_DADDR] = MNL_TYPE_UNSPEC,
> + [RDMA_NLDEV_ATTR_RES_IPV6_SADDR] = MNL_TYPE_UNSPEC,
> + [RDMA_NLDEV_ATTR_RES_IPV6_DADDR] = MNL_TYPE_UNSPEC,
> + [RDMA_NLDEV_ATTR_RES_IP_SPORT] = MNL_TYPE_U16,
> + [RDMA_NLDEV_ATTR_RES_IP_DPORT] = MNL_TYPE_U16,
> + [RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
> + [RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
> + [RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
> };
>
> int rd_attr_cb(const struct nlattr *attr, void *data)
> --
> 1.8.3.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH RFC iproute-next 3/5] rdma: Add CQ resource tracking information
2018-02-14 21:07 ` [PATCH RFC iproute-next 3/5] rdma: Add CQ " Steve Wise
@ 2018-02-20 13:09 ` Leon Romanovsky
2018-02-26 15:06 ` Steve Wise
0 siblings, 1 reply; 16+ messages in thread
From: Leon Romanovsky @ 2018-02-20 13:09 UTC (permalink / raw)
To: Steve Wise; +Cc: dsahern, stephen, netdev, linux-rdma
[-- Attachment #1: Type: text/plain, Size: 6477 bytes --]
On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> Sample output:
>
> # rdma resource show cq
> link cxgb4_0/- cqe 46 usecnt 2 pid 30503 comm rping
> link cxgb4_0/- cqe 46 usecnt 2 pid 30498 comm rping
> link mlx4_0/- cqe 63 usecnt 2 pid 30494 comm rping
> link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
> link mlx4_0/- cqe 1023 usecnt 2 poll_ctx WORKQUEUE pid 0 comm [ib_core]
>
> # rdma resource show cq pid 30489
> link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
>
> Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> ---
> rdma/res.c | 123 +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> rdma/utils.c | 5 +++
> 2 files changed, 128 insertions(+)
>
> diff --git a/rdma/res.c b/rdma/res.c
> index beae7dc..27c1efd 100644
> --- a/rdma/res.c
> +++ b/rdma/res.c
> @@ -21,6 +21,8 @@ static int res_help(struct rd *rd)
> pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> pr_out(" resource show cm_id link [DEV/PORT]\n");
> pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> + pr_out(" resource show cq link [DEV/PORT]\n");
> + pr_out(" resource show cq link [DEV/PORT] [FILTER-NAME FILTER-VALUE]\n");
> return 0;
> }
>
> @@ -705,6 +707,118 @@ static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
> return MNL_CB_OK;
> }
>
> +static void print_cqe(struct rd *rd, uint32_t val)
> +{
> + if (rd->json_output)
> + jsonw_uint_field(rd->jw, "cqe", val);
> + else
> + pr_out("cqe %u ", val);
> +}
> +
> +static void print_usecnt(struct rd *rd, uint64_t val)
> +{
> + if (rd->json_output)
> + jsonw_uint_field(rd->jw, "usecnt", val);
> + else
> + pr_out("usecnt %" PRIu64 " ", val);
Interesting, how many users are actually know what the "usecnt" actually means?
Will it be more clear to call it "users" instead of "usecnt"?
> +}
> +
> +static const char *poll_ctx_to_str(uint8_t idx)
> +{
> + static const char * const cm_id_states_str[] = { "DIRECT", "SOFTIRQ",
> + "WORKQUEUE"};
> +
> + if (idx < ARRAY_SIZE(cm_id_states_str))
> + return cm_id_states_str[idx];
> + return "UNKNOWN";
> +}
> +
> +static void print_poll_ctx(struct rd *rd, uint8_t poll_ctx)
> +{
> + if (rd->json_output) {
> + jsonw_string_field(rd->jw, "poll_ctx", poll_ctx_to_str(poll_ctx));
> + return;
> + }
> + pr_out("poll_ctx %s ", poll_ctx_to_str(poll_ctx));
> +}
> +
> +static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
> +{
> + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> + struct nlattr *nla_table, *nla_entry;
> + struct rd *rd = data;
> + const char *name;
> + uint32_t idx;
> +
> + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> + !tb[RDMA_NLDEV_ATTR_RES_CQ])
> + return MNL_CB_ERROR;
> +
> + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> + nla_table = tb[RDMA_NLDEV_ATTR_RES_CQ];
> +
> + mnl_attr_for_each_nested(nla_entry, nla_table) {
> + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> + char *comm = NULL;
> + uint32_t pid = 0;
> + uint8_t poll_ctx = 0;
> + uint64_t usecnt;
> + uint32_t cqe;
> + int err;
> +
> + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
> + if (err != MNL_CB_OK)
> + return MNL_CB_ERROR;
> +
> + if (!nla_line[RDMA_NLDEV_ATTR_RES_CQE] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
I'm not sure that we will have USECNT in the future, let's not put
requirement for RDMA_NLDEV_ATTR_RES_USECNT here.
> + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> + return MNL_CB_ERROR;
> + }
> +
> + cqe = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_CQE]);
> + usecnt = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
> + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
> + poll_ctx = mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]);
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> + pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> + comm = get_task_name(pid);
> + }
> +
> + if (rd_check_is_filtered(rd, "pid", pid))
free(comm);
> + continue;
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
> + /* discard const from mnl_attr_get_str */
> + comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> +
> + if (rd->json_output)
> + jsonw_start_array(rd->jw);
> +
> + print_link(rd, idx, name, 0, nla_line);
> + print_cqe(rd, cqe);
> + print_usecnt(rd, usecnt);
> + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
> + print_poll_ctx(rd, poll_ctx);
> + print_pid(rd, pid);
> + print_comm(rd, comm, nla_line);
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> + free(comm);
> +
> + if (rd->json_output)
> + jsonw_end_array(rd->jw);
> + else
> + pr_out("\n");
> + }
> + return MNL_CB_OK;
> +}
> +
> RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
>
> static const struct
> @@ -758,12 +872,21 @@ filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET, cm_id_valid_filters,
> false);
>
> +static const struct
> +filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> + .is_number = false },
> + { .name = "pid",
> + .is_number = true }};
Can you please add filter of usecnt too? It will give us easy view on
"over crowded" CQs.
> +
> +RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
> +
> static int res_show(struct rd *rd)
> {
> const struct rd_cmd cmds[] = {
> { NULL, res_no_args },
> { "qp", res_qp },
> { "cm_id", res_cm_id },
> + { "cq", res_cq },
> { 0 }
> };
>
> diff --git a/rdma/utils.c b/rdma/utils.c
> index 906ca73..11b34fe 100644
> --- a/rdma/utils.c
> +++ b/rdma/utils.c
> @@ -387,6 +387,11 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> [RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
> [RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
> [RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
> + [RDMA_NLDEV_ATTR_RES_CQ] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_CQ_ENTRY] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
> + [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
> + [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
> };
>
> int rd_attr_cb(const struct nlattr *attr, void *data)
> --
> 1.8.3.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH RFC iproute-next 4/5] rdma: Add MR resource tracking information
2018-02-14 21:07 ` [PATCH RFC iproute-next 4/5] rdma: Add MR " Steve Wise
@ 2018-02-20 14:12 ` Leon Romanovsky
2018-02-26 15:08 ` Steve Wise
0 siblings, 1 reply; 16+ messages in thread
From: Leon Romanovsky @ 2018-02-20 14:12 UTC (permalink / raw)
To: Steve Wise; +Cc: dsahern, stephen, netdev, linux-rdma
[-- Attachment #1: Type: text/plain, Size: 9316 bytes --]
On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> Sample output:
>
> # rdma resource show mr
> link cxgb4_0/- rkey 0x5e0e lkey 0x5e0e iova 0x7f9b60001c80 mrlen 64 pgsize 4096 pid 30503 comm rping
> link cxgb4_0/- rkey 0x510d lkey 0x510d iova 0x17c7ee0 mrlen 16 pgsize 4096 pid 30503 comm rping
> link cxgb4_0/- rkey 0x4a0c lkey 0x4a0c iova 0x17c7e38 mrlen 16 pgsize 4096 pid 30503 comm rping
> link cxgb4_0/- rkey 0x480b lkey 0x480b iova 0x224b3f0 mrlen 64 pgsize 4096 pid 30498 comm rping
> link cxgb4_0/- rkey 0x460a lkey 0x460a iova 0x224b350 mrlen 64 pgsize 4096 pid 30498 comm rping
> link cxgb4_0/- rkey 0x4509 lkey 0x4509 iova 0x2245890 mrlen 16 pgsize 4096 pid 30498 comm rping
> link cxgb4_0/- rkey 0x4208 lkey 0x4208 iova 0x22457e8 mrlen 16 pgsize 4096 pid 30498 comm rping
> link mlx4_0/- rkey 0x38010700 lkey 0x38010700 iova 0x7f9b5c002f90 mrlen 64 pgsize 4096 pid 30494 comm rping
> link mlx4_0/- rkey 0x38010600 lkey 0x38010600 iova 0x17c6c80 mrlen 16 pgsize 4096 pid 30494 comm rping
> link mlx4_0/- rkey 0x38010500 lkey 0x38010500 iova 0x17c6bd8 mrlen 16 pgsize 4096 pid 30494 comm rping
> link mlx4_0/- rkey 0x38010400 lkey 0x38010400 iova 0x1b68430 mrlen 64 pgsize 4096 pid 30489 comm rping
> link mlx4_0/- rkey 0x38010300 lkey 0x38010300 iova 0x1b683a0 mrlen 64 pgsize 4096 pid 30489 comm rping
> link mlx4_0/- rkey 0x38010200 lkey 0x38010200 iova 0x1b62890 mrlen 16 pgsize 4096 pid 30489 comm rping
> link mlx4_0/- rkey 0x38010100 lkey 0x38010100 iova 0x1b627e8 mrlen 16 pgsize 4096 pid 30489 comm rping
>
> Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> ---
> include/json_writer.h | 2 +
> lib/json_writer.c | 11 +++++
> rdma/res.c | 126 ++++++++++++++++++++++++++++++++++++++++++++++++++
> rdma/utils.c | 7 +++
> 4 files changed, 146 insertions(+)
>
> diff --git a/include/json_writer.h b/include/json_writer.h
> index 1516aaf..34f2ccc 100644
> --- a/include/json_writer.h
> +++ b/include/json_writer.h
> @@ -39,6 +39,7 @@ void jsonw_bool(json_writer_t *self, bool value);
> void jsonw_float(json_writer_t *self, double number);
> void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num);
> void jsonw_uint(json_writer_t *self, uint64_t number);
> +void jsonw_xint(json_writer_t *self, uint64_t number);
> void jsonw_hu(json_writer_t *self, unsigned short number);
> void jsonw_int(json_writer_t *self, int64_t number);
> void jsonw_null(json_writer_t *self);
> @@ -49,6 +50,7 @@ void jsonw_string_field(json_writer_t *self, const char *prop, const char *val);
> void jsonw_bool_field(json_writer_t *self, const char *prop, bool value);
> void jsonw_float_field(json_writer_t *self, const char *prop, double num);
> void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num);
> +void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num);
> void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num);
> void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num);
> void jsonw_null_field(json_writer_t *self, const char *prop);
> diff --git a/lib/json_writer.c b/lib/json_writer.c
> index f3eeaf7..6d73a1b 100644
> --- a/lib/json_writer.c
> +++ b/lib/json_writer.c
> @@ -224,6 +224,11 @@ void jsonw_uint(json_writer_t *self, uint64_t num)
> jsonw_printf(self, "%"PRIu64, num);
> }
>
> +void jsonw_xint(json_writer_t *self, uint64_t num)
> +{
> + jsonw_printf(self, "%"PRIx64, num);
> +}
> +
> void jsonw_lluint(json_writer_t *self, unsigned long long int num)
> {
> jsonw_printf(self, "%llu", num);
> @@ -268,6 +273,12 @@ void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num)
> jsonw_uint(self, num);
> }
>
> +void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t num)
> +{
> + jsonw_name(self, prop);
> + jsonw_xint(self, num);
> +}
> +
> void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num)
> {
> jsonw_name(self, prop);
> diff --git a/rdma/res.c b/rdma/res.c
> index 27c1efd..2b67d25 100644
> --- a/rdma/res.c
> +++ b/rdma/res.c
> @@ -819,6 +819,119 @@ static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
> return MNL_CB_OK;
> }
>
> +static void print_key(struct rd *rd, const char *name, uint32_t val)
> +{
> + if (rd->json_output)
> + jsonw_xint_field(rd->jw, name, val);
> + else
> + pr_out("%s 0x%x ", name, val);
> +}
> +
> +static void print_iova(struct rd *rd, uint64_t val)
> +{
> + if (rd->json_output)
> + jsonw_xint_field(rd->jw, "iova", val);
> + else
> + pr_out("iova 0x%" PRIx64 " ", val);
> +}
> +
> +static void print_mrlen(struct rd *rd, uint64_t val)
> +{
> + if (rd->json_output)
> + jsonw_uint_field(rd->jw, "mrlen", val);
> + else
> + pr_out("mrlen %" PRIu64 " ", val);
> +}
> +
> +static void print_pgsize(struct rd *rd, uint32_t val)
> +{
> + if (rd->json_output)
> + jsonw_uint_field(rd->jw, "pgsize", val);
> + else
> + pr_out("pgsize %u ", val);
> +}
> +
> +static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)
> +{
> + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> + struct nlattr *nla_table, *nla_entry;
> + struct rd *rd = data;
> + const char *name;
> + uint32_t idx;
> +
> + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> + !tb[RDMA_NLDEV_ATTR_RES_MR])
> + return MNL_CB_ERROR;
> +
> + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> + nla_table = tb[RDMA_NLDEV_ATTR_RES_MR];
> +
> + mnl_attr_for_each_nested(nla_entry, nla_table) {
> + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> + uint32_t rkey, lkey, pgsize;
> + uint64_t iova, mrlen;
> + char *comm = NULL;
> + uint32_t pid = 0;
> + int err;
> +
> + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
> + if (err != MNL_CB_OK)
> + return MNL_CB_ERROR;
> +
> + if (!nla_line[RDMA_NLDEV_ATTR_RES_RKEY] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_LKEY] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_IOVA] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_MRLEN] ||
> + !nla_line[RDMA_NLDEV_ATTR_RES_PGSIZE] ||
> + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> + return MNL_CB_ERROR;
> + }
I'm not sure that all this ATTR are needed.
> +
> + rkey = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_RKEY]);
> + lkey = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LKEY]);
> + iova = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_IOVA]);
> + mrlen = mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_MRLEN]);
> + pgsize = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PGSIZE]);
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> + pid = mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> + comm = get_task_name(pid);
> + }
> +
> + if (rd_check_is_filtered(rd, "pid", pid))
free(comm);
> + continue;
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
> + /* discard const from mnl_attr_get_str */
> + comm = (char *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> +
> + if (rd->json_output)
> + jsonw_start_array(rd->jw);
> +
> + print_link(rd, idx, name, 0, nla_line);
> + print_key(rd, "rkey", rkey);
> + print_key(rd, "lkey", lkey);
> + print_iova(rd, iova);
> + print_mrlen(rd, mrlen);
> + print_pgsize(rd, pgsize);
> + print_pid(rd, pid);
> + print_comm(rd, comm, nla_line);
> +
> + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> + free(comm);
> +
> + if (rd->json_output)
> + jsonw_end_array(rd->jw);
> + else
> + pr_out("\n");
> + }
> + return MNL_CB_OK;
> +}
> +
> RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
>
> static const struct
> @@ -880,6 +993,18 @@ filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
>
> RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters, true);
>
> +static const struct
> +filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> + .is_number = false },
> + { .name = "rkey",
> + .is_number = true },
> + { .name = "lkey",
> + .is_number = true },
> + { .name = "pid",
> + .is_number = true }};
> +
> +RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters, true);
> +
> static int res_show(struct rd *rd)
> {
> const struct rd_cmd cmds[] = {
> @@ -887,6 +1012,7 @@ static int res_show(struct rd *rd)
> { "qp", res_qp },
> { "cm_id", res_cm_id },
> { "cq", res_cq },
> + { "mr", res_mr },
> { 0 }
> };
>
> diff --git a/rdma/utils.c b/rdma/utils.c
> index 11b34fe..34b195a 100644
> --- a/rdma/utils.c
> +++ b/rdma/utils.c
> @@ -392,6 +392,13 @@ static const enum mnl_attr_data_type nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
> [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
> [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
> + [RDMA_NLDEV_ATTR_RES_MR] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_MR_ENTRY] = MNL_TYPE_NESTED,
> + [RDMA_NLDEV_ATTR_RES_RKEY] = MNL_TYPE_U32,
> + [RDMA_NLDEV_ATTR_RES_LKEY] = MNL_TYPE_U32,
> + [RDMA_NLDEV_ATTR_RES_IOVA] = MNL_TYPE_U64,
> + [RDMA_NLDEV_ATTR_RES_MRLEN] = MNL_TYPE_U64,
> + [RDMA_NLDEV_ATTR_RES_PGSIZE] = MNL_TYPE_U32,
> };
>
> int rd_attr_cb(const struct nlattr *attr, void *data)
> --
> 1.8.3.1
>
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information
2018-02-20 12:57 ` Leon Romanovsky
@ 2018-02-20 15:15 ` Parav Pandit
2018-02-26 15:05 ` Steve Wise
1 sibling, 0 replies; 16+ messages in thread
From: Parav Pandit @ 2018-02-20 15:15 UTC (permalink / raw)
To: Leon Romanovsky, Steve Wise; +Cc: dsahern, stephen, netdev, linux-rdma
> -----Original Message-----
> From: linux-rdma-owner@vger.kernel.org [mailto:linux-rdma-
> owner@vger.kernel.org] On Behalf Of Leon Romanovsky
> Sent: Tuesday, February 20, 2018 6:58 AM
> To: Steve Wise <swise@opengridcomputing.com>
> Cc: dsahern@gmail.com; stephen@networkplumber.org;
> netdev@vger.kernel.org; linux-rdma@vger.kernel.org
> Subject: Re: [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking
> information
>
> On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> > Sample output:
> >
> > # rdma resource
> > 2: cxgb4_0: pd 5 cq 2 qp 2 cm_id 3 mr 7
> > 3: mlx4_0: pd 7 cq 3 qp 3 cm_id 3 mr 7
> >
> > # rdma resource show cm_id
> > [root@stevo1 iproute2]# /root/stevo/iproute2/rdma/rdma resource show
> > cm_id link cxgb4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type
> > --- transport-type IWARP pid 30485 comm rping src-addr 0.0.0.0
> > src-port 7174 dst-addr 0.0.0.0 dst-port 0 link cxgb4_0/2 lqpn 1048
> > qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid
> > 30503 comm rping src-addr 172.16.2.1 src-port 7174 dst-addr 172.16.2.1
> > dst-port 38246 link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps
> > TCP dev-type ETH transport-type IWARP pid 30498 comm rping src-addr
> > 172.16.2.1 src-port 38246 dst-addr 172.16.2.1 dst-port 7174 link
> > mlx4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type ---
> > transport-type IB pid 30485 comm rping src-addr 0.0.0.0 src-port 7174
> > dst-addr 0.0.0.0 dst-port 0 link mlx4_0/1 lqpn 539 qp-type RC state
> > CONNECT ps TCP dev-type ETH transport-type IB pid 30494 comm rping
> > src-addr 172.16.99.1 src-port 7174 dst-addr 172.16.99.1 dst-port 43670
> > link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH
> > transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port
> > 43670 dst-addr 172.16.99.1 dst-port 7174
> >
> > # rdma resource show cm_id dst-port 7174 link cxgb4_0/2 lqpn 1040
> > qp-type RC state CONNECT ps TCP dev-type ETH transport-type IWARP pid
> > 30498 comm rping src-addr 172.16.2.1 src-port 38246 dst-addr
> > 172.16.2.1 dst-port 7174 link mlx4_0/1 lqpn 538 qp-type RC state
> > CONNECT ps TCP dev-type ETH transport-type IB pid 30492 comm rping
> > src-addr 172.16.99.1 src-port 43670 dst-addr 172.16.99.1 dst-port 7174
> >
> > Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> > ---
> > rdma/rdma.h | 1 +
> > rdma/res.c | 312
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> > rdma/utils.c | 12 +++
> > 3 files changed, 321 insertions(+), 4 deletions(-)
>
> Thanks, for doing it.
>
> >
> > diff --git a/rdma/rdma.h b/rdma/rdma.h index 5809f70..1ef0942 100644
> > --- a/rdma/rdma.h
> > +++ b/rdma/rdma.h
> > @@ -18,6 +18,7 @@
> > #include <libmnl/libmnl.h>
> > #include <rdma/rdma_netlink.h>
> > #include <time.h>
> > +#include <net/if_arp.h>
> >
> > #include "list.h"
> > #include "utils.h"
> > diff --git a/rdma/res.c b/rdma/res.c
> > index 2a63e71..beae7dc 100644
> > --- a/rdma/res.c
> > +++ b/rdma/res.c
> > @@ -16,9 +16,11 @@ static int res_help(struct rd *rd) {
> > pr_out("Usage: %s resource\n", rd->filename);
> > pr_out(" resource show [DEV]\n");
> > - pr_out(" resource show [qp]\n");
> > + pr_out(" resource show [qp|cm_id]\n");
> > pr_out(" resource show qp link [DEV/PORT]\n");
> > pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME FILTER-
> VALUE]\n");
> > + pr_out(" resource show cm_id link [DEV/PORT]\n");
> > + pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME
> FILTER-VALUE]\n");
> > return 0;
> > }
> >
> > @@ -431,6 +433,278 @@ static int res_qp_parse_cb(const struct nlmsghdr
> *nlh, void *data)
> > return MNL_CB_OK;
> > }
> >
> > +static void print_qp_type(struct rd *rd, uint32_t val) {
> > + if (rd->json_output)
> > + jsonw_string_field(rd->jw, "qp-type",
> > + qp_types_to_str(val));
> > + else
> > + pr_out("qp-type %s ", qp_types_to_str(val)); }
> > +
> > +static const char *cm_id_state_to_str(uint8_t idx) {
> > + static const char * const cm_id_states_str[] = { "IDLE", "ADDR_QUERY",
> > + "ADDR_RESOLVED",
> "ROUTE_QUERY", "ROUTE_RESOLVED",
> > + "CONNECT",
> "DISCONNECT",
> > + "ADDR_BOUND", "LISTEN",
> "DEVICE_REMOVAL", "DESTROYING" };
> > +
> > + if (idx < ARRAY_SIZE(cm_id_states_str))
> > + return cm_id_states_str[idx];
> > + return "UNKNOWN";
> > +}
> > +
> > +enum rdma_port_space {
> > + RDMA_PS_SDP = 0x0001,
>
> Do we still support this PS? It is not set in the kernel and Parav posted internal
> patch to remove it.
>
> > + RDMA_PS_IPOIB = 0x0002,
> > + RDMA_PS_IB = 0x013F,
> > + RDMA_PS_TCP = 0x0106,
> > + RDMA_PS_UDP = 0x0111,
> > +};
> > +
> > +static const char *cm_id_ps_to_str(uint32_t ps) {
> > + switch (ps) {
> > + case RDMA_PS_SDP:
> > + return "SDP";
>
> The same question
>
> > + case RDMA_PS_IPOIB:
> > + return "IPoIB";
> > + case RDMA_PS_IB:
> > + return "IPoIB";
> > + case RDMA_PS_TCP:
> > + return "TCP";
> > + case RDMA_PS_UDP:
> > + return "UDP";
> > + default:
> > + return "---";
> > + }
> > +}
> > +
> > +static const char *cm_id_dev_type_to_str(uint8_t dev_type) {
> > + switch (dev_type) {
> > + case ARPHRD_INFINIBAND:
> > + return "IB";
> > + case ARPHRD_ETHER:
> > + return "ETH";
> > + default:
> > + return "---";
> > + }
> > +}
Since link information is given as {device,port}, not sure this is needed on how ARP is done.
I think we should rely to device,port information and skip this in NL.
^ permalink raw reply [flat|nested] 16+ messages in thread
* Re: [PATCH RFC iproute-next 5/5] rdma: Add PD resource tracking information
2018-02-14 21:07 ` [PATCH RFC iproute-next 5/5] rdma: Add PD " Steve Wise
@ 2018-02-23 14:22 ` Leon Romanovsky
2018-02-26 15:09 ` Steve Wise
0 siblings, 1 reply; 16+ messages in thread
From: Leon Romanovsky @ 2018-02-23 14:22 UTC (permalink / raw)
To: Steve Wise; +Cc: dsahern, stephen, netdev, linux-rdma
[-- Attachment #1: Type: text/plain, Size: 306 bytes --]
On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> Sample output:
>
> # rdma resource show pd
> link cxgb4_0/- local_dma_lkey 0x0 usecnt 4 flags 0x0 pid 30503 comm rping
One more thing, flags need to be pre-parsed and accessible with "-d" command,
as we did with dev,link capabilities.
Thanks
[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information
2018-02-20 12:57 ` Leon Romanovsky
2018-02-20 15:15 ` Parav Pandit
@ 2018-02-26 15:05 ` Steve Wise
1 sibling, 0 replies; 16+ messages in thread
From: Steve Wise @ 2018-02-26 15:05 UTC (permalink / raw)
To: 'Leon Romanovsky'; +Cc: dsahern, stephen, netdev, linux-rdma
>
> On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> > Sample output:
> >
> > # rdma resource
> > 2: cxgb4_0: pd 5 cq 2 qp 2 cm_id 3 mr 7
> > 3: mlx4_0: pd 7 cq 3 qp 3 cm_id 3 mr 7
> >
> > # rdma resource show cm_id
> > [root@stevo1 iproute2]# /root/stevo/iproute2/rdma/rdma resource show
> cm_id
> > link cxgb4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type ---
transport-
> type IWARP pid 30485 comm rping src-addr 0.0.0.0 src-port 7174 dst-addr
> 0.0.0.0 dst-port 0
> > link cxgb4_0/2 lqpn 1048 qp-type RC state CONNECT ps TCP dev-type ETH
> transport-type IWARP pid 30503 comm rping src-addr 172.16.2.1 src-port
> 7174 dst-addr 172.16.2.1 dst-port 38246
> > link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP dev-type ETH
> transport-type IWARP pid 30498 comm rping src-addr 172.16.2.1 src-port
> 38246 dst-addr 172.16.2.1 dst-port 7174
> > link mlx4_0/- lqpn 0 qp-type RC state LISTEN ps TCP dev-type ---
transport-
> type IB pid 30485 comm rping src-addr 0.0.0.0 src-port 7174 dst-addr
0.0.0.0
> dst-port 0
> > link mlx4_0/1 lqpn 539 qp-type RC state CONNECT ps TCP dev-type ETH
> transport-type IB pid 30494 comm rping src-addr 172.16.99.1 src-port 7174
> dst-addr 172.16.99.1 dst-port 43670
> > link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH
> transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port 43670
> dst-addr 172.16.99.1 dst-port 7174
> >
> > # rdma resource show cm_id dst-port 7174
> > link cxgb4_0/2 lqpn 1040 qp-type RC state CONNECT ps TCP dev-type ETH
> transport-type IWARP pid 30498 comm rping src-addr 172.16.2.1 src-port
> 38246 dst-addr 172.16.2.1 dst-port 7174
> > link mlx4_0/1 lqpn 538 qp-type RC state CONNECT ps TCP dev-type ETH
> transport-type IB pid 30492 comm rping src-addr 172.16.99.1 src-port 43670
> dst-addr 172.16.99.1 dst-port 7174
> >
> > Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> > ---
> > rdma/rdma.h | 1 +
> > rdma/res.c | 312
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++-
> > rdma/utils.c | 12 +++
> > 3 files changed, 321 insertions(+), 4 deletions(-)
>
> Thanks, for doing it.
>
> >
> > diff --git a/rdma/rdma.h b/rdma/rdma.h
> > index 5809f70..1ef0942 100644
> > --- a/rdma/rdma.h
> > +++ b/rdma/rdma.h
> > @@ -18,6 +18,7 @@
> > #include <libmnl/libmnl.h>
> > #include <rdma/rdma_netlink.h>
> > #include <time.h>
> > +#include <net/if_arp.h>
> >
> > #include "list.h"
> > #include "utils.h"
> > diff --git a/rdma/res.c b/rdma/res.c
> > index 2a63e71..beae7dc 100644
> > --- a/rdma/res.c
> > +++ b/rdma/res.c
> > @@ -16,9 +16,11 @@ static int res_help(struct rd *rd)
> > {
> > pr_out("Usage: %s resource\n", rd->filename);
> > pr_out(" resource show [DEV]\n");
> > - pr_out(" resource show [qp]\n");
> > + pr_out(" resource show [qp|cm_id]\n");
> > pr_out(" resource show qp link [DEV/PORT]\n");
> > pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME
> FILTER-VALUE]\n");
> > + pr_out(" resource show cm_id link [DEV/PORT]\n");
> > + pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME
> FILTER-VALUE]\n");
> > return 0;
> > }
> >
> > @@ -431,6 +433,278 @@ static int res_qp_parse_cb(const struct nlmsghdr
> *nlh, void *data)
> > return MNL_CB_OK;
> > }
> >
> > +static void print_qp_type(struct rd *rd, uint32_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_string_field(rd->jw, "qp-type",
> > + qp_types_to_str(val));
> > + else
> > + pr_out("qp-type %s ", qp_types_to_str(val));
> > +}
> > +
> > +static const char *cm_id_state_to_str(uint8_t idx)
> > +{
> > + static const char * const cm_id_states_str[] = { "IDLE",
> "ADDR_QUERY",
> > + "ADDR_RESOLVED",
> "ROUTE_QUERY", "ROUTE_RESOLVED",
> > + "CONNECT",
> "DISCONNECT",
> > + "ADDR_BOUND",
> "LISTEN", "DEVICE_REMOVAL", "DESTROYING" };
> > +
> > + if (idx < ARRAY_SIZE(cm_id_states_str))
> > + return cm_id_states_str[idx];
> > + return "UNKNOWN";
> > +}
> > +
> > +enum rdma_port_space {
> > + RDMA_PS_SDP = 0x0001,
>
> Do we still support this PS? It is not set in the kernel and Parav
> posted internal patch to remove it.
>
> > + RDMA_PS_IPOIB = 0x0002,
> > + RDMA_PS_IB = 0x013F,
> > + RDMA_PS_TCP = 0x0106,
> > + RDMA_PS_UDP = 0x0111,
> > +};
> > +
> > +static const char *cm_id_ps_to_str(uint32_t ps)
> > +{
> > + switch (ps) {
> > + case RDMA_PS_SDP:
> > + return "SDP";
>
> The same question
You're right. No more PS_SDP. Also, rdma tool should get the
rdma_port_space enum from <include/rdma/rdma_cma.h>.
>
> > + case RDMA_PS_IPOIB:
> > + return "IPoIB";
> > + case RDMA_PS_IB:
> > + return "IPoIB";
> > + case RDMA_PS_TCP:
> > + return "TCP";
> > + case RDMA_PS_UDP:
> > + return "UDP";
> > + default:
> > + return "---";
> > + }
> > +}
> > +
> > +static const char *cm_id_dev_type_to_str(uint8_t dev_type)
> > +{
> > + switch (dev_type) {
> > + case ARPHRD_INFINIBAND:
> > + return "IB";
> > + case ARPHRD_ETHER:
> > + return "ETH";
> > + default:
> > + return "---";
> > + }
> > +}
> > +
> > +static const char *cm_id_transport_type_to_str(uint8_t transport_type)
> > +{
> > + static const char * const transport_type_str[] = { "IB", "IWARP",
> "USNIC", "USNIC/UDP" };
> > +
>
> I know that it is part of CM_ID, but wonder if node_type of device is
> not enough. The same question goes for device type, isn't it part of
> "rdma dev .." output?
>
I've gotten rid off dev_type, transport_type, and network_type based on
feedback from the kernel patches. So I'll remove it from this series as
well.
> > + if (transport_type < ARRAY_SIZE(transport_type_str))
> > + return transport_type_str[transport_type];
> > + return "---";
> > +}
> > +
> > +static void print_cm_id_state(struct rd *rd, uint8_t state)
> > +{
> > + if (rd->json_output) {
> > + jsonw_string_field(rd->jw, "state",
> cm_id_state_to_str(state));
> > + return;
> > + }
> > + pr_out("state %s ", cm_id_state_to_str(state));
> > +}
> > +
> > +static void print_ps(struct rd *rd, uint32_t ps)
> > +{
> > + if (rd->json_output) {
> > + jsonw_string_field(rd->jw, "ps", cm_id_ps_to_str(ps));
> > + return;
> > + }
> > + pr_out("ps %s ", cm_id_ps_to_str(ps));
> > +}
> > +
> > +static void print_dev_type(struct rd *rd, uint8_t dev_type)
> > +{
> > + if (rd->json_output) {
> > + jsonw_string_field(rd->jw, "dev-type",
> cm_id_dev_type_to_str(dev_type));
> > + return;
> > + }
> > + pr_out("dev-type %s ", cm_id_dev_type_to_str(dev_type));
> > +}
> > +
> > +static void print_transport_type(struct rd *rd, uint8_t transport_type)
> > +{
> > + if (rd->json_output) {
> > + jsonw_string_field(rd->jw, "transport-type",
> cm_id_transport_type_to_str(transport_type));
> > + return;
> > + }
> > + pr_out("transport-type %s ",
> cm_id_transport_type_to_str(transport_type));
> > +}
> > +
> > +static void print_ipaddr(struct rd *rd, const char *key, char *addrstr)
> > +{
> > + if (rd->json_output) {
> > + jsonw_string_field(rd->jw, key, addrstr);
> > + return;
> > + }
> > + pr_out("%s %s ", key, addrstr);
> > +}
> > +
> > +static void print_ipport(struct rd *rd, const char *key, uint16_t
ipport)
> > +{
> > + if (rd->json_output) {
> > + jsonw_uint_field(rd->jw, key, ipport);
> > + return;
> > + }
> > + pr_out("%s %u ", key, ipport);
> > +}
> > +
> > +static int res_cm_id_parse_cb(const struct nlmsghdr *nlh, void *data)
> > +{
> > + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> > + struct nlattr *nla_table, *nla_entry;
> > + struct rd *rd = data;
> > + const char *name;
> > + int idx;
> > +
> > + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> > + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> > + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> > + !tb[RDMA_NLDEV_ATTR_RES_CM_ID])
> > + return MNL_CB_ERROR;
> > +
> > + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> > + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> > + nla_table = tb[RDMA_NLDEV_ATTR_RES_CM_ID];
> > + mnl_attr_for_each_nested(nla_entry, nla_table) {
> > + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> > + uint8_t dev_type, transport_type;
> > + char src_addr_str[INET6_ADDRSTRLEN];
> > + char dst_addr_str[INET6_ADDRSTRLEN];
> > + uint8_t *src_addr, *dst_addr;
> > + uint16_t src_port, dst_port;
> > + uint32_t port = 0, pid = 0;
> > + uint8_t type, state;
> > + uint32_t lqpn = 0, ps;
> > + char *comm = NULL;
> > + int err;
> > +
> > + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb,
nla_line);
> > + if (err != MNL_CB_OK)
> > + return -EINVAL;
> > +
> > + if (!nla_line[RDMA_NLDEV_ATTR_RES_TYPE] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_STATE] ||
> > + (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR] &&
> > + !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]) ||
> > + (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR] &&
> > + !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR]) ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_IP_SPORT] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_IP_DPORT] ||
> > + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> > + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> > + return MNL_CB_ERROR;
>
> It is unreadable, any chances to use intermediate variables with
> descriptive names?
This gets cleaned up because the kernel driver will now pass only the
sockaddr_storage up for the addresses. It simplifies this code, so I think
it'll be cleaner on the next version (I'll send it out today or tomorrow
hopefully).
>
> > + }
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_PORT_INDEX])
> > + port =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_PORT_INDEX]);
> > +
> > + if (port && port != rd->port_idx)
> > + continue;
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_LQPN])
> > + lqpn =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LQPN]);
> > + if (rd_check_is_filtered(rd, "lqpn", lqpn))
> > + continue;
> > +
> > + ps =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PS]);
> > + if (rd_check_is_string_filtered(rd, "ps",
cm_id_ps_to_str(ps)))
> > + continue;
> > +
> > + type =
> mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TYPE]);
> > + if (rd_check_is_string_filtered(rd, "qp-type",
> qp_types_to_str(type)))
> > + continue;
> > +
> > + state =
> mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_STATE]);
> > + if (rd_check_is_string_filtered(rd, "state",
> cm_id_state_to_str(state)))
> > + continue;
> > +
> > + dev_type =
> mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_DEV_TYPE]);
> > + if (rd_check_is_string_filtered(rd, "dev-type",
> cm_id_dev_type_to_str(dev_type)))
> > + continue;
> > +
> > + transport_type =
> mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE]);
> > + if (rd_check_is_string_filtered(rd, "transport-type",
> cm_id_transport_type_to_str(transport_type)))
> > + continue;
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> > + pid =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> > + comm = get_task_name(pid);
> > + }
> > + if (rd_check_is_filtered(rd, "pid", pid))
>
> free(comm) here
>
Oops! Thanks!
> > + continue;
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]) {
> > + /* discard const from mnl_attr_get_str */
> > + comm = (char
> *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> > + }
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR]) {
> > + if (!nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR])
> > + return -EINVAL;
> > + src_addr =
> mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV4_SADDR]);
> > + if (!inet_ntop(AF_INET, src_addr, src_addr_str,
> INET6_ADDRSTRLEN))
> > + return -EINVAL;
> > + dst_addr =
> mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV4_DADDR]);
> > + if (!inet_ntop(AF_INET, dst_addr, dst_addr_str,
> INET6_ADDRSTRLEN))
> > + return -EINVAL;
> > + } else {
> > + if (!nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]
> ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR])
> > + return -EINVAL;
> > + src_addr =
> mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV6_SADDR]);
> > + if (!inet_ntop(AF_INET6, src_addr, src_addr_str,
> INET6_ADDRSTRLEN))
> > + return -EINVAL;
> > + dst_addr =
> mnl_attr_get_payload(nla_line[RDMA_NLDEV_ATTR_RES_IPV6_DADDR]);
> > + if (!inet_ntop(AF_INET6, dst_addr, dst_addr_str,
> INET6_ADDRSTRLEN))
> > + return -EINVAL;
> > + }
> > + if (rd_check_is_string_filtered(rd, "src-addr",
src_addr_str))
> > + continue;
> > + if (rd_check_is_string_filtered(rd, "dst-addr",
dst_addr_str))
> > + continue;
> > +
> > + src_port =
> mnl_attr_get_u16(nla_line[RDMA_NLDEV_ATTR_RES_IP_SPORT]);
> > + dst_port =
> mnl_attr_get_u16(nla_line[RDMA_NLDEV_ATTR_RES_IP_DPORT]);
> > + if (rd_check_is_filtered(rd, "src-port", src_port))
> > + continue;
> > + if (rd_check_is_filtered(rd, "dst-port", dst_port))
> > + continue;
> > +
>
> The same memory leaks as above, I put get_task_name() to the end of
> QP parsing code with purpose to avoid dealing with free() calls.
Sounds good. I'll look at your qp code.
>
> > + if (rd->json_output)
> > + jsonw_start_array(rd->jw);
> > +
> > + print_link(rd, idx, name, port, nla_line);
> > + print_lqpn(rd, lqpn);
> > + print_qp_type(rd, type);
> > + print_cm_id_state(rd, state);
> > + print_ps(rd, ps);
> > + print_dev_type(rd, dev_type);
> > + print_transport_type(rd, transport_type);
> > + print_pid(rd, pid);
> > + print_comm(rd, comm, nla_line);
> > + print_ipaddr(rd, "src-addr", src_addr_str);
> > + print_ipport(rd, "src-port", src_port);
> > + print_ipaddr(rd, "dst-addr", dst_addr_str);
> > + print_ipport(rd, "dst-port", dst_port);
>
> Does "ip tool" have standard Re presentation for addr<->port tupples?
> What about the following format src 1.1.1.1:1234 dst 2.2.2.2:6789?
>
That looks fine for me. I'm not sure if there's an 'ip' command that
displays ports along with addresses. But let us go with add:port.
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> > + free(comm);
> > +
> > + if (rd->json_output)
> > + jsonw_end_array(rd->jw);
> > + else
> > + pr_out("\n");
> > + }
> > + return MNL_CB_OK;
> > +}
> > +
> > RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
> >
> > static const struct
> > @@ -438,9 +712,9 @@ filters qp_valid_filters[MAX_NUMBER_OF_FILTERS]
> = {{ .name = "link",
> > .is_number = false },
> > { .name = "lqpn",
> > .is_number = true },
> > - { .name = "rqpn",
> > - .is_number = true },
> > - { .name = "pid",
> > + { .name = "type",
> > + .is_number = false },
> > + { .name = "cm_id_state",
> > .is_number = true },
> > { .name = "sq-psn",
> > .is_number = true },
>
> Why did you change qp_valid_filters?
Oops...I'll fix this.
>
> > @@ -455,11 +729,41 @@ filters
> qp_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> >
> > RES_FUNC(res_qp, RDMA_NLDEV_CMD_RES_QP_GET, qp_valid_filters,
> false);
> >
> > +static const struct
> > +filters cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> > + .is_number = false },
> > + { .name = "lqpn",
> > + .is_number = true },
> > + { .name = "qp-type",
> > + .is_number = false },
> > + { .name = "state",
> > + .is_number = false },
> > + { .name = "ps",
> > + .is_number = false },
> > + { .name = "dev-type",
> > + .is_number = false },
> > + { .name =
"transport-type",
> > + .is_number = false },
> > + { .name = "pid",
> > + .is_number = true },
> > + { .name = "src-addr",
> > + .is_number = false },
> > + { .name = "src-port",
> > + .is_number = true },
> > + { .name = "dst-addr",
> > + .is_number = false },
> > + { .name = "dst-port",
> > + .is_number = true }};
> > +
> > +RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET,
> cm_id_valid_filters,
> > + false);
> > +
> > static int res_show(struct rd *rd)
> > {
> > const struct rd_cmd cmds[] = {
> > { NULL, res_no_args },
> > { "qp", res_qp },
> > + { "cm_id", res_cm_id },
> > { 0 }
> > };
> >
> > diff --git a/rdma/utils.c b/rdma/utils.c
> > index f946016..906ca73 100644
> > --- a/rdma/utils.c
> > +++ b/rdma/utils.c
> > @@ -375,6 +375,18 @@ static const enum mnl_attr_data_type
> nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> > [RDMA_NLDEV_ATTR_RES_STATE] = MNL_TYPE_U8,
> > [RDMA_NLDEV_ATTR_RES_PID] = MNL_TYPE_U32,
> > [RDMA_NLDEV_ATTR_RES_KERN_NAME] =
> MNL_TYPE_NUL_STRING,
> > + [RDMA_NLDEV_ATTR_RES_CM_ID] = MNL_TYPE_NESTED,
> > + [RDMA_NLDEV_ATTR_RES_CM_ID_ENTRY] = MNL_TYPE_NESTED,
> > + [RDMA_NLDEV_ATTR_RES_PS] = MNL_TYPE_U32,
> > + [RDMA_NLDEV_ATTR_RES_IPV4_SADDR] =
> MNL_TYPE_UNSPEC,
> > + [RDMA_NLDEV_ATTR_RES_IPV4_DADDR] =
> MNL_TYPE_UNSPEC,
> > + [RDMA_NLDEV_ATTR_RES_IPV6_SADDR] =
> MNL_TYPE_UNSPEC,
> > + [RDMA_NLDEV_ATTR_RES_IPV6_DADDR] =
> MNL_TYPE_UNSPEC,
> > + [RDMA_NLDEV_ATTR_RES_IP_SPORT] = MNL_TYPE_U16,
> > + [RDMA_NLDEV_ATTR_RES_IP_DPORT] = MNL_TYPE_U16,
> > + [RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
> > + [RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
> > + [RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
> > };
> >
> > int rd_attr_cb(const struct nlattr *attr, void *data)
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH RFC iproute-next 3/5] rdma: Add CQ resource tracking information
2018-02-20 13:09 ` Leon Romanovsky
@ 2018-02-26 15:06 ` Steve Wise
0 siblings, 0 replies; 16+ messages in thread
From: Steve Wise @ 2018-02-26 15:06 UTC (permalink / raw)
To: 'Leon Romanovsky'; +Cc: dsahern, stephen, netdev, linux-rdma
>
> On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> > Sample output:
> >
> > # rdma resource show cq
> > link cxgb4_0/- cqe 46 usecnt 2 pid 30503 comm rping
> > link cxgb4_0/- cqe 46 usecnt 2 pid 30498 comm rping
> > link mlx4_0/- cqe 63 usecnt 2 pid 30494 comm rping
> > link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
> > link mlx4_0/- cqe 1023 usecnt 2 poll_ctx WORKQUEUE pid 0 comm [ib_core]
> >
> > # rdma resource show cq pid 30489
> > link mlx4_0/- cqe 63 usecnt 2 pid 30489 comm rping
> >
> > Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> > ---
> > rdma/res.c | 123
> +++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> > rdma/utils.c | 5 +++
> > 2 files changed, 128 insertions(+)
> >
> > diff --git a/rdma/res.c b/rdma/res.c
> > index beae7dc..27c1efd 100644
> > --- a/rdma/res.c
> > +++ b/rdma/res.c
> > @@ -21,6 +21,8 @@ static int res_help(struct rd *rd)
> > pr_out(" resource show qp link [DEV/PORT] [FILTER-NAME
> FILTER-VALUE]\n");
> > pr_out(" resource show cm_id link [DEV/PORT]\n");
> > pr_out(" resource show cm_id link [DEV/PORT] [FILTER-NAME
> FILTER-VALUE]\n");
> > + pr_out(" resource show cq link [DEV/PORT]\n");
> > + pr_out(" resource show cq link [DEV/PORT] [FILTER-NAME
> FILTER-VALUE]\n");
> > return 0;
> > }
> >
> > @@ -705,6 +707,118 @@ static int res_cm_id_parse_cb(const struct
> nlmsghdr *nlh, void *data)
> > return MNL_CB_OK;
> > }
> >
> > +static void print_cqe(struct rd *rd, uint32_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_uint_field(rd->jw, "cqe", val);
> > + else
> > + pr_out("cqe %u ", val);
> > +}
> > +
> > +static void print_usecnt(struct rd *rd, uint64_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_uint_field(rd->jw, "usecnt", val);
> > + else
> > + pr_out("usecnt %" PRIu64 " ", val);
>
> Interesting, how many users are actually know what the "usecnt" actually
> means?
> Will it be more clear to call it "users" instead of "usecnt"?
>
Users is good.
> > +}
> > +
> > +static const char *poll_ctx_to_str(uint8_t idx)
> > +{
> > + static const char * const cm_id_states_str[] = { "DIRECT", "SOFTIRQ",
> > + "WORKQUEUE"};
> > +
> > + if (idx < ARRAY_SIZE(cm_id_states_str))
> > + return cm_id_states_str[idx];
> > + return "UNKNOWN";
> > +}
> > +
> > +static void print_poll_ctx(struct rd *rd, uint8_t poll_ctx)
> > +{
> > + if (rd->json_output) {
> > + jsonw_string_field(rd->jw, "poll_ctx",
> poll_ctx_to_str(poll_ctx));
> > + return;
> > + }
> > + pr_out("poll_ctx %s ", poll_ctx_to_str(poll_ctx));
> > +}
> > +
> > +static int res_cq_parse_cb(const struct nlmsghdr *nlh, void *data)
> > +{
> > + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> > + struct nlattr *nla_table, *nla_entry;
> > + struct rd *rd = data;
> > + const char *name;
> > + uint32_t idx;
> > +
> > + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> > + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> > + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> > + !tb[RDMA_NLDEV_ATTR_RES_CQ])
> > + return MNL_CB_ERROR;
> > +
> > + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> > + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> > + nla_table = tb[RDMA_NLDEV_ATTR_RES_CQ];
> > +
> > + mnl_attr_for_each_nested(nla_entry, nla_table) {
> > + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> > + char *comm = NULL;
> > + uint32_t pid = 0;
> > + uint8_t poll_ctx = 0;
> > + uint64_t usecnt;
> > + uint32_t cqe;
> > + int err;
> > +
> > + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb, nla_line);
> > + if (err != MNL_CB_OK)
> > + return MNL_CB_ERROR;
> > +
> > + if (!nla_line[RDMA_NLDEV_ATTR_RES_CQE] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_USECNT] ||
>
> I'm not sure that we will have USECNT in the future, let's not put
> requirement for RDMA_NLDEV_ATTR_RES_USECNT here.
Agreed.
>
> > + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> > + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> > + return MNL_CB_ERROR;
> > + }
> > +
> > + cqe =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_CQE]);
> > + usecnt =
> mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_USECNT]);
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
> > + poll_ctx =
> mnl_attr_get_u8(nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX]);
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> > + pid =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> > + comm = get_task_name(pid);
> > + }
> > +
> > + if (rd_check_is_filtered(rd, "pid", pid))
>
> free(comm);
>
😊
> > + continue;
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
> > + /* discard const from mnl_attr_get_str */
> > + comm = (char
> *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> > +
> > + if (rd->json_output)
> > + jsonw_start_array(rd->jw);
> > +
> > + print_link(rd, idx, name, 0, nla_line);
> > + print_cqe(rd, cqe);
> > + print_usecnt(rd, usecnt);
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_POLL_CTX])
> > + print_poll_ctx(rd, poll_ctx);
> > + print_pid(rd, pid);
> > + print_comm(rd, comm, nla_line);
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> > + free(comm);
> > +
> > + if (rd->json_output)
> > + jsonw_end_array(rd->jw);
> > + else
> > + pr_out("\n");
> > + }
> > + return MNL_CB_OK;
> > +}
> > +
> > RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
> >
> > static const struct
> > @@ -758,12 +872,21 @@ filters
> cm_id_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> > RES_FUNC(res_cm_id, RDMA_NLDEV_CMD_RES_CM_ID_GET,
> cm_id_valid_filters,
> > false);
> >
> > +static const struct
> > +filters cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> > + .is_number = false },
> > + { .name = "pid",
> > + .is_number = true }};
>
> Can you please add filter of usecnt too? It will give us easy view on
> "over crowded" CQs.
Good idea.
>
> > +
> > +RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters,
> true);
> > +
> > static int res_show(struct rd *rd)
> > {
> > const struct rd_cmd cmds[] = {
> > { NULL, res_no_args },
> > { "qp", res_qp },
> > { "cm_id", res_cm_id },
> > + { "cq", res_cq },
> > { 0 }
> > };
> >
> > diff --git a/rdma/utils.c b/rdma/utils.c
> > index 906ca73..11b34fe 100644
> > --- a/rdma/utils.c
> > +++ b/rdma/utils.c
> > @@ -387,6 +387,11 @@ static const enum mnl_attr_data_type
> nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> > [RDMA_NLDEV_ATTR_RES_DEV_TYPE] = MNL_TYPE_U8,
> > [RDMA_NLDEV_ATTR_RES_TRANSPORT_TYPE] = MNL_TYPE_U8,
> > [RDMA_NLDEV_ATTR_RES_NETWORK_TYPE] = MNL_TYPE_U8,
> > + [RDMA_NLDEV_ATTR_RES_CQ] = MNL_TYPE_NESTED,
> > + [RDMA_NLDEV_ATTR_RES_CQ_ENTRY] = MNL_TYPE_NESTED,
> > + [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
> > + [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
> > + [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
> > };
> >
> > int rd_attr_cb(const struct nlattr *attr, void *data)
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH RFC iproute-next 4/5] rdma: Add MR resource tracking information
2018-02-20 14:12 ` Leon Romanovsky
@ 2018-02-26 15:08 ` Steve Wise
0 siblings, 0 replies; 16+ messages in thread
From: Steve Wise @ 2018-02-26 15:08 UTC (permalink / raw)
To: 'Leon Romanovsky'; +Cc: dsahern, stephen, netdev, linux-rdma
> -----Original Message-----
> From: Leon Romanovsky [mailto:leon@kernel.org]
> Sent: Tuesday, February 20, 2018 8:12 AM
> To: Steve Wise <swise@opengridcomputing.com>
> Cc: dsahern@gmail.com; stephen@networkplumber.org;
> netdev@vger.kernel.org; linux-rdma@vger.kernel.org
> Subject: Re: [PATCH RFC iproute-next 4/5] rdma: Add MR resource tracking
> information
>
> On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> > Sample output:
> >
> > # rdma resource show mr
> > link cxgb4_0/- rkey 0x5e0e lkey 0x5e0e iova 0x7f9b60001c80 mrlen 64
> pgsize 4096 pid 30503 comm rping
> > link cxgb4_0/- rkey 0x510d lkey 0x510d iova 0x17c7ee0 mrlen 16 pgsize
> 4096 pid 30503 comm rping
> > link cxgb4_0/- rkey 0x4a0c lkey 0x4a0c iova 0x17c7e38 mrlen 16 pgsize
4096
> pid 30503 comm rping
> > link cxgb4_0/- rkey 0x480b lkey 0x480b iova 0x224b3f0 mrlen 64 pgsize
> 4096 pid 30498 comm rping
> > link cxgb4_0/- rkey 0x460a lkey 0x460a iova 0x224b350 mrlen 64 pgsize
> 4096 pid 30498 comm rping
> > link cxgb4_0/- rkey 0x4509 lkey 0x4509 iova 0x2245890 mrlen 16 pgsize
> 4096 pid 30498 comm rping
> > link cxgb4_0/- rkey 0x4208 lkey 0x4208 iova 0x22457e8 mrlen 16 pgsize
> 4096 pid 30498 comm rping
> > link mlx4_0/- rkey 0x38010700 lkey 0x38010700 iova 0x7f9b5c002f90 mrlen
> 64 pgsize 4096 pid 30494 comm rping
> > link mlx4_0/- rkey 0x38010600 lkey 0x38010600 iova 0x17c6c80 mrlen 16
> pgsize 4096 pid 30494 comm rping
> > link mlx4_0/- rkey 0x38010500 lkey 0x38010500 iova 0x17c6bd8 mrlen 16
> pgsize 4096 pid 30494 comm rping
> > link mlx4_0/- rkey 0x38010400 lkey 0x38010400 iova 0x1b68430 mrlen 64
> pgsize 4096 pid 30489 comm rping
> > link mlx4_0/- rkey 0x38010300 lkey 0x38010300 iova 0x1b683a0 mrlen 64
> pgsize 4096 pid 30489 comm rping
> > link mlx4_0/- rkey 0x38010200 lkey 0x38010200 iova 0x1b62890 mrlen 16
> pgsize 4096 pid 30489 comm rping
> > link mlx4_0/- rkey 0x38010100 lkey 0x38010100 iova 0x1b627e8 mrlen 16
> pgsize 4096 pid 30489 comm rping
> >
> > Signed-off-by: Steve Wise <swise@opengridcomputing.com>
> > ---
> > include/json_writer.h | 2 +
> > lib/json_writer.c | 11 +++++
> > rdma/res.c | 126
> ++++++++++++++++++++++++++++++++++++++++++++++++++
> > rdma/utils.c | 7 +++
> > 4 files changed, 146 insertions(+)
> >
> > diff --git a/include/json_writer.h b/include/json_writer.h
> > index 1516aaf..34f2ccc 100644
> > --- a/include/json_writer.h
> > +++ b/include/json_writer.h
> > @@ -39,6 +39,7 @@ void jsonw_bool(json_writer_t *self, bool value);
> > void jsonw_float(json_writer_t *self, double number);
> > void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num);
> > void jsonw_uint(json_writer_t *self, uint64_t number);
> > +void jsonw_xint(json_writer_t *self, uint64_t number);
> > void jsonw_hu(json_writer_t *self, unsigned short number);
> > void jsonw_int(json_writer_t *self, int64_t number);
> > void jsonw_null(json_writer_t *self);
> > @@ -49,6 +50,7 @@ void jsonw_string_field(json_writer_t *self, const
char
> *prop, const char *val);
> > void jsonw_bool_field(json_writer_t *self, const char *prop, bool
value);
> > void jsonw_float_field(json_writer_t *self, const char *prop, double
num);
> > void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t
> num);
> > +void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t
> num);
> > void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned
short
> num);
> > void jsonw_int_field(json_writer_t *self, const char *prop, int64_t
num);
> > void jsonw_null_field(json_writer_t *self, const char *prop);
> > diff --git a/lib/json_writer.c b/lib/json_writer.c
> > index f3eeaf7..6d73a1b 100644
> > --- a/lib/json_writer.c
> > +++ b/lib/json_writer.c
> > @@ -224,6 +224,11 @@ void jsonw_uint(json_writer_t *self, uint64_t
> num)
> > jsonw_printf(self, "%"PRIu64, num);
> > }
> >
> > +void jsonw_xint(json_writer_t *self, uint64_t num)
> > +{
> > + jsonw_printf(self, "%"PRIx64, num);
> > +}
> > +
> > void jsonw_lluint(json_writer_t *self, unsigned long long int num)
> > {
> > jsonw_printf(self, "%llu", num);
> > @@ -268,6 +273,12 @@ void jsonw_uint_field(json_writer_t *self, const
> char *prop, uint64_t num)
> > jsonw_uint(self, num);
> > }
> >
> > +void jsonw_xint_field(json_writer_t *self, const char *prop, uint64_t
num)
> > +{
> > + jsonw_name(self, prop);
> > + jsonw_xint(self, num);
> > +}
> > +
> > void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned
short
> num)
> > {
> > jsonw_name(self, prop);
> > diff --git a/rdma/res.c b/rdma/res.c
> > index 27c1efd..2b67d25 100644
> > --- a/rdma/res.c
> > +++ b/rdma/res.c
> > @@ -819,6 +819,119 @@ static int res_cq_parse_cb(const struct nlmsghdr
> *nlh, void *data)
> > return MNL_CB_OK;
> > }
> >
> > +static void print_key(struct rd *rd, const char *name, uint32_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_xint_field(rd->jw, name, val);
> > + else
> > + pr_out("%s 0x%x ", name, val);
> > +}
> > +
> > +static void print_iova(struct rd *rd, uint64_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_xint_field(rd->jw, "iova", val);
> > + else
> > + pr_out("iova 0x%" PRIx64 " ", val);
> > +}
> > +
> > +static void print_mrlen(struct rd *rd, uint64_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_uint_field(rd->jw, "mrlen", val);
> > + else
> > + pr_out("mrlen %" PRIu64 " ", val);
> > +}
> > +
> > +static void print_pgsize(struct rd *rd, uint32_t val)
> > +{
> > + if (rd->json_output)
> > + jsonw_uint_field(rd->jw, "pgsize", val);
> > + else
> > + pr_out("pgsize %u ", val);
> > +}
> > +
> > +static int res_mr_parse_cb(const struct nlmsghdr *nlh, void *data)
> > +{
> > + struct nlattr *tb[RDMA_NLDEV_ATTR_MAX] = {};
> > + struct nlattr *nla_table, *nla_entry;
> > + struct rd *rd = data;
> > + const char *name;
> > + uint32_t idx;
> > +
> > + mnl_attr_parse(nlh, 0, rd_attr_cb, tb);
> > + if (!tb[RDMA_NLDEV_ATTR_DEV_INDEX] ||
> > + !tb[RDMA_NLDEV_ATTR_DEV_NAME] ||
> > + !tb[RDMA_NLDEV_ATTR_RES_MR])
> > + return MNL_CB_ERROR;
> > +
> > + name = mnl_attr_get_str(tb[RDMA_NLDEV_ATTR_DEV_NAME]);
> > + idx = mnl_attr_get_u32(tb[RDMA_NLDEV_ATTR_DEV_INDEX]);
> > + nla_table = tb[RDMA_NLDEV_ATTR_RES_MR];
> > +
> > + mnl_attr_for_each_nested(nla_entry, nla_table) {
> > + struct nlattr *nla_line[RDMA_NLDEV_ATTR_MAX] = {};
> > + uint32_t rkey, lkey, pgsize;
> > + uint64_t iova, mrlen;
> > + char *comm = NULL;
> > + uint32_t pid = 0;
> > + int err;
> > +
> > + err = mnl_attr_parse_nested(nla_entry, rd_attr_cb,
nla_line);
> > + if (err != MNL_CB_OK)
> > + return MNL_CB_ERROR;
> > +
> > + if (!nla_line[RDMA_NLDEV_ATTR_RES_RKEY] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_LKEY] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_IOVA] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_MRLEN] ||
> > + !nla_line[RDMA_NLDEV_ATTR_RES_PGSIZE] ||
> > + (!nla_line[RDMA_NLDEV_ATTR_RES_PID] &&
> > + !nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])) {
> > + return MNL_CB_ERROR;
> > + }
>
> I'm not sure that all this ATTR are needed.
>
Yea you're right. I'll revisit all these and decide which minimal ones are
really required.
> > +
> > + rkey =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_RKEY]);
> > + lkey =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_LKEY]);
> > + iova =
> mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_IOVA]);
> > + mrlen =
> mnl_attr_get_u64(nla_line[RDMA_NLDEV_ATTR_RES_MRLEN]);
> > + pgsize =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PGSIZE]);
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_PID]) {
> > + pid =
> mnl_attr_get_u32(nla_line[RDMA_NLDEV_ATTR_RES_PID]);
> > + comm = get_task_name(pid);
> > + }
> > +
> > + if (rd_check_is_filtered(rd, "pid", pid))
>
> free(comm);
>
yup
> > + continue;
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME])
> > + /* discard const from mnl_attr_get_str */
> > + comm = (char
> *)mnl_attr_get_str(nla_line[RDMA_NLDEV_ATTR_RES_KERN_NAME]);
> > +
> > + if (rd->json_output)
> > + jsonw_start_array(rd->jw);
> > +
> > + print_link(rd, idx, name, 0, nla_line);
> > + print_key(rd, "rkey", rkey);
> > + print_key(rd, "lkey", lkey);
> > + print_iova(rd, iova);
> > + print_mrlen(rd, mrlen);
> > + print_pgsize(rd, pgsize);
> > + print_pid(rd, pid);
> > + print_comm(rd, comm, nla_line);
> > +
> > + if (nla_line[RDMA_NLDEV_ATTR_RES_PID])
> > + free(comm);
> > +
> > + if (rd->json_output)
> > + jsonw_end_array(rd->jw);
> > + else
> > + pr_out("\n");
> > + }
> > + return MNL_CB_OK;
> > +}
> > +
> > RES_FUNC(res_no_args, RDMA_NLDEV_CMD_RES_GET, NULL, true);
> >
> > static const struct
> > @@ -880,6 +993,18 @@ filters
> cq_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> >
> > RES_FUNC(res_cq, RDMA_NLDEV_CMD_RES_CQ_GET, cq_valid_filters,
> true);
> >
> > +static const struct
> > +filters mr_valid_filters[MAX_NUMBER_OF_FILTERS] = {{ .name = "link",
> > + .is_number = false },
> > + { .name = "rkey",
> > + .is_number = true },
> > + { .name = "lkey",
> > + .is_number = true },
> > + { .name = "pid",
> > + .is_number = true }};
> > +
> > +RES_FUNC(res_mr, RDMA_NLDEV_CMD_RES_MR_GET, mr_valid_filters,
> true);
> > +
> > static int res_show(struct rd *rd)
> > {
> > const struct rd_cmd cmds[] = {
> > @@ -887,6 +1012,7 @@ static int res_show(struct rd *rd)
> > { "qp", res_qp },
> > { "cm_id", res_cm_id },
> > { "cq", res_cq },
> > + { "mr", res_mr },
> > { 0 }
> > };
> >
> > diff --git a/rdma/utils.c b/rdma/utils.c
> > index 11b34fe..34b195a 100644
> > --- a/rdma/utils.c
> > +++ b/rdma/utils.c
> > @@ -392,6 +392,13 @@ static const enum mnl_attr_data_type
> nldev_policy[RDMA_NLDEV_ATTR_MAX] = {
> > [RDMA_NLDEV_ATTR_RES_CQE] = MNL_TYPE_U32,
> > [RDMA_NLDEV_ATTR_RES_USECNT] = MNL_TYPE_U64,
> > [RDMA_NLDEV_ATTR_RES_POLL_CTX] = MNL_TYPE_U8,
> > + [RDMA_NLDEV_ATTR_RES_MR] = MNL_TYPE_NESTED,
> > + [RDMA_NLDEV_ATTR_RES_MR_ENTRY] = MNL_TYPE_NESTED,
> > + [RDMA_NLDEV_ATTR_RES_RKEY] = MNL_TYPE_U32,
> > + [RDMA_NLDEV_ATTR_RES_LKEY] = MNL_TYPE_U32,
> > + [RDMA_NLDEV_ATTR_RES_IOVA] = MNL_TYPE_U64,
> > + [RDMA_NLDEV_ATTR_RES_MRLEN] = MNL_TYPE_U64,
> > + [RDMA_NLDEV_ATTR_RES_PGSIZE] = MNL_TYPE_U32,
> > };
> >
> > int rd_attr_cb(const struct nlattr *attr, void *data)
> > --
> > 1.8.3.1
> >
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH RFC iproute-next 5/5] rdma: Add PD resource tracking information
2018-02-23 14:22 ` Leon Romanovsky
@ 2018-02-26 15:09 ` Steve Wise
2018-02-27 0:47 ` Steve Wise
0 siblings, 1 reply; 16+ messages in thread
From: Steve Wise @ 2018-02-26 15:09 UTC (permalink / raw)
To: 'Leon Romanovsky'; +Cc: dsahern, stephen, netdev, linux-rdma
>
> On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> > Sample output:
> >
> > # rdma resource show pd
> > link cxgb4_0/- local_dma_lkey 0x0 usecnt 4 flags 0x0 pid 30503 comm
rping
>
> One more thing, flags need to be pre-parsed and accessible with "-d"
> command,
> as we did with dev,link capabilities.
Will do.
Thanks for reviewing this series!
Steve.
^ permalink raw reply [flat|nested] 16+ messages in thread
* RE: [PATCH RFC iproute-next 5/5] rdma: Add PD resource tracking information
2018-02-26 15:09 ` Steve Wise
@ 2018-02-27 0:47 ` Steve Wise
0 siblings, 0 replies; 16+ messages in thread
From: Steve Wise @ 2018-02-27 0:47 UTC (permalink / raw)
To: 'Leon Romanovsky'; +Cc: dsahern, stephen, netdev, linux-rdma
> > On Wed, Feb 14, 2018 at 01:07:01PM -0800, Steve Wise wrote:
> > > Sample output:
> > >
> > > # rdma resource show pd
> > > link cxgb4_0/- local_dma_lkey 0x0 usecnt 4 flags 0x0 pid 30503 comm
> rping
> >
> > One more thing, flags need to be pre-parsed and accessible with "-d"
> > command,
> > as we did with dev,link capabilities.
>
> Will do.
>
> Thanks for reviewing this series!
>
Turns out the only flags field was in the pd, and pd->flags only has one
flag, currently, indicating that the global_dma_rkey is in use. Since the
kernel side only sends up the global_dma_rkey if that flag is set, I just
dropped the PD_FLAGS attribute in both patch series. If pd flags grow, we
can add it, but I don't expect that.
Steve.
^ permalink raw reply [flat|nested] 16+ messages in thread
end of thread, other threads:[~2018-02-27 0:47 UTC | newest]
Thread overview: 16+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2018-02-19 20:10 [PATCH RFC iproute-next 0/5] cm_id, cq, mr, and pd resource tracking Steve Wise
2018-02-14 21:05 ` [PATCH RFC iproute-next 1/5] rdma: update rdma_netlink.h Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 2/5] rdma: Add CM_ID resource tracking information Steve Wise
2018-02-20 12:57 ` Leon Romanovsky
2018-02-20 15:15 ` Parav Pandit
2018-02-26 15:05 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 5/5] rdma: Add PD " Steve Wise
2018-02-23 14:22 ` Leon Romanovsky
2018-02-26 15:09 ` Steve Wise
2018-02-27 0:47 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 3/5] rdma: Add CQ " Steve Wise
2018-02-20 13:09 ` Leon Romanovsky
2018-02-26 15:06 ` Steve Wise
2018-02-14 21:07 ` [PATCH RFC iproute-next 4/5] rdma: Add MR " Steve Wise
2018-02-20 14:12 ` Leon Romanovsky
2018-02-26 15:08 ` Steve Wise
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.