[RFC iproute2 0/8] RDMA tool

[RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

----
This is initial phase to understand if user experience for this tool fits
RDMA and netdev communities exepectations. Also I would like to get feedback
if it is really worth to provide legacy sysfs for old kernels, or maybe I should
implement netlink from the beginning and abandon sysfs completely.
-----

Hi,

Please find below, the patch set with initial implementation of configuration
tool for RDMA subsystem, which will be supplementary tool to already existed
tools in netdev community (ip, devlink, ethtool, ..).

In opposite to netdev community, where standard tools exist to configure and
present different devices abilities, RDMA subsystem historically lacked it.

Following our discussion both in mailing list [1] and at the LPC 2016 [2],
we would like to propose this RDMA tool to be part of iproute2 package
and finally improve this situation.

The development of tool was influenced by ip and devlink tools. This implies
to the object->command interface and naming convention.

In order to close object model, ensure reuse of existing code and make this
tool usable from day one, we decided to implement wrappers over legacy sysfs
prior to implementing netlink functionality. As a nice bonus, it will allow
to use this tool with old kernels too.

It is important to mention that any future extension will be required to be
done with netlink, so for already existing objects small conversion to netlink
will be unavoidable.

  # rdma -h
  Usage: rdma [ OPTIONS ] OBJECT { COMMAND | help }
  where  OBJECT := { dev | link | ipoib | memory | stats | protocols | providers | monitor }
  OPTIONS := { -V[ersion] }

* DEV object equals to CA in IBTA specification and will provide
  a way to configure/present settings relevant to specific struct ib_device.

* LINK object represents port in IBTA specification and will give access to
  struct ib_port_immutable. From the day one, It prints netdev name of
  the corresponding IB port that makes ibdev2netdev script redundant.

* IPoIB object is supposed to be specific for IP-over-Infiniband upper
  layer protocol [3]. This ULP was mainly configured by combination
  of various sysfs knobs together with ethtool. Such situation adds
  challenges to add new and expose old configuration settings due to
  the mix between different subsystems.

* MEMORY object will be used to configure memory related settings,
  e.g. on-demand-paging (ODP), force-mr (force usage of MRs for
  RDMA READ/WRITE operation).

* STATS object is needed for everything related to statistics
  (per-PID, per-QP, per-device etc.). Despite the fact that RDMA
  devices provide extensive set of counters, the decision was to
  implement it in netlink directly, because there is a need to add
  filter mechanism to them, which doesn't exist now.

* PROTOCOLS object is going to be used for device special treatment
  of global to protocol settings (e.g. set device in RoCEv2 mode as
  a default, instead of RoCEv1, instead of configfs).

* PROVIDERS objects gives ability to get specific to the device
  information, like supported kABI objects [4].

* MONITOR object is needed to debug netlink communication and will
  follow standard functionality, which exists in ip and devlink tools.

There are number of ULPs which are not covered by this tool yet:
 * HFI-VNIC - I have no access to the HW and believe that Intel
   will add native object support for it.
 * Other storage related ULPs (iSER and SRP) were not introduced too,
   because they have special tools (scci-target-utils) to configure them.
   However it will be pretty straightforward to introduce new object,
   if there is demand for it.

At the initial stage, we implemented infrastructure to read legacy
sysfs entries (Patch #1), initial man pages (Patch #7) and provided
future object examples (Patch #2-6) to allow parallel development.

Following patches will focus on cleaning user interface, parsing other
relevant entries in similar fashion to the link capability mask (Patch #8)
and providing netlink interface.

These patches were tested with two following setups:
 * Setup A:
   - Two Mellanox ConnectX-4 devices (one port)
   - One Mellanox Connect-IB device (two ports)
 * Setup B:
   - One Mellanox ConnectX-4 device (one port)
   - One Mellanox ConnectX-3 Pro device (two ports)

Please consider the inclusion of the RDMA tool into iproute2 package,
so other participants will be able to speed up development.

[1] https://www.mail-archive.com/netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org/msg148523.html
[2] http://www.medkio.com/talks/lpc_debug.pdf
[3] https://tools.ietf.org/html/rfc4392
[4] http://marc.info/?l=linux-rdma&m=149261526916544&w=2

TODO: Add json output

Cc: Stephen Hemminger <stephen-OTpzqLSitTUnbdJkjeBofR2eb7JE58TQ@public.gmane.org>
Cc: Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
Cc: Jiri Pirko <jiri-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Cc: Ariel Almog <ariela-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Cc: Dennis Dalessandro <dennis.dalessandro-ral2JQCrhuEAvxtiuMwx3w@public.gmane.org>
Cc: Ram Amrani <ram.amrani-YGCgFSpz5w/QT0dZR+AlfA@public.gmane.org>
Cc: Bart Van Assche <Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org>
Cc: Sagi Grimberg <sagi-NQWnxTmZq1alnMjI0IkVqw@public.gmane.org>
Cc: Jason Gunthorpe <jgunthorpe-ePGOBjL8dl3ta4EC/59zMFaTQe2KTcn/@public.gmane.org>
Cc: Christoph Hellwig <hch-jcswGhMUV9g@public.gmane.org>
Cc: Or Gerlitz <ogerlitz-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
Cc: Linux RDMA <linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
Cc: Linux Netdev <netdev-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>

Leon Romanovsky (8):
  rdma: Add basic infrastructure for RDMA tool
  rdma: Add dev object
  rdma: Add link object
  rdma: Add IPoIB object
  rdma: Add memory object
  rdma: add stubs for future objects
  man: rdma.8: Document objects and commands
  rdma: Add link capability parsing

 Makefile          |   2 +-
 man/man8/Makefile |   3 +-
 man/man8/rdma.8   | 109 +++++++++++++++++++
 rdma/.gitignore   |   1 +
 rdma/Makefile     |  15 +++
 rdma/dev.c        | 101 ++++++++++++++++++
 rdma/ipoib.c      |  54 ++++++++++
 rdma/link.c       | 160 ++++++++++++++++++++++++++++
 rdma/memory.c     |  30 ++++++
 rdma/monitor.c    |  22 ++++
 rdma/protocols.c  |  22 ++++
 rdma/providers.c  |  28 +++++
 rdma/rdma.c       | 104 ++++++++++++++++++
 rdma/rdma.h       |  93 ++++++++++++++++
 rdma/stats.c      |  22 ++++
 rdma/utils.c      | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 16 files changed, 1077 insertions(+), 2 deletions(-)
 create mode 100644 man/man8/rdma.8
 create mode 100644 rdma/.gitignore
 create mode 100644 rdma/Makefile
 create mode 100644 rdma/dev.c
 create mode 100644 rdma/ipoib.c
 create mode 100644 rdma/link.c
 create mode 100644 rdma/memory.c
 create mode 100644 rdma/monitor.c
 create mode 100644 rdma/protocols.c
 create mode 100644 rdma/providers.c
 create mode 100644 rdma/rdma.c
 create mode 100644 rdma/rdma.h
 create mode 100644 rdma/stats.c
 create mode 100644 rdma/utils.c

--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC iproute2 1/8] rdma: Add basic infrastructure for RDMA tool

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

RDMA devices are cross-functional devices from one side,
but very tailored for the specific markets from another.

Such diversity caused to spread of RDMA related configuration
across various tools, e.g. devlink, ip, ethtool, ib specific and
vendor specific solutions.

This patch adds ability to fill device and port information
by reading sysfs entries (legacy).

All future configuration settings will be implemented in netlink format,
to be aligned with iproute2 package.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 Makefile        |   2 +-
 rdma/.gitignore |   1 +
 rdma/Makefile   |  15 +++
 rdma/rdma.c     |  96 +++++++++++++++++
 rdma/rdma.h     |  77 ++++++++++++++
 rdma/utils.c    | 313 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 6 files changed, 503 insertions(+), 1 deletion(-)
 create mode 100644 rdma/.gitignore
 create mode 100644 rdma/Makefile
 create mode 100644 rdma/rdma.c
 create mode 100644 rdma/rdma.h
 create mode 100644 rdma/utils.c

diff --git a/Makefile b/Makefile
index 18de7dcb..c255063b 100644
--- a/Makefile
+++ b/Makefile
@@ -52,7 +52,7 @@ WFLAGS += -Wmissing-declarations -Wold-style-definition -Wformat=2
 CFLAGS := $(WFLAGS) $(CCOPTS) -I../include $(DEFINES) $(CFLAGS)
 YACCFLAGS = -d -t -v

-SUBDIRS=lib ip tc bridge misc netem genl tipc devlink man
+SUBDIRS=lib ip tc bridge misc netem genl tipc devlink rdma man

 LIBNETLINK=../lib/libnetlink.a ../lib/libutil.a
 LDLIBS += $(LIBNETLINK)
diff --git a/rdma/.gitignore b/rdma/.gitignore
new file mode 100644
index 00000000..51fb172b
--- /dev/null
+++ b/rdma/.gitignore
@@ -0,0 +1 @@
+rdma
diff --git a/rdma/Makefile b/rdma/Makefile
new file mode 100644
index 00000000..65248b31
--- /dev/null
+++ b/rdma/Makefile
@@ -0,0 +1,15 @@
+include ../Config
+
+RDMA_OBJ = rdma.o utils.o
+TARGETS=rdma
+
+all:	$(TARGETS) $(LIBS)
+
+rdma:	$(RDMA_OBJ)
+	$(QUIET_LINK)$(CC) $^ -o $@
+
+install: all
+	install -m 0755 $(TARGETS) $(DESTDIR)$(SBINDIR)
+
+clean:
+	rm -f $(RDMA_OBJ) $(TARGETS)
diff --git a/rdma/rdma.c b/rdma/rdma.c
new file mode 100644
index 00000000..bc7d1483
--- /dev/null
+++ b/rdma/rdma.c
@@ -0,0 +1,96 @@
+/*
+ * rdma.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+
+#include <limits.h>
+
+#include "rdma.h"
+#include "SNAPSHOT.h"
+
+static void help(char *name)
+{
+	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
+	       "where  OBJECT := { }\n"
+	       "       OPTIONS := { -V[ersion] }\n", name);
+}
+
+static int obj_help(struct rdma *rd)
+{
+	help(rd->filename);
+	return 0;
+}
+
+static int rd_cmd(struct rdma *rd)
+{
+	const struct rdma_obj objs[] = {
+		{ NULL,		obj_help },
+		{ "help",	obj_help },
+		{ 0 }
+	};
+
+	return rdma_exec_cmd(rd, objs, "object");
+}
+
+static int rd_init(struct rdma *rd, int argc, char **argv, char *filename)
+{
+	rd->filename = filename;
+	rd->argc = argc;
+	rd->argv = argv;
+	INIT_LIST_HEAD(&rd->dev_map_list);
+	return 0;
+}
+static void rd_free(struct rdma *rd)
+{
+	dev_map_cleanup(rd);
+}
+int main(int argc, char **argv)
+{
+	char *filename;
+	static const struct option long_options[] = {
+		{ "version",		no_argument,		NULL, 'V' },
+		{ "help",		no_argument,		NULL, 'h' },
+		{ NULL, 0, NULL, 0 }
+	};
+	struct rdma rd;
+	int opt;
+	int err;
+
+	filename = basename(argv[0]);
+
+	while ((opt = getopt_long(argc, argv, "Vh",
+				  long_options, NULL)) >= 0) {
+
+		switch (opt) {
+		case 'V':
+			printf("%s utility, iproute2-ss%s\n", filename, SNAPSHOT);
+			return EXIT_SUCCESS;
+		case 'h':
+			help(filename);
+			return EXIT_SUCCESS;
+		default:
+			pr_err("Unknown option.\n");
+			help(filename);
+			return EXIT_FAILURE;
+		}
+	}
+
+	argc -= optind;
+	argv += optind;
+
+	err = rd_init(&rd, argc, argv, filename);
+	if (err)
+		goto out;
+
+	err = rd_cmd(&rd);
+	/* Always cleanup */
+	rd_free(&rd);
+
+out:	return (err) ? EXIT_FAILURE:EXIT_SUCCESS;
+}
diff --git a/rdma/rdma.h b/rdma/rdma.h
new file mode 100644
index 00000000..156bb74c
--- /dev/null
+++ b/rdma/rdma.h
@@ -0,0 +1,77 @@
+/*
+ * rdma.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+#ifndef _RDMA_TOOL_H_
+#define _RDMA_TOOL_H_
+
+#include <stdio.h>
+#include <stdlib.h>
+#include <stdint.h>
+#include <string.h>
+#include <errno.h>
+#include <getopt.h>
+#include <stdbool.h>
+
+#include "list.h"
+
+#define pr_err(args...) fprintf(stderr, ##args)
+#define pr_out(args...) fprintf(stdout, ##args)
+
+enum rt_protocols {
+	RT_PROTOCOL_IB,
+	RT_PROTOCOL_IWARP,
+	RT_PROTOCOL_ROCE_V1,
+	RT_PROTOCOL_ROCE_V2,
+	RT_PROTOCOL_OPA,
+	RT_NO_PROTOCOL
+};
+
+struct port_map {
+	struct list_head list;
+	char *ifname;
+	uint32_t idx;
+};
+
+struct dev_map {
+	struct list_head list;
+	char *dev_name;
+	uint32_t num_ports;
+	struct list_head port_map_list;
+	uint32_t idx;
+};
+
+struct rdma {
+	int argc;
+	char **argv;
+	char *filename;
+	struct list_head dev_map_list;
+};
+
+struct rdma_obj {
+	const char *cmd;
+	int (*func)(struct rdma *rd);
+};
+
+/*
+ * Parser interface
+ */
+bool rd_no_arg(struct rdma *rd);
+uint32_t get_port_from_argv(struct rdma *rd);
+
+int rdma_exec_cmd(struct rdma *rd, const struct rdma_obj *o, const char *str);
+int rdma_sysfs_read_ib(const char *name, int port, const char *field, char *res);
+
+/*
+ * Device manipulation
+ */
+int dev_map_init(struct rdma *rd);
+void dev_map_cleanup(struct rdma *rd);
+struct dev_map *dev_map_lookup(struct rdma *rd, bool allow_port_index);
+#endif /* _RDMA_TOOL_H_ */
diff --git a/rdma/utils.c b/rdma/utils.c
new file mode 100644
index 00000000..568d7c0a
--- /dev/null
+++ b/rdma/utils.c
@@ -0,0 +1,313 @@
+/*
+ * utils.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+
+#include <sys/types.h>
+#include <dirent.h>
+
+#include "rdma.h"
+
+/*
+ * This macro will be moved to generic layer,
+ * after code will be accepted.
+ * it is placed here to avoid rebases with upstream code.
+ */
+#define MAX(a, b)	((a) > (b) ? (a) : (b))
+
+static int rd_argc(struct rdma *rd)
+{
+	return rd->argc;
+}
+
+static char *rd_argv(struct rdma *rd)
+{
+	if (!rd_argc(rd))
+		return NULL;
+	return *rd->argv;
+}
+
+static int strcmpx(const char *str1, const char *str2)
+{
+	if (strlen(str1) > strlen(str2))
+			return -1;
+	return strncmp(str1, str2, strlen(str1));
+}
+
+static bool rd_argv_match(struct rdma *rd, const char *pattern)
+{
+	if (!rd_argc(rd))
+		return false;
+	return strcmpx(rd_argv(rd), pattern) == 0;
+}
+
+static void rd_arg_inc(struct rdma *rd)
+{
+	if (!rd_argc(rd))
+		return;
+	rd->argc--;
+	rd->argv++;
+}
+
+bool rd_no_arg(struct rdma *rd)
+{
+	return rd_argc(rd) == 0;
+}
+
+#define SYSFS_INFINIBAND "/sys/class/infiniband"
+#define SYSFS_NET "/sys/class/net"
+static int sysfs_read(const char *prefix, const char *name, int port, const char *field, char *res)
+{
+	char fpath[64];
+	FILE *fentry;
+	int result;
+
+	if (port == 0)
+		snprintf(fpath, 64, "%s/%s/%s", prefix, name, field);
+	else
+		snprintf(fpath, 64, "%s/%s/ports/%d/%s", prefix, name, port, field);
+
+	fentry = fopen(fpath, "r");
+	if (!fentry)
+		return -ENOENT;
+
+	result = fread(res, 1, 4096, fentry);
+	/* Remove last "\n" */
+	res[result-1] = '\0';
+	fclose(fentry);
+	return 0;
+}
+
+int rdma_sysfs_read_ib(const char *name, int port, const char *field, char *res)
+{
+	return sysfs_read(SYSFS_INFINIBAND, name, port, field, res);
+}
+
+static int sysfs_read_net(const char *name,  const char *field, char *res)
+{
+	return sysfs_read(SYSFS_NET, name, 0, field, res);
+}
+
+static struct dev_map *dev_map_alloc(char *dev_name)
+{
+	struct dev_map *dev_map;
+
+	dev_map = calloc(1, sizeof(*dev_map));
+	if (!dev_map)
+		return NULL;
+	dev_map->dev_name = strdup(dev_name);
+	INIT_LIST_HEAD(&dev_map->port_map_list);
+
+	return dev_map;
+}
+
+static void port_map_free(struct port_map *port_map)
+{
+	free(port_map->ifname);
+	free(port_map);
+}
+
+static void dev_map_free(struct dev_map *dev_map)
+{
+	struct port_map *port_map, *tmp;
+
+	list_for_each_entry_safe(port_map, tmp,
+				 &dev_map->port_map_list, list) {
+		list_del(&port_map->list);
+		port_map_free(port_map);
+	}
+
+	free(dev_map->dev_name);
+	free(dev_map);
+}
+
+void dev_map_cleanup(struct rdma *rd)
+{
+	struct dev_map *dev_map, *tmp;
+
+	list_for_each_entry_safe(dev_map, tmp,
+				 &rd->dev_map_list, list) {
+		list_del(&dev_map->list);
+		dev_map_free(dev_map);
+	}
+}
+
+uint32_t get_port_from_argv(struct rdma *rd)
+{
+	char *slash;
+
+	slash = strchr(rd_argv(rd), '/');
+	/* if no port found, return 0 */
+	return (slash) ? (atoi(slash + 1)):0;
+}
+
+struct dev_map *dev_map_lookup(struct rdma *rd, bool allow_port_index)
+{
+	struct dev_map *dev_map;
+	char *dev_name;
+	char *slash;
+
+	dev_name = strdup(rd_argv(rd));
+	if (allow_port_index) {
+		slash = strrchr(dev_name, '/');
+		if (slash)
+			*slash = '\0';
+	}
+
+	list_for_each_entry(dev_map, &rd->dev_map_list, list)
+		if (strcmp(dev_name, dev_map->dev_name) == 0) {
+			free(dev_name);
+			return dev_map;
+		}
+
+	free(dev_name);
+	return NULL;
+}
+
+static bool is_dots(char *name)
+{
+	if (!strcmp(name, ".") || !strcmp(name, ".."))
+		return true;
+	return false;
+}
+
+static char *find_ifname(struct dev_map *dev_map, uint32_t port)
+{
+	struct dirent *dentry, *drentry;
+	char data[4096];
+	char drpath[64];
+	uint32_t net_port;
+	DIR *dir, *drdir;
+	char *ifname = NULL;
+
+	dir = opendir(SYSFS_NET);
+	if (!dir)
+		return NULL;
+
+	while ((dentry = readdir(dir))) {
+		if (is_dots(dentry->d_name))
+			continue;
+
+		if (sysfs_read_net(dentry->d_name, "dev_id", data))
+			continue;
+
+		/* handle up to 9 ports, due to insufficient handling of hex  */
+		net_port = strtoul(data, NULL, 16);
+
+		snprintf(drpath, 64, "%s/%s/device/infiniband/", SYSFS_NET, dentry->d_name);
+
+		drdir = opendir(drpath);
+		if (!drdir)
+			continue;
+
+		while ((drentry = readdir(drdir))) {
+			if (is_dots(drentry->d_name))
+				continue;
+			if (!strcmp(drentry->d_name, dev_map->dev_name) &&
+			    net_port == port - 1) {
+				ifname = strdup(dentry->d_name);
+				closedir(drdir);
+				return ifname;
+			}
+		}
+
+		closedir(drdir);
+
+	}
+
+	closedir(dir);
+	return NULL;
+}
+
+static struct port_map *port_map_alloc(struct dev_map *dev_map, uint32_t port)
+{
+	struct port_map *port_map;
+
+	port_map = calloc(1, sizeof(*port_map));
+	if (!port_map)
+		return NULL;
+
+	port_map->idx = port;
+	/*
+	 * Expensive operation in sysfs world, need to rescan all net devices.
+	 * Hopefuly, it is one time operation.
+	 * In netlink, it will be simpler
+	 */
+	port_map->ifname = find_ifname(dev_map, port);
+	return port_map;
+}
+
+int dev_map_init(struct rdma *rd)
+{
+	struct dev_map *dev_map;
+	struct port_map *port_map;
+	struct dirent *dentry, *pentry;
+	uint32_t i = 1;
+	uint32_t num_ports = 0;
+	char ports_name[64];
+	DIR *dir, *pdir;
+	dir = opendir(SYSFS_INFINIBAND);
+	if (!dir)
+		return -ENOENT;
+
+	while ((dentry = readdir(dir))) {
+		num_ports = 0;
+		if (is_dots(dentry->d_name))
+			continue;
+
+		dev_map = dev_map_alloc(dentry->d_name);
+		if (!dev_map)
+			/* The main function will cleanup the allocations */
+			return -ENOMEM;
+		list_add_tail(&dev_map->list, &rd->dev_map_list);
+		dev_map->idx = i;
+		i++;
+
+		snprintf(ports_name, 64, "%s/%s/ports/", SYSFS_INFINIBAND, dentry->d_name);
+		pdir = opendir(ports_name);
+		while ((pentry = readdir(pdir))) {
+			int port;
+			if (is_dots(pentry->d_name))
+				continue;
+
+			port = atoi(pentry->d_name);
+			port_map = port_map_alloc(dev_map, port);
+			if (!port_map)
+				return -ENOENT;
+			list_add_tail(&port_map->list, &dev_map->port_map_list);
+			num_ports = MAX(num_ports, port);
+		}
+		closedir(pdir);
+		dev_map->num_ports = num_ports;
+	}
+	closedir(dir);
+
+	/* num_ports == 0 => no devices in infiniband folder */
+	return (num_ports) ? 0:(-ENOENT);
+}
+
+int rdma_exec_cmd(struct rdma *rd, const struct rdma_obj *objs, const char *str)
+{
+	const struct rdma_obj *o;
+
+	/* First argument in objs table is default variant */
+	if (rd_no_arg(rd))
+		return objs->func(rd);
+
+	for (o = objs + 1; o->cmd; ++o) {
+		if (rd_argv_match(rd, o->cmd)) {
+			/* Move to next argument */
+			rd_arg_inc(rd);
+			return o->func(rd);
+		}
+	}
+
+	pr_err("Unknown %s '%s'.\n", str, rd_argv(rd));
+	return 0;
+}
--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC iproute2 2/8] rdma: Add dev object

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Device (dev) object represents struct ib_device to user space.

The supported commands are show, set and help.

Print all devices:
 # rdma dev
1: mlx5_0: board_id MT_2190110032 fw_pages 261002 fw_ver 12.17.2046 hca_type MT4115 hw_rev 0
	node_desc hpchead HCA-1 node_guid e41d:2d03:0066:dee6 node_type 1: CA reg_pages 0
	sys_image_guid e41d:2d03:0066:dee6
2: mlx5_1: board_id MT_2190110032 fw_pages 250793 fw_ver 12.17.2046 hca_type MT4115 hw_rev 0
	node_desc hpchead HCA-2 node_guid e41d:2d03:0066:dee7 node_type 1: CA reg_pages 0
	sys_image_guid e41d:2d03:0066:dee6
3: mlx5_2: board_id MT_1210110019 fw_pages 68067 fw_ver 10.16.1020 hca_type MT4113 hw_rev 0
	node_desc hpchead HCA-3 node_guid 0002:c903:0016:75b0 node_type 1: CA reg_pages 0
	sys_image_guid 0002:c903:0016:75b0

Print specific device:
 # rdma dev show mlx5_1
2: mlx5_1: board_id MT_2190110032 fw_pages 250793 fw_ver 12.17.2046 hca_type MT4115 hw_rev 0
	node_desc hpchead HCA-2 node_guid e41d:2d03:0066:dee7 node_type 1: CA reg_pages 0
	sys_image_guid e41d:2d03:0066:dee6

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/Makefile |   2 +-
 rdma/dev.c    | 101 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |   3 +-
 rdma/rdma.h   |   5 +++
 4 files changed, 109 insertions(+), 2 deletions(-)
 create mode 100644 rdma/dev.c

diff --git a/rdma/Makefile b/rdma/Makefile
index 65248b31..67e349b0 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -1,6 +1,6 @@
 include ../Config

-RDMA_OBJ = rdma.o utils.o
+RDMA_OBJ = rdma.o utils.o dev.o
 TARGETS=rdma

 all:	$(TARGETS) $(LIBS)
diff --git a/rdma/dev.c b/rdma/dev.c
new file mode 100644
index 00000000..e6d71035
--- /dev/null
+++ b/rdma/dev.c
@@ -0,0 +1,101 @@
+/*
+ * dev.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+static int dev_help(struct rdma *rd)
+{
+	pr_out("Usage: %s dev show [DEV]\n", rd->filename);
+	pr_out("       %s dev set DEV [ node_desc { DESCRIPTION } ]\n", rd->filename);
+	/* Add masking of device capabilities */
+	return 0;
+}
+
+static void dev_one_show(const struct dev_map *dev_map)
+{
+	char *nodes[] = { "board_id",
+			  "fw_pages",
+			  "fw_ver",
+			  "hca_type",
+			  "hw_rev",
+			  "node_desc",
+			  "node_guid",
+			  "node_type",
+			  "reg_pages",
+			  "sys_image_guid",
+			  /* hfi1 specific */
+			  "nctxts",
+			  "nfreectxts",
+			  "serial",
+			  "boardversion",
+			  "tempsense",
+			  NULL };
+
+	char data[4096];
+	int i;
+	pr_out("%u: %s:", dev_map->idx, dev_map->dev_name);
+	for (i = 0 ; nodes[i] ; i++) {
+		if (rdma_sysfs_read_ib(dev_map->dev_name, 0, nodes[i], data))
+			continue;
+
+		/* Split line before "node_desc" */
+		if (!strcmp(nodes[i], "node_desc") ||
+		    !strcmp(nodes[i], "sys_image_guid"))
+			printf("\n\t");
+
+		pr_out(" %s %s", nodes[i], data);
+	}
+	pr_out("\n");
+}
+
+static int dev_show(struct rdma *rd)
+{
+	struct dev_map *dev_map;
+
+	if (rd_no_arg(rd)) {
+		list_for_each_entry(dev_map, &rd->dev_map_list, list)
+			dev_one_show(dev_map);
+	}
+	else {
+		dev_map = dev_map_lookup(rd, false);
+		if (!dev_map) {
+			pr_err("Wrong device name\n");
+			return -ENOENT;
+		}
+		dev_one_show(dev_map);
+	}
+	return 0;
+}
+
+static int dev_set(struct rdma *rd)
+{
+	/* Not implemented yet */
+	return 0;
+}
+
+int obj_dev(struct rdma *rd)
+{
+	const struct rdma_obj objs[] = {
+		{ NULL,		dev_show },
+		{ "show",	dev_show },
+		{ "list",	dev_show },
+		{ "set",	dev_set },
+		{ "help",	dev_help },
+		{ 0 }
+	};
+
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return rdma_exec_cmd(rd, objs, "dev command");
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index bc7d1483..7c537c5e 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { }\n"
+	       "where  OBJECT := { dev }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }

@@ -31,6 +31,7 @@ static int rd_cmd(struct rdma *rd)
 {
 	const struct rdma_obj objs[] = {
 		{ NULL,		obj_help },
+		{ "dev",	obj_dev },
 		{ "help",	obj_help },
 		{ 0 }
 	};
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 156bb74c..2d81cd92 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -60,6 +60,11 @@ struct rdma_obj {
 };

 /*
+ * Command interfaces
+ */
+int obj_dev(struct rdma *rd);
+
+/*
  * Parser interface
  */
 bool rd_no_arg(struct rdma *rd);

[RFC iproute2 3/8] rdma: Add link object

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Link object represents port of struct ib_device.

Supported commands are show, set and help.

Print all links for all devices:
 # rdma link
1/1: mlx5_0/1: ifname ib0 cap_mask 0x2651e848 lid 0x13 lid_mask_count 0 link_layer InfiniBand
	phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x2 sm_sl 0 state 4: ACTIVE
2/1: mlx5_1/1: ifname ib1 cap_mask 0x2651e848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN
3/1: mlx5_2/1: ifname ib2 cap_mask 0x26516848 lid 0x1a lid_mask_count 0 link_layer InfiniBand
	phys_state 5: LinkUp rate 56 Gb/sec (4X FDR) sm_lid 0x2 sm_sl 0 state 4: ACTIVE
3/2: mlx5_2/2: ifname ib3 cap_mask 0x26516848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN

Print all links for specific device:
 # rdma link show mlx5_2
3/1: mlx5_2/1: ifname ib2 cap_mask 0x26516848 lid 0x1a lid_mask_count 0 link_layer InfiniBand
	phys_state 5: LinkUp rate 56 Gb/sec (4X FDR) sm_lid 0x2 sm_sl 0 state 4: ACTIVE
3/2: mlx5_2/2: ifname ib3 cap_mask 0x26516848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN

Print specific link:
 # rdma link show mlx5_2/2
3/2: mlx5_2/2: ifname ib3 cap_mask 0x26516848 lid 0xffff lid_mask_count 0 link_layer InfiniBand
	phys_state 3: Disabled rate 10 Gb/sec (4X) sm_lid 0x0 sm_sl 0 state 1: DOWN

Set parameter;
 # rdma link set mlx5_2/2 type auto lb_unicast off

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/Makefile |   2 +-
 rdma/link.c   | 112 ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |   3 +-
 rdma/rdma.h   |   1 +
 4 files changed, 116 insertions(+), 2 deletions(-)
 create mode 100644 rdma/link.c

diff --git a/rdma/Makefile b/rdma/Makefile
index 67e349b0..cf54ed36 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -1,6 +1,6 @@
 include ../Config

-RDMA_OBJ = rdma.o utils.o dev.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o
 TARGETS=rdma

 all:	$(TARGETS) $(LIBS)
diff --git a/rdma/link.c b/rdma/link.c
new file mode 100644
index 00000000..e86ff399
--- /dev/null
+++ b/rdma/link.c
@@ -0,0 +1,112 @@
+/*
+ * link.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+static int link_help(struct rdma *rd)
+{
+	pr_out("Usage: %s link show [ DEV | DEV/PORT ]\n", rd->filename);
+	pr_out("       %s link set DEV/PORT { type { eth | ib | auto } |\n", rd->filename);
+	pr_out("                                lb_unicast { on | off } |\n");
+	pr_out("                                lb_multicast { on | off } }\n");
+	return 0;
+}
+
+static void dev_one_show(const struct dev_map *dev_map, uint32_t port_idx_first, uint32_t port_idx_last)
+{
+	char *nodes[] = { "cap_mask",
+			  "lid",
+			  "lid_mask_count",
+			  "link_layer",
+			  "phys_state",
+			  "rate",
+			  "sm_lid",
+			  "sm_sl",
+			  "state",
+       			  NULL };
+
+	struct port_map *port_map;
+	char data[4096];
+	int i, j;
+
+	for(j = port_idx_first ; j <= port_idx_last; j++) {
+		pr_out("%u/%u: %s/%u:", dev_map->idx, j, dev_map->dev_name, j);
+		list_for_each_entry(port_map, &dev_map->port_map_list, list)
+			if (j == port_map->idx)
+			       printf(" ifname %s", (port_map->ifname)?:"NONE");
+
+		for (i = 0 ; nodes[i] ; i++) {
+			if (rdma_sysfs_read_ib(dev_map->dev_name, j, nodes[i], data))
+				continue;
+
+			/* Split line before "phys_state" */
+			if (!strcmp(nodes[i], "phys_state"))
+				printf("\n\t");
+
+			pr_out(" %s %s", nodes[i], data);
+		}
+		pr_out("\n");
+	}
+}
+
+static int link_show(struct rdma *rd)
+{
+	struct dev_map *dev_map;
+
+	if (rd_no_arg(rd)) {
+		list_for_each_entry(dev_map, &rd->dev_map_list, list)
+			dev_one_show(dev_map, 1, dev_map->num_ports);
+	}
+	else {
+		uint32_t port_idx;
+		uint32_t num_ports;
+		dev_map = dev_map_lookup(rd, true);
+		port_idx = get_port_from_argv(rd);
+		if (!dev_map || port_idx > dev_map->num_ports) {
+			pr_err("Wrong device name\n");
+			return -EINVAL;
+		}
+		if (port_idx)
+			num_ports = port_idx;
+		else {
+			port_idx = 1;
+			num_ports = dev_map->num_ports;
+		}
+
+		dev_one_show(dev_map, port_idx, num_ports);
+	}
+	return 0;
+}
+
+static int link_set(struct rdma *rd)
+{
+	/* Not supported yet */
+	return 0;
+}
+
+int obj_link(struct rdma *rd)
+{
+	const struct rdma_obj objs[] = {
+		{ NULL, 	link_show },
+		{ "show",	link_show },
+		{ "list",	link_show },
+		{ "set",	link_set },
+		{ "help",	link_help },
+		{ 0 }
+	};
+
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return rdma_exec_cmd(rd, objs, "link command");
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index 7c537c5e..55cbf0e3 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev }\n"
+	       "where  OBJECT := { dev | link }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }

@@ -32,6 +32,7 @@ static int rd_cmd(struct rdma *rd)
 	const struct rdma_obj objs[] = {
 		{ NULL,		obj_help },
 		{ "dev",	obj_dev },
+		{ "link",	obj_link },
 		{ "help",	obj_help },
 		{ 0 }
 	};
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 2d81cd92..bdb77b5e 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -63,6 +63,7 @@ struct rdma_obj {
  * Command interfaces
  */
 int obj_dev(struct rdma *rd);
+int obj_link(struct rdma *rd);

 /*
  * Parser interface

[RFC iproute2 4/8] rdma: Add IPoIB object

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

IPoIB object allows configuration and presentation of information for
IP-over-Infiniband user level protocol.

Supported commands are show, set and help.

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 rdma/Makefile |  2 +-
 rdma/ipoib.c  | 54 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 rdma/rdma.c   |  3 ++-
 rdma/rdma.h   |  1 +
 4 files changed, 58 insertions(+), 2 deletions(-)
 create mode 100644 rdma/ipoib.c

diff --git a/rdma/Makefile b/rdma/Makefile
index cf54ed36..dd702b9f 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -1,6 +1,6 @@
 include ../Config

-RDMA_OBJ = rdma.o utils.o dev.o link.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o
 TARGETS=rdma

 all:	$(TARGETS) $(LIBS)
diff --git a/rdma/ipoib.c b/rdma/ipoib.c
new file mode 100644
index 00000000..dd0d0285
--- /dev/null
+++ b/rdma/ipoib.c
@@ -0,0 +1,54 @@
+/*
+ * ipoib.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
+ */
+
+#include "rdma.h"
+
+static int ipoib_help(struct rdma *rd)
+{
+	pr_out("Usage: %s ipoib show NAME | DEV | DEV/PORT\n", rd->filename);
+	pr_out("       %s ipoib set NAME [ accel { off | on } ]\n", rd->filename);
+	pr_out("       %s ipoib start NAME dev DEV\n", rd->filename);
+	pr_out("       %s ipoib stop NAME\n", rd->filename);
+	return 0;
+}
+
+static int ipoib_show(struct rdma *rd)
+{
+	if (rd_no_arg(rd))
+		ipoib_help(rd);
+
+	return 0;
+}
+
+static int ipoib_set(struct rdma *rd)
+{
+	/* Not supported yet */
+	return 0;
+}
+
+int obj_ipoib(struct rdma *rd)
+{
+	const struct rdma_obj objs[] = {
+		{ NULL,		ipoib_show },
+		{ "show",	ipoib_show },
+		{ "list",	ipoib_show },
+		{ "set",	ipoib_set },
+		{ "help",	ipoib_help },
+		{ 0 }
+	};
+
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return rdma_exec_cmd(rd, objs, "Uknown ipoib command");
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index 55cbf0e3..ffd70899 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev | link }\n"
+	       "where  OBJECT := { dev | link | ipoib }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }

@@ -33,6 +33,7 @@ static int rd_cmd(struct rdma *rd)
 		{ NULL,		obj_help },
 		{ "dev",	obj_dev },
 		{ "link",	obj_link },
+		{ "ipoib",	obj_ipoib },
 		{ "help",	obj_help },
 		{ 0 }
 	};
diff --git a/rdma/rdma.h b/rdma/rdma.h
index bdb77b5e..1fef4eb8 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -64,6 +64,7 @@ struct rdma_obj {
  */
 int obj_dev(struct rdma *rd);
 int obj_link(struct rdma *rd);
+int obj_ipoib(struct rdma *rd);

 /*
  * Parser interface
--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC iproute2 5/8] rdma: Add memory object

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

Memory object gives to the user ability to manipulate over general
properties of memory for the specific devices. The memory properties
have broader usage than dev object can provide.

For example, on-demand-paging (ODP) configurations are mostly software related.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/Makefile |  2 +-
 rdma/memory.c | 30 ++++++++++++++++++++++++++++++
 rdma/rdma.c   |  3 ++-
 rdma/rdma.h   |  1 +
 4 files changed, 34 insertions(+), 2 deletions(-)
 create mode 100644 rdma/memory.c

diff --git a/rdma/Makefile b/rdma/Makefile
index dd702b9f..5cf0d29f 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -1,6 +1,6 @@
 include ../Config

-RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o memory.o
 TARGETS=rdma

 all:	$(TARGETS) $(LIBS)
diff --git a/rdma/memory.c b/rdma/memory.c
new file mode 100644
index 00000000..68fd5dd3
--- /dev/null
+++ b/rdma/memory.c
@@ -0,0 +1,30 @@
+/*
+ * memory.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+static void memory_help(char *filename)
+{
+	pr_out("Usage: %s memory show [ DEV ]\n", filename);
+	pr_out("       %s memory set DEV { odp { off | on } |\n", filename);
+	pr_out("       %s                  memic SIZE }\n", filename);
+}
+
+int obj_memory(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	memory_help(rd->filename);
+	return 0;
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index ffd70899..094d490d 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev | link | ipoib }\n"
+	       "where  OBJECT := { dev | link | ipoib | memory }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }

@@ -34,6 +34,7 @@ static int rd_cmd(struct rdma *rd)
 		{ "dev",	obj_dev },
 		{ "link",	obj_link },
 		{ "ipoib",	obj_ipoib },
+		{ "memory",	obj_memory },
 		{ "help",	obj_help },
 		{ 0 }
 	};
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 1fef4eb8..dcff066f 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -65,6 +65,7 @@ struct rdma_obj {
 int obj_dev(struct rdma *rd);
 int obj_link(struct rdma *rd);
 int obj_ipoib(struct rdma *rd);
+int obj_memory(struct rdma *rd);

 /*
  * Parser interface

[RFC iproute2 6/8] rdma: add stubs for future objects

[Leon Romanovsky]

From: Leon Romanovsky <leonro@mellanox.com>

The following objects (monitor, providers, stats and protocols) are not
implemented yet, however it is worth to place their stubs in the code.

This will serve as an initial starting point for other developers to
extend RDMA tool.

Signed-off-by: Leon Romanovsky <leonro@mellanox.com>
---
 rdma/Makefile    |  2 +-
 rdma/monitor.c   | 22 ++++++++++++++++++++++
 rdma/protocols.c | 22 ++++++++++++++++++++++
 rdma/providers.c | 28 ++++++++++++++++++++++++++++
 rdma/rdma.c      |  6 +++++-
 rdma/rdma.h      |  4 ++++
 rdma/stats.c     | 22 ++++++++++++++++++++++
 7 files changed, 104 insertions(+), 2 deletions(-)
 create mode 100644 rdma/monitor.c
 create mode 100644 rdma/protocols.c
 create mode 100644 rdma/providers.c
 create mode 100644 rdma/stats.c

diff --git a/rdma/Makefile b/rdma/Makefile
index 5cf0d29f..eb71da68 100644
--- a/rdma/Makefile
+++ b/rdma/Makefile
@@ -1,6 +1,6 @@
 include ../Config

-RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o memory.o
+RDMA_OBJ = rdma.o utils.o dev.o link.o ipoib.o memory.o stats.o protocols.o providers.o monitor.o
 TARGETS=rdma

 all:	$(TARGETS) $(LIBS)
diff --git a/rdma/monitor.c b/rdma/monitor.c
new file mode 100644
index 00000000..99d4b042
--- /dev/null
+++ b/rdma/monitor.c
@@ -0,0 +1,22 @@
+/*
+ * monitor.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+int obj_monitor(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return 0;
+}
diff --git a/rdma/protocols.c b/rdma/protocols.c
new file mode 100644
index 00000000..26de7d2b
--- /dev/null
+++ b/rdma/protocols.c
@@ -0,0 +1,22 @@
+/*
+ * protocols.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+int obj_protocols(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return 0;
+}
diff --git a/rdma/providers.c b/rdma/providers.c
new file mode 100644
index 00000000..8d516cca
--- /dev/null
+++ b/rdma/providers.c
@@ -0,0 +1,28 @@
+/*
+ * providers.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+static void providers_help(char *filename)
+{
+	pr_out("Usage: %s providers show [ DEV ]\n", filename);
+}
+
+int obj_providers(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	providers_help(rd->filename);
+	return 0;
+}
diff --git a/rdma/rdma.c b/rdma/rdma.c
index 094d490d..a0a3ec81 100644
--- a/rdma/rdma.c
+++ b/rdma/rdma.c
@@ -17,7 +17,7 @@
 static void help(char *name)
 {
 	pr_out("Usage: %s [ OPTIONS ] OBJECT { COMMAND | help }\n"
-	       "where  OBJECT := { dev | link | ipoib | memory }\n"
+	       "where  OBJECT := { dev | link | ipoib | memory | stats | protocols | providers | monitor }\n"
 	       "       OPTIONS := { -V[ersion] }\n", name);
 }

@@ -35,6 +35,10 @@ static int rd_cmd(struct rdma *rd)
 		{ "link",	obj_link },
 		{ "ipoib",	obj_ipoib },
 		{ "memory",	obj_memory },
+		{ "stats",	obj_stats },
+		{ "providers",	obj_providers },
+		{ "protocols",	obj_protocols },
+		{ "monitor",	obj_monitor },
 		{ "help",	obj_help },
 		{ 0 }
 	};
diff --git a/rdma/rdma.h b/rdma/rdma.h
index dcff066f..11d940d7 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -66,6 +66,10 @@ int obj_dev(struct rdma *rd);
 int obj_link(struct rdma *rd);
 int obj_ipoib(struct rdma *rd);
 int obj_memory(struct rdma *rd);
+int obj_protocols(struct rdma *rd);
+int obj_stats(struct rdma *rd);
+int obj_providers(struct rdma *rd);
+int obj_monitor(struct rdma *rd);

 /*
  * Parser interface
diff --git a/rdma/stats.c b/rdma/stats.c
new file mode 100644
index 00000000..a557e59b
--- /dev/null
+++ b/rdma/stats.c
@@ -0,0 +1,22 @@
+/*
+ * stats.c	RDMA tool
+ *
+ *              This program is free software; you can redistribute it and/or
+ *              modify it under the terms of the GNU General Public License
+ *              as published by the Free Software Foundation; either version
+ *              2 of the License, or (at your option) any later version.
+ *
+ * Authors:     Leon Romanovsky <leonro@mellanox.com>
+ */
+
+#include "rdma.h"
+
+int obj_stats(struct rdma *rd)
+{
+	if (dev_map_init(rd)) {
+		pr_err("There are no RDMA devices\n");
+		return -ENOENT;
+	}
+
+	return 0;
+}

[RFC iproute2 V1 7/8] man: rdma.8: Document objects and commands

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 man/man8/Makefile |   3 +-
 man/man8/rdma.8   | 109 ++++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 111 insertions(+), 1 deletion(-)
 create mode 100644 man/man8/rdma.8

diff --git a/man/man8/Makefile b/man/man8/Makefile
index f3318644..81979a07 100644
--- a/man/man8/Makefile
+++ b/man/man8/Makefile
@@ -19,7 +19,8 @@ MAN8PAGES = $(TARGETS) ip.8 arpd.8 lnstat.8 routel.8 rtacct.8 rtmon.8 rtpr.8 ss.
 	tc-simple.8 tc-skbedit.8 tc-vlan.8 tc-xt.8 tc-skbmod.8 tc-ife.8 \
 	tc-tunnel_key.8 tc-sample.8 \
 	devlink.8 devlink-dev.8 devlink-monitor.8 devlink-port.8 devlink-sb.8 \
-	ifstat.8
+	ifstat.8 \
+	rdma.8

 all: $(TARGETS)

diff --git a/man/man8/rdma.8 b/man/man8/rdma.8
new file mode 100644
index 00000000..410b3d7d
--- /dev/null
+++ b/man/man8/rdma.8
@@ -0,0 +1,109 @@
+.TH RDMA 8 "28 Mar 2017" "iproute2" "Linux"
+.SH NAME
+rdma \- RDMA tool
+.SH SYNOPSIS
+.sp
+.ad l
+.in +8
+.ti -8
+.B rdma
+.RI "[ " OPTIONS " ] " OBJECT " { " COMMAND " | "
+.BR help " }"
+.sp
+
+.ti -8
+.IR OBJECT " := { "
+.BR dev " | " link " | " protocol " | " stats " | " monitor " | " memory " | " ipoib " | " provider "}"
+.sp
+
+.ti -8
+.IR OPTIONS " := { "
+\fB\-V\fR[\fIersion\fR] }
+
+.SH OPTIONS
+
+.TP
+.BR "\-V" , " -Version"
+Print the version of the
+.B rdma
+tool and exit.
+
+.SS
+.I OBJECT
+
+.TP
+.B dev
+- RDMA device.
+
+.TP
+.B link
+- RDMA port related.
+
+.TP
+.B protocol
+- RDMA protocol.
+
+.TP
+.B monitor
+- watch for netlink messages.
+
+.TP
+.B memory
+- configure memory related operations.
+
+.TP
+.B ipoib
+- configure IPoIB.
+
+.TP
+.B provider
+- provider specific configurations.
+
+.PP
+The names of all objects may be written in full or
+abbreviated form, for example
+.B stats
+can be abbreviated as
+.B stat
+or just
+.B s.
+
+.SS
+.I COMMAND
+
+Specifies the action to perform on the object.
+The set of possible actions depends on the object type.
+As a rule, it is possible to
+.B show
+(or
+.B list
+) objects, but some objects do not allow all of these operations
+or have some additional commands. The
+.B help
+command is available for all objects. It prints
+out a list of available commands and argument syntax conventions.
+.sp
+If no command is given, some default command is assumed.
+Usually it is
+.B list
+or, if the objects of this class cannot be listed,
+.BR "help" .
+
+.SH EXIT STATUS
+Exit status is 0 if command was successful or a positive integer upon failure.
+
+.SH SEE ALSO
+.BR rdma-link (8),
+.BR rdma-protocol (8),
+.BR rdma-monitor (8),
+.BR rdma-stats (8),
+.br
+
+.SH REPORTING BUGS
+Report any bugs to the Linux RDMA mailing list
+.B <linux-rdma-u79uwXL29TY76Z2rM5mHXA@public.gmane.org>
+where the development and maintenance is primarily done.
+You do not have to be subscribed to the list to send a message there.
+
+.SH AUTHOR
+Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[RFC iproute2 8/8] rdma: Add link capability parsing

[Leon Romanovsky]

From: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>

Add parsing interface for the cap_mask

$./rdma/rdma link show mlx5_2/2 cap_mask
3/2: mlx5_2/2: sm off notice off trap on opt_ipd off auto_migr off sl_map on mkey_nvram off
	pkey_nvram off led_info off sm_disabled off sys_image_guid on pkey_sw_ext_port_trap off
	extended_speeds on cm on snmp_tunnel off reinit off device_mgmt off vendor_class on dr_notice off
	cap_mask_notice on boot_mgmt off link_latency off client_reg on ip_based_gids on

Signed-off-by: Leon Romanovsky <leonro-VPRAkNaXOzVWk0Htik3J/w@public.gmane.org>
---
 rdma/link.c  | 74 +++++++++++++++++++++++++++++++++++++++++++++++++-----------
 rdma/rdma.h  |  4 ++++
 rdma/utils.c |  4 ++--
 3 files changed, 67 insertions(+), 15 deletions(-)

diff --git a/rdma/link.c b/rdma/link.c
index e86ff399..e9880914 100644
--- a/rdma/link.c
+++ b/rdma/link.c
@@ -14,13 +14,48 @@
 static int link_help(struct rdma *rd)
 {
 	pr_out("Usage: %s link show [ DEV | DEV/PORT ]\n", rd->filename);
+	pr_out("       %s link show [ DEV | DEV/PORT ] cap_mask\n", rd->filename);
 	pr_out("       %s link set DEV/PORT { type { eth | ib | auto } |\n", rd->filename);
 	pr_out("                                lb_unicast { on | off } |\n");
 	pr_out("                                lb_multicast { on | off } }\n");
 	return 0;
 }

-static void dev_one_show(const struct dev_map *dev_map, uint32_t port_idx_first, uint32_t port_idx_last)
+static void print_cap_mask(uint32_t cap_mask)
+{
+#define PRINT_PORT_CAP(name, val, offset)		(printf(" %s %s", name, (((val) >> (offset))&0x1)?"on":"off"))
+
+	/* Naive copy/paste from include/rdma/ib_verbs.h */
+ 	PRINT_PORT_CAP("sm", cap_mask, 1);
+ 	PRINT_PORT_CAP("notice", cap_mask, 2);
+ 	PRINT_PORT_CAP("trap", cap_mask, 3);
+ 	PRINT_PORT_CAP("opt_ipd", cap_mask, 4);
+ 	PRINT_PORT_CAP("auto_migr", cap_mask, 5);
+	PRINT_PORT_CAP("sl_map", cap_mask, 6);
+ 	PRINT_PORT_CAP("mkey_nvram", cap_mask, 7);
+	printf("\n\t");
+ 	PRINT_PORT_CAP("pkey_nvram", cap_mask, 8);
+ 	PRINT_PORT_CAP("led_info", cap_mask, 9);
+ 	PRINT_PORT_CAP("sm_disabled", cap_mask, 10);
+ 	PRINT_PORT_CAP("sys_image_guid", cap_mask, 11);
+	PRINT_PORT_CAP("pkey_sw_ext_port_trap", cap_mask, 12);
+	printf("\n\t");
+	PRINT_PORT_CAP("extended_speeds", cap_mask, 14);
+ 	PRINT_PORT_CAP("cm", cap_mask, 16);
+ 	PRINT_PORT_CAP("snmp_tunnel", cap_mask, 17);
+ 	PRINT_PORT_CAP("reinit", cap_mask, 18);
+ 	PRINT_PORT_CAP("device_mgmt", cap_mask, 19);
+ 	PRINT_PORT_CAP("vendor_class", cap_mask, 20);
+ 	PRINT_PORT_CAP("dr_notice", cap_mask, 21);
+	printf("\n\t");
+	PRINT_PORT_CAP("cap_mask_notice", cap_mask, 22);
+ 	PRINT_PORT_CAP("boot_mgmt", cap_mask, 23);
+ 	PRINT_PORT_CAP("link_latency", cap_mask, 24);
+ 	PRINT_PORT_CAP("client_reg", cap_mask, 25);
+ 	PRINT_PORT_CAP("ip_based_gids", cap_mask, 26);
+}
+static void dev_one_show(struct rdma *rd, const struct dev_map *dev_map,
+			 uint32_t port_idx_first, uint32_t port_idx_last)
 {
 	char *nodes[] = { "cap_mask",
 			  "lid",
@@ -35,23 +70,36 @@ static void dev_one_show(const struct dev_map *dev_map, uint32_t port_idx_first,

 	struct port_map *port_map;
 	char data[4096];
+	uint32_t cap_mask;
+	bool cap_mask_r = false;
 	int i, j;

+	rd_arg_inc(rd);
+	if (rd_argv_match(rd, "cap_mask"))
+		cap_mask_r = true;
+
 	for(j = port_idx_first ; j <= port_idx_last; j++) {
 		pr_out("%u/%u: %s/%u:", dev_map->idx, j, dev_map->dev_name, j);
-		list_for_each_entry(port_map, &dev_map->port_map_list, list)
-			if (j == port_map->idx)
-			       printf(" ifname %s", (port_map->ifname)?:"NONE");
+		if (cap_mask_r) {
+			rdma_sysfs_read_ib(dev_map->dev_name, 1, nodes[0], data);
+			cap_mask = strtoul(data, NULL, 16);
+			print_cap_mask(cap_mask);
+		}
+		else {
+			list_for_each_entry(port_map, &dev_map->port_map_list, list)
+				if (j == port_map->idx)
+				       printf(" ifname %s", (port_map->ifname)?:"NONE");

-		for (i = 0 ; nodes[i] ; i++) {
-			if (rdma_sysfs_read_ib(dev_map->dev_name, j, nodes[i], data))
-				continue;
+			for (i = 0 ; nodes[i] ; i++) {
+				if (rdma_sysfs_read_ib(dev_map->dev_name, j, nodes[i], data))
+					continue;

-			/* Split line before "phys_state" */
-			if (!strcmp(nodes[i], "phys_state"))
-				printf("\n\t");
+				/* Split line before "phys_state" */
+				if (!strcmp(nodes[i], "phys_state"))
+					printf("\n\t");

-			pr_out(" %s %s", nodes[i], data);
+				pr_out(" %s %s", nodes[i], data);
+			}
 		}
 		pr_out("\n");
 	}
@@ -63,7 +111,7 @@ static int link_show(struct rdma *rd)

 	if (rd_no_arg(rd)) {
 		list_for_each_entry(dev_map, &rd->dev_map_list, list)
-			dev_one_show(dev_map, 1, dev_map->num_ports);
+			dev_one_show(rd, dev_map, 1, dev_map->num_ports);
 	}
 	else {
 		uint32_t port_idx;
@@ -81,7 +129,7 @@ static int link_show(struct rdma *rd)
 			num_ports = dev_map->num_ports;
 		}

-		dev_one_show(dev_map, port_idx, num_ports);
+		dev_one_show(rd, dev_map, port_idx, num_ports);
 	}
 	return 0;
 }
diff --git a/rdma/rdma.h b/rdma/rdma.h
index 11d940d7..12c87048 100644
--- a/rdma/rdma.h
+++ b/rdma/rdma.h
@@ -76,6 +76,10 @@ int obj_monitor(struct rdma *rd);
  */
 bool rd_no_arg(struct rdma *rd);
 uint32_t get_port_from_argv(struct rdma *rd);
+bool rd_no_arg(struct rdma *rd);
+bool rd_argv_match(struct rdma *rd, const char *pattern);
+void rd_arg_inc(struct rdma *rd);
+uint32_t get_port_from_argv(struct rdma *rd);

 int rdma_exec_cmd(struct rdma *rd, const struct rdma_obj *o, const char *str);
 int rdma_sysfs_read_ib(const char *name, int port, const char *field, char *res);
diff --git a/rdma/utils.c b/rdma/utils.c
index 568d7c0a..fd5fe77f 100644
--- a/rdma/utils.c
+++ b/rdma/utils.c
@@ -40,14 +40,14 @@ static int strcmpx(const char *str1, const char *str2)
 	return strncmp(str1, str2, strlen(str1));
 }

-static bool rd_argv_match(struct rdma *rd, const char *pattern)
+bool rd_argv_match(struct rdma *rd, const char *pattern)
 {
 	if (!rd_argc(rd))
 		return false;
 	return strcmpx(rd_argv(rd), pattern) == 0;
 }

-static void rd_arg_inc(struct rdma *rd)
+void rd_arg_inc(struct rdma *rd)
 {
 	if (!rd_argc(rd))
 		return;
--
2.12.2

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Bart Van Assche]

On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> we would like to propose this RDMA tool to be part of iproute2 package
> and finally improve this situation.

Hello Leon,

Although I really appreciate your work: can you clarify why you would like to
add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
for adding RDMA functionality to iproute2 in [1].

Thanks,

Bart.

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1299 bytes --]

On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
> On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > we would like to propose this RDMA tool to be part of iproute2 package
> > and finally improve this situation.
>
> Hello Leon,
>
> Although I really appreciate your work: can you clarify why you would like to
> add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> for adding RDMA functionality to iproute2 in [1].

We are planning to reuse the same infrastructure provided by iproute2,
like netlink parsing, access to distributions, same CLI and same standards.

Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
RDMA.

I do expect that iproute2 will be installed on every machine with any
type of connection, including IB and OPA.

So I think that it is enough to be part of that suite and don't invent
our own for one specific tool.

Thanks

>
> Thanks,
>
> Bart.--
> To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Bart Van Assche]

On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
> On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
> > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > we would like to propose this RDMA tool to be part of iproute2 package
> > > and finally improve this situation.
> > 
> > Hello Leon,
> > 
> > Although I really appreciate your work: can you clarify why you would like to
> > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > for adding RDMA functionality to iproute2 in [1].
> 
> We are planning to reuse the same infrastructure provided by iproute2,
> like netlink parsing, access to distributions, same CLI and same standards.
> 
> Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
> Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
> RDMA.
> 
> I do expect that iproute2 will be installed on every machine with any
> type of connection, including IB and OPA.
> 
> So I think that it is enough to be part of that suite and don't invent
> our own for one specific tool.

Hello Leon,

Sorry but to me that sounds like a weak argument for including RDMA functionality
in iproute2. There is already a library for communication over netlink sockets,
namely libnl. Is there functionality that is in iproute2 but not in libnl and
that is needed for the new tool? If so, have you considered to create a new
library for that functionality?

Thanks,

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2022 bytes --]

On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:
> On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
> > On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
> > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > > we would like to propose this RDMA tool to be part of iproute2 package
> > > > and finally improve this situation.
> > >
> > > Hello Leon,
> > >
> > > Although I really appreciate your work: can you clarify why you would like to
> > > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > > for adding RDMA functionality to iproute2 in [1].
> >
> > We are planning to reuse the same infrastructure provided by iproute2,
> > like netlink parsing, access to distributions, same CLI and same standards.
> >
> > Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
> > Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
> > RDMA.
> >
> > I do expect that iproute2 will be installed on every machine with any
> > type of connection, including IB and OPA.
> >
> > So I think that it is enough to be part of that suite and don't invent
> > our own for one specific tool.
>
> Hello Leon,
>
> Sorry but to me that sounds like a weak argument for including RDMA functionality
> in iproute2. There is already a library for communication over netlink sockets,
> namely libnl. Is there functionality that is in iproute2 but not in libnl and
> that is needed for the new tool? If so, have you considered to create a new
> library for that functionality?

It is not hard to create new tool, the hardest part is to ensure that it is
part of the distributions. Did you count how many months we are trying to
add rdma-core to debian?

I have enough headache with that and don't want another one.

Do you have situation in mind where you will have RDMA device without
iproute2 installed?

Thanks

>
> Thanks,
>
> Bart.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Dennis Dalessandro]

On 05/04/2017 02:45 PM, Leon Romanovsky wrote:
> On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:
>> On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
>>> On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
>>>> On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
>>>>> Following our discussion both in mailing list [1] and at the LPC 2016 [2],
>>>>> we would like to propose this RDMA tool to be part of iproute2 package
>>>>> and finally improve this situation.
>>>>
>>>> Hello Leon,
>>>>
>>>> Although I really appreciate your work: can you clarify why you would like to
>>>> add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
>>>> for adding RDMA functionality to iproute2 in [1].
>>>
>>> We are planning to reuse the same infrastructure provided by iproute2,
>>> like netlink parsing, access to distributions, same CLI and same standards.
>>>
>>> Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
>>> Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
>>> RDMA.
>>>
>>> I do expect that iproute2 will be installed on every machine with any
>>> type of connection, including IB and OPA.
>>>
>>> So I think that it is enough to be part of that suite and don't invent
>>> our own for one specific tool.
>>
>> Hello Leon,
>>
>> Sorry but to me that sounds like a weak argument for including RDMA functionality
>> in iproute2. There is already a library for communication over netlink sockets,
>> namely libnl. Is there functionality that is in iproute2 but not in libnl and
>> that is needed for the new tool? If so, have you considered to create a new
>> library for that functionality?
>
> It is not hard to create new tool, the hardest part is to ensure that it is
> part of the distributions. Did you count how many months we are trying to
> add rdma-core to debian?

I do agree that it is a strange pairing and am not really a fan. However 
at the end of the day it's just a name for a repo/package. If the 
iproute folks are fine to include rdma in their repo/package, great we 
can leverage their code for CLI and other common stuff.

Now if the interface was something like "ip -FlagForRdma ..." I would 
object to that, but the interface is "rdma ... " so from users 
perspective it's different tools. They don't need to care that it was 
sourced from a common git repo.

Just as an aside this already works a bit with OPA:

  $ ./rdma link
1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0 
link_layer InfiniBand
          phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 
0 state 4: ACTIVE

Leon I'll get you more feedback and testing, I've just been really 
bogged down this week, sorry.

-Denny

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 3598 bytes --]

On Thu, May 04, 2017 at 03:26:13PM -0400, Dennis Dalessandro wrote:
> On 05/04/2017 02:45 PM, Leon Romanovsky wrote:
> > On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:
> > > On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
> > > > On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
> > > > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > > > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > > > > we would like to propose this RDMA tool to be part of iproute2 package
> > > > > > and finally improve this situation.
> > > > >
> > > > > Hello Leon,
> > > > >
> > > > > Although I really appreciate your work: can you clarify why you would like to
> > > > > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > > > > for adding RDMA functionality to iproute2 in [1].
> > > >
> > > > We are planning to reuse the same infrastructure provided by iproute2,
> > > > like netlink parsing, access to distributions, same CLI and same standards.
> > > >
> > > > Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
> > > > Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
> > > > RDMA.
> > > >
> > > > I do expect that iproute2 will be installed on every machine with any
> > > > type of connection, including IB and OPA.
> > > >
> > > > So I think that it is enough to be part of that suite and don't invent
> > > > our own for one specific tool.
> > >
> > > Hello Leon,
> > >
> > > Sorry but to me that sounds like a weak argument for including RDMA functionality
> > > in iproute2. There is already a library for communication over netlink sockets,
> > > namely libnl. Is there functionality that is in iproute2 but not in libnl and
> > > that is needed for the new tool? If so, have you considered to create a new
> > > library for that functionality?
> >
> > It is not hard to create new tool, the hardest part is to ensure that it is
> > part of the distributions. Did you count how many months we are trying to
> > add rdma-core to debian?
>
> I do agree that it is a strange pairing and am not really a fan. However at
> the end of the day it's just a name for a repo/package. If the iproute folks
> are fine to include rdma in their repo/package, great we can leverage their
> code for CLI and other common stuff.
>
> Now if the interface was something like "ip -FlagForRdma ..." I would object
> to that, but the interface is "rdma ... " so from users perspective it's
> different tools. They don't need to care that it was sourced from a common
> git repo.
>
> Just as an aside this already works a bit with OPA:
>
>  $ ./rdma link
> 1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
> link_layer InfiniBand
>          phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
> state 4: ACTIVE
>
> Leon I'll get you more feedback and testing, I've just been really bogged
> down this week, sorry.

Thanks Denny,

Before you are starting to test it, can you please provide your feedback
on my initial questions? Usability and need of sysfs.

----
This is initial phase to understand if user experience for this tool fits
RDMA and netdev communities exepectations. Also I would like to get feedback
if it is really worth to provide legacy sysfs for old kernels, or maybe I should
implement netlink from the beginning and abandon sysfs completely.
-----

P.S. I believe this will give you wrong output, because it parses IB port cap_mask.
$./rdma link show hfi1_0/1 cap_mask

Thanks

>
> -Denny
>
>
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Doug Ledford]

On Thu, 2017-05-04 at 15:26 -0400, Dennis Dalessandro wrote:
> On 05/04/2017 02:45 PM, Leon Romanovsky wrote:
> > 
> > On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:
> > > 
> > > On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
> > > > 
> > > > On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche
> > > > wrote:
> > > > > 
> > > > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > > > > 
> > > > > > Following our discussion both in mailing list [1] and at
> > > > > > the LPC 2016 [2],
> > > > > > we would like to propose this RDMA tool to be part of
> > > > > > iproute2 package
> > > > > > and finally improve this situation.
> > > > > 
> > > > > Hello Leon,
> > > > > 
> > > > > Although I really appreciate your work: can you clarify why
> > > > > you would like to
> > > > > add *RDMA* functionality to an *IP routing* tool? I haven't
> > > > > found any motivation
> > > > > for adding RDMA functionality to iproute2 in [1].
> > > > 
> > > > We are planning to reuse the same infrastructure provided by
> > > > iproute2,
> > > > like netlink parsing, access to distributions, same CLI and
> > > > same standards.
> > > > 
> > > > Right now, RDMA is already tightened to netdev: iWARP, RoCE,
> > > > IPoIB, HFI-VNIC.
> > > > Many drivers (mlx, qed, i40, cxgb) are sharing code between net
> > > > and
> > > > RDMA.
> > > > 
> > > > I do expect that iproute2 will be installed on every machine
> > > > with any
> > > > type of connection, including IB and OPA.
> > > > 
> > > > So I think that it is enough to be part of that suite and don't
> > > > invent
> > > > our own for one specific tool.
> > > 
> > > Hello Leon,
> > > 
> > > Sorry but to me that sounds like a weak argument for including
> > > RDMA functionality
> > > in iproute2. There is already a library for communication over
> > > netlink sockets,
> > > namely libnl. Is there functionality that is in iproute2 but not
> > > in libnl and
> > > that is needed for the new tool? If so, have you considered to
> > > create a new
> > > library for that functionality?
> > 
> > It is not hard to create new tool, the hardest part is to ensure
> > that it is
> > part of the distributions. Did you count how many months we are
> > trying to
> > add rdma-core to debian?
> 
> I do agree that it is a strange pairing and am not really a fan.
> However 
> at the end of the day it's just a name for a repo/package. If the 
> iproute folks are fine to include rdma in their repo/package, great
> we 
> can leverage their code for CLI and other common stuff.

If you look into the iproute2 package, it becomes clear that the name
iproute2 is historical and not really accurate any more.  It contains
things like the bridge control software, tc for controlling send
queues, and many things network related but not routing related.  The
rdma tool is a perfectly fine fit in the sense that it is an additional
network management tool IMO.

For reference, here's the list of stuff already in iproute on my Fedora
24 box:

/usr/sbin/arpd
/usr/sbin/bridge
/usr/sbin/cbq
/usr/sbin/ctstat
/usr/sbin/genl
/usr/sbin/ifcfg
/usr/sbin/ifstat
/usr/sbin/ip
/usr/sbin/lnstat
/usr/sbin/nstat
/usr/sbin/routef
/usr/sbin/routel
/usr/sbin/rtacct
/usr/sbin/rtmon
/usr/sbin/rtpr
/usr/sbin/rtstat
/usr/sbin/ss
/usr/sbin/tc
/usr/sbin/tipc

And in fact, if you check, tipc is almost similar to RDMA ;-)  So, I
suggest people not get hung up on the name iproute2, the fit is fine
when you look deeper into the nature of the package.

-- 
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>
    GPG KeyID: B826A3330E572FDD
   
Key fingerprint = AE6B 1BDA 122B 23B4 265B  1274 B826 A333 0E57 2FDD

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Stephen Hemminger]

On Thu, 04 May 2017 16:45:58 -0400
Doug Ledford <dledford-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org> wrote:

> On Thu, 2017-05-04 at 15:26 -0400, Dennis Dalessandro wrote:
> > On 05/04/2017 02:45 PM, Leon Romanovsky wrote:  
> > > 
> > > On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:  
> > > > 
> > > > On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:  
> > > > > 
> > > > > On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche
> > > > > wrote:  
> > > > > > 
> > > > > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:  
> > > > > > > 
> > > > > > > Following our discussion both in mailing list [1] and at
> > > > > > > the LPC 2016 [2],
> > > > > > > we would like to propose this RDMA tool to be part of
> > > > > > > iproute2 package
> > > > > > > and finally improve this situation.  
> > > > > > 
> > > > > > Hello Leon,
> > > > > > 
> > > > > > Although I really appreciate your work: can you clarify why
> > > > > > you would like to
> > > > > > add *RDMA* functionality to an *IP routing* tool? I haven't
> > > > > > found any motivation
> > > > > > for adding RDMA functionality to iproute2 in [1].  
> > > > > 
> > > > > We are planning to reuse the same infrastructure provided by
> > > > > iproute2,
> > > > > like netlink parsing, access to distributions, same CLI and
> > > > > same standards.
> > > > > 
> > > > > Right now, RDMA is already tightened to netdev: iWARP, RoCE,
> > > > > IPoIB, HFI-VNIC.
> > > > > Many drivers (mlx, qed, i40, cxgb) are sharing code between net
> > > > > and
> > > > > RDMA.
> > > > > 
> > > > > I do expect that iproute2 will be installed on every machine
> > > > > with any
> > > > > type of connection, including IB and OPA.
> > > > > 
> > > > > So I think that it is enough to be part of that suite and don't
> > > > > invent
> > > > > our own for one specific tool.  
> > > > 
> > > > Hello Leon,
> > > > 
> > > > Sorry but to me that sounds like a weak argument for including
> > > > RDMA functionality
> > > > in iproute2. There is already a library for communication over
> > > > netlink sockets,
> > > > namely libnl. Is there functionality that is in iproute2 but not
> > > > in libnl and
> > > > that is needed for the new tool? If so, have you considered to
> > > > create a new
> > > > library for that functionality?  
> > > 
> > > It is not hard to create new tool, the hardest part is to ensure
> > > that it is
> > > part of the distributions. Did you count how many months we are
> > > trying to
> > > add rdma-core to debian?  
> > 
> > I do agree that it is a strange pairing and am not really a fan.
> > However 
> > at the end of the day it's just a name for a repo/package. If the 
> > iproute folks are fine to include rdma in their repo/package, great
> > we 
> > can leverage their code for CLI and other common stuff.  
> 
> If you look into the iproute2 package, it becomes clear that the name
> iproute2 is historical and not really accurate any more.  It contains
> things like the bridge control software, tc for controlling send
> queues, and many things network related but not routing related.  The
> rdma tool is a perfectly fine fit in the sense that it is an additional
> network management tool IMO.
> 
> For reference, here's the list of stuff already in iproute on my Fedora
> 24 box:
> 
> /usr/sbin/arpd
> /usr/sbin/bridge
> /usr/sbin/cbq
> /usr/sbin/ctstat
> /usr/sbin/genl
> /usr/sbin/ifcfg
> /usr/sbin/ifstat
> /usr/sbin/ip
> /usr/sbin/lnstat
> /usr/sbin/nstat
> /usr/sbin/routef
> /usr/sbin/routel
> /usr/sbin/rtacct
> /usr/sbin/rtmon
> /usr/sbin/rtpr
> /usr/sbin/rtstat
> /usr/sbin/ss
> /usr/sbin/tc
> /usr/sbin/tipc
> 
> And in fact, if you check, tipc is almost similar to RDMA ;-)  So, I
> suggest people not get hung up on the name iproute2, the fit is fine
> when you look deeper into the nature of the package.
> 

Iproute2 is a collection like busybox. It has bridging, devlink and tipc already.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Jiri Benc]

On Thu,  4 May 2017 21:02:08 +0300, Leon Romanovsky wrote:
> In order to close object model, ensure reuse of existing code and make this
> tool usable from day one, we decided to implement wrappers over legacy sysfs
> prior to implementing netlink functionality. As a nice bonus, it will allow
> to use this tool with old kernels too.

This sounds wrong. We don't support legacy ioctl interface for the 'ip'
command, either. I think rdma should be converted to netlink first and
the new tool should only use netlink.

 Jiri
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1131 bytes --]

On Fri, May 05, 2017 at 08:54:57AM +0200, Jiri Benc wrote:
> On Thu,  4 May 2017 21:02:08 +0300, Leon Romanovsky wrote:
> > In order to close object model, ensure reuse of existing code and make this
> > tool usable from day one, we decided to implement wrappers over legacy sysfs
> > prior to implementing netlink functionality. As a nice bonus, it will allow
> > to use this tool with old kernels too.
>
> This sounds wrong. We don't support legacy ioctl interface for the 'ip'
> command, either. I think rdma should be converted to netlink first and
> the new tool should only use netlink.

RDMA in slightly different situation than "ip" tool was. "ip" was implemented
when tools like ifconfig existed. It allowed to old and new systems to be
configured to some degree. In RDMA community, there are no similar tools like
"ifconfig". Implementation in netlink-only interface will leave old systems without
common tool at all.

As an upstream-oriented person, I personally fine with that, but anyway would
like to get wider agreement/disagreement on that, before removing sysfs
parsing logic from the rdmatool.

Thanks

>
>  Jiri

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Bart Van Assche]

On Thu, 2017-05-04 at 21:45 +0300, Leon Romanovsky wrote:
> It is not hard to create new tool, the hardest part is to ensure that it is
> part of the distributions. Did you count how many months we are trying to
> add rdma-core to debian?

Hello Leon,

Sorry but I was not aware that the effort to add rdma-core to Debian was taking
that long. Please let me know if I can help with that effort.

Bart.

Re: [RFC iproute2 0/8] RDMA tool

[Jiri Pirko]

Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche@sandisk.com wrote:
>On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
>> Following our discussion both in mailing list [1] and at the LPC 2016 [2],
>> we would like to propose this RDMA tool to be part of iproute2 package
>> and finally improve this situation.
>
>Hello Leon,
>
>Although I really appreciate your work: can you clarify why you would like to
>add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
>for adding RDMA functionality to iproute2 in [1].

Bart, please realize that iproute2 is much more than "*IP routing* tool".
I understand you got confused by the name. Please see sources. Your comment
is totally pointless...

Re: [RFC iproute2 0/8] RDMA tool

[Jiri Pirko]

Fri, May 05, 2017 at 03:17:54PM CEST, leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org wrote:
>On Fri, May 05, 2017 at 08:54:57AM +0200, Jiri Benc wrote:
>> On Thu,  4 May 2017 21:02:08 +0300, Leon Romanovsky wrote:
>> > In order to close object model, ensure reuse of existing code and make this
>> > tool usable from day one, we decided to implement wrappers over legacy sysfs
>> > prior to implementing netlink functionality. As a nice bonus, it will allow
>> > to use this tool with old kernels too.
>>
>> This sounds wrong. We don't support legacy ioctl interface for the 'ip'
>> command, either. I think rdma should be converted to netlink first and
>> the new tool should only use netlink.
>
>RDMA in slightly different situation than "ip" tool was. "ip" was implemented
>when tools like ifconfig existed. It allowed to old and new systems to be
>configured to some degree. In RDMA community, there are no similar tools like
>"ifconfig". Implementation in netlink-only interface will leave old systems without
>common tool at all.
>
>As an upstream-oriented person, I personally fine with that, but anyway would
>like to get wider agreement/disagreement on that, before removing sysfs
>parsing logic from the rdmatool.

I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink
api later on for the same things will make the code unnecessary complex.
Also, the legacy sysfs will most likely stay there forever so there will
be no actual motivation to port the existing things to the new netlink
api.

For the prototyping purposes, I belive that what you did makes perfect
sense. But for the actual mergable version, my feeling is that we need
to strictly stick with new netlink rdma interface and just forget about
the old sysfs one. Distros would have to backport the new kernel
rdma netlink api.

Yes, this will be little bit more painful at the beginning, but in the
long run, I believe it will save some severe headaches.

--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Bart Van Assche]

[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1: Type: text/plain; charset="utf-8", Size: 1243 bytes --]

On Sat, 2017-05-06 at 12:40 +0200, Jiri Pirko wrote:
> Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche@sandisk.com wrote:
> > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > we would like to propose this RDMA tool to be part of iproute2 package
> > > and finally improve this situation.
> > 
> > Although I really appreciate your work: can you clarify why you would like to
> > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > for adding RDMA functionality to iproute2 in [1].
> 
> Bart, please realize that iproute2 is much more than "*IP routing* tool".
> I understand you got confused by the name. Please see sources. Your comment
> is totally pointless...

I asked for a clarification that should have been in the cover letter but that
was missing from that cover letter. So I think that was the right thing to do
instead of pointless. BTW, can you explain why you are using an e-mail address
that is hiding that you are a Mellanox employee?

Bart.N‹§²æìr¸›yúèšØb²X¬¶Ç§vØ^–)Þº{.nÇ+‰·¥Š{±­ÙšŠ{ayº\x1dʇڙë,j\a­¢f£¢·hš‹»öì\x17/oSc¾™Ú³9˜uÀ¦æå‰È&jw¨®\x03(­éšŽŠÝ¢j"ú\x1a¶^[m§ÿïêäz¹Þ–Šàþf£¢·hšˆ§~ˆmš

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 1519 bytes --]

On Sat, May 06, 2017 at 02:40:24PM +0000, Bart Van Assche wrote:
> On Sat, 2017-05-06 at 12:40 +0200, Jiri Pirko wrote:
> > Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche@sandisk.com wrote:
> > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > > we would like to propose this RDMA tool to be part of iproute2 package
> > > > and finally improve this situation.
> > >
> > > Although I really appreciate your work: can you clarify why you would like to
> > > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > > for adding RDMA functionality to iproute2 in [1].
> >
> > Bart, please realize that iproute2 is much more than "*IP routing* tool".
> > I understand you got confused by the name. Please see sources. Your comment
> > is totally pointless...
>
> I asked for a clarification that should have been in the cover letter but that
> was missing from that cover letter. So I think that was the right thing to do
> instead of pointless. BTW, can you explain why you are using an e-mail address
> that is hiding that you are a Mellanox employee?

For the same reason as I do. It is much easier to use outside servers
than Mellanox's IT infrastructure.

Right now, I'm speaking for myself, but for me, to properly answer with @mellanox.com,
I need to use very specific mail setup, while for any other addresses I can and use any
sane setup, including mobile application.

>
> Bart.

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 2162 bytes --]

On Sat, May 06, 2017 at 12:48:26PM +0200, Jiri Pirko wrote:
> Fri, May 05, 2017 at 03:17:54PM CEST, leon-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org wrote:
> >On Fri, May 05, 2017 at 08:54:57AM +0200, Jiri Benc wrote:
> >> On Thu,  4 May 2017 21:02:08 +0300, Leon Romanovsky wrote:
> >> > In order to close object model, ensure reuse of existing code and make this
> >> > tool usable from day one, we decided to implement wrappers over legacy sysfs
> >> > prior to implementing netlink functionality. As a nice bonus, it will allow
> >> > to use this tool with old kernels too.
> >>
> >> This sounds wrong. We don't support legacy ioctl interface for the 'ip'
> >> command, either. I think rdma should be converted to netlink first and
> >> the new tool should only use netlink.
> >
> >RDMA in slightly different situation than "ip" tool was. "ip" was implemented
> >when tools like ifconfig existed. It allowed to old and new systems to be
> >configured to some degree. In RDMA community, there are no similar tools like
> >"ifconfig". Implementation in netlink-only interface will leave old systems without
> >common tool at all.
> >
> >As an upstream-oriented person, I personally fine with that, but anyway would
> >like to get wider agreement/disagreement on that, before removing sysfs
> >parsing logic from the rdmatool.
>
> I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink
> api later on for the same things will make the code unnecessary complex.
> Also, the legacy sysfs will most likely stay there forever so there will
> be no actual motivation to port the existing things to the new netlink
> api.
>
> For the prototyping purposes, I belive that what you did makes perfect
> sense. But for the actual mergable version, my feeling is that we need
> to strictly stick with new netlink rdma interface and just forget about
> the old sysfs one. Distros would have to backport the new kernel
> rdma netlink api.

Thanks,
It looks like that most of the comments are in favor of netlink-only
solution.

>
> Yes, this will be little bit more painful at the beginning, but in the
> long run, I believe it will save some severe headaches.
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Jiri Pirko]

Sat, May 06, 2017 at 04:40:24PM CEST, Bart.VanAssche@sandisk.com wrote:
>On Sat, 2017-05-06 at 12:40 +0200, Jiri Pirko wrote:
>> Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche@sandisk.com wrote:
>> > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
>> > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
>> > > we would like to propose this RDMA tool to be part of iproute2 package
>> > > and finally improve this situation.
>> > 
>> > Although I really appreciate your work: can you clarify why you would like to
>> > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
>> > for adding RDMA functionality to iproute2 in [1].
>> 
>> Bart, please realize that iproute2 is much more than "*IP routing* tool".
>> I understand you got confused by the name. Please see sources. Your comment
>> is totally pointless...
>
>I asked for a clarification that should have been in the cover letter but that
>was missing from that cover letter. So I think that was the right thing to do

I think that was just complete misunderstanding about what iproute2 is.


>instead of pointless. BTW, can you explain why you are using an e-mail address
>that is hiding that you are a Mellanox employee?

Who sais I have to do that? This is funny...

Re: [RFC iproute2 0/8] RDMA tool

[Stephen Hemminger]

[-- Attachment #1: Type: text/plain, Size: 2370 bytes --]

On Sun, 7 May 2017 09:33:29 +0300
Leon Romanovsky <leon@kernel.org> wrote:

> On Sat, May 06, 2017 at 12:48:26PM +0200, Jiri Pirko wrote:
> > Fri, May 05, 2017 at 03:17:54PM CEST, leon@kernel.org wrote:  
> > >On Fri, May 05, 2017 at 08:54:57AM +0200, Jiri Benc wrote:  
> > >> On Thu,  4 May 2017 21:02:08 +0300, Leon Romanovsky wrote:  
> > >> > In order to close object model, ensure reuse of existing code and make this
> > >> > tool usable from day one, we decided to implement wrappers over legacy sysfs
> > >> > prior to implementing netlink functionality. As a nice bonus, it will allow
> > >> > to use this tool with old kernels too.  
> > >>
> > >> This sounds wrong. We don't support legacy ioctl interface for the 'ip'
> > >> command, either. I think rdma should be converted to netlink first and
> > >> the new tool should only use netlink.  
> > >
> > >RDMA in slightly different situation than "ip" tool was. "ip" was implemented
> > >when tools like ifconfig existed. It allowed to old and new systems to be
> > >configured to some degree. In RDMA community, there are no similar tools like
> > >"ifconfig". Implementation in netlink-only interface will leave old systems without
> > >common tool at all.
> > >
> > >As an upstream-oriented person, I personally fine with that, but anyway would
> > >like to get wider agreement/disagreement on that, before removing sysfs
> > >parsing logic from the rdmatool.  
> >
> > I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink
> > api later on for the same things will make the code unnecessary complex.
> > Also, the legacy sysfs will most likely stay there forever so there will
> > be no actual motivation to port the existing things to the new netlink
> > api.
> >
> > For the prototyping purposes, I belive that what you did makes perfect
> > sense. But for the actual mergable version, my feeling is that we need
> > to strictly stick with new netlink rdma interface and just forget about
> > the old sysfs one. Distros would have to backport the new kernel
> > rdma netlink api.  
> 
> Thanks,
> It looks like that most of the comments are in favor of netlink-only
> solution.

If current (like 4.10 or later) kernel support netlink only solution, that makes sense.
When I created bridge command; it also abandoned the old ioctl interface.

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 833 bytes --]

Re: [RFC iproute2 0/8] RDMA tool

[Knut Omang]

On Sun, 2017-05-07 at 09:33 +0300, Leon Romanovsky wrote:
> On Sat, May 06, 2017 at 12:48:26PM +0200, Jiri Pirko wrote:
> > Fri, May 05, 2017 at 03:17:54PM CEST, leon@kernel.org wrote:
> > >On Fri, May 05, 2017 at 08:54:57AM +0200, Jiri Benc wrote:
> > >> On Thu,  4 May 2017 21:02:08 +0300, Leon Romanovsky wrote:
> > >> > In order to close object model, ensure reuse of existing code and make this
> > >> > tool usable from day one, we decided to implement wrappers over legacy sysfs
> > >> > prior to implementing netlink functionality. As a nice bonus, it will allow
> > >> > to use this tool with old kernels too.
> > >>
> > >> This sounds wrong. We don't support legacy ioctl interface for the 'ip'
> > >> command, either. I think rdma should be converted to netlink first and
> > >> the new tool should only use netlink.
> > >
> > >RDMA in slightly different situation than "ip" tool was. "ip" was implemented
> > >when tools like ifconfig existed. It allowed to old and new systems to be
> > >configured to some degree. In RDMA community, there are no similar tools like
> > >"ifconfig". Implementation in netlink-only interface will leave old systems without
> > >common tool at all.
> > >
> > >As an upstream-oriented person, I personally fine with that, but anyway would
> > >like to get wider agreement/disagreement on that, before removing sysfs
> > >parsing logic from the rdmatool.
> >
> > I tend to agree with Jiri Benc. I fear that supporting sysfs + netlink
> > api later on for the same things will make the code unnecessary complex.
> > Also, the legacy sysfs will most likely stay there forever so there will
> > be no actual motivation to port the existing things to the new netlink
> > api.
> >
> > For the prototyping purposes, I belive that what you did makes perfect
> > sense. But for the actual mergable version, my feeling is that we need
> > to strictly stick with new netlink rdma interface and just forget about
> > the old sysfs one. Distros would have to backport the new kernel
> > rdma netlink api.
> 
> Thanks,
> It looks like that most of the comments are in favor of netlink-only
> solution.

Leon, I like the thought bw comp support. After all this is a user level tool so it should
be possible to make a clean implementation that makes the old stuff easy to remove 
at some point. It will also attract users much sooner than if they have to have 
their own if-then-else logic around everything to be able to support old and new.

> > Yes, this will be little bit more painful at the beginning, but in the
> > long run, I believe it will save some severe headaches.
> >

IMHO, some headache will be there anyway, just a matter of how how far out it gets.

Knut

Re: [RFC iproute2 0/8] RDMA tool

[Bart Van Assche]

On Sun, 2017-05-07 at 12:20 +0200, Jiri Pirko wrote:
> Sat, May 06, 2017 at 04:40:24PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org wrote:
> > On Sat, 2017-05-06 at 12:40 +0200, Jiri Pirko wrote:
> > > Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org wrote:
> > > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > > > we would like to propose this RDMA tool to be part of iproute2 package
> > > > > and finally improve this situation.
> > > > 
> > > > Although I really appreciate your work: can you clarify why you would like to
> > > > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > > > for adding RDMA functionality to iproute2 in [1].
> > > 
> > > Bart, please realize that iproute2 is much more than "*IP routing* tool".
> > > I understand you got confused by the name. Please see sources. Your comment
> > > is totally pointless...
> > 
> > I asked for a clarification that should have been in the cover letter but that
> > was missing from that cover letter. So I think that was the right thing to do
> 
> I think that was just complete misunderstanding about what iproute2 is.

Hello Jiri,

I do not agree with your reply. The abbreviation "IP" occurs in the package
name and that is a reference to the "Internet Protocol". As far as I know
originally the iproute2 package contained only tools related to the Internet
Protocol. Other tools, e.g. the tipc tool, were added later on. What I'm
wondering about is whether it really is a good idea to add tools like tipc
and rdma to the iproute2 package. The iproute2 package is so essential that
it gets installed on every Linux system, including embedded systems and
smartphones based on Linux. Several companies maintain embedded Linux
distributions and tools to build software images. These tools provide a user
interface that allows to select what packages go into such an image. Adding
tools like tipc and rdma to the iproute2 package makes it harder than
necessary for those who build software images for embedded devices to
minimize the size of such an image. As you probably know even today the size
of a software image still matters for embedded devices. Something else I have
been wondering about is whether bundling the tipc and rdma tools in the
iproute2 package will make the job harder of people who build Android ROMs?
The ip tool is present in every Android ROM, and the size of these ROMs matters
because the larger these ROMs are the less space remains for apps.

Bart.--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Stephen Hemminger]

On Mon, 8 May 2017 15:19:28 +0000
Bart Van Assche <Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org> wrote:

> On Sun, 2017-05-07 at 12:20 +0200, Jiri Pirko wrote:
> > Sat, May 06, 2017 at 04:40:24PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org wrote:  
> > > On Sat, 2017-05-06 at 12:40 +0200, Jiri Pirko wrote:  
> > > > Thu, May 04, 2017 at 08:10:54PM CEST, Bart.VanAssche-XdAiOPVOjttBDgjK7y7TUQ@public.gmane.org wrote:  
> > > > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:  
> > > > > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > > > > we would like to propose this RDMA tool to be part of iproute2 package
> > > > > > and finally improve this situation.  
> > > > > 
> > > > > Although I really appreciate your work: can you clarify why you would like to
> > > > > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > > > > for adding RDMA functionality to iproute2 in [1].  
> > > > 
> > > > Bart, please realize that iproute2 is much more than "*IP routing* tool".
> > > > I understand you got confused by the name. Please see sources. Your comment
> > > > is totally pointless...  
> > > 
> > > I asked for a clarification that should have been in the cover letter but that
> > > was missing from that cover letter. So I think that was the right thing to do  
> > 
> > I think that was just complete misunderstanding about what iproute2 is.  
> 
> Hello Jiri,
> 
> I do not agree with your reply. The abbreviation "IP" occurs in the package
> name and that is a reference to the "Internet Protocol". As far as I know
> originally the iproute2 package contained only tools related to the Internet
> Protocol. Other tools, e.g. the tipc tool, were added later on. What I'm
> wondering about is whether it really is a good idea to add tools like tipc
> and rdma to the iproute2 package. The iproute2 package is so essential that
> it gets installed on every Linux system, including embedded systems and
> smartphones based on Linux. Several companies maintain embedded Linux
> distributions and tools to build software images. These tools provide a user
> interface that allows to select what packages go into such an image. Adding
> tools like tipc and rdma to the iproute2 package makes it harder than
> necessary for those who build software images for embedded devices to
> minimize the size of such an image. As you probably know even today the size
> of a software image still matters for embedded devices. Something else I have
> been wondering about is whether bundling the tipc and rdma tools in the
> iproute2 package will make the job harder of people who build Android ROMs?
> The ip tool is present in every Android ROM, and the size of these ROMs matters
> because the larger these ROMs are the less space remains for apps.
> 
> Bart.

I would assume embedded world does not use the standard distro model of "got to
have them all". The sane way to do builds for embedded is build everything
in the source, but selectively install components based on a Bill Of Materials
file.
--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Dennis Dalessandro]

On 05/04/2017 03:42 PM, Leon Romanovsky wrote:
> On Thu, May 04, 2017 at 03:26:13PM -0400, Dennis Dalessandro wrote:
>> On 05/04/2017 02:45 PM, Leon Romanovsky wrote:
>>> On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:
>>>> On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
>>>>> On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
>>>>>> On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
>>>>>>> Following our discussion both in mailing list [1] and at the LPC 2016 [2],
>>>>>>> we would like to propose this RDMA tool to be part of iproute2 package
>>>>>>> and finally improve this situation.
>>>>>>
>>>>>> Hello Leon,
>>>>>>
>>>>>> Although I really appreciate your work: can you clarify why you would like to
>>>>>> add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
>>>>>> for adding RDMA functionality to iproute2 in [1].
>>>>>
>>>>> We are planning to reuse the same infrastructure provided by iproute2,
>>>>> like netlink parsing, access to distributions, same CLI and same standards.
>>>>>
>>>>> Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
>>>>> Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
>>>>> RDMA.
>>>>>
>>>>> I do expect that iproute2 will be installed on every machine with any
>>>>> type of connection, including IB and OPA.
>>>>>
>>>>> So I think that it is enough to be part of that suite and don't invent
>>>>> our own for one specific tool.
>>>>
>>>> Hello Leon,
>>>>
>>>> Sorry but to me that sounds like a weak argument for including RDMA functionality
>>>> in iproute2. There is already a library for communication over netlink sockets,
>>>> namely libnl. Is there functionality that is in iproute2 but not in libnl and
>>>> that is needed for the new tool? If so, have you considered to create a new
>>>> library for that functionality?
>>>
>>> It is not hard to create new tool, the hardest part is to ensure that it is
>>> part of the distributions. Did you count how many months we are trying to
>>> add rdma-core to debian?
>>
>> I do agree that it is a strange pairing and am not really a fan. However at
>> the end of the day it's just a name for a repo/package. If the iproute folks
>> are fine to include rdma in their repo/package, great we can leverage their
>> code for CLI and other common stuff.
>>
>> Now if the interface was something like "ip -FlagForRdma ..." I would object
>> to that, but the interface is "rdma ... " so from users perspective it's
>> different tools. They don't need to care that it was sourced from a common
>> git repo.
>>
>> Just as an aside this already works a bit with OPA:
>>
>>  $ ./rdma link
>> 1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
>> link_layer InfiniBand
>>          phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
>> state 4: ACTIVE
>>
>> Leon I'll get you more feedback and testing, I've just been really bogged
>> down this week, sorry.
>
> Thanks Denny,
>
> Before you are starting to test it, can you please provide your feedback
> on my initial questions? Usability and need of sysfs.
> ----
> This is initial phase to understand if user experience for this tool fits
> RDMA and netdev communities exepectations. Also I would like to get feedback
> if it is really worth to provide legacy sysfs for old kernels, or maybe I should
> implement netlink from the beginning and abandon sysfs completely.
> -----

For the initial phase I think you did the right thing by reading sysfs. 
I like the ability for the tool to be compatible with legacy kernels, 
but at the same time I don't know if it's worth the hassle. I won't 
fight too hard either way.

Perhaps we should take a stab and seeing what a dual sysfs/netlink 
interface would look like, and see just how hard and complicated it 
really is. Then we can make that call. You already have the sysfs 
version, and have to do netink anyway, so let's not rip out what's there 
just yet, this is an RFC after all, not like you are asking for this 
exact version to be merged yet.

As far as usability, I think what's here is a great start and we can 
continue to refine. I'm particularly interested in the stats 
capabilities. In particular being able to filter what stats are shown 
and watch as they change over time. RDMA devices have lots of counters 
and stats. A common tool for users to be able to get that data across HW 
types would be a really good thing.

-Denny

















--
To unsubscribe from this list: send the line "unsubscribe linux-rdma" in
the body of a message to majordomo-u79uwXL29TY76Z2rM5mHXA@public.gmane.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [RFC iproute2 0/8] RDMA tool

[Andrew Lunn]

> Several companies maintain embedded Linux
> distributions and tools to build software images. These tools provide a user
> interface that allows to select what packages go into such an image.

The tools allow you to select what binary packages are placed into the
image. You can build multiple binary packages from one source package.
Desktop distributions are not likely to do this for something as small
as iproute2. But embedded distributions can easily break up iproute2
into a number of smaller packages, as you suggested, tipc, devlink,
tc, bridge, ss, etc.

Openwrt does exactly this:

https://github.com/openwrt-mirror/openwrt/blob/master/package/network/utils/iproute2/Makefile

    Andrew

Re: [RFC iproute2 0/8] RDMA tool

[Leon Romanovsky]

[-- Attachment #1: Type: text/plain, Size: 5043 bytes --]

On Mon, May 08, 2017 at 11:55:25AM -0400, Dennis Dalessandro wrote:
> On 05/04/2017 03:42 PM, Leon Romanovsky wrote:
> > On Thu, May 04, 2017 at 03:26:13PM -0400, Dennis Dalessandro wrote:
> > > On 05/04/2017 02:45 PM, Leon Romanovsky wrote:
> > > > On Thu, May 04, 2017 at 06:30:27PM +0000, Bart Van Assche wrote:
> > > > > On Thu, 2017-05-04 at 21:25 +0300, Leon Romanovsky wrote:
> > > > > > On Thu, May 04, 2017 at 06:10:54PM +0000, Bart Van Assche wrote:
> > > > > > > On Thu, 2017-05-04 at 21:02 +0300, Leon Romanovsky wrote:
> > > > > > > > Following our discussion both in mailing list [1] and at the LPC 2016 [2],
> > > > > > > > we would like to propose this RDMA tool to be part of iproute2 package
> > > > > > > > and finally improve this situation.
> > > > > > >
> > > > > > > Hello Leon,
> > > > > > >
> > > > > > > Although I really appreciate your work: can you clarify why you would like to
> > > > > > > add *RDMA* functionality to an *IP routing* tool? I haven't found any motivation
> > > > > > > for adding RDMA functionality to iproute2 in [1].
> > > > > >
> > > > > > We are planning to reuse the same infrastructure provided by iproute2,
> > > > > > like netlink parsing, access to distributions, same CLI and same standards.
> > > > > >
> > > > > > Right now, RDMA is already tightened to netdev: iWARP, RoCE, IPoIB, HFI-VNIC.
> > > > > > Many drivers (mlx, qed, i40, cxgb) are sharing code between net and
> > > > > > RDMA.
> > > > > >
> > > > > > I do expect that iproute2 will be installed on every machine with any
> > > > > > type of connection, including IB and OPA.
> > > > > >
> > > > > > So I think that it is enough to be part of that suite and don't invent
> > > > > > our own for one specific tool.
> > > > >
> > > > > Hello Leon,
> > > > >
> > > > > Sorry but to me that sounds like a weak argument for including RDMA functionality
> > > > > in iproute2. There is already a library for communication over netlink sockets,
> > > > > namely libnl. Is there functionality that is in iproute2 but not in libnl and
> > > > > that is needed for the new tool? If so, have you considered to create a new
> > > > > library for that functionality?
> > > >
> > > > It is not hard to create new tool, the hardest part is to ensure that it is
> > > > part of the distributions. Did you count how many months we are trying to
> > > > add rdma-core to debian?
> > >
> > > I do agree that it is a strange pairing and am not really a fan. However at
> > > the end of the day it's just a name for a repo/package. If the iproute folks
> > > are fine to include rdma in their repo/package, great we can leverage their
> > > code for CLI and other common stuff.
> > >
> > > Now if the interface was something like "ip -FlagForRdma ..." I would object
> > > to that, but the interface is "rdma ... " so from users perspective it's
> > > different tools. They don't need to care that it was sourced from a common
> > > git repo.
> > >
> > > Just as an aside this already works a bit with OPA:
> > >
> > >  $ ./rdma link
> > > 1/1: hfi1_0/1: ifname NONE cap_mask 0x00410022 lid 0x1 lid_mask_count 0
> > > link_layer InfiniBand
> > >          phys_state 5: LinkUp rate 100 Gb/sec (4X EDR) sm_lid 0x1 sm_sl 0
> > > state 4: ACTIVE
> > >
> > > Leon I'll get you more feedback and testing, I've just been really bogged
> > > down this week, sorry.
> >
> > Thanks Denny,
> >
> > Before you are starting to test it, can you please provide your feedback
> > on my initial questions? Usability and need of sysfs.
> > ----
> > This is initial phase to understand if user experience for this tool fits
> > RDMA and netdev communities exepectations. Also I would like to get feedback
> > if it is really worth to provide legacy sysfs for old kernels, or maybe I should
> > implement netlink from the beginning and abandon sysfs completely.
> > -----
>
> For the initial phase I think you did the right thing by reading sysfs. I
> like the ability for the tool to be compatible with legacy kernels, but at
> the same time I don't know if it's worth the hassle. I won't fight too hard
> either way.
>
> Perhaps we should take a stab and seeing what a dual sysfs/netlink interface
> would look like, and see just how hard and complicated it really is. Then we
> can make that call. You already have the sysfs version, and have to do
> netink anyway, so let's not rip out what's there just yet, this is an RFC
> after all, not like you are asking for this exact version to be merged yet.

Not at all, it is too early to merge it, at the end it is POC.

>
> As far as usability, I think what's here is a great start and we can
> continue to refine. I'm particularly interested in the stats capabilities.
> In particular being able to filter what stats are shown and watch as they
> change over time. RDMA devices have lots of counters and stats. A common
> tool for users to be able to get that data across HW types would be a really
> good thing.

Yeah, the statistics will be tough nut to crack.


>
> -Denny
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 833 bytes --]