[PATCH net-next 00/12] tools: bpftool: Add JSON output to bpftool

[PATCH net-next 00/12] tools: bpftool: Add JSON output to bpftool

[Jakub Kicinski]

Quentin says:

This series introduces support for JSON output to all bpftool commands. It
adds option parsing, and several options are created:

  * -j, --json     Switch to JSON output.
  * -p, --pretty   Switch to JSON and print it in a human-friendly fashion.
  * -h, --help     Print generic help message.
  * -V, --version  Print version number.

This code uses a "json_writer", which is a copy of the one written by
Stephen Hemminger in iproute2.
---
I don't know if there is an easy way to share the code for json_write
without copying the file, so I am very open to suggestions on this matter.


Quentin Monnet (12):
  tools: bpftool: copy JSON writer from iproute2 repository
  tools: bpftool: add option parsing to bpftool, --help and --version
  tools: bpftool: introduce --json and --pretty options
  tools: bpftool: add JSON output for `bpftool prog show *` command
  tools: bpftool: add JSON output for `bpftool prog dump jited *`
    command
  tools: bpftool: add JSON output for `bpftool prog dump xlated *`
    command
  tools: bpftool: add JSON output for `bpftool map *` commands
  tools: bpftool: add JSON output for `bpftool batch file FILE` command
  tools: bpftool: turn err() and info() macros into functions
  tools: bpftool: provide JSON output for all possible commands
  tools: bpftool: add cosmetic changes for the manual pages
  tools: bpftool: update documentation for --json and --pretty usage

 tools/bpf/bpftool/Documentation/bpftool-map.rst  |  44 ++-
 tools/bpf/bpftool/Documentation/bpftool-prog.rst |  81 +++++-
 tools/bpf/bpftool/Documentation/bpftool.rst      |  30 +-
 tools/bpf/bpftool/common.c                       |  36 ++-
 tools/bpf/bpftool/jit_disasm.c                   |  86 +++++-
 tools/bpf/bpftool/json_writer.c                  | 356 +++++++++++++++++++++++
 tools/bpf/bpftool/json_writer.h                  |  72 +++++
 tools/bpf/bpftool/main.c                         | 121 +++++++-
 tools/bpf/bpftool/main.h                         |  43 ++-
 tools/bpf/bpftool/map.c                          | 253 ++++++++++++----
 tools/bpf/bpftool/prog.c                         | 271 +++++++++++++----
 11 files changed, 1209 insertions(+), 184 deletions(-)
 create mode 100644 tools/bpf/bpftool/json_writer.c
 create mode 100644 tools/bpf/bpftool/json_writer.h

-- 
2.14.1

[PATCH net-next 01/12] tools: bpftool: copy JSON writer from iproute2 repository

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

In prevision of following commits, supposed to add JSON output to the
tool, two files are copied from the iproute2 repository (taken at commit
268a9eee985f): lib/json_writer.c and include/json_writer.h.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/json_writer.c | 348 ++++++++++++++++++++++++++++++++++++++++
 tools/bpf/bpftool/json_writer.h |  70 ++++++++
 2 files changed, 418 insertions(+)
 create mode 100644 tools/bpf/bpftool/json_writer.c
 create mode 100644 tools/bpf/bpftool/json_writer.h

diff --git a/tools/bpf/bpftool/json_writer.c b/tools/bpf/bpftool/json_writer.c
new file mode 100644
index 000000000000..6b77d288cce2
--- /dev/null
+++ b/tools/bpf/bpftool/json_writer.c
@@ -0,0 +1,348 @@
+/*
+ * Simple streaming JSON writer
+ *
+ * This takes care of the annoying bits of JSON syntax like the commas
+ * after elements
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ * Authors:	Stephen Hemminger <stephen@networkplumber.org>
+ */
+
+#include <stdio.h>
+#include <stdbool.h>
+#include <stdarg.h>
+#include <assert.h>
+#include <malloc.h>
+#include <inttypes.h>
+#include <stdint.h>
+
+#include "json_writer.h"
+
+struct json_writer {
+	FILE		*out;	/* output file */
+	unsigned	depth;  /* nesting */
+	bool		pretty; /* optional whitepace */
+	char		sep;	/* either nul or comma */
+};
+
+/* indentation for pretty print */
+static void jsonw_indent(json_writer_t *self)
+{
+	unsigned i;
+	for (i = 0; i < self->depth; ++i)
+		fputs("    ", self->out);
+}
+
+/* end current line and indent if pretty printing */
+static void jsonw_eol(json_writer_t *self)
+{
+	if (!self->pretty)
+		return;
+
+	putc('\n', self->out);
+	jsonw_indent(self);
+}
+
+/* If current object is not empty print a comma */
+static void jsonw_eor(json_writer_t *self)
+{
+	if (self->sep != '\0')
+		putc(self->sep, self->out);
+	self->sep = ',';
+}
+
+
+/* Output JSON encoded string */
+/* Handles C escapes, does not do Unicode */
+static void jsonw_puts(json_writer_t *self, const char *str)
+{
+	putc('"', self->out);
+	for (; *str; ++str)
+		switch (*str) {
+		case '\t':
+			fputs("\\t", self->out);
+			break;
+		case '\n':
+			fputs("\\n", self->out);
+			break;
+		case '\r':
+			fputs("\\r", self->out);
+			break;
+		case '\f':
+			fputs("\\f", self->out);
+			break;
+		case '\b':
+			fputs("\\b", self->out);
+			break;
+		case '\\':
+			fputs("\\n", self->out);
+			break;
+		case '"':
+			fputs("\\\"", self->out);
+			break;
+		case '\'':
+			fputs("\\\'", self->out);
+			break;
+		default:
+			putc(*str, self->out);
+		}
+	putc('"', self->out);
+}
+
+/* Create a new JSON stream */
+json_writer_t *jsonw_new(FILE *f)
+{
+	json_writer_t *self = malloc(sizeof(*self));
+	if (self) {
+		self->out = f;
+		self->depth = 0;
+		self->pretty = false;
+		self->sep = '\0';
+	}
+	return self;
+}
+
+/* End output to JSON stream */
+void jsonw_destroy(json_writer_t **self_p)
+{
+	json_writer_t *self = *self_p;
+
+	assert(self->depth == 0);
+	fputs("\n", self->out);
+	fflush(self->out);
+	free(self);
+	*self_p = NULL;
+}
+
+void jsonw_pretty(json_writer_t *self, bool on)
+{
+	self->pretty = on;
+}
+
+/* Basic blocks */
+static void jsonw_begin(json_writer_t *self, int c)
+{
+	jsonw_eor(self);
+	putc(c, self->out);
+	++self->depth;
+	self->sep = '\0';
+}
+
+static void jsonw_end(json_writer_t *self, int c)
+{
+	assert(self->depth > 0);
+
+	--self->depth;
+	if (self->sep != '\0')
+		jsonw_eol(self);
+	putc(c, self->out);
+	self->sep = ',';
+}
+
+
+/* Add a JSON property name */
+void jsonw_name(json_writer_t *self, const char *name)
+{
+	jsonw_eor(self);
+	jsonw_eol(self);
+	self->sep = '\0';
+	jsonw_puts(self, name);
+	putc(':', self->out);
+	if (self->pretty)
+		putc(' ', self->out);
+}
+
+void jsonw_printf(json_writer_t *self, const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	jsonw_eor(self);
+	vfprintf(self->out, fmt, ap);
+	va_end(ap);
+}
+
+/* Collections */
+void jsonw_start_object(json_writer_t *self)
+{
+	jsonw_begin(self, '{');
+}
+
+void jsonw_end_object(json_writer_t *self)
+{
+	jsonw_end(self, '}');
+}
+
+void jsonw_start_array(json_writer_t *self)
+{
+	jsonw_begin(self, '[');
+}
+
+void jsonw_end_array(json_writer_t *self)
+{
+	jsonw_end(self, ']');
+}
+
+/* JSON value types */
+void jsonw_string(json_writer_t *self, const char *value)
+{
+	jsonw_eor(self);
+	jsonw_puts(self, value);
+}
+
+void jsonw_bool(json_writer_t *self, bool val)
+{
+	jsonw_printf(self, "%s", val ? "true" : "false");
+}
+
+void jsonw_null(json_writer_t *self)
+{
+	jsonw_printf(self, "null");
+}
+
+void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num)
+{
+	jsonw_printf(self, fmt, num);
+}
+
+#ifdef notused
+void jsonw_float(json_writer_t *self, double num)
+{
+	jsonw_printf(self, "%g", num);
+}
+#endif
+
+void jsonw_hu(json_writer_t *self, unsigned short num)
+{
+	jsonw_printf(self, "%hu", num);
+}
+
+void jsonw_uint(json_writer_t *self, uint64_t num)
+{
+	jsonw_printf(self, "%"PRIu64, num);
+}
+
+void jsonw_lluint(json_writer_t *self, unsigned long long int num)
+{
+	jsonw_printf(self, "%llu", num);
+}
+
+void jsonw_int(json_writer_t *self, int64_t num)
+{
+	jsonw_printf(self, "%"PRId64, num);
+}
+
+/* Basic name/value objects */
+void jsonw_string_field(json_writer_t *self, const char *prop, const char *val)
+{
+	jsonw_name(self, prop);
+	jsonw_string(self, val);
+}
+
+void jsonw_bool_field(json_writer_t *self, const char *prop, bool val)
+{
+	jsonw_name(self, prop);
+	jsonw_bool(self, val);
+}
+
+#ifdef notused
+void jsonw_float_field(json_writer_t *self, const char *prop, double val)
+{
+	jsonw_name(self, prop);
+	jsonw_float(self, val);
+}
+#endif
+
+void jsonw_float_field_fmt(json_writer_t *self,
+			   const char *prop,
+			   const char *fmt,
+			   double val)
+{
+	jsonw_name(self, prop);
+	jsonw_float_fmt(self, fmt, val);
+}
+
+void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num)
+{
+	jsonw_name(self, prop);
+	jsonw_uint(self, num);
+}
+
+void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num)
+{
+	jsonw_name(self, prop);
+	jsonw_hu(self, num);
+}
+
+void jsonw_lluint_field(json_writer_t *self,
+			const char *prop,
+			unsigned long long int num)
+{
+	jsonw_name(self, prop);
+	jsonw_lluint(self, num);
+}
+
+void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num)
+{
+	jsonw_name(self, prop);
+	jsonw_int(self, num);
+}
+
+void jsonw_null_field(json_writer_t *self, const char *prop)
+{
+	jsonw_name(self, prop);
+	jsonw_null(self);
+}
+
+#ifdef TEST
+int main(int argc, char **argv)
+{
+	json_writer_t *wr = jsonw_new(stdout);
+
+	jsonw_start_object(wr);
+	jsonw_pretty(wr, true);
+	jsonw_name(wr, "Vyatta");
+	jsonw_start_object(wr);
+	jsonw_string_field(wr, "url", "http://vyatta.com");
+	jsonw_uint_field(wr, "downloads", 2000000ul);
+	jsonw_float_field(wr, "stock", 8.16);
+
+	jsonw_name(wr, "ARGV");
+	jsonw_start_array(wr);
+	while (--argc)
+		jsonw_string(wr, *++argv);
+	jsonw_end_array(wr);
+
+	jsonw_name(wr, "empty");
+	jsonw_start_array(wr);
+	jsonw_end_array(wr);
+
+	jsonw_name(wr, "NIL");
+	jsonw_start_object(wr);
+	jsonw_end_object(wr);
+
+	jsonw_null_field(wr, "my_null");
+
+	jsonw_name(wr, "special chars");
+	jsonw_start_array(wr);
+	jsonw_string_field(wr, "slash", "/");
+	jsonw_string_field(wr, "newline", "\n");
+	jsonw_string_field(wr, "tab", "\t");
+	jsonw_string_field(wr, "ff", "\f");
+	jsonw_string_field(wr, "quote", "\"");
+	jsonw_string_field(wr, "tick", "\'");
+	jsonw_string_field(wr, "backslash", "\\");
+	jsonw_end_array(wr);
+
+	jsonw_end_object(wr);
+
+	jsonw_end_object(wr);
+	jsonw_destroy(&wr);
+	return 0;
+}
+
+#endif
diff --git a/tools/bpf/bpftool/json_writer.h b/tools/bpf/bpftool/json_writer.h
new file mode 100644
index 000000000000..1516aafba59d
--- /dev/null
+++ b/tools/bpf/bpftool/json_writer.h
@@ -0,0 +1,70 @@
+/*
+ * Simple streaming JSON writer
+ *
+ * This takes care of the annoying bits of JSON syntax like the commas
+ * after elements
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ *
+ * Authors:	Stephen Hemminger <stephen@networkplumber.org>
+ */
+
+#ifndef _JSON_WRITER_H_
+#define _JSON_WRITER_H_
+
+#include <stdbool.h>
+#include <stdint.h>
+
+/* Opaque class structure */
+typedef struct json_writer json_writer_t;
+
+/* Create a new JSON stream */
+json_writer_t *jsonw_new(FILE *f);
+/* End output to JSON stream */
+void jsonw_destroy(json_writer_t **self_p);
+
+/* Cause output to have pretty whitespace */
+void jsonw_pretty(json_writer_t *self, bool on);
+
+/* Add property name */
+void jsonw_name(json_writer_t *self, const char *name);
+
+/* Add value  */
+void jsonw_printf(json_writer_t *self, const char *fmt, ...);
+void jsonw_string(json_writer_t *self, const char *value);
+void jsonw_bool(json_writer_t *self, bool value);
+void jsonw_float(json_writer_t *self, double number);
+void jsonw_float_fmt(json_writer_t *self, const char *fmt, double num);
+void jsonw_uint(json_writer_t *self, uint64_t number);
+void jsonw_hu(json_writer_t *self, unsigned short number);
+void jsonw_int(json_writer_t *self, int64_t number);
+void jsonw_null(json_writer_t *self);
+void jsonw_lluint(json_writer_t *self, unsigned long long int num);
+
+/* Useful Combinations of name and value */
+void jsonw_string_field(json_writer_t *self, const char *prop, const char *val);
+void jsonw_bool_field(json_writer_t *self, const char *prop, bool value);
+void jsonw_float_field(json_writer_t *self, const char *prop, double num);
+void jsonw_uint_field(json_writer_t *self, const char *prop, uint64_t num);
+void jsonw_hu_field(json_writer_t *self, const char *prop, unsigned short num);
+void jsonw_int_field(json_writer_t *self, const char *prop, int64_t num);
+void jsonw_null_field(json_writer_t *self, const char *prop);
+void jsonw_lluint_field(json_writer_t *self, const char *prop,
+			unsigned long long int num);
+void jsonw_float_field_fmt(json_writer_t *self, const char *prop,
+			   const char *fmt, double val);
+
+/* Collections */
+void jsonw_start_object(json_writer_t *self);
+void jsonw_end_object(json_writer_t *self);
+
+void jsonw_start_array(json_writer_t *self);
+void jsonw_end_array(json_writer_t *self);
+
+/* Override default exception handling */
+typedef void (jsonw_err_handler_fn)(const char *);
+
+#endif /* _JSON_WRITER_H_ */
-- 
2.14.1

Re: [PATCH net-next 01/12] tools: bpftool: copy JSON writer from iproute2 repository

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> In prevision of following commits, supposed to add JSON output to the
> tool, two files are copied from the iproute2 repository (taken at commit
> 268a9eee985f): lib/json_writer.c and include/json_writer.h.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

Re: [PATCH net-next 01/12] tools: bpftool: copy JSON writer from iproute2 repository

[Stephen Hemminger]

On Mon, 23 Oct 2017 09:24:05 -0700
Jakub Kicinski <jakub.kicinski@netronome.com> wrote:

> From: Quentin Monnet <quentin.monnet@netronome.com>
> 
> In prevision of following commits, supposed to add JSON output to the
> tool, two files are copied from the iproute2 repository (taken at commit
> 268a9eee985f): lib/json_writer.c and include/json_writer.h.
> 
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Stephen Hemminger <stephen@networkplumber.org>

[PATCH net-next 02/12] tools: bpftool: add option parsing to bpftool, --help and --version

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Add an option parsing facility to bpftool, in prevision of future
options for demanding JSON output. Currently, two options are added:
--help and --version, that act the same as the respective commands
`help` and `version`.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/Documentation/bpftool-map.rst  |  8 +++++++
 tools/bpf/bpftool/Documentation/bpftool-prog.rst |  8 +++++++
 tools/bpf/bpftool/Documentation/bpftool.rst      |  8 +++++++
 tools/bpf/bpftool/main.c                         | 27 +++++++++++++++++++++++-
 4 files changed, 50 insertions(+), 1 deletion(-)

diff --git a/tools/bpf/bpftool/Documentation/bpftool-map.rst b/tools/bpf/bpftool/Documentation/bpftool-map.rst
index ff63e89e4b6c..5210c4fab356 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-map.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-map.rst
@@ -68,6 +68,14 @@ DESCRIPTION
 	**bpftool map help**
 		  Print short help message.
 
+OPTIONS
+=======
+	-h, --help
+		  Print short generic help message (similar to **bpftool help**).
+
+	-v, --version
+		  Print version number (similar to **bpftool version**).
+
 EXAMPLES
 ========
 **# bpftool map show**
diff --git a/tools/bpf/bpftool/Documentation/bpftool-prog.rst b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
index 69b3770370c8..6620a81d9dc9 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-prog.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
@@ -50,6 +50,14 @@ DESCRIPTION
 	**bpftool prog help**
 		  Print short help message.
 
+OPTIONS
+=======
+	-h, --help
+		  Print short generic help message (similar to **bpftool help**).
+
+	-v, --version
+		  Print version number (similar to **bpftool version**).
+
 EXAMPLES
 ========
 **# bpftool prog show**
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index 45ad8baf1915..9c04cd6677bd 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -31,6 +31,14 @@ DESCRIPTION
 	Note that format of the output of all tools is not guaranteed to be
 	stable and should not be depended upon.
 
+OPTIONS
+=======
+	-h, --help
+		  Print short help message (similar to **bpftool help**).
+
+	-v, --version
+		  Print version number (similar to **bpftool version**).
+
 SEE ALSO
 ========
 	**bpftool-map**\ (8), **bpftool-prog**\ (8)
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 814d19e1b53f..613e3c75f78a 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -36,6 +36,7 @@
 #include <bfd.h>
 #include <ctype.h>
 #include <errno.h>
+#include <getopt.h>
 #include <linux/bpf.h>
 #include <linux/version.h>
 #include <stdio.h>
@@ -215,8 +216,32 @@ static int do_batch(int argc, char **argv)
 
 int main(int argc, char **argv)
 {
+	static const struct option options[] = {
+		{ "help",	no_argument,	NULL,	'h' },
+		{ "version",	no_argument,	NULL,	'V' },
+		{ 0 }
+	};
+	int opt;
+
+	last_do_help = do_help;
 	bin_name = argv[0];
-	NEXT_ARG();
+
+	while ((opt = getopt_long(argc, argv, "Vh",
+				  options, NULL)) >= 0) {
+		switch (opt) {
+		case 'V':
+			return do_version(argc, argv);
+		case 'h':
+			return do_help(argc, argv);
+		default:
+			usage();
+		}
+	}
+
+	argc -= optind;
+	argv += optind;
+	if (argc < 0)
+		usage();
 
 	bfd_init();
 
-- 
2.14.1

Re: [PATCH net-next 02/12] tools: bpftool: add option parsing to bpftool, --help and --version

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Add an option parsing facility to bpftool, in prevision of future
> options for demanding JSON output. Currently, two options are added:
> --help and --version, that act the same as the respective commands
> `help` and `version`.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 03/12] tools: bpftool: introduce --json and --pretty options

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

These two options can be used to ask for a JSON output (--j or -json),
and to make this JSON human-readable (-p or --pretty).

A json_writer object is created when JSON is required, and will be used
in follow-up commits to produce JSON output.

Note that --pretty implies --json.

Update for the manual pages and interactive help messages comes in a
later patch of the series.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/main.c | 33 ++++++++++++++++++++++++++++++---
 tools/bpf/bpftool/main.h |  5 +++++
 2 files changed, 35 insertions(+), 3 deletions(-)

diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 613e3c75f78a..14bfc17cd4de 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -51,6 +51,9 @@ const char *bin_name;
 static int last_argc;
 static char **last_argv;
 static int (*last_do_help)(int argc, char **argv);
+json_writer_t *json_wtr;
+bool pretty_output;
+bool json_output;
 
 void usage(void)
 {
@@ -217,22 +220,32 @@ static int do_batch(int argc, char **argv)
 int main(int argc, char **argv)
 {
 	static const struct option options[] = {
+		{ "json",	no_argument,	NULL,	'j' },
 		{ "help",	no_argument,	NULL,	'h' },
+		{ "pretty",	no_argument,	NULL,	'p' },
 		{ "version",	no_argument,	NULL,	'V' },
 		{ 0 }
 	};
-	int opt;
+	int opt, ret;
 
 	last_do_help = do_help;
+	pretty_output = false;
+	json_output = false;
 	bin_name = argv[0];
 
-	while ((opt = getopt_long(argc, argv, "Vh",
+	while ((opt = getopt_long(argc, argv, "Vhpj",
 				  options, NULL)) >= 0) {
 		switch (opt) {
 		case 'V':
 			return do_version(argc, argv);
 		case 'h':
 			return do_help(argc, argv);
+		case 'p':
+			pretty_output = true;
+			/* fall through */
+		case 'j':
+			json_output = true;
+			break;
 		default:
 			usage();
 		}
@@ -243,7 +256,21 @@ int main(int argc, char **argv)
 	if (argc < 0)
 		usage();
 
+	if (json_output) {
+		json_wtr = jsonw_new(stdout);
+		if (!json_wtr) {
+			err("failed to create JSON writer\n");
+			return -1;
+		}
+		jsonw_pretty(json_wtr, pretty_output);
+	}
+
 	bfd_init();
 
-	return cmd_select(cmds, argc, argv, do_help);
+	ret = cmd_select(cmds, argc, argv, do_help);
+
+	if (json_output)
+		jsonw_destroy(&json_wtr);
+
+	return ret;
 }
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 41e6c7d3fcad..15927fc9fb31 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -43,6 +43,8 @@
 #include <linux/bpf.h>
 #include <linux/kernel.h>
 
+#include "json_writer.h"
+
 #define err(msg...)	fprintf(stderr, "Error: " msg)
 #define warn(msg...)	fprintf(stderr, "Warning: " msg)
 #define info(msg...)	fprintf(stderr, msg)
@@ -66,6 +68,9 @@ enum bpf_obj_type {
 
 extern const char *bin_name;
 
+extern json_writer_t *json_wtr;
+extern bool json_output;
+
 bool is_prefix(const char *pfx, const char *str);
 void fprint_hex(FILE *f, void *arg, unsigned int n, const char *sep);
 void usage(void) __attribute__((noreturn));
-- 
2.14.1

Re: [PATCH net-next 03/12] tools: bpftool: introduce --json and --pretty options

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> These two options can be used to ask for a JSON output (--j or -json),
> and to make this JSON human-readable (-p or --pretty).
>
> A json_writer object is created when JSON is required, and will be used
> in follow-up commits to produce JSON output.
>
> Note that --pretty implies --json.
>
> Update for the manual pages and interactive help messages comes in a
> later patch of the series.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 04/12] tools: bpftool: add JSON output for `bpftool prog show *` command

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Reuse the json_writer API introduced in an earlier commit to make
bpftool able to generate JSON output on `bpftool prog show *` commands.

For readability, the code from show_prog() has been split into two
functions, one for plain output, one for JSON.

Outputs from sample programs have been successfully tested against a
JSON validator.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/prog.c | 139 ++++++++++++++++++++++++++++++++++++-----------
 1 file changed, 107 insertions(+), 32 deletions(-)

diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 7838206a455b..f373f2baef5a 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -195,51 +195,100 @@ static void show_prog_maps(int fd, u32 num_maps)
 	if (err || !info.nr_map_ids)
 		return;
 
-	printf("  map_ids ");
-	for (i = 0; i < info.nr_map_ids; i++)
-		printf("%u%s", map_ids[i],
-		       i == info.nr_map_ids - 1 ? "" : ",");
+	if (json_output) {
+		jsonw_name(json_wtr, "map_ids");
+		jsonw_start_array(json_wtr);
+		for (i = 0; i < info.nr_map_ids; i++)
+			jsonw_uint(json_wtr, map_ids[i]);
+		jsonw_end_array(json_wtr);
+	} else {
+		printf("  map_ids ");
+		for (i = 0; i < info.nr_map_ids; i++)
+			printf("%u%s", map_ids[i],
+			       i == info.nr_map_ids - 1 ? "" : ",");
+	}
 }
 
-static int show_prog(int fd)
+static void print_prog_json(struct bpf_prog_info *info, int fd)
 {
-	struct bpf_prog_info info = {};
-	__u32 len = sizeof(info);
 	char *memlock;
-	int err;
 
-	err = bpf_obj_get_info_by_fd(fd, &info, &len);
-	if (err) {
-		err("can't get prog info: %s\n", strerror(errno));
-		return -1;
+	jsonw_start_object(json_wtr);
+	jsonw_uint_field(json_wtr, "id", info->id);
+	if (info->type < ARRAY_SIZE(prog_type_name))
+		jsonw_string_field(json_wtr, "type",
+				   prog_type_name[info->type]);
+	else
+		jsonw_uint_field(json_wtr, "type", info->type);
+
+	if (*info->name)
+		jsonw_string_field(json_wtr, "name", info->name);
+
+	jsonw_name(json_wtr, "tag");
+	jsonw_printf(json_wtr, "\"" BPF_TAG_FMT "\"",
+		     info->tag[0], info->tag[1], info->tag[2], info->tag[3],
+		     info->tag[4], info->tag[5], info->tag[6], info->tag[7]);
+
+	if (info->load_time) {
+		char buf[32];
+
+		print_boot_time(info->load_time, buf, sizeof(buf));
+
+		/* Piggy back on load_time, since 0 uid is a valid one */
+		jsonw_string_field(json_wtr, "loaded_at", buf);
+		jsonw_uint_field(json_wtr, "uid", info->created_by_uid);
 	}
 
-	printf("%u: ", info.id);
-	if (info.type < ARRAY_SIZE(prog_type_name))
-		printf("%s  ", prog_type_name[info.type]);
+	jsonw_uint_field(json_wtr, "bytes_xlated", info->xlated_prog_len);
+
+	if (info->jited_prog_len) {
+		jsonw_bool_field(json_wtr, "jited", true);
+		jsonw_uint_field(json_wtr, "bytes_jited", info->jited_prog_len);
+	} else {
+		jsonw_bool_field(json_wtr, "jited", false);
+	}
+
+	memlock = get_fdinfo(fd, "memlock");
+	if (memlock)
+		jsonw_int_field(json_wtr, "bytes_memlock", atoi(memlock));
+	free(memlock);
+
+	if (info->nr_map_ids)
+		show_prog_maps(fd, info->nr_map_ids);
+
+	jsonw_end_object(json_wtr);
+}
+
+static void print_prog_plain(struct bpf_prog_info *info, int fd)
+{
+	char *memlock;
+
+	printf("%u: ", info->id);
+	if (info->type < ARRAY_SIZE(prog_type_name))
+		printf("%s  ", prog_type_name[info->type]);
 	else
-		printf("type %u  ", info.type);
+		printf("type %u  ", info->type);
 
-	if (*info.name)
-		printf("name %s  ", info.name);
+	if (*info->name)
+		printf("name %s  ", info->name);
 
 	printf("tag ");
-	fprint_hex(stdout, info.tag, BPF_TAG_SIZE, "");
+	fprint_hex(stdout, info->tag, BPF_TAG_SIZE, "");
 	printf("\n");
 
-	if (info.load_time) {
+	if (info->load_time) {
 		char buf[32];
 
-		print_boot_time(info.load_time, buf, sizeof(buf));
+		print_boot_time(info->load_time, buf, sizeof(buf));
 
 		/* Piggy back on load_time, since 0 uid is a valid one */
-		printf("\tloaded_at %s  uid %u\n", buf, info.created_by_uid);
+		printf("\tloaded_at %s  uid %u\n", buf, info->created_by_uid);
 	}
 
-	printf("\txlated %uB", info.xlated_prog_len);
+	printf("\txlated %uB", info->xlated_prog_len);
 
-	if (info.jited_prog_len)
-		printf("  jited %uB", info.jited_prog_len);
+	if (info->jited_prog_len)
+		printf("  jited %uB", info->jited_prog_len);
 	else
 		printf("  not jited");
 
@@ -248,16 +297,35 @@ static int show_prog(int fd)
 		printf("  memlock %sB", memlock);
 	free(memlock);
 
-	if (info.nr_map_ids)
-		show_prog_maps(fd, info.nr_map_ids);
+	if (info->nr_map_ids)
+		show_prog_maps(fd, info->nr_map_ids);
 
 	printf("\n");
+}
+
+static int show_prog(int fd)
+{
+	struct bpf_prog_info info = {};
+	__u32 len = sizeof(info);
+	int err;
+
+	err = bpf_obj_get_info_by_fd(fd, &info, &len);
+	if (err) {
+		err("can't get prog info: %s\n", strerror(errno));
+		return -1;
+	}
+
+	if (json_output)
+		print_prog_json(&info, fd);
+	else
+		print_prog_plain(&info, fd);
 
 	return 0;
 }
 
 static int do_show(int argc, char **argv)
-{	__u32 id = 0;
+{
+	__u32 id = 0;
 	int err;
 	int fd;
 
@@ -272,6 +340,8 @@ static int do_show(int argc, char **argv)
 	if (argc)
 		return BAD_ARG();
 
+	if (json_output)
+		jsonw_start_array(json_wtr);
 	while (true) {
 		err = bpf_prog_get_next_id(id, &id);
 		if (err) {
@@ -282,23 +352,28 @@ static int do_show(int argc, char **argv)
 			err("can't get next program: %s\n", strerror(errno));
 			if (errno == EINVAL)
 				err("kernel too old?\n");
-			return -1;
+			err = -1;
+			break;
 		}
 
 		fd = bpf_prog_get_fd_by_id(id);
 		if (fd < 0) {
 			err("can't get prog by id (%u): %s\n",
 			    id, strerror(errno));
-			return -1;
+			err = -1;
+			break;
 		}
 
 		err = show_prog(fd);
 		close(fd);
 		if (err)
-			return err;
+			break;
 	}
 
-	return 0;
+	if (json_output)
+		jsonw_end_array(json_wtr);
+
+	return err;
 }
 
 static void print_insn(struct bpf_verifier_env *env, const char *fmt, ...)
-- 
2.14.1

Re: [PATCH net-next 04/12] tools: bpftool: add JSON output for `bpftool prog show *` command

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Reuse the json_writer API introduced in an earlier commit to make
> bpftool able to generate JSON output on `bpftool prog show *` commands.
>
> For readability, the code from show_prog() has been split into two
> functions, one for plain output, one for JSON.
>
> Outputs from sample programs have been successfully tested against a
> JSON validator.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 05/12] tools: bpftool: add JSON output for `bpftool prog dump jited *` command

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Reuse the json_writer API introduced in an earlier commit to make
bpftool able to generate JSON output on `bpftool prog show *` commands.
A new printing function is created to be passed as an argument to the
disassembler.

Similarly to plain output, opcodes are printed on request.

Outputs from sample programs have been successfully tested against a
JSON validator.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/jit_disasm.c | 86 +++++++++++++++++++++++++++++++++++++++---
 1 file changed, 80 insertions(+), 6 deletions(-)

diff --git a/tools/bpf/bpftool/jit_disasm.c b/tools/bpf/bpftool/jit_disasm.c
index 70e480b59e9d..5937e134e408 100644
--- a/tools/bpf/bpftool/jit_disasm.c
+++ b/tools/bpf/bpftool/jit_disasm.c
@@ -10,6 +10,7 @@
  * Licensed under the GNU General Public License, version 2.0 (GPLv2)
  */
 
+#include <stdarg.h>
 #include <stdint.h>
 #include <stdio.h>
 #include <stdlib.h>
@@ -21,6 +22,9 @@
 #include <sys/types.h>
 #include <sys/stat.h>
 
+#include "json_writer.h"
+#include "main.h"
+
 static void get_exec_path(char *tpath, size_t size)
 {
 	ssize_t len;
@@ -39,6 +43,38 @@ static void get_exec_path(char *tpath, size_t size)
 	free(path);
 }
 
+static int oper_count;
+static int fprintf_json(void *out, const char *fmt, ...)
+{
+	va_list ap;
+	char *s;
+
+	va_start(ap, fmt);
+	if (!oper_count) {
+		int i;
+
+		s = va_arg(ap, char *);
+
+		/* Strip trailing spaces */
+		i = strlen(s) - 1;
+		while (s[i] == ' ')
+			s[i--] = '\0';
+
+		jsonw_string_field(json_wtr, "operation", s);
+		jsonw_name(json_wtr, "operands");
+		jsonw_start_array(json_wtr);
+		oper_count++;
+	} else if (!strcmp(fmt, ",")) {
+		   /* Skip */
+	} else {
+		s = va_arg(ap, char *);
+		jsonw_string(json_wtr, s);
+		oper_count++;
+	}
+	va_end(ap);
+	return 0;
+}
+
 void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes)
 {
 	disassembler_ftype disassemble;
@@ -57,7 +93,12 @@ void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes)
 	assert(bfdf);
 	assert(bfd_check_format(bfdf, bfd_object));
 
-	init_disassemble_info(&info, stdout, (fprintf_ftype) fprintf);
+	if (json_output)
+		init_disassemble_info(&info, stdout,
+				      (fprintf_ftype) fprintf_json);
+	else
+		init_disassemble_info(&info, stdout,
+				      (fprintf_ftype) fprintf);
 	info.arch = bfd_get_arch(bfdf);
 	info.mach = bfd_get_mach(bfdf);
 	info.buffer = image;
@@ -68,20 +109,53 @@ void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes)
 	disassemble = disassembler(bfdf);
 	assert(disassemble);
 
+	if (json_output)
+		jsonw_start_array(json_wtr);
 	do {
-		printf("%4x:\t", pc);
+		if (json_output) {
+			jsonw_start_object(json_wtr);
+			oper_count = 0;
+			jsonw_name(json_wtr, "pc");
+			jsonw_printf(json_wtr, "\"0x%x\"", pc);
+		} else {
+			printf("%4x:\t", pc);
+		}
 
 		count = disassemble(pc, &info);
+		if (json_output) {
+			/* Operand array, was started in fprintf_json. Before
+			 * that, make sure we have a _null_ value if no operand
+			 * other than operation code was present.
+			 */
+			if (oper_count == 1)
+				jsonw_null(json_wtr);
+			jsonw_end_array(json_wtr);
+		}
 
 		if (opcodes) {
-			printf("\n\t");
-			for (i = 0; i < count; ++i)
-				printf("%02x ", (uint8_t) image[pc + i]);
+			if (json_output) {
+				jsonw_name(json_wtr, "opcodes");
+				jsonw_start_array(json_wtr);
+				for (i = 0; i < count; ++i)
+					jsonw_printf(json_wtr, "\"0x%02hhx\"",
+						     (uint8_t)image[pc + i]);
+				jsonw_end_array(json_wtr);
+			} else {
+				printf("\n\t");
+				for (i = 0; i < count; ++i)
+					printf("%02x ",
+					       (uint8_t)image[pc + i]);
+			}
 		}
-		printf("\n");
+		if (json_output)
+			jsonw_end_object(json_wtr);
+		else
+			printf("\n");
 
 		pc += count;
 	} while (count > 0 && pc < len);
+	if (json_output)
+		jsonw_end_array(json_wtr);
 
 	bfd_close(bfdf);
 }
-- 
2.14.1

Re: [PATCH net-next 05/12] tools: bpftool: add JSON output for `bpftool prog dump jited *` command

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Reuse the json_writer API introduced in an earlier commit to make
> bpftool able to generate JSON output on `bpftool prog show *` commands.
> A new printing function is created to be passed as an argument to the
> disassembler.
>
> Similarly to plain output, opcodes are printed on request.
>
> Outputs from sample programs have been successfully tested against a
> JSON validator.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 06/12] tools: bpftool: add JSON output for `bpftool prog dump xlated *` command

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Add a new printing function to dump translated eBPF instructions as
JSON. As for plain output, opcodes are printed only on request (when
`opcodes` is provided on the command line).

The disassembled output is generated by the same code that is used by
the kernel verifier.

Example output:

    $ bpftool --json --pretty prog dump xlated id 1
    [{
            "disasm": "(bf) r6 = r1"
        },{
            "disasm": "(61) r7 = *(u32 *)(r6 +16)"
        },{
            "disasm": "(95) exit"
        }
    ]

    $ bpftool --json --pretty prog dump xlated id 1 opcodes
    [{
            "disasm": "(bf) r6 = r1",
            "opcodes": {
                "code": "0xbf",
                "src_reg": "0x1",
                "dst_reg": "0x6",
                "off": ["0x00","0x00"
                ],
                "imm": ["0x00","0x00","0x00","0x00"
                ]
            }
        },{
            "disasm": "(61) r7 = *(u32 *)(r6 +16)",
            "opcodes": {
                "code": "0x61",
                "src_reg": "0x6",
                "dst_reg": "0x7",
                "off": ["0x10","0x00"
                ],
                "imm": ["0x00","0x00","0x00","0x00"
                ]
            }
        },{
            "disasm": "(95) exit",
            "opcodes": {
                "code": "0x95",
                "src_reg": "0x0",
                "dst_reg": "0x0",
                "off": ["0x00","0x00"
                ],
                "imm": ["0x00","0x00","0x00","0x00"
                ]
            }
        }
    ]

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/common.c      | 10 ++++++
 tools/bpf/bpftool/json_writer.c |  8 +++++
 tools/bpf/bpftool/json_writer.h |  2 ++
 tools/bpf/bpftool/main.h        |  1 +
 tools/bpf/bpftool/prog.c        | 70 +++++++++++++++++++++++++++++++++++++++--
 5 files changed, 89 insertions(+), 2 deletions(-)

diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c
index df8396a0c400..e7a756b8ee21 100644
--- a/tools/bpf/bpftool/common.c
+++ b/tools/bpf/bpftool/common.c
@@ -214,3 +214,13 @@ char *get_fdinfo(int fd, const char *key)
 	fclose(fdi);
 	return NULL;
 }
+
+void print_hex_data_json(uint8_t *data, size_t len)
+{
+	unsigned int i;
+
+	jsonw_start_array(json_wtr);
+	for (i = 0; i < len; i++)
+		jsonw_printf(json_wtr, "\"0x%02hhx\"", data[i]);
+	jsonw_end_array(json_wtr);
+}
diff --git a/tools/bpf/bpftool/json_writer.c b/tools/bpf/bpftool/json_writer.c
index 6b77d288cce2..c6eef76322ae 100644
--- a/tools/bpf/bpftool/json_writer.c
+++ b/tools/bpf/bpftool/json_writer.c
@@ -156,6 +156,14 @@ void jsonw_name(json_writer_t *self, const char *name)
 		putc(' ', self->out);
 }
 
+void jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap)
+{
+	jsonw_eor(self);
+	putc('"', self->out);
+	vfprintf(self->out, fmt, ap);
+	putc('"', self->out);
+}
+
 void jsonw_printf(json_writer_t *self, const char *fmt, ...)
 {
 	va_list ap;
diff --git a/tools/bpf/bpftool/json_writer.h b/tools/bpf/bpftool/json_writer.h
index 1516aafba59d..0fa2fb1b6351 100644
--- a/tools/bpf/bpftool/json_writer.h
+++ b/tools/bpf/bpftool/json_writer.h
@@ -17,6 +17,7 @@
 
 #include <stdbool.h>
 #include <stdint.h>
+#include <stdarg.h>
 
 /* Opaque class structure */
 typedef struct json_writer json_writer_t;
@@ -33,6 +34,7 @@ void jsonw_pretty(json_writer_t *self, bool on);
 void jsonw_name(json_writer_t *self, const char *name);
 
 /* Add value  */
+void jsonw_vprintf_enquote(json_writer_t *self, const char *fmt, va_list ap);
 void jsonw_printf(json_writer_t *self, const char *fmt, ...);
 void jsonw_string(json_writer_t *self, const char *value);
 void jsonw_bool(json_writer_t *self, bool value);
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 15927fc9fb31..693fc9710be1 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -95,5 +95,6 @@ int do_map(int argc, char **arg);
 int prog_parse_fd(int *argc, char ***argv);
 
 void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes);
+void print_hex_data_json(uint8_t *data, size_t len);
 
 #endif
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index f373f2baef5a..43e49799a624 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -385,7 +385,7 @@ static void print_insn(struct bpf_verifier_env *env, const char *fmt, ...)
 	va_end(args);
 }
 
-static void dump_xlated(void *buf, unsigned int len, bool opcodes)
+static void dump_xlated_plain(void *buf, unsigned int len, bool opcodes)
 {
 	struct bpf_insn *insn = buf;
 	bool double_insn = false;
@@ -414,6 +414,69 @@ static void dump_xlated(void *buf, unsigned int len, bool opcodes)
 	}
 }
 
+static void print_insn_json(struct bpf_verifier_env *env, const char *fmt, ...)
+{
+	unsigned int l = strlen(fmt);
+	char chomped_fmt[l];
+	va_list args;
+
+	va_start(args, fmt);
+	if (l > 0) {
+		strncpy(chomped_fmt, fmt, l - 1);
+		chomped_fmt[l - 1] = '\0';
+	}
+	jsonw_vprintf_enquote(json_wtr, chomped_fmt, args);
+	va_end(args);
+}
+
+static void dump_xlated_json(void *buf, unsigned int len, bool opcodes)
+{
+	struct bpf_insn *insn = buf;
+	bool double_insn = false;
+	unsigned int i;
+
+	jsonw_start_array(json_wtr);
+	for (i = 0; i < len / sizeof(*insn); i++) {
+		if (double_insn) {
+			double_insn = false;
+			continue;
+		}
+		double_insn = insn[i].code == (BPF_LD | BPF_IMM | BPF_DW);
+
+		jsonw_start_object(json_wtr);
+		jsonw_name(json_wtr, "disasm");
+		print_bpf_insn(print_insn_json, NULL, insn + i, true);
+
+		if (opcodes) {
+			jsonw_name(json_wtr, "opcodes");
+			jsonw_start_object(json_wtr);
+
+			jsonw_name(json_wtr, "code");
+			jsonw_printf(json_wtr, "\"0x%02hhx\"", insn[i].code);
+
+			jsonw_name(json_wtr, "src_reg");
+			jsonw_printf(json_wtr, "\"0x%hhx\"", insn[i].src_reg);
+
+			jsonw_name(json_wtr, "dst_reg");
+			jsonw_printf(json_wtr, "\"0x%hhx\"", insn[i].dst_reg);
+
+			jsonw_name(json_wtr, "off");
+			print_hex_data_json((uint8_t *)(&insn[i].off), 2);
+
+			jsonw_name(json_wtr, "imm");
+			if (double_insn && i < len - 1)
+				print_hex_data_json((uint8_t *)(&insn[i].imm),
+						    12);
+			else
+				print_hex_data_json((uint8_t *)(&insn[i].imm),
+						    4);
+			jsonw_end_object(json_wtr);
+		}
+		jsonw_end_object(json_wtr);
+	}
+	jsonw_end_array(json_wtr);
+}
+
 static int do_dump(int argc, char **argv)
 {
 	struct bpf_prog_info info = {};
@@ -523,7 +586,10 @@ static int do_dump(int argc, char **argv)
 		if (member_len == &info.jited_prog_len)
 			disasm_print_insn(buf, *member_len, opcodes);
 		else
-			dump_xlated(buf, *member_len, opcodes);
+			if (json_output)
+				dump_xlated_json(buf, *member_len, opcodes);
+			else
+				dump_xlated_plain(buf, *member_len, opcodes);
 	}
 
 	free(buf);
-- 
2.14.1

Re: [PATCH net-next 06/12] tools: bpftool: add JSON output for `bpftool prog dump xlated *` command

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Add a new printing function to dump translated eBPF instructions as
> JSON. As for plain output, opcodes are printed only on request (when
> `opcodes` is provided on the command line).
>
> The disassembled output is generated by the same code that is used by
> the kernel verifier.
>
> Example output:
>
>      $ bpftool --json --pretty prog dump xlated id 1
>      [{
>              "disasm": "(bf) r6 = r1"
>          },{
>              "disasm": "(61) r7 = *(u32 *)(r6 +16)"
>          },{
>              "disasm": "(95) exit"
>          }
>      ]
>
>      $ bpftool --json --pretty prog dump xlated id 1 opcodes
>      [{
>              "disasm": "(bf) r6 = r1",
>              "opcodes": {
>                  "code": "0xbf",
>                  "src_reg": "0x1",
>                  "dst_reg": "0x6",
>                  "off": ["0x00","0x00"
>                  ],
>                  "imm": ["0x00","0x00","0x00","0x00"
>                  ]
>              }
>          },{
>              "disasm": "(61) r7 = *(u32 *)(r6 +16)",
>              "opcodes": {
>                  "code": "0x61",
>                  "src_reg": "0x6",
>                  "dst_reg": "0x7",
>                  "off": ["0x10","0x00"
>                  ],
>                  "imm": ["0x00","0x00","0x00","0x00"
>                  ]
>              }
>          },{
>              "disasm": "(95) exit",
>              "opcodes": {
>                  "code": "0x95",
>                  "src_reg": "0x0",
>                  "dst_reg": "0x0",
>                  "off": ["0x00","0x00"
>                  ],
>                  "imm": ["0x00","0x00","0x00","0x00"
>                  ]
>              }
>          }
>      ]
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 07/12] tools: bpftool: add JSON output for `bpftool map *` commands

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Reuse the json_writer API introduced in an earlier commit to make
bpftool able to generate JSON output on
`bpftool map { show | dump | lookup | getnext }` commands. Remaining
commands produce no output.

Some functions have been spit into plain-output and JSON versions in
order to remain readable.

Outputs for sample maps have been successfully tested against a JSON
validator.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/map.c | 149 +++++++++++++++++++++++++++++++++++++++++-------
 1 file changed, 129 insertions(+), 20 deletions(-)

diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index e1004d825392..14d89bfabc66 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -205,8 +205,45 @@ map_parse_fd_and_info(int *argc, char ***argv, void *info, __u32 *info_len)
 	return fd;
 }
 
-static void print_entry(struct bpf_map_info *info, unsigned char *key,
-			unsigned char *value)
+static void print_entry_json(struct bpf_map_info *info, unsigned char *key,
+			     unsigned char *value)
+{
+	jsonw_start_object(json_wtr);
+
+	if (!map_is_per_cpu(info->type)) {
+		jsonw_name(json_wtr, "key");
+		print_hex_data_json(key, info->key_size);
+		jsonw_name(json_wtr, "value");
+		print_hex_data_json(value, info->value_size);
+	} else {
+		unsigned int i, n;
+
+		n = get_possible_cpus();
+
+		jsonw_name(json_wtr, "key");
+		print_hex_data_json(key, info->key_size);
+
+		jsonw_name(json_wtr, "values");
+		jsonw_start_array(json_wtr);
+		for (i = 0; i < n; i++) {
+			jsonw_start_object(json_wtr);
+
+			jsonw_int_field(json_wtr, "cpu", i);
+
+			jsonw_name(json_wtr, "value");
+			print_hex_data_json(value + i * info->value_size,
+					    info->value_size);
+
+			jsonw_end_object(json_wtr);
+		}
+		jsonw_end_array(json_wtr);
+	}
+
+	jsonw_end_object(json_wtr);
+}
+
+static void print_entry_plain(struct bpf_map_info *info, unsigned char *key,
+			      unsigned char *value)
 {
 	if (!map_is_per_cpu(info->type)) {
 		bool single_line, break_names;
@@ -370,7 +407,41 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 	return -1;
 }
 
-static int show_map_close(int fd, struct bpf_map_info *info)
+static int show_map_close_json(int fd, struct bpf_map_info *info)
+{
+	char *memlock;
+
+	memlock = get_fdinfo(fd, "memlock");
+	close(fd);
+
+	jsonw_start_object(json_wtr);
+
+	jsonw_uint_field(json_wtr, "id", info->id);
+	if (info->type < ARRAY_SIZE(map_type_name))
+		jsonw_string_field(json_wtr, "type",
+				   map_type_name[info->type]);
+	else
+		jsonw_uint_field(json_wtr, "type", info->type);
+
+	if (*info->name)
+		jsonw_string_field(json_wtr, "name", info->name);
+
+	jsonw_name(json_wtr, "flags");
+	jsonw_printf(json_wtr, "%#x", info->map_flags);
+	jsonw_uint_field(json_wtr, "bytes_key", info->key_size);
+	jsonw_uint_field(json_wtr, "bytes_value", info->value_size);
+	jsonw_uint_field(json_wtr, "max_entries", info->max_entries);
+
+	if (memlock)
+		jsonw_int_field(json_wtr, "bytes_memlock", atoi(memlock));
+	free(memlock);
+
+	jsonw_end_object(json_wtr);
+
+	return 0;
+}
+
+static int show_map_close_plain(int fd, struct bpf_map_info *info)
 {
 	char *memlock;
 
@@ -412,12 +483,17 @@ static int do_show(int argc, char **argv)
 		if (fd < 0)
 			return -1;
 
-		return show_map_close(fd, &info);
+		if (json_output)
+			return show_map_close_json(fd, &info);
+		else
+			return show_map_close_plain(fd, &info);
 	}
 
 	if (argc)
 		return BAD_ARG();
 
+	if (json_output)
+		jsonw_start_array(json_wtr);
 	while (true) {
 		err = bpf_map_get_next_id(id, &id);
 		if (err) {
@@ -443,8 +519,13 @@ static int do_show(int argc, char **argv)
 			return -1;
 		}
 
-		show_map_close(fd, &info);
+		if (json_output)
+			show_map_close_json(fd, &info);
+		else
+			show_map_close_plain(fd, &info);
 	}
+	if (json_output)
+		jsonw_end_array(json_wtr);
 
 	return errno == ENOENT ? 0 : -1;
 }
@@ -480,6 +561,8 @@ static int do_dump(int argc, char **argv)
 	}
 
 	prev_key = NULL;
+	if (json_output)
+		jsonw_start_array(json_wtr);
 	while (true) {
 		err = bpf_map_get_next_key(fd, prev_key, key);
 		if (err) {
@@ -489,7 +572,10 @@ static int do_dump(int argc, char **argv)
 		}
 
 		if (!bpf_map_lookup_elem(fd, key, value)) {
-			print_entry(&info, key, value);
+			if (json_output)
+				print_entry_json(&info, key, value);
+			else
+				print_entry_plain(&info, key, value);
 		} else {
 			info("can't lookup element with key: ");
 			fprint_hex(stderr, key, info.key_size, " ");
@@ -500,7 +586,11 @@ static int do_dump(int argc, char **argv)
 		num_elems++;
 	}
 
-	printf("Found %u element%s\n", num_elems, num_elems != 1 ? "s" : "");
+	if (json_output)
+		jsonw_end_array(json_wtr);
+	else
+		printf("Found %u element%s\n", num_elems,
+		       num_elems != 1 ? "s" : "");
 
 exit_free:
 	free(key);
@@ -584,11 +674,18 @@ static int do_lookup(int argc, char **argv)
 
 	err = bpf_map_lookup_elem(fd, key, value);
 	if (!err) {
-		print_entry(&info, key, value);
+		if (json_output)
+			print_entry_json(&info, key, value);
+		else
+			print_entry_plain(&info, key, value);
 	} else if (errno == ENOENT) {
-		printf("key:\n");
-		fprint_hex(stdout, key, info.key_size, " ");
-		printf("\n\nNot found\n");
+		if (json_output) {
+			jsonw_null(json_wtr);
+		} else {
+			printf("key:\n");
+			fprint_hex(stdout, key, info.key_size, " ");
+			printf("\n\nNot found\n");
+		}
 	} else {
 		err("lookup failed: %s\n", strerror(errno));
 	}
@@ -640,18 +737,30 @@ static int do_getnext(int argc, char **argv)
 		goto exit_free;
 	}
 
-	if (key) {
-		printf("key:\n");
-		fprint_hex(stdout, key, info.key_size, " ");
-		printf("\n");
+	if (json_output) {
+		jsonw_start_object(json_wtr);
+		if (key) {
+			jsonw_name(json_wtr, "key");
+			print_hex_data_json(key, info.key_size);
+		} else {
+			jsonw_null_field(json_wtr, "key");
+		}
+		jsonw_name(json_wtr, "next_key");
+		print_hex_data_json(nextkey, info.key_size);
+		jsonw_end_object(json_wtr);
 	} else {
-		printf("key: None\n");
+		if (key) {
+			printf("key:\n");
+			fprint_hex(stdout, key, info.key_size, " ");
+			printf("\n");
+		} else {
+			printf("key: None\n");
+		}
+		printf("next key:\n");
+		fprint_hex(stdout, nextkey, info.key_size, " ");
+		printf("\n");
 	}
 
-	printf("next key:\n");
-	fprint_hex(stdout, nextkey, info.key_size, " ");
-	printf("\n");
-
 exit_free:
 	free(nextkey);
 	free(key);
-- 
2.14.1

Re: [PATCH net-next 07/12] tools: bpftool: add JSON output for `bpftool map *` commands

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Reuse the json_writer API introduced in an earlier commit to make
> bpftool able to generate JSON output on
> `bpftool map { show | dump | lookup | getnext }` commands. Remaining
> commands produce no output.
>
> Some functions have been spit into plain-output and JSON versions in
> order to remain readable.
>
> Outputs for sample maps have been successfully tested against a JSON
> validator.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 08/12] tools: bpftool: add JSON output for `bpftool batch file FILE` command

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

`bpftool batch file FILE` takes FILE as an argument and executes all the
bpftool commands it finds inside (or stops if an error occurs).

To obtain a consistent JSON output, create a root JSON array, then for
each command create a new object containing two fields: one with the
command arguments, the other with the output (which is the JSON object
that the command would have produced, if called on its own).

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/main.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)

diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 14bfc17cd4de..71b01bf73912 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -155,6 +155,7 @@ static int do_batch(int argc, char **argv)
 	int n_argc;
 	FILE *fp;
 	int err;
+	int i;
 
 	if (argc < 2) {
 		err("too few parameters for batch\n");
@@ -174,6 +175,8 @@ static int do_batch(int argc, char **argv)
 		return -1;
 	}
 
+	if (json_output)
+		jsonw_start_array(json_wtr);
 	while (fgets(buf, sizeof(buf), fp)) {
 		if (strlen(buf) == sizeof(buf) - 1) {
 			errno = E2BIG;
@@ -197,7 +200,21 @@ static int do_batch(int argc, char **argv)
 		if (!n_argc)
 			continue;
 
+		if (json_output) {
+			jsonw_start_object(json_wtr);
+			jsonw_name(json_wtr, "command");
+			jsonw_start_array(json_wtr);
+			for (i = 0; i < n_argc; i++)
+				jsonw_string(json_wtr, n_argv[i]);
+			jsonw_end_array(json_wtr);
+			jsonw_name(json_wtr, "output");
+		}
+
 		err = cmd_select(cmds, n_argc, n_argv, do_help);
+
+		if (json_output)
+			jsonw_end_object(json_wtr);
+
 		if (err)
 			goto err_close;
 
@@ -214,6 +231,9 @@ static int do_batch(int argc, char **argv)
 err_close:
 	fclose(fp);
 
+	if (json_output)
+		jsonw_end_array(json_wtr);
+
 	return err;
 }
 
-- 
2.14.1

Re: [PATCH net-next 08/12] tools: bpftool: add JSON output for `bpftool batch file FILE` command

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> `bpftool batch file FILE` takes FILE as an argument and executes all the
> bpftool commands it finds inside (or stops if an error occurs).
>
> To obtain a consistent JSON output, create a root JSON array, then for
> each command create a new object containing two fields: one with the
> command arguments, the other with the output (which is the JSON object
> that the command would have produced, if called on its own).
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 09/12] tools: bpftool: turn err() and info() macros into functions

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Turn err() and info() macros into functions.

In order to avoid naming conflicts with variables in the code, rename
them as p_err() and p_info() respectively.

The behavior of these functions is similar to the one of the macros for
plain output. However, when JSON output is requested, these macros
return a JSON-formatted "error" object instead of printing a message to
stderr.

To handle error messages correctly with JSON, a modification was brought
to their behavior nonetheless: the functions now append a end-of-line
character at the end of the message. This way, we can remove end-of-line
characters at the end of the argument strings, and not have them in the
JSON output.

All error messages are formatted to hold in a single call to p_err(), in
order to produce a single JSON field.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
---
 tools/bpf/bpftool/common.c | 26 +++++++-------
 tools/bpf/bpftool/main.c   | 16 ++++-----
 tools/bpf/bpftool/main.h   | 37 +++++++++++++++++---
 tools/bpf/bpftool/map.c    | 87 +++++++++++++++++++++++++---------------------
 tools/bpf/bpftool/prog.c   | 51 +++++++++++++--------------
 5 files changed, 126 insertions(+), 91 deletions(-)

diff --git a/tools/bpf/bpftool/common.c b/tools/bpf/bpftool/common.c
index e7a756b8ee21..b2533f1cae3e 100644
--- a/tools/bpf/bpftool/common.c
+++ b/tools/bpf/bpftool/common.c
@@ -66,8 +66,8 @@ int open_obj_pinned_any(char *path, enum bpf_obj_type exp_type)
 
 	fd = bpf_obj_get(path);
 	if (fd < 0) {
-		err("bpf obj get (%s): %s\n", path,
-		    errno == EACCES && !is_bpffs(dirname(path)) ?
+		p_err("bpf obj get (%s): %s", path,
+		      errno == EACCES && !is_bpffs(dirname(path)) ?
 		    "directory not in bpf file system (bpffs)" :
 		    strerror(errno));
 		return -1;
@@ -79,7 +79,7 @@ int open_obj_pinned_any(char *path, enum bpf_obj_type exp_type)
 		return type;
 	}
 	if (type != exp_type) {
-		err("incorrect object type: %s\n", get_fd_type_name(type));
+		p_err("incorrect object type: %s", get_fd_type_name(type));
 		close(fd);
 		return -1;
 	}
@@ -95,14 +95,14 @@ int do_pin_any(int argc, char **argv, int (*get_fd_by_id)(__u32))
 	int fd;
 
 	if (!is_prefix(*argv, "id")) {
-		err("expected 'id' got %s\n", *argv);
+		p_err("expected 'id' got %s", *argv);
 		return -1;
 	}
 	NEXT_ARG();
 
 	id = strtoul(*argv, &endptr, 0);
 	if (*endptr) {
-		err("can't parse %s as ID\n", *argv);
+		p_err("can't parse %s as ID", *argv);
 		return -1;
 	}
 	NEXT_ARG();
@@ -112,15 +112,15 @@ int do_pin_any(int argc, char **argv, int (*get_fd_by_id)(__u32))
 
 	fd = get_fd_by_id(id);
 	if (fd < 0) {
-		err("can't get prog by id (%u): %s\n", id, strerror(errno));
+		p_err("can't get prog by id (%u): %s", id, strerror(errno));
 		return -1;
 	}
 
 	err = bpf_obj_pin(fd, *argv);
 	close(fd);
 	if (err) {
-		err("can't pin the object (%s): %s\n", *argv,
-		    errno == EACCES && !is_bpffs(dirname(*argv)) ?
+		p_err("can't pin the object (%s): %s", *argv,
+		      errno == EACCES && !is_bpffs(dirname(*argv)) ?
 		    "directory not in bpf file system (bpffs)" :
 		    strerror(errno));
 		return -1;
@@ -153,11 +153,11 @@ int get_fd_type(int fd)
 
 	n = readlink(path, buf, sizeof(buf));
 	if (n < 0) {
-		err("can't read link type: %s\n", strerror(errno));
+		p_err("can't read link type: %s", strerror(errno));
 		return -1;
 	}
 	if (n == sizeof(path)) {
-		err("can't read link type: path too long!\n");
+		p_err("can't read link type: path too long!");
 		return -1;
 	}
 
@@ -181,7 +181,7 @@ char *get_fdinfo(int fd, const char *key)
 
 	fdi = fopen(path, "r");
 	if (!fdi) {
-		err("can't open fdinfo: %s\n", strerror(errno));
+		p_err("can't open fdinfo: %s", strerror(errno));
 		return NULL;
 	}
 
@@ -196,7 +196,7 @@ char *get_fdinfo(int fd, const char *key)
 
 		value = strchr(line, '\t');
 		if (!value || !value[1]) {
-			err("malformed fdinfo!?\n");
+			p_err("malformed fdinfo!?");
 			free(line);
 			return NULL;
 		}
@@ -209,7 +209,7 @@ char *get_fdinfo(int fd, const char *key)
 		return line;
 	}
 
-	err("key '%s' not found in fdinfo\n", key);
+	p_err("key '%s' not found in fdinfo", key);
 	free(line);
 	fclose(fdi);
 	return NULL;
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 71b01bf73912..9989a77fdc4a 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -158,20 +158,20 @@ static int do_batch(int argc, char **argv)
 	int i;
 
 	if (argc < 2) {
-		err("too few parameters for batch\n");
+		p_err("too few parameters for batch");
 		return -1;
 	} else if (!is_prefix(*argv, "file")) {
-		err("expected 'file', got: %s\n", *argv);
+		p_err("expected 'file', got: %s", *argv);
 		return -1;
 	} else if (argc > 2) {
-		err("too many parameters for batch\n");
+		p_err("too many parameters for batch");
 		return -1;
 	}
 	NEXT_ARG();
 
 	fp = fopen(*argv, "r");
 	if (!fp) {
-		err("Can't open file (%s): %s\n", *argv, strerror(errno));
+		p_err("Can't open file (%s): %s", *argv, strerror(errno));
 		return -1;
 	}
 
@@ -189,8 +189,8 @@ static int do_batch(int argc, char **argv)
 		while (n_argv[n_argc]) {
 			n_argc++;
 			if (n_argc == ARRAY_SIZE(n_argv)) {
-				err("line %d has too many arguments, skip\n",
-				    lines);
+				p_err("line %d has too many arguments, skip",
+				      lines);
 				n_argc = 0;
 				break;
 			}
@@ -225,7 +225,7 @@ static int do_batch(int argc, char **argv)
 		perror("reading batch file failed");
 		err = -1;
 	} else {
-		info("processed %d lines\n", lines);
+		p_info("processed %d lines", lines);
 		err = 0;
 	}
 err_close:
@@ -279,7 +279,7 @@ int main(int argc, char **argv)
 	if (json_output) {
 		json_wtr = jsonw_new(stdout);
 		if (!json_wtr) {
-			err("failed to create JSON writer\n");
+			p_err("failed to create JSON writer");
 			return -1;
 		}
 		jsonw_pretty(json_wtr, pretty_output);
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 693fc9710be1..04c88b55d8c7 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -45,15 +45,11 @@
 
 #include "json_writer.h"
 
-#define err(msg...)	fprintf(stderr, "Error: " msg)
-#define warn(msg...)	fprintf(stderr, "Warning: " msg)
-#define info(msg...)	fprintf(stderr, msg)
-
 #define ptr_to_u64(ptr)	((__u64)(unsigned long)(ptr))
 
 #define NEXT_ARG()	({ argc--; argv++; if (argc < 0) usage(); })
 #define NEXT_ARGP()	({ (*argc)--; (*argv)++; if (*argc < 0) usage(); })
-#define BAD_ARG()	({ err("what is '%s'?\n", *argv); -1; })
+#define BAD_ARG()	({ p_err("what is '%s'?\n", *argv); -1; })
 
 #define BPF_TAG_FMT	"%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx%02hhx"
 
@@ -97,4 +93,35 @@ int prog_parse_fd(int *argc, char ***argv);
 void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes);
 void print_hex_data_json(uint8_t *data, size_t len);
 
+static inline void p_err(const char *fmt, ...)
+{
+	va_list ap;
+
+	va_start(ap, fmt);
+	if (json_output) {
+		jsonw_start_object(json_wtr);
+		jsonw_name(json_wtr, "error");
+		jsonw_vprintf_enquote(json_wtr, fmt, ap);
+		jsonw_end_object(json_wtr);
+	} else {
+		fprintf(stderr, "Error: ");
+		vfprintf(stderr, fmt, ap);
+		fprintf(stderr, "\n");
+	}
+	va_end(ap);
+}
+
+static inline void p_info(const char *fmt, ...)
+{
+	va_list ap;
+
+	if (json_output)
+		return;
+
+	va_start(ap, fmt);
+	vfprintf(stderr, fmt, ap);
+	fprintf(stderr, "\n");
+	va_end(ap);
+}
+
 #endif
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 14d89bfabc66..86c128c433ba 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -81,19 +81,19 @@ static unsigned int get_possible_cpus(void)
 
 	fd = open("/sys/devices/system/cpu/possible", O_RDONLY);
 	if (fd < 0) {
-		err("can't open sysfs possible cpus\n");
+		p_err("can't open sysfs possible cpus");
 		exit(-1);
 	}
 
 	n = read(fd, buf, sizeof(buf));
 	if (n < 2) {
-		err("can't read sysfs possible cpus\n");
+		p_err("can't read sysfs possible cpus");
 		exit(-1);
 	}
 	close(fd);
 
 	if (n == sizeof(buf)) {
-		err("read sysfs possible cpus overflow\n");
+		p_err("read sysfs possible cpus overflow");
 		exit(-1);
 	}
 
@@ -161,14 +161,14 @@ static int map_parse_fd(int *argc, char ***argv)
 
 		id = strtoul(**argv, &endptr, 0);
 		if (*endptr) {
-			err("can't parse %s as ID\n", **argv);
+			p_err("can't parse %s as ID", **argv);
 			return -1;
 		}
 		NEXT_ARGP();
 
 		fd = bpf_map_get_fd_by_id(id);
 		if (fd < 0)
-			err("get map by id (%u): %s\n", id, strerror(errno));
+			p_err("get map by id (%u): %s", id, strerror(errno));
 		return fd;
 	} else if (is_prefix(**argv, "pinned")) {
 		char *path;
@@ -181,7 +181,7 @@ static int map_parse_fd(int *argc, char ***argv)
 		return open_obj_pinned_any(path, BPF_OBJ_MAP);
 	}
 
-	err("expected 'id' or 'pinned', got: '%s'?\n", **argv);
+	p_err("expected 'id' or 'pinned', got: '%s'?", **argv);
 	return -1;
 }
 
@@ -197,7 +197,7 @@ map_parse_fd_and_info(int *argc, char ***argv, void *info, __u32 *info_len)
 
 	err = bpf_obj_get_info_by_fd(fd, info, info_len);
 	if (err) {
-		err("can't get map info: %s\n", strerror(errno));
+		p_err("can't get map info: %s", strerror(errno));
 		close(fd);
 		return err;
 	}
@@ -288,14 +288,14 @@ static char **parse_bytes(char **argv, const char *name, unsigned char *val,
 	while (i < n && argv[i]) {
 		val[i] = strtoul(argv[i], &endptr, 0);
 		if (*endptr) {
-			err("error parsing byte: %s\n", argv[i]);
+			p_err("error parsing byte: %s", argv[i]);
 			return NULL;
 		}
 		i++;
 	}
 
 	if (i != n) {
-		err("%s expected %d bytes got %d\n", name, n, i);
+		p_err("%s expected %d bytes got %d", name, n, i);
 		return NULL;
 	}
 
@@ -309,16 +309,16 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 	if (!*argv) {
 		if (!key && !value)
 			return 0;
-		err("did not find %s\n", key ? "key" : "value");
+		p_err("did not find %s", key ? "key" : "value");
 		return -1;
 	}
 
 	if (is_prefix(*argv, "key")) {
 		if (!key) {
 			if (key_size)
-				err("duplicate key\n");
+				p_err("duplicate key");
 			else
-				err("unnecessary key\n");
+				p_err("unnecessary key");
 			return -1;
 		}
 
@@ -333,9 +333,9 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 
 		if (!value) {
 			if (value_size)
-				err("duplicate value\n");
+				p_err("duplicate value");
 			else
-				err("unnecessary value\n");
+				p_err("unnecessary value");
 			return -1;
 		}
 
@@ -345,11 +345,11 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 			int argc = 2;
 
 			if (value_size != 4) {
-				err("value smaller than 4B for map in map?\n");
+				p_err("value smaller than 4B for map in map?");
 				return -1;
 			}
 			if (!argv[0] || !argv[1]) {
-				err("not enough value arguments for map in map\n");
+				p_err("not enough value arguments for map in map");
 				return -1;
 			}
 
@@ -363,11 +363,11 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 			int argc = 2;
 
 			if (value_size != 4) {
-				err("value smaller than 4B for map of progs?\n");
+				p_err("value smaller than 4B for map of progs?");
 				return -1;
 			}
 			if (!argv[0] || !argv[1]) {
-				err("not enough value arguments for map of progs\n");
+				p_err("not enough value arguments for map of progs");
 				return -1;
 			}
 
@@ -388,7 +388,7 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 	} else if (is_prefix(*argv, "any") || is_prefix(*argv, "noexist") ||
 		   is_prefix(*argv, "exist")) {
 		if (!flags) {
-			err("flags specified multiple times: %s\n", *argv);
+			p_err("flags specified multiple times: %s", *argv);
 			return -1;
 		}
 
@@ -403,7 +403,7 @@ static int parse_elem(char **argv, struct bpf_map_info *info,
 				  value_size, NULL, value_fd);
 	}
 
-	err("expected key or value, got: %s\n", *argv);
+	p_err("expected key or value, got: %s", *argv);
 	return -1;
 }
 
@@ -499,22 +499,21 @@ static int do_show(int argc, char **argv)
 		if (err) {
 			if (errno == ENOENT)
 				break;
-			err("can't get next map: %s\n", strerror(errno));
-			if (errno == EINVAL)
-				err("kernel too old?\n");
+			p_err("can't get next map: %s%s", strerror(errno),
+			      errno == EINVAL ? " -- kernel too old?" : "");
 			return -1;
 		}
 
 		fd = bpf_map_get_fd_by_id(id);
 		if (fd < 0) {
-			err("can't get map by id (%u): %s\n",
-			    id, strerror(errno));
+			p_err("can't get map by id (%u): %s",
+			      id, strerror(errno));
 			return -1;
 		}
 
 		err = bpf_obj_get_info_by_fd(fd, &info, &len);
 		if (err) {
-			err("can't get map info: %s\n", strerror(errno));
+			p_err("can't get map info: %s", strerror(errno));
 			close(fd);
 			return -1;
 		}
@@ -547,7 +546,7 @@ static int do_dump(int argc, char **argv)
 		return -1;
 
 	if (map_is_map_of_maps(info.type) || map_is_map_of_progs(info.type)) {
-		err("Dumping maps of maps and program maps not supported\n");
+		p_err("Dumping maps of maps and program maps not supported");
 		close(fd);
 		return -1;
 	}
@@ -555,7 +554,7 @@ static int do_dump(int argc, char **argv)
 	key = malloc(info.key_size);
 	value = alloc_value(&info);
 	if (!key || !value) {
-		err("mem alloc failed\n");
+		p_err("mem alloc failed");
 		err = -1;
 		goto exit_free;
 	}
@@ -577,9 +576,19 @@ static int do_dump(int argc, char **argv)
 			else
 				print_entry_plain(&info, key, value);
 		} else {
-			info("can't lookup element with key: ");
-			fprint_hex(stderr, key, info.key_size, " ");
-			fprintf(stderr, "\n");
+			if (json_output) {
+				jsonw_name(json_wtr, "key");
+				print_hex_data_json(key, info.key_size);
+				jsonw_name(json_wtr, "value");
+				jsonw_start_object(json_wtr);
+				jsonw_string_field(json_wtr, "error",
+						   "can't lookup element");
+				jsonw_end_object(json_wtr);
+			} else {
+				p_info("can't lookup element with key: ");
+				fprint_hex(stderr, key, info.key_size, " ");
+				fprintf(stderr, "\n");
+			}
 		}
 
 		prev_key = key;
@@ -619,7 +628,7 @@ static int do_update(int argc, char **argv)
 	key = malloc(info.key_size);
 	value = alloc_value(&info);
 	if (!key || !value) {
-		err("mem alloc failed");
+		p_err("mem alloc failed");
 		err = -1;
 		goto exit_free;
 	}
@@ -631,7 +640,7 @@ static int do_update(int argc, char **argv)
 
 	err = bpf_map_update_elem(fd, key, value, flags);
 	if (err) {
-		err("update failed: %s\n", strerror(errno));
+		p_err("update failed: %s", strerror(errno));
 		goto exit_free;
 	}
 
@@ -663,7 +672,7 @@ static int do_lookup(int argc, char **argv)
 	key = malloc(info.key_size);
 	value = alloc_value(&info);
 	if (!key || !value) {
-		err("mem alloc failed");
+		p_err("mem alloc failed");
 		err = -1;
 		goto exit_free;
 	}
@@ -687,7 +696,7 @@ static int do_lookup(int argc, char **argv)
 			printf("\n\nNot found\n");
 		}
 	} else {
-		err("lookup failed: %s\n", strerror(errno));
+		p_err("lookup failed: %s", strerror(errno));
 	}
 
 exit_free:
@@ -716,7 +725,7 @@ static int do_getnext(int argc, char **argv)
 	key = malloc(info.key_size);
 	nextkey = malloc(info.key_size);
 	if (!key || !nextkey) {
-		err("mem alloc failed");
+		p_err("mem alloc failed");
 		err = -1;
 		goto exit_free;
 	}
@@ -733,7 +742,7 @@ static int do_getnext(int argc, char **argv)
 
 	err = bpf_map_get_next_key(fd, key, nextkey);
 	if (err) {
-		err("can't get next key: %s\n", strerror(errno));
+		p_err("can't get next key: %s", strerror(errno));
 		goto exit_free;
 	}
 
@@ -786,7 +795,7 @@ static int do_delete(int argc, char **argv)
 
 	key = malloc(info.key_size);
 	if (!key) {
-		err("mem alloc failed");
+		p_err("mem alloc failed");
 		err = -1;
 		goto exit_free;
 	}
@@ -797,7 +806,7 @@ static int do_delete(int argc, char **argv)
 
 	err = bpf_map_delete_elem(fd, key);
 	if (err)
-		err("delete failed: %s\n", strerror(errno));
+		p_err("delete failed: %s", strerror(errno));
 
 exit_free:
 	free(key);
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 43e49799a624..41bd5390b4fc 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -104,21 +104,21 @@ static int prog_fd_by_tag(unsigned char *tag)
 	while (true) {
 		err = bpf_prog_get_next_id(id, &id);
 		if (err) {
-			err("%s\n", strerror(errno));
+			p_err("%s", strerror(errno));
 			return -1;
 		}
 
 		fd = bpf_prog_get_fd_by_id(id);
 		if (fd < 0) {
-			err("can't get prog by id (%u): %s\n",
-			    id, strerror(errno));
+			p_err("can't get prog by id (%u): %s",
+			      id, strerror(errno));
 			return -1;
 		}
 
 		err = bpf_obj_get_info_by_fd(fd, &info, &len);
 		if (err) {
-			err("can't get prog info (%u): %s\n",
-			    id, strerror(errno));
+			p_err("can't get prog info (%u): %s",
+			      id, strerror(errno));
 			close(fd);
 			return -1;
 		}
@@ -142,14 +142,14 @@ int prog_parse_fd(int *argc, char ***argv)
 
 		id = strtoul(**argv, &endptr, 0);
 		if (*endptr) {
-			err("can't parse %s as ID\n", **argv);
+			p_err("can't parse %s as ID", **argv);
 			return -1;
 		}
 		NEXT_ARGP();
 
 		fd = bpf_prog_get_fd_by_id(id);
 		if (fd < 0)
-			err("get by id (%u): %s\n", id, strerror(errno));
+			p_err("get by id (%u): %s", id, strerror(errno));
 		return fd;
 	} else if (is_prefix(**argv, "tag")) {
 		unsigned char tag[BPF_TAG_SIZE];
@@ -159,7 +159,7 @@ int prog_parse_fd(int *argc, char ***argv)
 		if (sscanf(**argv, BPF_TAG_FMT, tag, tag + 1, tag + 2,
 			   tag + 3, tag + 4, tag + 5, tag + 6, tag + 7)
 		    != BPF_TAG_SIZE) {
-			err("can't parse tag\n");
+			p_err("can't parse tag");
 			return -1;
 		}
 		NEXT_ARGP();
@@ -176,7 +176,7 @@ int prog_parse_fd(int *argc, char ***argv)
 		return open_obj_pinned_any(path, BPF_OBJ_PROG);
 	}
 
-	err("expected 'id', 'tag' or 'pinned', got: '%s'?\n", **argv);
+	p_err("expected 'id', 'tag' or 'pinned', got: '%s'?", **argv);
 	return -1;
 }
 
@@ -311,7 +311,7 @@ static int show_prog(int fd)
 
 	err = bpf_obj_get_info_by_fd(fd, &info, &len);
 	if (err) {
-		err("can't get prog info: %s\n", strerror(errno));
+		p_err("can't get prog info: %s", strerror(errno));
 		return -1;
 	}
 
@@ -349,17 +349,16 @@ static int do_show(int argc, char **argv)
 				err = 0;
 				break;
 			}
-			err("can't get next program: %s\n", strerror(errno));
-			if (errno == EINVAL)
-				err("kernel too old?\n");
+			p_err("can't get next program: %s%s", strerror(errno),
+			      errno == EINVAL ? " -- kernel too old?" : "");
 			err = -1;
 			break;
 		}
 
 		fd = bpf_prog_get_fd_by_id(id);
 		if (fd < 0) {
-			err("can't get prog by id (%u): %s\n",
-			    id, strerror(errno));
+			p_err("can't get prog by id (%u): %s",
+			      id, strerror(errno));
 			err = -1;
 			break;
 		}
@@ -498,7 +497,7 @@ static int do_dump(int argc, char **argv)
 		member_len = &info.xlated_prog_len;
 		member_ptr = &info.xlated_prog_insns;
 	} else {
-		err("expected 'xlated' or 'jited', got: %s\n", *argv);
+		p_err("expected 'xlated' or 'jited', got: %s", *argv);
 		return -1;
 	}
 	NEXT_ARG();
@@ -513,7 +512,7 @@ static int do_dump(int argc, char **argv)
 	if (is_prefix(*argv, "file")) {
 		NEXT_ARG();
 		if (!argc) {
-			err("expected file path\n");
+			p_err("expected file path");
 			return -1;
 		}
 
@@ -531,12 +530,12 @@ static int do_dump(int argc, char **argv)
 
 	err = bpf_obj_get_info_by_fd(fd, &info, &len);
 	if (err) {
-		err("can't get prog info: %s\n", strerror(errno));
+		p_err("can't get prog info: %s", strerror(errno));
 		return -1;
 	}
 
 	if (!*member_len) {
-		info("no instructions returned\n");
+		p_info("no instructions returned");
 		close(fd);
 		return 0;
 	}
@@ -545,7 +544,7 @@ static int do_dump(int argc, char **argv)
 
 	buf = malloc(buf_size);
 	if (!buf) {
-		err("mem alloc failed\n");
+		p_err("mem alloc failed");
 		close(fd);
 		return -1;
 	}
@@ -558,28 +557,28 @@ static int do_dump(int argc, char **argv)
 	err = bpf_obj_get_info_by_fd(fd, &info, &len);
 	close(fd);
 	if (err) {
-		err("can't get prog info: %s\n", strerror(errno));
+		p_err("can't get prog info: %s", strerror(errno));
 		goto err_free;
 	}
 
 	if (*member_len > buf_size) {
-		err("too many instructions returned\n");
+		p_err("too many instructions returned");
 		goto err_free;
 	}
 
 	if (filepath) {
 		fd = open(filepath, O_WRONLY | O_CREAT | O_TRUNC, 0600);
 		if (fd < 0) {
-			err("can't open file %s: %s\n", filepath,
-			    strerror(errno));
+			p_err("can't open file %s: %s", filepath,
+			      strerror(errno));
 			goto err_free;
 		}
 
 		n = write(fd, buf, *member_len);
 		close(fd);
 		if (n != *member_len) {
-			err("error writing output file: %s\n",
-			    n < 0 ? strerror(errno) : "short write");
+			p_err("error writing output file: %s",
+			      n < 0 ? strerror(errno) : "short write");
 			goto err_free;
 		}
 	} else {
-- 
2.14.1

Re: [PATCH net-next 09/12] tools: bpftool: turn err() and info() macros into functions

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Turn err() and info() macros into functions.
>
> In order to avoid naming conflicts with variables in the code, rename
> them as p_err() and p_info() respectively.
>
> The behavior of these functions is similar to the one of the macros for
> plain output. However, when JSON output is requested, these macros
> return a JSON-formatted "error" object instead of printing a message to
> stderr.
>
> To handle error messages correctly with JSON, a modification was brought
> to their behavior nonetheless: the functions now append a end-of-line
> character at the end of the message. This way, we can remove end-of-line
> characters at the end of the argument strings, and not have them in the
> JSON output.
>
> All error messages are formatted to hold in a single call to p_err(), in
> order to produce a single JSON field.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
> Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

Re: [PATCH net-next 09/12] tools: bpftool: turn err() and info() macros into functions

[Joe Perches]

On Mon, 2017-10-23 at 09:24 -0700, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
> 
> Turn err() and info() macros into functions.
> 
> In order to avoid naming conflicts with variables in the code, rename
> them as p_err() and p_info() respectively.
> 
> The behavior of these functions is similar to the one of the macros for
> plain output. However, when JSON output is requested, these macros
> return a JSON-formatted "error" object instead of printing a message to
> stderr.
> 
> To handle error messages correctly with JSON, a modification was brought
> to their behavior nonetheless: the functions now append a end-of-line
> character at the end of the message. This way, we can remove end-of-line
> characters at the end of the argument strings, and not have them in the
> JSON output.
> 
> All error messages are formatted to hold in a single call to p_err(), in
> order to produce a single JSON field.

> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
> Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
[]
> diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
[]
> @@ -97,4 +93,35 @@ int prog_parse_fd(int *argc, char ***argv);
>  void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes);
>  void print_hex_data_json(uint8_t *data, size_t len);
>  
> +static inline void p_err(const char *fmt, ...)
> +{
> +	va_list ap;
> +
> +	va_start(ap, fmt);
> +	if (json_output) {
> +		jsonw_start_object(json_wtr);
> +		jsonw_name(json_wtr, "error");
> +		jsonw_vprintf_enquote(json_wtr, fmt, ap);
> +		jsonw_end_object(json_wtr);
> +	} else {
> +		fprintf(stderr, "Error: ");
> +		vfprintf(stderr, fmt, ap);
> +		fprintf(stderr, "\n");
> +	}
> +	va_end(ap);
> +}

inline seems very wasteful.

Why not move p_err and p_info to common.c ?

> +
> +static inline void p_info(const char *fmt, ...)
> +{
> +	va_list ap;
> +
> +	if (json_output)
> +		return;
> +
> +	va_start(ap, fmt);
> +	vfprintf(stderr, fmt, ap);
> +	fprintf(stderr, "\n");
> +	va_end(ap);
> +}
> 

Re: [PATCH net-next 09/12] tools: bpftool: turn err() and info() macros into functions

[Quentin Monnet]

2017-11-002 17:59 UTC-0700 ~ Joe Perches <joe@perches.com>
> On Mon, 2017-10-23 at 09:24 -0700, Jakub Kicinski wrote:
>> From: Quentin Monnet <quentin.monnet@netronome.com>
>>
>> Turn err() and info() macros into functions.
>>
>> In order to avoid naming conflicts with variables in the code, rename
>> them as p_err() and p_info() respectively.
>>
>> The behavior of these functions is similar to the one of the macros for
>> plain output. However, when JSON output is requested, these macros
>> return a JSON-formatted "error" object instead of printing a message to
>> stderr.
>>
>> To handle error messages correctly with JSON, a modification was brought
>> to their behavior nonetheless: the functions now append a end-of-line
>> character at the end of the message. This way, we can remove end-of-line
>> characters at the end of the argument strings, and not have them in the
>> JSON output.
>>
>> All error messages are formatted to hold in a single call to p_err(), in
>> order to produce a single JSON field.
>> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
>> Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
> []
>> diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
> []
>> @@ -97,4 +93,35 @@ int prog_parse_fd(int *argc, char ***argv);
>>  void disasm_print_insn(unsigned char *image, ssize_t len, int opcodes);
>>  void print_hex_data_json(uint8_t *data, size_t len);
>>  
>> +static inline void p_err(const char *fmt, ...)
>> +{
>> +	va_list ap;
>> +
>> +	va_start(ap, fmt);
>> +	if (json_output) {
>> +		jsonw_start_object(json_wtr);
>> +		jsonw_name(json_wtr, "error");
>> +		jsonw_vprintf_enquote(json_wtr, fmt, ap);
>> +		jsonw_end_object(json_wtr);
>> +	} else {
>> +		fprintf(stderr, "Error: ");
>> +		vfprintf(stderr, fmt, ap);
>> +		fprintf(stderr, "\n");
>> +	}
>> +	va_end(ap);
>> +}
> inline seems very wasteful.
>
> Why not move p_err and p_info to common.c ?

Hi Joe,
That's a good point. I wrote a patch to change that, Jakub posted it
some minutes ago. Thanks for your feedback!

Quentin

[PATCH net-next 10/12] tools: bpftool: provide JSON output for all possible commands

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

As all commands can now return JSON output (possibly just a "null"
value), output of `bpftool --json batch file FILE` should also be fully
JSON compliant.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/main.c | 25 +++++++++++++++++++++----
 tools/bpf/bpftool/map.c  | 16 +++++++++++++++-
 tools/bpf/bpftool/prog.c | 12 +++++++++++-
 3 files changed, 47 insertions(+), 6 deletions(-)

diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 9989a77fdc4a..55ba0a04c102 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -64,6 +64,11 @@ void usage(void)
 
 static int do_help(int argc, char **argv)
 {
+	if (json_output) {
+		jsonw_null(json_wtr);
+		return 0;
+	}
+
 	fprintf(stderr,
 		"Usage: %s OBJECT { COMMAND | help }\n"
 		"       %s batch file FILE\n"
@@ -77,10 +82,22 @@ static int do_help(int argc, char **argv)
 
 static int do_version(int argc, char **argv)
 {
-	printf("%s v%d.%d.%d\n", bin_name,
-	       LINUX_VERSION_CODE >> 16,
-	       LINUX_VERSION_CODE >> 8 & 0xf,
-	       LINUX_VERSION_CODE & 0xf);
+	unsigned int version[3];
+
+	version[0] = LINUX_VERSION_CODE >> 16;
+	version[1] = LINUX_VERSION_CODE >> 8 & 0xf;
+	version[2] = LINUX_VERSION_CODE & 0xf;
+
+	if (json_output) {
+		jsonw_start_object(json_wtr);
+		jsonw_name(json_wtr, "version");
+		jsonw_printf(json_wtr, "\"%u.%u.%u\"",
+			     version[0], version[1], version[2]);
+		jsonw_end_object(json_wtr);
+	} else {
+		printf("%s v%u.%u.%u\n", bin_name,
+		       version[0], version[1], version[2]);
+	}
 	return 0;
 }
 
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index 86c128c433ba..a611f31f574f 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -651,6 +651,8 @@ static int do_update(int argc, char **argv)
 	free(value);
 	close(fd);
 
+	if (!err && json_output)
+		jsonw_null(json_wtr);
 	return err;
 }
 
@@ -812,16 +814,28 @@ static int do_delete(int argc, char **argv)
 	free(key);
 	close(fd);
 
+	if (!err && json_output)
+		jsonw_null(json_wtr);
 	return err;
 }
 
 static int do_pin(int argc, char **argv)
 {
-	return do_pin_any(argc, argv, bpf_map_get_fd_by_id);
+	int err;
+
+	err = do_pin_any(argc, argv, bpf_map_get_fd_by_id);
+	if (!err && json_output)
+		jsonw_null(json_wtr);
+	return err;
 }
 
 static int do_help(int argc, char **argv)
 {
+	if (json_output) {
+		jsonw_null(json_wtr);
+		return 0;
+	}
+
 	fprintf(stderr,
 		"Usage: %s %s show   [MAP]\n"
 		"       %s %s dump    MAP\n"
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index 41bd5390b4fc..e07f35ff80d1 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -602,11 +602,21 @@ static int do_dump(int argc, char **argv)
 
 static int do_pin(int argc, char **argv)
 {
-	return do_pin_any(argc, argv, bpf_prog_get_fd_by_id);
+	int err;
+
+	err = do_pin_any(argc, argv, bpf_prog_get_fd_by_id);
+	if (!err && json_output)
+		jsonw_null(json_wtr);
+	return err;
 }
 
 static int do_help(int argc, char **argv)
 {
+	if (json_output) {
+		jsonw_null(json_wtr);
+		return 0;
+	}
+
 	fprintf(stderr,
 		"Usage: %s %s show [PROG]\n"
 		"       %s %s dump xlated PROG [{ file FILE | opcodes }]\n"
-- 
2.14.1

Re: [PATCH net-next 10/12] tools: bpftool: provide JSON output for all possible commands

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> As all commands can now return JSON output (possibly just a "null"
> value), output of `bpftool --json batch file FILE` should also be fully
> JSON compliant.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 11/12] tools: bpftool: add cosmetic changes for the manual pages

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Make the look-and-feel of the manual pages somewhat closer to other
manual pages, such as the ones from the utilities from iproute2, by
highlighting more keywords.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>
---
 tools/bpf/bpftool/Documentation/bpftool-map.rst  | 25 ++++++++++++------------
 tools/bpf/bpftool/Documentation/bpftool-prog.rst | 12 ++++++------
 tools/bpf/bpftool/Documentation/bpftool.rst      | 12 +++++++-----
 3 files changed, 26 insertions(+), 23 deletions(-)

diff --git a/tools/bpf/bpftool/Documentation/bpftool-map.rst b/tools/bpf/bpftool/Documentation/bpftool-map.rst
index 5210c4fab356..3954b3ea4f26 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-map.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-map.rst
@@ -13,23 +13,24 @@ SYNOPSIS
 	**bpftool** **map** *COMMAND*
 
 	*COMMANDS* :=
-	{ show | dump | update | lookup | getnext | delete | pin | help }
+	{ **show** | **dump** | **update** | **lookup** | **getnext** | **delete**
+	| **pin** | **help** }
 
 MAP COMMANDS
 =============
 
-|	**bpftool** map show   [*MAP*]
-|	**bpftool** map dump    *MAP*
-|	**bpftool** map update  *MAP*  key *BYTES*   value *VALUE* [*UPDATE_FLAGS*]
-|	**bpftool** map lookup  *MAP*  key *BYTES*
-|	**bpftool** map getnext *MAP* [key *BYTES*]
-|	**bpftool** map delete  *MAP*  key *BYTES*
-|	**bpftool** map pin     *MAP*  *FILE*
-|	**bpftool** map help
+|	**bpftool** **map show**   [*MAP*]
+|	**bpftool** **map dump**    *MAP*
+|	**bpftool** **map update**  *MAP*  **key** *BYTES*   **value** *VALUE* [*UPDATE_FLAGS*]
+|	**bpftool** **map lookup**  *MAP*  **key** *BYTES*
+|	**bpftool** **map getnext** *MAP* [**key** *BYTES*]
+|	**bpftool** **map delete**  *MAP*  **key** *BYTES*
+|	**bpftool** **map pin**     *MAP*  *FILE*
+|	**bpftool** **map help**
 |
-|	*MAP* := { id MAP_ID | pinned FILE }
-|	*VALUE* := { BYTES | MAP | PROGRAM }
-|	*UPDATE_FLAGS* := { any | exist | noexist }
+|	*MAP* := { **id** *MAP_ID* | **pinned** *FILE* }
+|	*VALUE* := { *BYTES* | *MAP* | *PROGRAM* }
+|	*UPDATE_FLAGS* := { **any** | **exist** | **noexist** }
 
 DESCRIPTION
 ===========
diff --git a/tools/bpf/bpftool/Documentation/bpftool-prog.rst b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
index 6620a81d9dc9..685a19e71fec 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-prog.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
@@ -10,13 +10,13 @@ tool for inspection and simple manipulation of eBPF progs
 SYNOPSIS
 ========
 
-|	**bpftool** prog show [*PROG*]
-|	**bpftool** prog dump xlated *PROG* [{file *FILE* | opcodes }]
-|	**bpftool** prog dump jited  *PROG* [{file *FILE* | opcodes }]
-|	**bpftool** prog pin *PROG* *FILE*
-|	**bpftool** prog help
+|	**bpftool** **prog show** [*PROG*]
+|	**bpftool** **prog dump xlated** *PROG* [{**file** *FILE* | **opcodes**}]
+|	**bpftool** **prog dump jited**  *PROG* [{**file** *FILE* | **opcodes**}]
+|	**bpftool** **prog pin** *PROG* *FILE*
+|	**bpftool** **prog help**
 |
-|	*PROG* := { id *PROG_ID* | pinned *FILE* | tag *PROG_TAG* }
+|	*PROG* := { **id** *PROG_ID* | **pinned** *FILE* | **tag** *PROG_TAG* }
 
 DESCRIPTION
 ===========
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index 9c04cd6677bd..44e07799d54d 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -10,18 +10,20 @@ tool for inspection and simple manipulation of eBPF programs and maps
 SYNOPSIS
 ========
 
-	**bpftool** *OBJECT* { *COMMAND* | help }
+	**bpftool** *OBJECT* { *COMMAND* | **help** }
 
-	**bpftool** batch file *FILE*
+	**bpftool** **batch file** *FILE*
 
-	**bpftool** version
+	**bpftool** **version**
 
 	*OBJECT* := { **map** | **program** }
 
 	*MAP-COMMANDS* :=
-	{ show | dump | update | lookup | getnext | delete | pin | help }
+	{ **show** | **dump** | **update** | **lookup** | **getnext** | **delete**
+	| **pin** | **help** }
 
-	*PROG-COMMANDS* := { show | dump jited | dump xlated | pin | help }
+	*PROG-COMMANDS* := { **show** | **dump jited** | **dump xlated** | **pin**
+	| **help** }
 
 DESCRIPTION
 ===========
-- 
2.14.1

Re: [PATCH net-next 11/12] tools: bpftool: add cosmetic changes for the manual pages

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Make the look-and-feel of the manual pages somewhat closer to other
> manual pages, such as the ones from the utilities from iproute2, by
> highlighting more keywords.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
> Acked-by: Jakub Kicinski <jakub.kicinski@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

[PATCH net-next 12/12] tools: bpftool: update documentation for --json and --pretty usage

[Jakub Kicinski]

From: Quentin Monnet <quentin.monnet@netronome.com>

Update the documentation to provide help about JSON output generation,
and add an example in bpftool-prog manual page.

Also reintroduce an example that was left aside when the tool was moved
from GitHub to the kernel sources, in order to show how to mount the
bpffs file system (to pin programs) inside the bpftool-prog manual page.

Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>
---
 tools/bpf/bpftool/Documentation/bpftool-map.rst  | 11 ++++-
 tools/bpf/bpftool/Documentation/bpftool-prog.rst | 61 +++++++++++++++++++++++-
 tools/bpf/bpftool/Documentation/bpftool.rst      | 12 ++++-
 tools/bpf/bpftool/main.c                         |  6 ++-
 tools/bpf/bpftool/main.h                         |  2 +
 tools/bpf/bpftool/map.c                          |  1 +
 tools/bpf/bpftool/prog.c                         |  1 +
 7 files changed, 88 insertions(+), 6 deletions(-)

diff --git a/tools/bpf/bpftool/Documentation/bpftool-map.rst b/tools/bpf/bpftool/Documentation/bpftool-map.rst
index 3954b3ea4f26..abb9ee940b15 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-map.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-map.rst
@@ -10,7 +10,9 @@ tool for inspection and simple manipulation of eBPF maps
 SYNOPSIS
 ========
 
-	**bpftool** **map** *COMMAND*
+	**bpftool** [*OPTIONS*] **map** *COMMAND*
+
+	*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
 
 	*COMMANDS* :=
 	{ **show** | **dump** | **update** | **lookup** | **getnext** | **delete**
@@ -77,6 +79,13 @@ OPTIONS
 	-v, --version
 		  Print version number (similar to **bpftool version**).
 
+	-j, --json
+		  Generate JSON output. For commands that cannot produce JSON, this
+		  option has no effect.
+
+	-p, --pretty
+		  Generate human-readable JSON output. Implies **-j**.
+
 EXAMPLES
 ========
 **# bpftool map show**
diff --git a/tools/bpf/bpftool/Documentation/bpftool-prog.rst b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
index 685a19e71fec..0f25d3c39e05 100644
--- a/tools/bpf/bpftool/Documentation/bpftool-prog.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool-prog.rst
@@ -10,6 +10,16 @@ tool for inspection and simple manipulation of eBPF progs
 SYNOPSIS
 ========
 
+	**bpftool** [*OPTIONS*] **prog** *COMMAND*
+
+	*OPTIONS* := { { **-j** | **--json** } [{ **-p** | **--pretty** }] }
+
+	*COMMANDS* :=
+	{ **show** | **dump xlated** | **dump jited** | **pin** | **help** }
+
+MAP COMMANDS
+=============
+
 |	**bpftool** **prog show** [*PROG*]
 |	**bpftool** **prog dump xlated** *PROG* [{**file** *FILE* | **opcodes**}]
 |	**bpftool** **prog dump jited**  *PROG* [{**file** *FILE* | **opcodes**}]
@@ -58,6 +68,13 @@ OPTIONS
 	-v, --version
 		  Print version number (similar to **bpftool version**).
 
+	-j, --json
+		  Generate JSON output. For commands that cannot produce JSON, this
+		  option has no effect.
+
+	-p, --pretty
+		  Generate human-readable JSON output. Implies **-j**.
+
 EXAMPLES
 ========
 **# bpftool prog show**
@@ -67,13 +84,33 @@ EXAMPLES
 	loaded_at Sep 29/20:11  uid 0
 	xlated 528B  jited 370B  memlock 4096B  map_ids 10
 
+**# bpftool --json --pretty prog show**
+
+::
+
+    {
+        "programs": [{
+                "id": 10,
+                "type": "xdp",
+                "tag": "005a3d2123620c8b",
+                "loaded_at": "Sep 29/20:11",
+                "uid": 0,
+                "bytes_xlated": 528,
+                "jited": true,
+                "bytes_jited": 370,
+                "bytes_memlock": 4096,
+                "map_ids": [10
+                ]
+            }
+        ]
+    }
+
 |
 | **# bpftool prog dump xlated id 10 file /tmp/t**
 | **# ls -l /tmp/t**
 |   -rw------- 1 root root 560 Jul 22 01:42 /tmp/t
 
-|
-| **# bpftool prog dum jited pinned /sys/fs/bpf/prog**
+**# bpftool prog dum jited tag 005a3d2123620c8b**
 
 ::
 
@@ -83,6 +120,26 @@ EXAMPLES
     sub    $0x28,%rbp
     mov    %rbx,0x0(%rbp)
 
+|
+| **# mount -t bpf none /sys/fs/bpf/**
+| **# bpftool prog pin id 10 /sys/fs/bpf/prog**
+| **# ls -l /sys/fs/bpf/**
+|   -rw------- 1 root root 0 Jul 22 01:43 prog
+
+**# bpftool prog dum jited pinned /sys/fs/bpf/prog opcodes**
+
+::
+
+    push   %rbp
+    55
+    mov    %rsp,%rbp
+    48 89 e5
+    sub    $0x228,%rsp
+    48 81 ec 28 02 00 00
+    sub    $0x28,%rbp
+    48 83 ed 28
+    mov    %rbx,0x0(%rbp)
+    48 89 5d 00
 
 
 SEE ALSO
diff --git a/tools/bpf/bpftool/Documentation/bpftool.rst b/tools/bpf/bpftool/Documentation/bpftool.rst
index 44e07799d54d..926c03d5a8da 100644
--- a/tools/bpf/bpftool/Documentation/bpftool.rst
+++ b/tools/bpf/bpftool/Documentation/bpftool.rst
@@ -10,7 +10,7 @@ tool for inspection and simple manipulation of eBPF programs and maps
 SYNOPSIS
 ========
 
-	**bpftool** *OBJECT* { *COMMAND* | **help** }
+	**bpftool** [*OPTIONS*] *OBJECT* { *COMMAND* | **help** }
 
 	**bpftool** **batch file** *FILE*
 
@@ -18,6 +18,9 @@ SYNOPSIS
 
 	*OBJECT* := { **map** | **program** }
 
+	*OPTIONS* := { { **-V** | **--version** } | { **-h** | **--help** }
+	| { **-j** | **--json** } [{ **-p** | **--pretty** }] }
+
 	*MAP-COMMANDS* :=
 	{ **show** | **dump** | **update** | **lookup** | **getnext** | **delete**
 	| **pin** | **help** }
@@ -41,6 +44,13 @@ OPTIONS
 	-v, --version
 		  Print version number (similar to **bpftool version**).
 
+	-j, --json
+		  Generate JSON output. For commands that cannot produce JSON, this
+		  option has no effect.
+
+	-p, --pretty
+		  Generate human-readable JSON output. Implies **-j**.
+
 SEE ALSO
 ========
 	**bpftool-map**\ (8), **bpftool-prog**\ (8)
diff --git a/tools/bpf/bpftool/main.c b/tools/bpf/bpftool/main.c
index 55ba0a04c102..78d9afb74ef4 100644
--- a/tools/bpf/bpftool/main.c
+++ b/tools/bpf/bpftool/main.c
@@ -70,11 +70,13 @@ static int do_help(int argc, char **argv)
 	}
 
 	fprintf(stderr,
-		"Usage: %s OBJECT { COMMAND | help }\n"
+		"Usage: %s [OPTIONS] OBJECT { COMMAND | help }\n"
 		"       %s batch file FILE\n"
 		"       %s version\n"
 		"\n"
-		"       OBJECT := { prog | map }\n",
+		"       OBJECT := { prog | map }\n"
+		"       " HELP_SPEC_OPTIONS "\n"
+		"",
 		bin_name, bin_name, bin_name);
 
 	return 0;
diff --git a/tools/bpf/bpftool/main.h b/tools/bpf/bpftool/main.h
index 04c88b55d8c7..2f94bed03a8d 100644
--- a/tools/bpf/bpftool/main.h
+++ b/tools/bpf/bpftool/main.h
@@ -55,6 +55,8 @@
 
 #define HELP_SPEC_PROGRAM						\
 	"PROG := { id PROG_ID | pinned FILE | tag PROG_TAG }"
+#define HELP_SPEC_OPTIONS						\
+	"OPTIONS := { {-j|--json} [{-p|--pretty}] }"
 
 enum bpf_obj_type {
 	BPF_OBJ_UNKNOWN,
diff --git a/tools/bpf/bpftool/map.c b/tools/bpf/bpftool/map.c
index a611f31f574f..e978ab23a77f 100644
--- a/tools/bpf/bpftool/map.c
+++ b/tools/bpf/bpftool/map.c
@@ -850,6 +850,7 @@ static int do_help(int argc, char **argv)
 		"       " HELP_SPEC_PROGRAM "\n"
 		"       VALUE := { BYTES | MAP | PROG }\n"
 		"       UPDATE_FLAGS := { any | exist | noexist }\n"
+		"       " HELP_SPEC_OPTIONS "\n"
 		"",
 		bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
 		bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
diff --git a/tools/bpf/bpftool/prog.c b/tools/bpf/bpftool/prog.c
index e07f35ff80d1..250f80fd46aa 100644
--- a/tools/bpf/bpftool/prog.c
+++ b/tools/bpf/bpftool/prog.c
@@ -625,6 +625,7 @@ static int do_help(int argc, char **argv)
 		"       %s %s help\n"
 		"\n"
 		"       " HELP_SPEC_PROGRAM "\n"
+		"       " HELP_SPEC_OPTIONS "\n"
 		"",
 		bin_name, argv[-2], bin_name, argv[-2], bin_name, argv[-2],
 		bin_name, argv[-2], bin_name, argv[-2]);
-- 
2.14.1

Re: [PATCH net-next 12/12] tools: bpftool: update documentation for --json and --pretty usage

[Daniel Borkmann]

On 10/23/2017 06:24 PM, Jakub Kicinski wrote:
> From: Quentin Monnet <quentin.monnet@netronome.com>
>
> Update the documentation to provide help about JSON output generation,
> and add an example in bpftool-prog manual page.
>
> Also reintroduce an example that was left aside when the tool was moved
> from GitHub to the kernel sources, in order to show how to mount the
> bpffs file system (to pin programs) inside the bpftool-prog manual page.
>
> Signed-off-by: Quentin Monnet <quentin.monnet@netronome.com>

Acked-by: Daniel Borkmann <daniel@iogearbox.net>

Re: [PATCH net-next 00/12] tools: bpftool: Add JSON output to bpftool

[Alexei Starovoitov]

On Mon, Oct 23, 2017 at 09:24:04AM -0700, Jakub Kicinski wrote:
> Quentin says:
> 
> This series introduces support for JSON output to all bpftool commands. It
> adds option parsing, and several options are created:
> 
>   * -j, --json     Switch to JSON output.
>   * -p, --pretty   Switch to JSON and print it in a human-friendly fashion.
>   * -h, --help     Print generic help message.
>   * -V, --version  Print version number.
> 
> This code uses a "json_writer", which is a copy of the one written by
> Stephen Hemminger in iproute2.
> ---
> I don't know if there is an easy way to share the code for json_write
> without copying the file, so I am very open to suggestions on this matter.

I think copy-paste is fine in this case.
If we ever decide to remove it from here and move it into iproute2
this json.[hc] can be unified, but I think kernel/tools/bpf/ is
a better place since introspection interface is still evolving a lot
and bpftool is not networking specific. Like would be great if it
can show what programs attached to which cgroups, tracepoints, kprobes.

Overall looks like a nice set of improvements.

Acked-by: Alexei Starovoitov <ast@kernel.org>

Re: [PATCH net-next 00/12] tools: bpftool: Add JSON output to bpftool

[David Miller]

From: Jakub Kicinski <jakub.kicinski@netronome.com>
Date: Mon, 23 Oct 2017 09:24:04 -0700

> Quentin says:
> 
> This series introduces support for JSON output to all bpftool commands. It
> adds option parsing, and several options are created:
> 
>   * -j, --json     Switch to JSON output.
>   * -p, --pretty   Switch to JSON and print it in a human-friendly fashion.
>   * -h, --help     Print generic help message.
>   * -V, --version  Print version number.
> 
> This code uses a "json_writer", which is a copy of the one written by
> Stephen Hemminger in iproute2.
> ---
> I don't know if there is an easy way to share the code for json_write
> without copying the file, so I am very open to suggestions on this matter.

Series applied.