All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH 0/8] Fast bulk transfers of large sets of ct entries
@ 2020-09-25 12:49 Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky
                   ` (8 more replies)
  0 siblings, 9 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Hi Fellows,

This is a set of patches to the conntrack tool that I think 
might be interesting to the community.

The PATCH 1 and PATCH 2 represent a fix to the icmp ct entry creation 
(a test case and a fix respectively).

The remaining patches represent an extension to the conntrack tool
functionality that enables fast bulk transfers of large sets of ct 
entries, which includes creating a large set of ct entries with
a single conntrack tool invocation by passing ct entry parameters
on stdin and making conntrack be able to dump ct entries in a new
"opts" format that could be later fed back to the conntrack.

To demonstrate the overall idea, this functionality makes it possible 
to e.g. copy all ct entries from one ct zone (15) to another (9915) 
with the following command:

  conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I -


In addition to this I have a question about the behavioural change
of the "conntrack -L" done after conntrack v1.4.5.
With the conntrack v1.4.5 used on Debian Buster the "conntrack -L"
dumps both ipv4 and ipv6 ct entries, while with the current master, 
presumably starting with the commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e 
"conntrack -L"  only dumps ipv4 entries.

So is this really the desired behavior? 
(I see the manual page was always saying it should be like that,
but since it behaved differently there might be multiple appliances 
out there relying on the "old" behavior).

And if the "new" behavior is desired, would it make sense to add a new 
-f option value, e.g. "any", that would actually explicitly allow the 
"old" behaviour, i.e. dump both ipv4 and ipv6 entries with one go?
If yes - I could create a small patch for that as well.

Thanks & Regards,
Mikhail

Mikhail Sennikovsky (8):
  tests: icmp entry create/delete
  conntrack: fix icmp entry creation
  conntrack: accept parameters from stdin
  conntrack.8: man update for stdin params support
  tests: conntrack parameters from stdin
  conntrack: implement options output format
  conntrack.8: man update for opts format support
  tests: dumping ct entries in opts format

 conntrack.8                         |  13 +-
 extensions/libct_proto_dccp.c       |  24 ++
 extensions/libct_proto_gre.c        |  16 +
 extensions/libct_proto_icmp.c       |  33 ++
 extensions/libct_proto_icmpv6.c     |  33 ++
 extensions/libct_proto_sctp.c       |  19 ++
 extensions/libct_proto_tcp.c        |  17 ++
 extensions/libct_proto_udp.c        |  16 +
 extensions/libct_proto_udplite.c    |  16 +
 include/conntrack.h                 |  38 +++
 src/conntrack.c                     | 457 +++++++++++++++++++++++++---
 tests/conntrack/test-conntrack.c    |  84 ++++-
 tests/conntrack/testsuite/00create  |   4 +
 tests/conntrack/testsuite/08stdin   |  62 ++++
 tests/conntrack/testsuite/09dumpopt |  77 +++++
 15 files changed, 857 insertions(+), 52 deletions(-)
 create mode 100644 tests/conntrack/testsuite/08stdin
 create mode 100644 tests/conntrack/testsuite/09dumpopt

-- 
2.25.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH 1/8] tests: icmp entry create/delete
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-10-13 18:28   ` Pablo Neira Ayuso
  2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky
                   ` (7 subsequent siblings)
  8 siblings, 1 reply; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Add test to cover icmp entry creation/deletion with conntrack

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 tests/conntrack/testsuite/00create | 4 ++++
 1 file changed, 4 insertions(+)

diff --git a/tests/conntrack/testsuite/00create b/tests/conntrack/testsuite/00create
index 4e55a7b..911e711 100644
--- a/tests/conntrack/testsuite/00create
+++ b/tests/conntrack/testsuite/00create
@@ -30,3 +30,7 @@
 -D -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
 # mismatched address family
 -I -s 2001:DB8::1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+# creae icmp ping request entry
+-I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# delete icmp ping request entry
+-D -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 2/8] conntrack: fix icmp entry creation
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-10-13 18:29   ` Pablo Neira Ayuso
  2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky
                   ` (6 subsequent siblings)
  8 siblings, 1 reply; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Creating icmp ct entry with command like

conntrack -I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 \
   -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226

results in nfct_query( NFCT_Q_CREATE ) request would fail
because reply L4 proto is not set while having reply data specified

Set reply L4 proto when reply data is given for the icmp ct entry

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 extensions/libct_proto_icmp.c   | 18 ++++++++++++++++++
 extensions/libct_proto_icmpv6.c | 18 ++++++++++++++++++
 2 files changed, 36 insertions(+)

diff --git a/extensions/libct_proto_icmp.c b/extensions/libct_proto_icmp.c
index 2ce1c65..16c2e2e 100644
--- a/extensions/libct_proto_icmp.c
+++ b/extensions/libct_proto_icmp.c
@@ -78,18 +78,36 @@ static int parse(char c,
 			tmp = atoi(optarg);
 			nfct_set_attr_u8(ct, ATTR_ICMP_TYPE, tmp);
 			nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMP);
+			/*
+			 * need to set the reply proto, otherwise the
+			 * NFCT_Q_CREATE call would fail
+			 */
+			if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO))
+				nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMP);
 			*flags |= CT_ICMP_TYPE;
 			break;
 		case '2':
 			tmp = atoi(optarg);
 			nfct_set_attr_u8(ct, ATTR_ICMP_CODE, tmp);
 			nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMP);
+			/*
+			 * need to set the reply proto, otherwise the
+			 * NFCT_Q_CREATE call would fail
+			 */
+			if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO))
+				nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMP);
 			*flags |= CT_ICMP_CODE;
 			break;
 		case '3':
 			id = htons(atoi(optarg));
 			nfct_set_attr_u16(ct, ATTR_ICMP_ID, id);
 			nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMP);
+			/*
+			 * need to set the reply proto, otherwise the
+			 * NFCT_Q_CREATE call would fail
+			 */
+			if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO))
+				nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMP);
 			*flags |= CT_ICMP_ID;
 			break;
 	}
diff --git a/extensions/libct_proto_icmpv6.c b/extensions/libct_proto_icmpv6.c
index 18dd3e5..7f5e637 100644
--- a/extensions/libct_proto_icmpv6.c
+++ b/extensions/libct_proto_icmpv6.c
@@ -81,18 +81,36 @@ static int parse(char c,
 			tmp = atoi(optarg);
 			nfct_set_attr_u8(ct, ATTR_ICMP_TYPE, tmp);
 			nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMPV6);
+			/*
+			 * need to set the reply proto, otherwise the
+			 * NFCT_Q_CREATE call would fail
+			 */
+			if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO))
+				nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMPV6);
 			*flags |= CT_ICMP_TYPE;
 			break;
 		case '2':
 			tmp = atoi(optarg);
 			nfct_set_attr_u8(ct, ATTR_ICMP_CODE, tmp);
 			nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMPV6);
+			/*
+			 * need to set the reply proto, otherwise the
+			 * NFCT_Q_CREATE call would fail
+			 */
+			if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO))
+				nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMPV6);
 			*flags |= CT_ICMP_CODE;
 			break;
 		case '3':
 			id = htons(atoi(optarg));
 			nfct_set_attr_u16(ct, ATTR_ICMP_ID, id);
 			nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMPV6);
+			/*
+			 * need to set the reply proto, otherwise the
+			 * NFCT_Q_CREATE call would fail
+			 */
+			if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO))
+				nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMPV6);
 			*flags |= CT_ICMP_ID;
 			break;
 	}
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 3/8] conntrack: accept parameters from stdin
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-11-04 13:56   ` Pablo Neira Ayuso
  2020-09-25 12:49 ` [PATCH 4/8] conntrack.8: man update for stdin params support Mikhail Sennikovsky
                   ` (5 subsequent siblings)
  8 siblings, 1 reply; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

This commit allows accepting multiple sets of ct entry-related
parameters on stdin.
This is useful when one needs to add/update/delete a large
set of ct entries with a single conntrack tool invocation.

Expected syntax is "conntrack [-I|-D|-U] [table] -".
When invoked like that, conntrack expects ct entry parameters
to be passed to the stdin, each line presenting a separate parameter
set.

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 src/conntrack.c | 196 +++++++++++++++++++++++++++++++++++++++---------
 1 file changed, 161 insertions(+), 35 deletions(-)

diff --git a/src/conntrack.c b/src/conntrack.c
index a26fa60..5834f2d 100644
--- a/src/conntrack.c
+++ b/src/conntrack.c
@@ -96,15 +96,18 @@ static struct {
 	struct nfct_bitmask *label_modify;
 } tmpl;
 
+int cur_argc;
+char **cur_argv;
+
 static int alloc_tmpl_objects(void)
 {
+	memset(&tmpl, 0, sizeof(tmpl));
+
 	tmpl.ct = nfct_new();
 	tmpl.exptuple = nfct_new();
 	tmpl.mask = nfct_new();
 	tmpl.exp = nfexp_new();
 
-	memset(&tmpl.mark, 0, sizeof(tmpl.mark));
-
 	return tmpl.ct != NULL && tmpl.exptuple != NULL &&
 	       tmpl.mask != NULL && tmpl.exp != NULL;
 }
@@ -685,12 +688,18 @@ static void free_options(void)
 void __attribute__((noreturn))
 exit_error(enum exittype status, const char *msg, ...)
 {
+	int i;
 	va_list args;
 
 	free_options();
 	va_start(args, msg);
 	fprintf(stderr,"%s v%s (conntrack-tools): ", PROGNAME, VERSION);
 	vfprintf(stderr, msg, args);
+	if (cur_argc) {
+		fprintf(stderr, "\nargs:");
+		for (i = 1; i < cur_argc; ++i)
+			fprintf(stderr, " %s", cur_argv[i]);
+	}
 	fprintf(stderr, "\n");
 	va_end(args);
 	if (status == PARAMETER_PROBLEM)
@@ -2317,23 +2326,63 @@ nfct_set_nat_details(const int opt, struct nf_conntrack *ct,
 		nfct_set_attr_u16(ct, ATTR_DNAT_PORT,
 				  ntohs((uint16_t)atoi(port_str)));
 	}
+}
+
+static int line_to_argcv(char *cmd, char *line, char ***pargv, size_t *pargv_size)
+{
+	char *arg;
+	int argc;
+	char **argv = *pargv;
+	size_t argv_size = *pargv_size;
+	int argc_max = argv_size / sizeof(*argv);
+
+#define _ARG_ADD(_arg) do { \
+		if (argc == argc_max) { \
+			argc_max += 20; \
+			argv_size = argc_max * sizeof (argv[0]); \
+			argv = realloc(argv, argv_size); \
+			if (!argv) \
+				exit_error(OTHER_PROBLEM, "out of memory"); \
+		} \
+		argv[argc] = _arg; \
+		++argc; \
+} while (0)
+
+#define _ARG_SEP " \t\n\r"
+	for (argc = 0, arg = strtok (line, _ARG_SEP);
+			arg;
+			arg = strtok (NULL, _ARG_SEP)) {
+		/*
+		 * getopt_long expects argv[0] to be the command name,
+		 * and would always skip it so we need to include it here
+		 */
+		if (!argc && cmd)
+			_ARG_ADD(cmd);
+		_ARG_ADD(arg);
+	}
+
+#undef _ARG_ADD
+#undef _ARG_SEP
 
+	*pargv = argv;
+	*pargv_size = argv_size;
+
+	return argc;
 }
 
 int main(int argc, char *argv[])
 {
 	int c, cmd;
-	unsigned int type = 0, event_mask = 0, l4flags = 0, status = 0;
+	unsigned int type = 0, event_mask = 0, l4flags, status = 0;
 	int res = 0, partial;
-	size_t socketbuffersize = 0;
-	int family = AF_UNSPEC;
-	int protonum = 0;
+	size_t socketbuffersize;
+	int family;
+	int protonum;
 	union ct_address ad;
 	unsigned int command = 0;
-
-	/* we release these objects in the exit_error() path. */
-	if (!alloc_tmpl_objects())
-		exit_error(OTHER_PROBLEM, "out of memory");
+	FILE *opts_file = NULL;
+	char **argv_buf = NULL, *getline_buf = NULL;
+	size_t argv_buf_size = 0, getline_buf_size = 0;
 
 	register_tcp();
 	register_udp();
@@ -2348,6 +2397,23 @@ int main(int argc, char *argv[])
 	/* disable explicit missing arguments error output from getopt_long */
 	opterr = 0;
 
+parse_opts:
+
+	options = 0;
+	filter_family = 0;
+	memset(dir2network, 0, sizeof(dir2network));
+	/* all allocate-able objects get freed zero-inited
+	 * at the end of each iteration */
+
+	l4flags = 0;
+	family = AF_UNSPEC;
+	socketbuffersize = 0;
+	protonum = 0;
+
+	/* we release these objects in the exit_error() path. */
+	if (!alloc_tmpl_objects())
+		exit_error(OTHER_PROBLEM, "out of memory");
+
 	while ((c = getopt_long(argc, argv, getopt_str, opts, NULL)) != -1) {
 	switch(c) {
 		/* commands */
@@ -2585,6 +2651,26 @@ int main(int argc, char *argv[])
 					      "`--dst-nat' with `--any-nat'");
 	}
 	cmd = bit2cmd(command);
+
+	if (!opts_file && optind == argc - 1 && !strcmp(argv[optind], "-")) {
+		switch (command) {
+		case CT_CREATE:
+		case EXP_CREATE:
+		case CT_UPDATE:
+		case CT_DELETE:
+		case EXP_DELETE:
+			break;
+		default:
+			exit_error(PARAMETER_PROBLEM, "stdin mode not supported "
+					"for this command!");
+		}
+		if (options)
+			exit_error(PARAMETER_PROBLEM, "no extra options are expected "
+					"with stdin read mode!");
+		opts_file = stdin;
+		goto next_opts;
+	}
+
 	res = generic_opt_check(options, NUMBER_OF_OPT,
 				commands_v_options[cmd], optflags,
 				addr_valid_flags, ADDR_VALID_FLAGS_MAX,
@@ -2670,8 +2756,8 @@ int main(int argc, char *argv[])
 			printf("</conntrack>\n");
 			fflush(stdout);
 		}
-
 		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case EXP_LIST:
@@ -2681,12 +2767,13 @@ int main(int argc, char *argv[])
 
 		nfexp_callback_register(cth, NFCT_T_ALL, dump_exp_cb, NULL);
 		res = nfexp_query(cth, NFCT_Q_DUMP, &family);
-		nfct_close(cth);
 
 		if (dump_xml_header_done == 0) {
 			printf("</expect>\n");
 			fflush(stdout);
 		}
+		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case CT_CREATE:
@@ -2702,14 +2789,15 @@ int main(int argc, char *argv[])
 			nfct_set_attr(tmpl.ct, ATTR_CONNLABELS,
 					xnfct_bitmask_clone(tmpl.label_modify));
 
-		cth = nfct_open(CONNTRACK, 0);
-		if (!cth)
-			exit_error(OTHER_PROBLEM, "Can't open handler");
+		if (!cth) {
+			cth = nfct_open(CONNTRACK, 0);
+			if (!cth)
+				exit_error(OTHER_PROBLEM, "Can't open handler");
+		}
 
 		res = nfct_query(cth, NFCT_Q_CREATE, tmpl.ct);
 		if (res != -1)
 			counter++;
-		nfct_close(cth);
 		break;
 
 	case EXP_CREATE:
@@ -2717,18 +2805,21 @@ int main(int argc, char *argv[])
 		nfexp_set_attr(tmpl.exp, ATTR_EXP_EXPECTED, tmpl.exptuple);
 		nfexp_set_attr(tmpl.exp, ATTR_EXP_MASK, tmpl.mask);
 
-		cth = nfct_open(EXPECT, 0);
-		if (!cth)
-			exit_error(OTHER_PROBLEM, "Can't open handler");
+		if (!cth) {
+			cth = nfct_open(EXPECT, 0);
+			if (!cth)
+				exit_error(OTHER_PROBLEM, "Can't open handler");
+		}
 
 		res = nfexp_query(cth, NFCT_Q_CREATE, tmpl.exp);
-		nfct_close(cth);
 		break;
 
 	case CT_UPDATE:
-		cth = nfct_open(CONNTRACK, 0);
+		if (!cth)
+			cth = nfct_open(CONNTRACK, 0);
 		/* internal handler for delete_cb, otherwise we hit EILSEQ */
-		ith = nfct_open(CONNTRACK, 0);
+		if (!ith)
+			ith = nfct_open(CONNTRACK, 0);
 		if (!cth || !ith)
 			exit_error(OTHER_PROBLEM, "Can't open handler");
 
@@ -2737,13 +2828,13 @@ int main(int argc, char *argv[])
 		nfct_callback_register(cth, NFCT_T_ALL, update_cb, tmpl.ct);
 
 		res = nfct_query(cth, NFCT_Q_DUMP, &family);
-		nfct_close(ith);
-		nfct_close(cth);
 		break;
 		
 	case CT_DELETE:
-		cth = nfct_open(CONNTRACK, 0);
-		ith = nfct_open(CONNTRACK, 0);
+		if (!cth)
+			cth = nfct_open(CONNTRACK, 0);
+		if (!ith)
+			ith = nfct_open(CONNTRACK, 0);
 		if (!cth || !ith)
 			exit_error(OTHER_PROBLEM, "Can't open handler");
 
@@ -2768,19 +2859,18 @@ int main(int argc, char *argv[])
 
 		nfct_filter_dump_destroy(filter_dump);
 
-		nfct_close(ith);
-		nfct_close(cth);
 		break;
 
 	case EXP_DELETE:
 		nfexp_set_attr(tmpl.exp, ATTR_EXP_EXPECTED, tmpl.ct);
 
-		cth = nfct_open(EXPECT, 0);
-		if (!cth)
-			exit_error(OTHER_PROBLEM, "Can't open handler");
+		if (!cth) {
+			cth = nfct_open(EXPECT, 0);
+			if (!cth)
+				exit_error(OTHER_PROBLEM, "Can't open handler");
+		}
 
 		res = nfexp_query(cth, NFCT_Q_DESTROY, tmpl.exp);
-		nfct_close(cth);
 		break;
 
 	case CT_GET:
@@ -2791,6 +2881,7 @@ int main(int argc, char *argv[])
 		nfct_callback_register(cth, NFCT_T_ALL, dump_cb, tmpl.ct);
 		res = nfct_query(cth, NFCT_Q_GET, tmpl.ct);
 		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case EXP_GET:
@@ -2803,6 +2894,7 @@ int main(int argc, char *argv[])
 		nfexp_callback_register(cth, NFCT_T_ALL, dump_exp_cb, NULL);
 		res = nfexp_query(cth, NFCT_Q_GET, tmpl.exp);
 		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case CT_FLUSH:
@@ -2810,9 +2902,10 @@ int main(int argc, char *argv[])
 		if (!cth)
 			exit_error(OTHER_PROBLEM, "Can't open handler");
 		res = nfct_query(cth, NFCT_Q_FLUSH, &family);
-		nfct_close(cth);
 		fprintf(stderr, "%s v%s (conntrack-tools): ",PROGNAME,VERSION);
 		fprintf(stderr,"connection tracking table has been emptied.\n");
+		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case EXP_FLUSH:
@@ -2820,9 +2913,10 @@ int main(int argc, char *argv[])
 		if (!cth)
 			exit_error(OTHER_PROBLEM, "Can't open handler");
 		res = nfexp_query(cth, NFCT_Q_FLUSH, &family);
-		nfct_close(cth);
 		fprintf(stderr, "%s v%s (conntrack-tools): ",PROGNAME,VERSION);
 		fprintf(stderr,"expectation table has been emptied.\n");
+		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case CT_EVENT:
@@ -2894,6 +2988,8 @@ int main(int argc, char *argv[])
 			res = mnl_cb_run(buf, res, 0, 0, event_cb, tmpl.ct);
 		}
 		mnl_socket_close(sock.mnl);
+		nfct_close(cth);
+		cth = NULL;
 		break;
 
 	case EXP_EVENT:
@@ -2922,6 +3018,7 @@ int main(int argc, char *argv[])
 		nfexp_callback_register(cth, NFCT_T_ALL, event_exp_cb, NULL);
 		res = nfexp_catch(cth);
 		nfct_close(cth);
+		cth = NULL;
 		break;
 	case CT_COUNT:
 		/* If we fail with netlink, fall back to /proc to ensure
@@ -2966,6 +3063,7 @@ try_proc_count:
 		nfexp_callback_register(cth, NFCT_T_ALL, count_exp_cb, NULL);
 		res = nfexp_query(cth, NFCT_Q_DUMP, &family);
 		nfct_close(cth);
+		cth = NULL;
 		printf("%d\n", counter);
 		break;
 	case CT_STATS:
@@ -3024,10 +3122,35 @@ try_proc:
 		exit_error(OTHER_PROBLEM, "Operation failed: %s",
 			   err2str(errno, command));
 
+next_opts:
 	free_tmpl_objects();
 	free_options();
-	if (labelmap)
+	if (labelmap) {
 		nfct_labelmap_destroy(labelmap);
+		labelmap = NULL;
+	}
+
+	if (opts_file) {
+		while ((res = getline(&getline_buf, &getline_buf_size, opts_file)) >= 0) {
+			if (!res)
+				continue;
+			argc = line_to_argcv(argv[0], getline_buf, &argv_buf, &argv_buf_size);
+			if (!argc)
+				continue;
+			argv = argv_buf;
+
+			cur_argc = argc;
+			cur_argv = argv;
+
+			optind = 0;
+			goto parse_opts;
+		}
+	}
+
+	if (ith)
+		nfct_close(ith);
+	if (cth)
+		nfct_close(cth);
 
 	if (command && exit_msg[cmd][0]) {
 		fprintf(stderr, "%s v%s (conntrack-tools): ",PROGNAME,VERSION);
@@ -3036,5 +3159,8 @@ try_proc:
 			return EXIT_FAILURE;
 	}
 
+	free(argv_buf);
+	free(getline_buf);
+
 	return EXIT_SUCCESS;
 }
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 4/8] conntrack.8: man update for stdin params support
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
                   ` (2 preceding siblings ...)
  2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 5/8] tests: conntrack parameters from stdin Mikhail Sennikovsky
                   ` (4 subsequent siblings)
  8 siblings, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 conntrack.8 | 8 ++++++++
 1 file changed, 8 insertions(+)

diff --git a/conntrack.8 b/conntrack.8
index 1174c6c..3db4849 100644
--- a/conntrack.8
+++ b/conntrack.8
@@ -135,6 +135,14 @@ to overrun the socket buffer. Note that using a big buffer reduces the chances
 to hit ENOBUFS, however, this results in more memory consumption.
 .
 This option can only be used in conjunction with "\-E, \-\-event".
+.TP
+.BI "-"
+Make conntrack accept multiple sets of ct entry-related parameters on stdin.
+This option is useful for the fast bulk conntrack entries updates.
+.
+This option can only be used in conjunction with "\-I, \-\-create",
+"\-U, \-\-update" and "\-D, \-\-delete".
+No other options are allowed to be present on the command line.
 
 .SS FILTER PARAMETERS
 .TP
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 5/8] tests: conntrack parameters from stdin
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
                   ` (3 preceding siblings ...)
  2020-09-25 12:49 ` [PATCH 4/8] conntrack.8: man update for stdin params support Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky
                   ` (3 subsequent siblings)
  8 siblings, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Add tests to cover handling parameters from stdin by conntrack

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 tests/conntrack/test-conntrack.c  | 70 +++++++++++++++++++++++++------
 tests/conntrack/testsuite/08stdin | 62 +++++++++++++++++++++++++++
 2 files changed, 120 insertions(+), 12 deletions(-)
 create mode 100644 tests/conntrack/testsuite/08stdin

diff --git a/tests/conntrack/test-conntrack.c b/tests/conntrack/test-conntrack.c
index 76ab051..90bdc5b 100644
--- a/tests/conntrack/test-conntrack.c
+++ b/tests/conntrack/test-conntrack.c
@@ -28,6 +28,23 @@ int main()
 	struct dirent *dent;
 	char file[1024];
 	int i,n;
+	char cmd_buf[1024 * 8];
+	int i_cmd_buf = 0;
+	char cmd, cur_cmd = 0;
+	char *cmd_opt;
+
+#define cmd_strappend(_s) do { \
+	char * pos = stpncpy(cmd_buf + i_cmd_buf, _s, sizeof(cmd_buf) - i_cmd_buf); \
+	i_cmd_buf = pos - cmd_buf; \
+	if (i_cmd_buf == sizeof(cmd_buf)) { \
+		printf("buffer full!\n"); \
+		exit(EXIT_FAILURE); \
+	} \
+} while (0)
+
+#define cmd_reset() do { \
+		i_cmd_buf = 0; \
+} while (0)
 
 	n = scandir("testsuite", &dents, NULL, alphasort);
 
@@ -48,9 +65,7 @@ int main()
 		}
 
 		while (fgets(buf, sizeof(buf), fp)) {
-			char tmp[1024] = CT_PROG, *res;
-			tmp[strlen(CT_PROG)] = ' ';
-
+			char *res;
 			line++;
 
 			if (buf[0] == '#' || buf[0] == ' ')
@@ -63,27 +78,58 @@ int main()
 				exit(EXIT_FAILURE);
 			}
 			*res = '\0';
-			res+=2;
+			res++;
+			for (; *res == ' ' || *res == '\t'; res++);
+			cmd = res[0];
+
+			if (cur_cmd && cmd != cur_cmd) {
+				/* complete current multi-line command */
+				switch (cur_cmd) {
+				case '\n':
+					cmd_strappend("\" | ");
+					break;
+				default:
+					printf("Internal Error: unexpected multiline command %c",
+							cur_cmd);
+					exit(EXIT_FAILURE);
+					break;
+				}
+
+				cur_cmd = 0;
+			}
+
+			switch (cmd) {
+			case '\n':
+				if (!cur_cmd) {
+					cmd_strappend("echo \"");
+					cur_cmd = cmd;
+				} else
+					cmd_strappend("\n");
+				cmd_strappend(buf);
+				continue;
+			default:
+				cmd_strappend(CT_PROG);
+				cmd_strappend(" ");
+				cmd_strappend(buf);
+				cmd_reset();
+				break;
+			}
 
-			strcpy(tmp + strlen(CT_PROG) + 1, buf);
-			printf("(%d) Executing: %s\n", line, tmp);
+			printf("(%d) Executing: %s\n", line, cmd_buf);
 
 			fflush(stdout);
-			ret = system(tmp);
+			ret = system(cmd_buf);
 
 			if (WIFEXITED(ret) &&
 			    WEXITSTATUS(ret) == EXIT_SUCCESS) {
-			    	if (res[0] == 'O' &&
-				    res[1] == 'K')
+				if (cmd == 'O')
 					ok++;
 				else {
 					bad++;
 					printf("^----- BAD\n");
 				}
 			} else {
-				if (res[0] == 'B' &&
-				    res[1] == 'A' &&
-				    res[2] == 'D')
+				if (cmd == 'B')
 					ok++;
 				else {
 					bad++;
diff --git a/tests/conntrack/testsuite/08stdin b/tests/conntrack/testsuite/08stdin
new file mode 100644
index 0000000..cf3eadd
--- /dev/null
+++ b/tests/conntrack/testsuite/08stdin
@@ -0,0 +1,62 @@
+# create
+# create a conntrack
+-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+# create from reply
+-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ;
+# create a v6 conntrack
+-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+# creae icmp ping request entry
+-t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ;
+-I - ; OK
+# create again
+-I -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+# make sure create again with stdio mode fails as well
+-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+-I - ; BAD
+-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ;
+-I - ; BAD
+# empty lines are ignored
+;
+-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+-I - ; BAD
+# spaces or tabs are ignored as well
+  ;
+		;
+-t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ;
+-I - ; BAD
+# delete
+-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ;
+# empty lines should be just ignored
+;
+;
+# delete reverse
+-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ;
+# empty lines with spaces or tabs should be ignored as well
+ ;
+	;
+		;
+  ;
+	    ;
+	    	    	;
+# delete v6 conntrack
+-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ;
+# delete icmp ping request entry
+-u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ;
+;
+;
+-D - ; OK
+# create again - should succeed now
+-I -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK
+-I -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; OK
+-I -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK
+-I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# delete again (for cleanup)
+-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ;
+-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ;
+-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ;
+-u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ;
+;
+-D - ; OK
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 6/8] conntrack: implement options output format
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
                   ` (4 preceding siblings ...)
  2020-09-25 12:49 ` [PATCH 5/8] tests: conntrack parameters from stdin Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-10-22 12:36   ` Pablo Neira Ayuso
  2020-09-25 12:49 ` [PATCH 7/8] conntrack.8: man update for opts format support Mikhail Sennikovsky
                   ` (2 subsequent siblings)
  8 siblings, 1 reply; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

As a counterpart to the "conntrack: accept parameters from stdin"
commit, this commit allows dumping conntrack entries in the format
used by the conntrack parameters.
This is useful for transfering a large set of ct entries between
hosts or between different ct zones in an efficient way.

To enable the "options" output the "-o opts" parameter needs to be
passed to the "contnrack -L" tool invocation.

To demonstrate the overall idea of the options output format works
in conjunction with the "stdin parameter"s mode,
the following command will copy all ct entries from one ct zone
to another.

conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I -

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 extensions/libct_proto_dccp.c    |  24 +++
 extensions/libct_proto_gre.c     |  16 ++
 extensions/libct_proto_icmp.c    |  15 ++
 extensions/libct_proto_icmpv6.c  |  15 ++
 extensions/libct_proto_sctp.c    |  19 +++
 extensions/libct_proto_tcp.c     |  17 ++
 extensions/libct_proto_udp.c     |  16 ++
 extensions/libct_proto_udplite.c |  16 ++
 include/conntrack.h              |  38 +++++
 src/conntrack.c                  | 261 ++++++++++++++++++++++++++++++-
 10 files changed, 433 insertions(+), 4 deletions(-)

diff --git a/extensions/libct_proto_dccp.c b/extensions/libct_proto_dccp.c
index f6258ad..cfe1313 100644
--- a/extensions/libct_proto_dccp.c
+++ b/extensions/libct_proto_dccp.c
@@ -198,6 +198,29 @@ static int parse_options(char c,
 	return 1;
 }
 
+
+static const char *dccp_roles[__DCCP_CONNTRACK_ROLE_MAX] = {
+	[DCCP_CONNTRACK_ROLE_CLIENT]	= "client",
+	[DCCP_CONNTRACK_ROLE_SERVER]	= "server",
+};
+
+static struct ctproto_attr attrs[] = {
+		{"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--state", ATTR_DCCP_STATE, CTPROTO_ATTR_U8, DCCP_CONNTRACK_MAX, dccp_states},
+		{"--role", ATTR_DCCP_ROLE, CTPROTO_ATTR_U8, __DCCP_CONNTRACK_ROLE_MAX, dccp_roles},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 #define DCCP_VALID_FLAGS_MAX	2
 static unsigned int dccp_valid_flags[DCCP_VALID_FLAGS_MAX] = {
 	CT_DCCP_ORIG_SPORT | CT_DCCP_ORIG_DPORT,
@@ -235,6 +258,7 @@ static struct ctproto_handler dccp = {
 	.protonum		= IPPROTO_DCCP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.snprintf_options	= snprintf_options,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_gre.c b/extensions/libct_proto_gre.c
index 2dc63d1..b03887b 100644
--- a/extensions/libct_proto_gre.c
+++ b/extensions/libct_proto_gre.c
@@ -144,6 +144,21 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--srckey", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--dstkey", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-key-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-key-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 #define GRE_VALID_FLAGS_MAX   2
 static unsigned int gre_valid_flags[GRE_VALID_FLAGS_MAX] = {
        CT_GRE_ORIG_SKEY | CT_GRE_ORIG_DKEY,
@@ -181,6 +196,7 @@ static struct ctproto_handler gre = {
 	.protonum		= IPPROTO_GRE,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.snprintf_options	= snprintf_options,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_icmp.c b/extensions/libct_proto_icmp.c
index 16c2e2e..e1af53f 100644
--- a/extensions/libct_proto_icmp.c
+++ b/extensions/libct_proto_icmp.c
@@ -114,6 +114,20 @@ static int parse(char c,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--icmp-type", ATTR_ICMP_TYPE, CTPROTO_ATTR_U8, 0, 0},
+		{"--icmp-code", ATTR_ICMP_CODE, CTPROTO_ATTR_U8, 0, 0},
+		{"--icmp-id", ATTR_ICMP_ID, CTPROTO_ATTR_U16_N, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 static void final_check(unsigned int flags,
 		        unsigned int cmd,
 		        struct nf_conntrack *ct)
@@ -129,6 +143,7 @@ static struct ctproto_handler icmp = {
 	.protonum	= IPPROTO_ICMP,
 	.parse_opts	= parse,
 	.final_check	= final_check,
+	.snprintf_options	= snprintf_options,
 	.help		= help,
 	.opts		= opts,
 	.version	= VERSION,
diff --git a/extensions/libct_proto_icmpv6.c b/extensions/libct_proto_icmpv6.c
index 7f5e637..cb2178e 100644
--- a/extensions/libct_proto_icmpv6.c
+++ b/extensions/libct_proto_icmpv6.c
@@ -117,6 +117,20 @@ static int parse(char c,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--icmpv6-type", ATTR_ICMP_TYPE, CTPROTO_ATTR_U8, 0, 0},
+		{"--icmpv6-code", ATTR_ICMP_CODE, CTPROTO_ATTR_U8, 0, 0},
+		{"--icmpv6-id", ATTR_ICMP_ID, CTPROTO_ATTR_U16_N, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 static void final_check(unsigned int flags,
 		        unsigned int cmd,
 		        struct nf_conntrack *ct)
@@ -131,6 +145,7 @@ static struct ctproto_handler icmpv6 = {
 	.protonum	= IPPROTO_ICMPV6,
 	.parse_opts	= parse,
 	.final_check	= final_check,
+	.snprintf_options	= snprintf_options,
 	.help		= help,
 	.opts		= opts,
 	.version	= VERSION,
diff --git a/extensions/libct_proto_sctp.c b/extensions/libct_proto_sctp.c
index 04828bf..5b7a43b 100644
--- a/extensions/libct_proto_sctp.c
+++ b/extensions/libct_proto_sctp.c
@@ -198,6 +198,24 @@ parse_options(char c, struct nf_conntrack *ct,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--state", ATTR_SCTP_STATE, CTPROTO_ATTR_U8, SCTP_CONNTRACK_MAX, sctp_states},
+		{"--orig-vtag", ATTR_SCTP_VTAG_ORIG, CTPROTO_ATTR_U32_N, 0, 0},
+		{"--reply-vtag", ATTR_SCTP_VTAG_REPL, CTPROTO_ATTR_U32_N, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 #define SCTP_VALID_FLAGS_MAX   2
 static unsigned int dccp_valid_flags[SCTP_VALID_FLAGS_MAX] = {
 	CT_SCTP_ORIG_SPORT | CT_SCTP_ORIG_DPORT,
@@ -235,6 +253,7 @@ static struct ctproto_handler sctp = {
 	.protonum		= IPPROTO_SCTP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.snprintf_options	= snprintf_options,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_tcp.c b/extensions/libct_proto_tcp.c
index 8a37a55..90fecf6 100644
--- a/extensions/libct_proto_tcp.c
+++ b/extensions/libct_proto_tcp.c
@@ -177,6 +177,22 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--state", ATTR_TCP_STATE, CTPROTO_ATTR_U8, TCP_CONNTRACK_MAX, tcp_states},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 #define TCP_VALID_FLAGS_MAX   2
 static unsigned int tcp_valid_flags[TCP_VALID_FLAGS_MAX] = {
        CT_TCP_ORIG_SPORT | CT_TCP_ORIG_DPORT,
@@ -228,6 +244,7 @@ static struct ctproto_handler tcp = {
 	.protonum		= IPPROTO_TCP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.snprintf_options	= snprintf_options,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_udp.c b/extensions/libct_proto_udp.c
index e30637c..a70cee0 100644
--- a/extensions/libct_proto_udp.c
+++ b/extensions/libct_proto_udp.c
@@ -144,6 +144,21 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 #define UDP_VALID_FLAGS_MAX   2
 static unsigned int udp_valid_flags[UDP_VALID_FLAGS_MAX] = {
        CT_UDP_ORIG_SPORT | CT_UDP_ORIG_DPORT,
@@ -181,6 +196,7 @@ static struct ctproto_handler udp = {
 	.protonum		= IPPROTO_UDP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.snprintf_options	= snprintf_options,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_udplite.c b/extensions/libct_proto_udplite.c
index f46cef0..382d990 100644
--- a/extensions/libct_proto_udplite.c
+++ b/extensions/libct_proto_udplite.c
@@ -148,6 +148,21 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ctproto_attr attrs[] = {
+		{"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0},
+		{"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int snprintf_options(char *buf,
+			 unsigned int len,
+			 const struct nf_conntrack *ct)
+{
+	return snprintf_attrs(buf, len, ct, attrs);
+}
+
 #define UDPLITE_VALID_FLAGS_MAX   2
 static unsigned int udplite_valid_flags[UDPLITE_VALID_FLAGS_MAX] = {
        CT_UDPLITE_ORIG_SPORT | CT_UDPLITE_ORIG_DPORT,
@@ -186,6 +201,7 @@ static struct ctproto_handler udplite = {
 	.protonum		= IPPROTO_UDPLITE,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.snprintf_options	= snprintf_options,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/include/conntrack.h b/include/conntrack.h
index 37ccf6e..b518096 100644
--- a/include/conntrack.h
+++ b/include/conntrack.h
@@ -8,6 +8,9 @@
 
 #include <netinet/in.h>
 
+#include <linux/netfilter/nf_conntrack_common.h>
+#include <libnetfilter_conntrack/libnetfilter_conntrack.h>
+
 #define NUMBER_OF_CMD   19
 #define NUMBER_OF_OPT   29
 
@@ -32,6 +35,10 @@ struct ctproto_handler {
 			    unsigned int command,
 			    struct nf_conntrack *ct);
 
+	int (*snprintf_options)(char *buf,
+				 unsigned int len,
+				 const struct nf_conntrack *ct);
+
 	void (*help)(void);
 
 	struct option 		*opts;
@@ -53,6 +60,37 @@ void exit_error(enum exittype status, const char *msg, ...);
 
 extern void register_proto(struct ctproto_handler *h);
 
+enum ctproto_attr_value_type {
+	CTPROTO_ATTR_U8 = 1,
+	CTPROTO_ATTR_U16_N,
+	CTPROTO_ATTR_U16_H,
+	CTPROTO_ATTR_U32_N,
+	CTPROTO_ATTR_U32_H,
+	CTPROTO_ATTR_U64_H,
+	CTPROTO_ATTR_U32_BITMAP,
+	CTPROTO_ATTR_IPV4,
+	CTPROTO_ATTR_IPV6,
+};
+
+struct ctproto_attr {
+	const char *name;
+	enum nf_conntrack_attr type;
+	short value_type;
+	short val_mapping_count;
+	const char **val_mapping;
+};
+
+extern int snprintf_attr(char *buf,
+				unsigned int len,
+				const struct nf_conntrack *ct,
+				const struct ctproto_attr *attr);
+
+extern int snprintf_attrs(char *buf,
+				unsigned int len,
+				const struct nf_conntrack *ct,
+				const struct ctproto_attr *attrs);
+
+
 extern void register_tcp(void);
 extern void register_udp(void);
 extern void register_udplite(void);
diff --git a/src/conntrack.c b/src/conntrack.c
index 5834f2d..a11958b 100644
--- a/src/conntrack.c
+++ b/src/conntrack.c
@@ -53,6 +53,7 @@
 #include <sys/socket.h>
 #include <sys/time.h>
 #include <time.h>
+#include <inttypes.h>
 #ifdef HAVE_ARPA_INET_H
 #include <arpa/inet.h>
 #endif
@@ -610,6 +611,222 @@ void register_proto(struct ctproto_handler *h)
 	list_add(&h->head, &proto_list);
 }
 
+#define BUFFER_SIZE(ret, size, len, offset)		do {\
+	size += ret;					\
+	if ((int)ret > (int)len)					\
+		ret = len;				\
+	offset += ret;					\
+	len -= ret;	\
+} while(0)
+
+int snprintf_attr(char *buf,
+				unsigned int len,
+				const struct nf_conntrack *ct,
+				const struct ctproto_attr *attr
+				)
+{
+	uint16_t u16;
+	uint32_t u32;
+	uint64_t u64_val = 0;
+	char ipstr[INET6_ADDRSTRLEN];
+
+	if (!nfct_attr_is_set(ct, attr->type))
+		return 0;
+
+	switch (attr->value_type) {
+	case CTPROTO_ATTR_U8:
+		u64_val = nfct_get_attr_u8(ct, attr->type);
+		break;
+	case CTPROTO_ATTR_U16_N:
+		u16 = nfct_get_attr_u16(ct, attr->type);
+		u64_val = ntohs(u16);
+		break;
+	case CTPROTO_ATTR_U16_H:
+		u64_val = nfct_get_attr_u16(ct, attr->type);
+		break;
+	case CTPROTO_ATTR_U32_N:
+		u32 = nfct_get_attr_u32(ct, attr->type);
+		u64_val = ntohl(u32);
+		break;
+	case CTPROTO_ATTR_U32_H:
+		u64_val = nfct_get_attr_u32(ct, attr->type);
+		break;
+	case CTPROTO_ATTR_U64_H:
+		u64_val = nfct_get_attr_u64(ct, attr->type);
+		break;
+	case CTPROTO_ATTR_IPV4:
+		inet_ntop(AF_INET, nfct_get_attr(ct, attr->type), ipstr, sizeof(ipstr));
+		break;
+	case CTPROTO_ATTR_IPV6:
+		inet_ntop(AF_INET6, nfct_get_attr(ct, attr->type), ipstr, sizeof(ipstr));
+		break;
+	case CTPROTO_ATTR_U32_BITMAP:
+		u64_val = nfct_get_attr_u32(ct, attr->type);
+		break;
+	default:
+		fprintf(stderr, "unsupported type %u\n", attr->value_type);
+		return 0;
+	}
+
+	if (attr->value_type == CTPROTO_ATTR_IPV4
+			|| attr->value_type == CTPROTO_ATTR_IPV6) {
+	    return snprintf(buf, len, "%s %s ", attr->name, ipstr);
+	}
+
+	if (attr->val_mapping) {
+		if (attr->value_type == CTPROTO_ATTR_U32_BITMAP) {
+			unsigned int size = 0, offset = 0, ret, i, imax, found = 0;
+
+			if (!u64_val)
+				return 0;
+
+			imax = 31 < attr->val_mapping_count ? 31 : attr->val_mapping_count;
+			for (i = 0; i < imax; ++i) {
+				if (!(u64_val & (1 << i)))
+					continue;
+				if (!attr->val_mapping[i])
+					continue;
+
+				if (!found) {
+					ret = snprintf(buf + offset, len, "%s %s", attr->name, attr->val_mapping[i]);
+					found = 1;
+				} else
+					ret = snprintf(buf + offset, len, ",%s", attr->val_mapping[i]);
+				BUFFER_SIZE(ret, size, len, offset);
+			}
+
+			if (found) {
+				ret = snprintf(buf + offset, len, " ");
+				BUFFER_SIZE(ret, size, len, offset);
+			}
+			return size;
+		}
+
+		if (u64_val >= (uint64_t)attr->val_mapping_count) {
+			fprintf(stderr, "too big value for mapping %s %" PRIu64 "\n",
+					attr->name, u64_val);
+			return 0;
+		}
+		if (!attr->val_mapping[u64_val]) {
+			fprintf(stderr, "no mapping for %s %" PRIu64 "\n",
+					attr->name, u64_val);
+			return 0;
+		}
+		return snprintf(buf, len, "%s %s ", attr->name, attr->val_mapping[u64_val]);
+	}
+
+	return snprintf(buf, len, "%s %" PRIu64 " ", attr->name, u64_val);
+}
+
+int snprintf_attrs(char *buf,
+				unsigned int len,
+				const struct nf_conntrack *ct,
+				const struct ctproto_attr *attrs)
+{
+	int i;
+	unsigned int size = 0, offset = 0, ret;
+
+	for (i = 0; attrs[i].name; ++i) {
+		ret = snprintf_attr(buf + offset, len, ct, &attrs[i]);
+		BUFFER_SIZE(ret, size, len, offset);
+	}
+
+	return size;
+}
+
+static struct ctproto_attr attrs_ipv4[] = {
+		{"-s", ATTR_ORIG_IPV4_SRC, CTPROTO_ATTR_IPV4, 0, 0},
+		{"-d", ATTR_ORIG_IPV4_DST, CTPROTO_ATTR_IPV4, 0, 0},
+		{"-g", ATTR_DNAT_IPV4, CTPROTO_ATTR_IPV4, 0, 0},
+		{"-n", ATTR_SNAT_IPV4, CTPROTO_ATTR_IPV4, 0, 0},
+		{"-r", ATTR_REPL_IPV4_SRC, CTPROTO_ATTR_IPV4, 0, 0},
+		{"-q", ATTR_REPL_IPV4_DST, CTPROTO_ATTR_IPV4, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static struct ctproto_attr attrs_ipv6[] = {
+		{"-s", ATTR_ORIG_IPV6_SRC, CTPROTO_ATTR_IPV6, 0, 0},
+		{"-d", ATTR_ORIG_IPV6_DST, CTPROTO_ATTR_IPV6, 0, 0},
+		{"-g", ATTR_DNAT_IPV6, CTPROTO_ATTR_IPV6, 0, 0},
+		{"-n", ATTR_SNAT_IPV6, CTPROTO_ATTR_IPV6, 0, 0},
+		{"-r", ATTR_REPL_IPV6_SRC, CTPROTO_ATTR_IPV6, 0, 0},
+		{"-q", ATTR_REPL_IPV6_DST, CTPROTO_ATTR_IPV6, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static const char *conntrack_status_map[] = {
+		[IPS_ASSURED_BIT] = "ASSURED",
+		[IPS_SEEN_REPLY_BIT] = "SEEN_REPLY",
+		[IPS_FIXED_TIMEOUT_BIT] = "FIXED_TIMEOUT",
+		[IPS_EXPECTED_BIT] = "EXPECTED"
+};
+
+static struct ctproto_attr attrs_generic[] = {
+		{"-t", ATTR_TIMEOUT, CTPROTO_ATTR_U32_H, 0, 0},
+		{"-u", ATTR_STATUS, CTPROTO_ATTR_U32_BITMAP,
+				sizeof(conntrack_status_map)/sizeof(conntrack_status_map[0]),
+				conntrack_status_map},
+		{"-c", ATTR_SECMARK, CTPROTO_ATTR_U32_H, 0, 0},
+/*		{"-i", ATTR_ID, CTPROTO_ATTR_U32_H, 0, 0}, */
+		{"-w", ATTR_ZONE, CTPROTO_ATTR_U16_H, 0, 0},
+		{"--orig-zone", ATTR_ORIG_ZONE, CTPROTO_ATTR_U16_H, 0, 0},
+		{"--reply-zone", ATTR_REPL_ZONE, CTPROTO_ATTR_U16_H, 0, 0},
+		{0, 0, 0, 0, 0},
+};
+
+static int nfct_snprintf_labels_opts(char *buf,
+				unsigned int len,
+				const struct nf_conntrack *ct,
+				struct nfct_labelmap *map)
+{
+	int ret = 0;
+	unsigned int size = 0, offset = 0;
+	uint8_t l3proto, l4proto;
+	static struct ctproto_attr *attrs_l3;
+	struct ctproto_handler *cur;
+
+	ret = snprintf_attrs(buf + offset, len, ct, attrs_generic);
+	BUFFER_SIZE(ret, size, len, offset);
+
+	l3proto = nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO);
+	if (!l3proto)
+		l3proto = nfct_get_attr_u8(ct, ATTR_REPL_L3PROTO);
+	switch (l3proto) {
+	case AF_INET:
+		attrs_l3 = attrs_ipv4;
+		break;
+	case AF_INET6:
+		attrs_l3 = attrs_ipv6;
+		break;
+	default:
+		fprintf(stderr,
+				"WARNING: unknown l3proto %d, skipping..\n", l3proto);
+		return 0;
+	}
+
+	ret = snprintf_attrs(buf + offset, len, ct, attrs_l3);
+	BUFFER_SIZE(ret, size, len, offset);
+
+	l4proto = nfct_get_attr_u8(ct, ATTR_L4PROTO);
+
+	/* is it in the list of supported protocol? */
+	list_for_each_entry(cur, &proto_list, head) {
+		if (cur->protonum == l4proto) {
+			ret = snprintf(buf + offset, len, "-p %s ", cur->name);
+			BUFFER_SIZE(ret, size, len, offset);
+			ret = cur->snprintf_options(buf + offset, len, ct);
+			BUFFER_SIZE(ret, size, len, offset);
+			break;
+		}
+	}
+
+	/* skip trailing space, if any */
+	for (;size && buf[size-1] == ' '; --size)
+		buf[size-1] = '\0';
+
+	return size;
+}
+
 extern struct ctproto_handler ct_proto_unknown;
 
 static struct ctproto_handler *findproto(char *name, int *pnum)
@@ -865,6 +1082,7 @@ enum {
 	_O_KTMS	= (1 << 4),
 	_O_CL	= (1 << 5),
 	_O_US	= (1 << 6),
+	_O_OPTS	= (1 << 7),
 };
 
 enum {
@@ -875,7 +1093,7 @@ enum {
 };
 
 static struct parse_parameter {
-	const char	*parameter[7];
+	const char	*parameter[8];
 	size_t  size;
 	unsigned int value[8];
 } parse_array[PARSE_MAX] = {
@@ -883,8 +1101,8 @@ static struct parse_parameter {
 	  { IPS_ASSURED, IPS_SEEN_REPLY, 0, IPS_FIXED_TIMEOUT, IPS_EXPECTED, IPS_OFFLOAD, IPS_HW_OFFLOAD} },
 	{ {"ALL", "NEW", "UPDATES", "DESTROY"}, 4,
 	  { CT_EVENT_F_ALL, CT_EVENT_F_NEW, CT_EVENT_F_UPD, CT_EVENT_F_DEL } },
-	{ {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace" }, 7,
-	  { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US },
+	{ {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace", "opts"}, 8,
+	  { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US, _O_OPTS },
 	},
 };
 
@@ -1467,6 +1685,11 @@ static int event_cb(const struct nlmsghdr *nlh, void *data)
 	if (nfct_filter(obj, ct))
 		goto out;
 
+	if (output_mask & _O_OPTS) {
+		nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap);
+		goto done;
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -1491,7 +1714,7 @@ static int event_cb(const struct nlmsghdr *nlh, void *data)
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf_labels(buf, sizeof(buf), ct, type, op_type, op_flags, labelmap);
-
+done:
 	if (output_mask & _O_US) {
 		if (nlh->nlmsg_pid)
 			userspace = true;
@@ -1520,6 +1743,11 @@ static int dump_cb(enum nf_conntrack_msg_type type,
 	if (nfct_filter(obj, ct))
 		return NFCT_CB_CONTINUE;
 
+	if (output_mask & _O_OPTS) {
+		nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap);
+		goto done;
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -1536,6 +1764,7 @@ static int dump_cb(enum nf_conntrack_msg_type type,
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap);
+done:
 	printf("%s\n", buf);
 
 	counter++;
@@ -1562,6 +1791,11 @@ static int delete_cb(enum nf_conntrack_msg_type type,
 			   "Operation failed: %s",
 			   err2str(errno, CT_DELETE));
 
+	if (output_mask & _O_OPTS) {
+		nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap);
+		goto done;
+	}
+
 	if (output_mask & _O_XML)
 		op_type = NFCT_O_XML;
 	if (output_mask & _O_EXT)
@@ -1570,6 +1804,7 @@ static int delete_cb(enum nf_conntrack_msg_type type,
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags);
+done:
 	printf("%s\n", buf);
 
 	counter++;
@@ -1585,6 +1820,11 @@ static int print_cb(enum nf_conntrack_msg_type type,
 	unsigned int op_type = NFCT_O_DEFAULT;
 	unsigned int op_flags = 0;
 
+	if (output_mask & _O_OPTS) {
+		nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap);
+		goto done;
+	}
+
 	if (output_mask & _O_XML)
 		op_type = NFCT_O_XML;
 	if (output_mask & _O_EXT)
@@ -1593,6 +1833,7 @@ static int print_cb(enum nf_conntrack_msg_type type,
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap);
+done:
 	printf("%s\n", buf);
 
 	return NFCT_CB_CONTINUE;
@@ -1757,6 +1998,12 @@ static int dump_exp_cb(enum nf_conntrack_msg_type type,
 	unsigned int op_type = NFCT_O_DEFAULT;
 	unsigned int op_flags = 0;
 
+	if (output_mask & _O_OPTS) {
+		/* not implemented at the moment */
+		exit_error(PARAMETER_PROBLEM,
+		   "opts output format is not supported for table of expectations");
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -1788,6 +2035,12 @@ static int event_exp_cb(enum nf_conntrack_msg_type type,
 	unsigned int op_type = NFCT_O_DEFAULT;
 	unsigned int op_flags = 0;
 
+	if (output_mask & _O_OPTS) {
+		/* not implemented at the moment */
+		exit_error(PARAMETER_PROBLEM,
+		   "opts output format is not supported for table of expectations");
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 7/8] conntrack.8: man update for opts format support
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
                   ` (5 preceding siblings ...)
  2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-09-25 12:49 ` [PATCH 8/8] tests: dumping ct entries in opts format Mikhail Sennikovsky
  2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal
  8 siblings, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 conntrack.8 | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/conntrack.8 b/conntrack.8
index 3db4849..2d354a7 100644
--- a/conntrack.8
+++ b/conntrack.8
@@ -109,7 +109,7 @@ Show the in-kernel connection tracking system statistics.
 Atomically zero counters after reading them.  This option is only valid in
 combination with the "\-L, \-\-dump" command options.
 .TP
-.BI "-o, --output [extended,xml,timestamp,id,ktimestamp,labels,userspace] "
+.BI "-o, --output [extended,xml,opts,timestamp,id,ktimestamp,labels,userspace] "
 Display output in a certain format. With the extended output option, this tool
 displays the layer 3 information. With ktimestamp, it displays the in-kernel
 timestamp available since 2.6.38 (you can enable it via the \fBsysctl(8)\fP
@@ -399,6 +399,9 @@ Delete all flow whose source address is 1.2.3.4
 .TP
 .B conntrack \-U \-s 1.2.3.4 \-m 1
 Set connmark to 1 of all the flows whose source address is 1.2.3.4
+.TP
+.B conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I -
+Copy all ct entries from one ct zone 15 to ct zone 9915
 
 .SH BUGS
 Please, report them to netfilter-devel@vger.kernel.org or file a bug in
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 8/8] tests: dumping ct entries in opts format
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
                   ` (6 preceding siblings ...)
  2020-09-25 12:49 ` [PATCH 7/8] conntrack.8: man update for opts format support Mikhail Sennikovsky
@ 2020-09-25 12:49 ` Mikhail Sennikovsky
  2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal
  8 siblings, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw)
  To: netfilter-devel; +Cc: Mikhail Sennikovsky

Add tests to cover dumping ct entries in "opts" format by conntrack

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 tests/conntrack/test-conntrack.c    | 14 ++++++
 tests/conntrack/testsuite/09dumpopt | 77 +++++++++++++++++++++++++++++
 2 files changed, 91 insertions(+)
 create mode 100644 tests/conntrack/testsuite/09dumpopt

diff --git a/tests/conntrack/test-conntrack.c b/tests/conntrack/test-conntrack.c
index 90bdc5b..372e025 100644
--- a/tests/conntrack/test-conntrack.c
+++ b/tests/conntrack/test-conntrack.c
@@ -81,6 +81,11 @@ int main()
 			res++;
 			for (; *res == ' ' || *res == '\t'; res++);
 			cmd = res[0];
+			cmd_opt = &res[1];
+			for (; *cmd_opt == ' ' || *cmd_opt == '\t'; cmd_opt++);
+			res = strchr(cmd_opt, '\n');
+			if (res)
+				*res = '\0';
 
 			if (cur_cmd && cmd != cur_cmd) {
 				/* complete current multi-line command */
@@ -111,6 +116,15 @@ int main()
 				cmd_strappend(CT_PROG);
 				cmd_strappend(" ");
 				cmd_strappend(buf);
+				if (cmd == '|') {
+					cmd_strappend(" | ");
+					if (cmd_opt[0]) {
+						cmd_strappend("sed \"");
+						cmd_strappend(cmd_opt);
+						cmd_strappend("\" | ");
+					}
+					continue;
+				}
 				cmd_reset();
 				break;
 			}
diff --git a/tests/conntrack/testsuite/09dumpopt b/tests/conntrack/testsuite/09dumpopt
new file mode 100644
index 0000000..0e3c649
--- /dev/null
+++ b/tests/conntrack/testsuite/09dumpopt
@@ -0,0 +1,77 @@
+# test opts output for -L
+# create
+# create a conntrack
+-w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+# create from reply
+-w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ;
+# create a v6 conntrack
+-w 10 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+# creae icmp ping request entry
+-w 10 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ;
+-I - ; OK
+# copy ipv4 bits to zone 11
+-L -w 10 -o opt -f ipv4 ; |s/-w 10/-w 11/g
+-I - ; OK
+# copy ipv6 bits to zone 11
+-L -w 10 -o opt -f ipv6 ; |s/-w 10/-w 11/g
+-I - ; OK
+# create again in zone 11
+-I -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -w 11 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+# delete new entries
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
+# delete reverse
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; OK
+# delete v6 conntrack
+-D -w 11-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
+# delete icmp ping request entry
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# delete old entries
+-D -w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
+# delete reverse
+-D -w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; OK
+# delete v6 conntrack
+-D -w 10-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
+# delete icmp ping request entry
+-D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+#
+# now test opts output for -D
+# create entries again
+# create a conntrack
+-w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+# create from reply
+-w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ;
+# create a v6 conntrack
+-w 10 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ;
+# creae icmp ping request entry
+-w 10 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ;
+-I - ; OK
+# move ipv4 bits to zone 11
+-D -w 10 -o opt -f ipv4 ; |s/-w 10/-w 11/g
+-I - ; OK
+# move ipv6 bits to zone 11
+-D -w 10 -o opt -f ipv6 ; |s/-w 10/-w 11/g
+-I - ; OK
+# create again in zone 11
+-I -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD
+-I -w 11 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+# delete new entries
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
+# delete reverse
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; OK
+# delete v6 conntrack
+-D -w 11-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK
+# delete icmp ping request entry
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# delete old entries
+-D -w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; BAD
+# delete reverse
+-D -w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; BAD
+# delete v6 conntrack
+-D -w 10-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; BAD
+# delete icmp ping request entry
+-D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
\ No newline at end of file
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH 0/8] Fast bulk transfers of large sets of ct entries
  2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
                   ` (7 preceding siblings ...)
  2020-09-25 12:49 ` [PATCH 8/8] tests: dumping ct entries in opts format Mikhail Sennikovsky
@ 2020-09-25 13:28 ` Florian Westphal
  2020-09-26 18:19   ` Pablo Neira Ayuso
  8 siblings, 1 reply; 20+ messages in thread
From: Florian Westphal @ 2020-09-25 13:28 UTC (permalink / raw)
  To: Mikhail Sennikovsky; +Cc: netfilter-devel

Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> wrote:
> In addition to this I have a question about the behavioural change
> of the "conntrack -L" done after conntrack v1.4.5.
> With the conntrack v1.4.5 used on Debian Buster the "conntrack -L"
> dumps both ipv4 and ipv6 ct entries, while with the current master, 
> presumably starting with the commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e 
> "conntrack -L"  only dumps ipv4 entries.
> 
> So is this really the desired behavior? 

I'd like conntrack to dump both by default.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 0/8] Fast bulk transfers of large sets of ct entries
  2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal
@ 2020-09-26 18:19   ` Pablo Neira Ayuso
  2020-09-29 11:20     ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky
  0 siblings, 1 reply; 20+ messages in thread
From: Pablo Neira Ayuso @ 2020-09-26 18:19 UTC (permalink / raw)
  To: Florian Westphal; +Cc: Mikhail Sennikovsky, netfilter-devel

On Fri, Sep 25, 2020 at 03:28:34PM +0200, Florian Westphal wrote:
> Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> wrote:
> > In addition to this I have a question about the behavioural change
> > of the "conntrack -L" done after conntrack v1.4.5.
> > With the conntrack v1.4.5 used on Debian Buster the "conntrack -L"
> > dumps both ipv4 and ipv6 ct entries, while with the current master, 
> > presumably starting with the commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e 
> > "conntrack -L"  only dumps ipv4 entries.
> > 
> > So is this really the desired behavior? 
> 
> I'd like conntrack to dump both by default.

Dumping both with -L is fine, so -f behaves as a way to filter.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given
  2020-09-26 18:19   ` Pablo Neira Ayuso
@ 2020-09-29 11:20     ` Mikhail Sennikovsky
  2020-09-29 11:20       ` [PATCH 1/2] " Mikhail Sennikovsky
  2020-09-29 11:20       ` [PATCH 2/2] tests: conntrack -L/-D ip family filtering Mikhail Sennikovsky
  0 siblings, 2 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-29 11:20 UTC (permalink / raw)
  To: netfilter-devel, pablo, fw; +Cc: Mikhail Sennikovsky

Hi all,

As we discussed in the "Fast bulk transfers of large sets of ct 
entries" mailing thread, conntrack -L (and presumably
conntracks -D as well) should dump/delete both IPv4 and IPv6
entries if no family is specified.

As a follow-up for that here is a patch which supposed to restore
this behavior and a patch with the test-cases covering both the
invocation with family is specified and not.

The patches are created on top of my previous set of patches,
mainly for the reason that the "opts" output format and stdio 
input introduced there among other thing allow testing the 
-L functionality easily.

I would really appreciate if someone could have a look into
those patches btw and give some feedback on whether they make
sense or not.

Thanks & Regards,
Mikhail

Mikhail Sennikovsky (2):
  conntrack: -L/-D both ipv4/6 if no family is given
  tests: conntrack -L/-D ip family filtering

 src/conntrack.c                     | 35 +++++++++++---
 tests/conntrack/testsuite/09dumpopt | 72 ++++++++++++++++++++++++++++-
 2 files changed, 99 insertions(+), 8 deletions(-)

-- 
2.25.1


^ permalink raw reply	[flat|nested] 20+ messages in thread

* [PATCH 1/2] conntrack: -L/-D both ipv4/6 if no family is given
  2020-09-29 11:20     ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky
@ 2020-09-29 11:20       ` Mikhail Sennikovsky
  2020-09-29 11:20       ` [PATCH 2/2] tests: conntrack -L/-D ip family filtering Mikhail Sennikovsky
  1 sibling, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-29 11:20 UTC (permalink / raw)
  To: netfilter-devel, pablo, fw; +Cc: Mikhail Sennikovsky

Starting commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e
conntrack -L as well as conntrack -D list/delete
IPv4 entries only if no family is specified.

Restore original behavior to list/delete both IPv4 and IPv6
entries if no family is specified.

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 src/conntrack.c | 35 ++++++++++++++++++++++++++++-------
 1 file changed, 28 insertions(+), 7 deletions(-)

diff --git a/src/conntrack.c b/src/conntrack.c
index a11958b..3f5eb37 100644
--- a/src/conntrack.c
+++ b/src/conntrack.c
@@ -1578,6 +1578,11 @@ nfct_filter_network_direction(const struct nf_conntrack *ct, enum ct_direction d
 	enum nf_conntrack_attr attr;
 	struct ct_network *net = &dir2network[dir];
 
+	if (family == AF_UNSPEC) {
+		exit_error(OTHER_PROBLEM,
+			   "Internal Error: unspecified Family!");
+	}
+
 	if (nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO) != family)
 		return 1;
 
@@ -2433,6 +2438,10 @@ nfct_filter_init(const int family)
 {
 	filter_family = family;
 	if (options & CT_OPT_MASK_SRC) {
+		if (family == AF_UNSPEC) {
+			exit_error(OTHER_PROBLEM,
+				   "Internal Error: unspecified Family!");
+		}
 		if (!(options & CT_OPT_ORIG_SRC))
 			exit_error(PARAMETER_PROBLEM,
 			           "Can't use --mask-src without --src");
@@ -2440,6 +2449,10 @@ nfct_filter_init(const int family)
 	}
 
 	if (options & CT_OPT_MASK_DST) {
+		if (family == AF_UNSPEC) {
+			exit_error(OTHER_PROBLEM,
+				   "Internal Error: unspecified Family!");
+		}
 		if (!(options & CT_OPT_ORIG_DST))
 			exit_error(PARAMETER_PROBLEM,
 			           "Can't use --mask-dst without --dst");
@@ -2894,7 +2907,9 @@ parse_opts:
 	}
 
 	/* default family */
-	if (family == AF_UNSPEC)
+	if (family == AF_UNSPEC
+			&& command != CT_LIST
+			&& command != CT_DELETE)
 		family = AF_INET;
 
 	/* we cannot check this combination with generic_opt_check. */
@@ -2993,9 +3008,12 @@ parse_opts:
 						  NFCT_FILTER_DUMP_MARK,
 						  &tmpl.filter_mark_kernel);
 		}
-		nfct_filter_dump_set_attr_u8(filter_dump,
-					     NFCT_FILTER_DUMP_L3NUM,
-					     family);
+
+		if (family != AF_UNSPEC) {
+			nfct_filter_dump_set_attr_u8(filter_dump,
+						     NFCT_FILTER_DUMP_L3NUM,
+						     family);
+		}
 
 		if (options & CT_OPT_ZERO)
 			res = nfct_query(cth, NFCT_Q_DUMP_FILTER_RESET,
@@ -3104,9 +3122,12 @@ parse_opts:
 						  NFCT_FILTER_DUMP_MARK,
 						  &tmpl.filter_mark_kernel);
 		}
-		nfct_filter_dump_set_attr_u8(filter_dump,
-					     NFCT_FILTER_DUMP_L3NUM,
-					     family);
+
+		if (family != AF_UNSPEC) {
+			nfct_filter_dump_set_attr_u8(filter_dump,
+							 NFCT_FILTER_DUMP_L3NUM,
+							 family);
+		}
 
 		res = nfct_query(cth, NFCT_Q_DUMP_FILTER, filter_dump);
 
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* [PATCH 2/2] tests: conntrack -L/-D ip family filtering
  2020-09-29 11:20     ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky
  2020-09-29 11:20       ` [PATCH 1/2] " Mikhail Sennikovsky
@ 2020-09-29 11:20       ` Mikhail Sennikovsky
  1 sibling, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-09-29 11:20 UTC (permalink / raw)
  To: netfilter-devel, pablo, fw; +Cc: Mikhail Sennikovsky

Tests to cover conntrack -L and conntrack -D with and w/o
family (-f) specfied.

conntrack -L and contnrack -D shold list/delete
both IPv4 and IPv6 entries if no family is specified,
and should ony display the corresponding entries if
the family is given.

Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com>
---
 tests/conntrack/testsuite/09dumpopt | 72 ++++++++++++++++++++++++++++-
 1 file changed, 71 insertions(+), 1 deletion(-)

diff --git a/tests/conntrack/testsuite/09dumpopt b/tests/conntrack/testsuite/09dumpopt
index 0e3c649..9c5121d 100644
--- a/tests/conntrack/testsuite/09dumpopt
+++ b/tests/conntrack/testsuite/09dumpopt
@@ -74,4 +74,74 @@
 # delete v6 conntrack
 -D -w 10-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; BAD
 # delete icmp ping request entry
--D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
\ No newline at end of file
+-D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+#
+# Additional tests to check that family attribute is treated properly
+# for -L and -D commands
+# namely:
+# - if family (-f) is given - only entries of the given family are dumped/deleted
+# - if no family is given - entries of both ipv4 and ipv6 families are dumped/deleted
+# First create some ipv4 and ipv6 entries
+-I -w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK
+-I -w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; OK
+-I -w 10 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK
+-I -w 10 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# dump all entries to zone 11
+-L -w 10 -o opt; |s/-w 10/-w 11/g
+-I - ; OK
+# ensure that both ipv4 and ipv6 entries get copied (delete for each of them should succeed)
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY ; OK
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY ; OK
+-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# dump only ipv4 entries to zone 11
+-L -w 10 -o opt -f ipv4; |s/-w 10/-w 11/g
+-I - ; OK
+# ensure that only ipv4 entries get copied (delete only for ipv4 entries should succeed)
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# dump only ipv6 entries to zone 11
+-L -w 10 -o opt -f ipv6; |s/-w 10/-w 11/g
+-I - ; OK
+# ensure that only ipv6 entries get copied (delete only for ipv6 entries should succeed)
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+# now test deleting w/ and /o family specified
+# for simplicity do it by re-creating entries in zone 11
+# by copying ezisting entries from zone 10 into it
+# re-create entries in ct zone 11
+-L -w 10 -o opt; |s/-w 10/-w 11/g
+-I - ; OK
+# delete all entries in zone 11
+-D -w 11 ; OK
+# both ipv4 and ipv6 should be deleted
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+# re-create entries in ct zone 11
+-L -w 10 -o opt; |s/-w 10/-w 11/g
+-I - ; OK
+# delete only ipv4 entries in zone 11
+-D -w 11 -f ipv4 ; OK
+# ipv6 should remain
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD
+ # re-create entries in ct zone 11
+-L -w 10 -o opt; |s/-w 10/-w 11/g
+-I - ; OK
+# delete only ipv6 entries in zone 11
+-D -w 11 -f ipv6 ; OK
+# ipv4 should remain
+-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; OK
+-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD
+-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK
+# clean up after yourself
+ -D -w 10 ; OK
\ No newline at end of file
-- 
2.25.1


^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH 1/8] tests: icmp entry create/delete
  2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky
@ 2020-10-13 18:28   ` Pablo Neira Ayuso
  0 siblings, 0 replies; 20+ messages in thread
From: Pablo Neira Ayuso @ 2020-10-13 18:28 UTC (permalink / raw)
  To: Mikhail Sennikovsky; +Cc: netfilter-devel

On Fri, Sep 25, 2020 at 02:49:12PM +0200, Mikhail Sennikovsky wrote:
> Add test to cover icmp entry creation/deletion with conntrack

Applied, thanks.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 2/8] conntrack: fix icmp entry creation
  2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky
@ 2020-10-13 18:29   ` Pablo Neira Ayuso
  0 siblings, 0 replies; 20+ messages in thread
From: Pablo Neira Ayuso @ 2020-10-13 18:29 UTC (permalink / raw)
  To: Mikhail Sennikovsky; +Cc: netfilter-devel

On Fri, Sep 25, 2020 at 02:49:13PM +0200, Mikhail Sennikovsky wrote:
> Creating icmp ct entry with command like
> 
> conntrack -I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 \
>    -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226
> 
> results in nfct_query( NFCT_Q_CREATE ) request would fail
> because reply L4 proto is not set while having reply data specified
> 
> Set reply L4 proto when reply data is given for the icmp ct entry

Also applied, thanks.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 6/8] conntrack: implement options output format
  2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky
@ 2020-10-22 12:36   ` Pablo Neira Ayuso
  2020-10-22 12:37     ` Pablo Neira Ayuso
  0 siblings, 1 reply; 20+ messages in thread
From: Pablo Neira Ayuso @ 2020-10-22 12:36 UTC (permalink / raw)
  To: Mikhail Sennikovsky; +Cc: netfilter-devel

Hi Mikhail,

Thanks for your patchset.

On Fri, Sep 25, 2020 at 02:49:17PM +0200, Mikhail Sennikovsky wrote:
> As a counterpart to the "conntrack: accept parameters from stdin"
> commit, this commit allows dumping conntrack entries in the format
> used by the conntrack parameters.
> This is useful for transfering a large set of ct entries between
> hosts or between different ct zones in an efficient way.
> 
> To enable the "options" output the "-o opts" parameter needs to be
> passed to the "contnrack -L" tool invocation.

I started slightly revisiting this 6/8 patch a bit (please find it
enclosed to this email), I have rename -o opts to -o save, to get this
aligned with iptables-save.

I have also added a check for -o xml,save , to reject this
combination.

I have extended it to display -I, -U, -D in the conntrack events.

I have removed several safety runtime checks, that can be done at
registration time (make sure the option description is well-formed
from there, otherwise rise an error message to spot buggy protocol
extensions).

This patch should also be extended to support for other existing
output flags combinations. Or just bail out if they are specified.

At this point I have concerns with NAT: I don't see how this can work
as is. There is also a conntrack helpers that might trigger NAT
sequence adjustments, this information would be lost.

We would need to expose all these details through the -o save, see
below. For some of this, there is no options from command line,
because it made no sense to expose them.

We have to discuss this before deciding where to go. See below for
details.

> To demonstrate the overall idea of the options output format works
> in conjunction with the "stdin parameter"s mode,
> the following command will copy all ct entries from one ct zone
> to another.
> 
> conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I -

For zone updates in the same host, probably conntrack can be extended
to support for:

        conntrack -U --zone 15 --set-zone 9915

If --set-zone is specified, then --zone is used a filter.

Then, for "zone transfers" *between hosts*, a different way to address
this is to extend conntrackd.

The idea is:

1) Add new "transfer" mode which does _not_ subscribe to
   conntrack events, it needs to register a new struct ct_mode
   (currently there is "sync" and "stats" ct_modes).

2) Add a new message type to request a zone transfer, e.g.

        conntrackd --from 192.168.10.20 --zone 15 --set-zone 9915

   This will make your local daemon send a request to the conntrackd
   instance running on host 192.168.10.20 to retrieve zone 1200. The
   remote conntrackd instance dumps the existing conntrack table from
   kernel and sends it to you.

   You can reuse the channel infrastructure to establish communications
   between conntrackd instances in the new "transfer mode". You can
   also reuse the sync protocol, see network.h, build.c and parse.c,
   which takes a conntrack object and it translates it to network
   message.

   Note that the struct internal_handler actually refers to the
   netlink handler for this new struct ct_mode that you would be
   registering.

Let me know, thanks.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 6/8] conntrack: implement options output format
  2020-10-22 12:36   ` Pablo Neira Ayuso
@ 2020-10-22 12:37     ` Pablo Neira Ayuso
  2020-10-23 11:07       ` Mikhail Sennikovsky
  0 siblings, 1 reply; 20+ messages in thread
From: Pablo Neira Ayuso @ 2020-10-22 12:37 UTC (permalink / raw)
  To: Mikhail Sennikovsky; +Cc: netfilter-devel

[-- Attachment #1: Type: text/plain, Size: 3489 bytes --]

On Thu, Oct 22, 2020 at 02:36:47PM +0200, Pablo Neira Ayuso wrote:
> Hi Mikhail,
> 
> Thanks for your patchset.
> 
> On Fri, Sep 25, 2020 at 02:49:17PM +0200, Mikhail Sennikovsky wrote:
> > As a counterpart to the "conntrack: accept parameters from stdin"
> > commit, this commit allows dumping conntrack entries in the format
> > used by the conntrack parameters.
> > This is useful for transfering a large set of ct entries between
> > hosts or between different ct zones in an efficient way.
> > 
> > To enable the "options" output the "-o opts" parameter needs to be
> > passed to the "contnrack -L" tool invocation.
> 
> I started slightly revisiting this 6/8 patch a bit (please find it
> enclosed to this email), I have rename -o opts to -o save, to get this
> aligned with iptables-save.

Attaching the revisited patch 6/8 to this email.

> I have also added a check for -o xml,save , to reject this
> combination.
> 
> I have extended it to display -I, -U, -D in the conntrack events.
> 
> I have removed several safety runtime checks, that can be done at
> registration time (make sure the option description is well-formed
> from there, otherwise rise an error message to spot buggy protocol
> extensions).
> 
> This patch should also be extended to support for other existing
> output flags combinations. Or just bail out if they are specified.
> 
> At this point I have concerns with NAT: I don't see how this can work
> as is. There is also a conntrack helpers that might trigger NAT
> sequence adjustments, this information would be lost.
> 
> We would need to expose all these details through the -o save, see
> below. For some of this, there is no options from command line,
> because it made no sense to expose them.
> 
> We have to discuss this before deciding where to go. See below for
> details.
> 
> > To demonstrate the overall idea of the options output format works
> > in conjunction with the "stdin parameter"s mode,
> > the following command will copy all ct entries from one ct zone
> > to another.
> > 
> > conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I -
> 
> For zone updates in the same host, probably conntrack can be extended
> to support for:
> 
>         conntrack -U --zone 15 --set-zone 9915
> 
> If --set-zone is specified, then --zone is used a filter.
> 
> Then, for "zone transfers" *between hosts*, a different way to address
> this is to extend conntrackd.
> 
> The idea is:
> 
> 1) Add new "transfer" mode which does _not_ subscribe to
>    conntrack events, it needs to register a new struct ct_mode
>    (currently there is "sync" and "stats" ct_modes).
> 
> 2) Add a new message type to request a zone transfer, e.g.
> 
>         conntrackd --from 192.168.10.20 --zone 15 --set-zone 9915
> 
>    This will make your local daemon send a request to the conntrackd
>    instance running on host 192.168.10.20 to retrieve zone 1200. The
>    remote conntrackd instance dumps the existing conntrack table from
>    kernel and sends it to you.
> 
>    You can reuse the channel infrastructure to establish communications
>    between conntrackd instances in the new "transfer mode". You can
>    also reuse the sync protocol, see network.h, build.c and parse.c,
>    which takes a conntrack object and it translates it to network
>    message.
> 
>    Note that the struct internal_handler actually refers to the
>    netlink handler for this new struct ct_mode that you would be
>    registering.
> 
> Let me know, thanks.

[-- Attachment #2: x.patch --]
[-- Type: text/x-diff, Size: 20886 bytes --]

diff --git a/extensions/libct_proto_dccp.c b/extensions/libct_proto_dccp.c
index f6258ad82a72..13cd7c53e292 100644
--- a/extensions/libct_proto_dccp.c
+++ b/extensions/libct_proto_dccp.c
@@ -198,6 +198,22 @@ static int parse_options(char c,
 	return 1;
 }
 
+
+static const char *dccp_roles[__DCCP_CONNTRACK_ROLE_MAX] = {
+	[DCCP_CONNTRACK_ROLE_CLIENT]	= "client",
+	[DCCP_CONNTRACK_ROLE_SERVER]	= "server",
+};
+
+static struct ct_print_opts dccp_print_opts[] = {
+	{ "--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, NULL },
+	{ "--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, NULL },
+	{ "--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, NULL },
+	{ "--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, NULL },
+	{ "--state", ATTR_DCCP_STATE, CT_ATTR_TYPE_U8, DCCP_CONNTRACK_MAX, dccp_states },
+	{ "--role", ATTR_DCCP_ROLE, CT_ATTR_TYPE_U8, __DCCP_CONNTRACK_ROLE_MAX, dccp_roles },
+	{},
+};
+
 #define DCCP_VALID_FLAGS_MAX	2
 static unsigned int dccp_valid_flags[DCCP_VALID_FLAGS_MAX] = {
 	CT_DCCP_ORIG_SPORT | CT_DCCP_ORIG_DPORT,
@@ -235,6 +251,7 @@ static struct ctproto_handler dccp = {
 	.protonum		= IPPROTO_DCCP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.print_opts		= dccp_print_opts,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_gre.c b/extensions/libct_proto_gre.c
index 2dc63d14f498..406eb0691b1b 100644
--- a/extensions/libct_proto_gre.c
+++ b/extensions/libct_proto_gre.c
@@ -144,6 +144,14 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ct_print_opts gre_print_opts[] = {
+	{ "--srckey", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--dstkey", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--reply-key-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--reply-key-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 },
+	{},
+};
+
 #define GRE_VALID_FLAGS_MAX   2
 static unsigned int gre_valid_flags[GRE_VALID_FLAGS_MAX] = {
        CT_GRE_ORIG_SKEY | CT_GRE_ORIG_DKEY,
@@ -181,6 +189,7 @@ static struct ctproto_handler gre = {
 	.protonum		= IPPROTO_GRE,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.print_opts		= gre_print_opts,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_icmp.c b/extensions/libct_proto_icmp.c
index 7fc82bdaddb6..a438d113f04a 100644
--- a/extensions/libct_proto_icmp.c
+++ b/extensions/libct_proto_icmp.c
@@ -102,6 +102,13 @@ static int parse(char c,
 	return 1;
 }
 
+static struct ct_print_opts icmp_print_opts[] = {
+	{ "--icmp-type", ATTR_ICMP_TYPE, CT_ATTR_TYPE_U8, 0, 0 },
+	{ "--icmp-code", ATTR_ICMP_CODE, CT_ATTR_TYPE_U8, 0, 0 },
+	{ "--icmp-id", ATTR_ICMP_ID, CT_ATTR_TYPE_BE16, 0, 0 },
+	{}
+};
+
 static void final_check(unsigned int flags,
 		        unsigned int cmd,
 		        struct nf_conntrack *ct)
@@ -117,6 +124,7 @@ static struct ctproto_handler icmp = {
 	.protonum	= IPPROTO_ICMP,
 	.parse_opts	= parse,
 	.final_check	= final_check,
+	.print_opts	= icmp_print_opts,
 	.help		= help,
 	.opts		= opts,
 	.version	= VERSION,
diff --git a/extensions/libct_proto_icmpv6.c b/extensions/libct_proto_icmpv6.c
index f872c23a1075..20b9d79aaf78 100644
--- a/extensions/libct_proto_icmpv6.c
+++ b/extensions/libct_proto_icmpv6.c
@@ -105,6 +105,13 @@ static int parse(char c,
 	return 1;
 }
 
+static struct ct_print_opts icmpv6_print_opts[] = {
+	{"--icmpv6-type", ATTR_ICMP_TYPE, CT_ATTR_TYPE_U8, 0, 0},
+	{"--icmpv6-code", ATTR_ICMP_CODE, CT_ATTR_TYPE_U8, 0, 0},
+	{"--icmpv6-id", ATTR_ICMP_ID, CT_ATTR_TYPE_BE16, 0, 0},
+	{0, 0, 0, 0, 0},
+};
+
 static void final_check(unsigned int flags,
 		        unsigned int cmd,
 		        struct nf_conntrack *ct)
@@ -119,6 +126,7 @@ static struct ctproto_handler icmpv6 = {
 	.protonum	= IPPROTO_ICMPV6,
 	.parse_opts	= parse,
 	.final_check	= final_check,
+	.print_opts	= icmpv6_print_opts,
 	.help		= help,
 	.opts		= opts,
 	.version	= VERSION,
diff --git a/extensions/libct_proto_sctp.c b/extensions/libct_proto_sctp.c
index 04828bf15815..e582e917d961 100644
--- a/extensions/libct_proto_sctp.c
+++ b/extensions/libct_proto_sctp.c
@@ -198,6 +198,17 @@ parse_options(char c, struct nf_conntrack *ct,
 	return 1;
 }
 
+static struct ct_print_opts sctp_print_opts[] = {
+	{ "--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--state", ATTR_SCTP_STATE, CT_ATTR_TYPE_U8, SCTP_CONNTRACK_MAX, sctp_states },
+	{ "--orig-vtag", ATTR_SCTP_VTAG_ORIG, CT_ATTR_TYPE_BE32, 0, 0 },
+	{ "--reply-vtag", ATTR_SCTP_VTAG_REPL, CT_ATTR_TYPE_BE32, 0, 0 },
+	{},
+};
+
 #define SCTP_VALID_FLAGS_MAX   2
 static unsigned int dccp_valid_flags[SCTP_VALID_FLAGS_MAX] = {
 	CT_SCTP_ORIG_SPORT | CT_SCTP_ORIG_DPORT,
@@ -235,6 +246,7 @@ static struct ctproto_handler sctp = {
 	.protonum		= IPPROTO_SCTP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.print_opts		= sctp_print_opts,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_tcp.c b/extensions/libct_proto_tcp.c
index 8a37a556327c..5e580e939fba 100644
--- a/extensions/libct_proto_tcp.c
+++ b/extensions/libct_proto_tcp.c
@@ -177,6 +177,15 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ct_print_opts tcp_print_opts[] = {
+	{"--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--state", ATTR_TCP_STATE, CT_ATTR_TYPE_U8, TCP_CONNTRACK_MAX, tcp_states},
+	{0, 0, 0, 0, 0},
+};
+
 #define TCP_VALID_FLAGS_MAX   2
 static unsigned int tcp_valid_flags[TCP_VALID_FLAGS_MAX] = {
        CT_TCP_ORIG_SPORT | CT_TCP_ORIG_DPORT,
@@ -228,6 +237,7 @@ static struct ctproto_handler tcp = {
 	.protonum		= IPPROTO_TCP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.print_opts		= tcp_print_opts,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_udp.c b/extensions/libct_proto_udp.c
index e30637c7bd1f..02e368b38073 100644
--- a/extensions/libct_proto_udp.c
+++ b/extensions/libct_proto_udp.c
@@ -144,6 +144,14 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ct_print_opts udp_print_opts[] = {
+	{"--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0},
+	{"--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0},
+	{},
+};
+
 #define UDP_VALID_FLAGS_MAX   2
 static unsigned int udp_valid_flags[UDP_VALID_FLAGS_MAX] = {
        CT_UDP_ORIG_SPORT | CT_UDP_ORIG_DPORT,
@@ -181,6 +189,7 @@ static struct ctproto_handler udp = {
 	.protonum		= IPPROTO_UDP,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.print_opts		= udp_print_opts,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/extensions/libct_proto_udplite.c b/extensions/libct_proto_udplite.c
index f46cef0c30f5..80de70c56179 100644
--- a/extensions/libct_proto_udplite.c
+++ b/extensions/libct_proto_udplite.c
@@ -148,6 +148,14 @@ static int parse_options(char c,
 	return 1;
 }
 
+static struct ct_print_opts udplite_print_opts[] = {
+	{ "--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 },
+	{ "--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 },
+	{},
+};
+
 #define UDPLITE_VALID_FLAGS_MAX   2
 static unsigned int udplite_valid_flags[UDPLITE_VALID_FLAGS_MAX] = {
        CT_UDPLITE_ORIG_SPORT | CT_UDPLITE_ORIG_DPORT,
@@ -186,6 +194,7 @@ static struct ctproto_handler udplite = {
 	.protonum		= IPPROTO_UDPLITE,
 	.parse_opts		= parse_options,
 	.final_check		= final_check,
+	.print_opts		= udplite_print_opts,
 	.help			= help,
 	.opts			= opts,
 	.version		= VERSION,
diff --git a/include/conntrack.h b/include/conntrack.h
index 37ccf6e9a87e..bc58d3be8f41 100644
--- a/include/conntrack.h
+++ b/include/conntrack.h
@@ -8,6 +8,9 @@
 
 #include <netinet/in.h>
 
+#include <linux/netfilter/nf_conntrack_common.h>
+#include <libnetfilter_conntrack/libnetfilter_conntrack.h>
+
 #define NUMBER_OF_CMD   19
 #define NUMBER_OF_OPT   29
 
@@ -32,6 +35,8 @@ struct ctproto_handler {
 			    unsigned int command,
 			    struct nf_conntrack *ct);
 
+	const struct ct_print_opts *print_opts;
+
 	void (*help)(void);
 
 	struct option 		*opts;
@@ -53,6 +58,31 @@ void exit_error(enum exittype status, const char *msg, ...);
 
 extern void register_proto(struct ctproto_handler *h);
 
+enum ct_attr_type {
+	CT_ATTR_TYPE_NONE = 0,
+	CT_ATTR_TYPE_U8,
+	CT_ATTR_TYPE_BE16,
+	CT_ATTR_TYPE_U16,
+	CT_ATTR_TYPE_BE32,
+	CT_ATTR_TYPE_U32,
+	CT_ATTR_TYPE_U64,
+	CT_ATTR_TYPE_U32_BITMAP,
+	CT_ATTR_TYPE_IPV4,
+	CT_ATTR_TYPE_IPV6,
+};
+
+struct ct_print_opts {
+	const char		*name;
+	enum nf_conntrack_attr	type;
+	short			value_type;
+	short			val_mapping_count;
+	const char		**val_mapping;
+};
+
+extern int ct_snprintf_opts(char *buf, unsigned int len,
+			    const struct nf_conntrack *ct,
+			    const struct ct_print_opts *attrs);
+
 extern void register_tcp(void);
 extern void register_udp(void);
 extern void register_udplite(void);
diff --git a/src/conntrack.c b/src/conntrack.c
index a26fa60bbbc9..4e1cb9fdc60c 100644
--- a/src/conntrack.c
+++ b/src/conntrack.c
@@ -53,6 +53,7 @@
 #include <sys/socket.h>
 #include <sys/time.h>
 #include <time.h>
+#include <inttypes.h>
 #ifdef HAVE_ARPA_INET_H
 #include <arpa/inet.h>
 #endif
@@ -607,6 +608,240 @@ void register_proto(struct ctproto_handler *h)
 	list_add(&h->head, &proto_list);
 }
 
+#define BUFFER_SIZE(ret, size, len, offset)		do {\
+	size += ret;					\
+	if ((int)ret > (int)len)					\
+		ret = len;				\
+	offset += ret;					\
+	len -= ret;	\
+} while(0)
+
+static int ct_snprintf_u32_bitmap(char *buf, size_t size,
+				  const struct nf_conntrack *ct,
+				  const struct ct_print_opts *attr)
+{
+	unsigned int offset = 0, ret, len = size;
+	bool found = false;
+	uint32_t val;
+	int i;
+
+	val = nfct_get_attr_u32(ct, attr->type);
+
+	for (i = 0; i < attr->val_mapping_count; i++) {
+		if (!(val & (1 << i)))
+			continue;
+		if (!attr->val_mapping[i])
+			continue;
+
+		ret = snprintf(buf + offset, len, "%s,", attr->val_mapping[i]);
+		BUFFER_SIZE(ret, size, len, offset);
+		found = true;
+	}
+
+	if (found) {
+		offset--;
+		ret = snprintf(buf + offset, len, " ");
+		BUFFER_SIZE(ret, size, len, offset);
+	}
+
+	return offset;
+}
+
+static int ct_snprintf_attr(char *buf, size_t size,
+			    const struct nf_conntrack *ct,
+			    const struct ct_print_opts *attr)
+{
+	char ipstr[INET6_ADDRSTRLEN] = {};
+	unsigned int offset = 0;
+	int type = attr->type;
+	int len = size;
+	int ret;
+
+	ret = snprintf(buf, len, "%s ", attr->name);
+	BUFFER_SIZE(ret, size, len, offset);
+
+	switch (attr->value_type) {
+	case CT_ATTR_TYPE_U8:
+		if (attr->val_mapping)
+			ret = snprintf(buf + offset, len, "%s ",
+				       attr->val_mapping[nfct_get_attr_u8(ct, type)]);
+		else
+			ret = snprintf(buf + offset, len, "%u ",
+				       nfct_get_attr_u8(ct, type));
+		break;
+	case CT_ATTR_TYPE_BE16:
+		ret = snprintf(buf + offset, len, "%u ",
+			       ntohs(nfct_get_attr_u16(ct, type)));
+		break;
+	case CT_ATTR_TYPE_U16:
+		ret = snprintf(buf + offset, len, "%u ",
+			       nfct_get_attr_u16(ct, type));
+		break;
+	case CT_ATTR_TYPE_BE32:
+		ret = snprintf(buf + offset, len, "%u ",
+			       ntohl(nfct_get_attr_u32(ct, type)));
+		break;
+	case CT_ATTR_TYPE_U32:
+		ret = snprintf(buf + offset, len, "%u ",
+			       nfct_get_attr_u32(ct, type));
+		break;
+	case CT_ATTR_TYPE_U64:
+		ret = snprintf(buf + offset, len, "%lu ",
+			       nfct_get_attr_u64(ct, type));
+		break;
+	case CT_ATTR_TYPE_IPV4:
+		inet_ntop(AF_INET, nfct_get_attr(ct, type),
+			  ipstr, sizeof(ipstr));
+		ret = snprintf(buf + offset, len, "%s ", ipstr);
+		break;
+	case CT_ATTR_TYPE_IPV6:
+		inet_ntop(AF_INET6, nfct_get_attr(ct, type),
+			  ipstr, sizeof(ipstr));
+		ret = snprintf(buf + offset, len, "%s ", ipstr);
+		break;
+	case CT_ATTR_TYPE_U32_BITMAP:
+		ret = ct_snprintf_u32_bitmap(buf + offset, len, ct, attr);
+		break;
+	default:
+		/* Unsupported datatype, should not ever happen */
+		ret = 0;
+		break;
+	}
+	BUFFER_SIZE(ret, size, len, offset);
+
+	return offset;
+}
+
+int ct_snprintf_opts(char *buf, unsigned int len, const struct nf_conntrack *ct,
+		     const struct ct_print_opts *attrs)
+{
+	unsigned int size = 0, offset = 0, ret;
+	int i;
+
+	for (i = 0; attrs[i].name; ++i) {
+		if (!nfct_attr_is_set(ct, attrs[i].type))
+			continue;
+
+		ret = ct_snprintf_attr(buf + offset, len, ct, &attrs[i]);
+		BUFFER_SIZE(ret, size, len, offset);
+	}
+
+	return offset;
+}
+
+static struct ct_print_opts attrs_ipv4[] = {
+	{"-s", ATTR_ORIG_IPV4_SRC, CT_ATTR_TYPE_IPV4, 0, 0},
+	{"-d", ATTR_ORIG_IPV4_DST, CT_ATTR_TYPE_IPV4, 0, 0},
+	{"-g", ATTR_DNAT_IPV4, CT_ATTR_TYPE_IPV4, 0, 0},
+	{"-n", ATTR_SNAT_IPV4, CT_ATTR_TYPE_IPV4, 0, 0},
+	{"-r", ATTR_REPL_IPV4_SRC, CT_ATTR_TYPE_IPV4, 0, 0},
+	{"-q", ATTR_REPL_IPV4_DST, CT_ATTR_TYPE_IPV4, 0, 0},
+	{0, 0, 0, 0, 0},
+};
+
+static struct ct_print_opts attrs_ipv6[] = {
+	{"-s", ATTR_ORIG_IPV6_SRC, CT_ATTR_TYPE_IPV6, 0, 0},
+	{"-d", ATTR_ORIG_IPV6_DST, CT_ATTR_TYPE_IPV6, 0, 0},
+	{"-g", ATTR_DNAT_IPV6, CT_ATTR_TYPE_IPV6, 0, 0},
+	{"-n", ATTR_SNAT_IPV6, CT_ATTR_TYPE_IPV6, 0, 0},
+	{"-r", ATTR_REPL_IPV6_SRC, CT_ATTR_TYPE_IPV6, 0, 0},
+	{"-q", ATTR_REPL_IPV6_DST, CT_ATTR_TYPE_IPV6, 0, 0},
+	{0, 0, 0, 0, 0},
+};
+
+static const char *conntrack_status_map[] = {
+	[IPS_ASSURED_BIT] = "ASSURED",
+	[IPS_SEEN_REPLY_BIT] = "SEEN_REPLY",
+	[IPS_FIXED_TIMEOUT_BIT] = "FIXED_TIMEOUT",
+	[IPS_EXPECTED_BIT] = "EXPECTED"
+};
+
+static struct ct_print_opts attrs_generic[] = {
+	{"-t", ATTR_TIMEOUT, CT_ATTR_TYPE_U32, 0, 0},
+	{"-u", ATTR_STATUS, CT_ATTR_TYPE_U32_BITMAP,
+		sizeof(conntrack_status_map)/sizeof(conntrack_status_map[0]),
+		conntrack_status_map},
+	{"-c", ATTR_SECMARK, CT_ATTR_TYPE_U32, 0, 0},
+/*		{"-i", ATTR_ID, CT_ATTR_TYPE_U32, 0, 0}, */
+	{"-w", ATTR_ZONE, CT_ATTR_TYPE_U16, 0, 0},
+	{"--orig-zone", ATTR_ORIG_ZONE, CT_ATTR_TYPE_U16, 0, 0},
+	{"--reply-zone", ATTR_REPL_ZONE, CT_ATTR_TYPE_U16, 0, 0},
+	{},
+};
+
+static int ct_save_snprintf(char *buf, size_t len,
+			    const struct nf_conntrack *ct,
+			    struct nfct_labelmap *map,
+			    enum nf_conntrack_msg_type type)
+{
+	static struct ct_print_opts *attrs_l3;
+	unsigned int size = 0, offset = 0;
+	struct ctproto_handler *cur;
+	uint8_t l3proto, l4proto;
+	int ret;
+
+	switch (type) {
+	case NFCT_T_NEW:
+		ret = snprintf(buf + offset, len, "-I ");
+		BUFFER_SIZE(ret, size, len, offset);
+		break;
+	case NFCT_T_UPDATE:
+		ret = snprintf(buf + offset, len, "-U ");
+		BUFFER_SIZE(ret, size, len, offset);
+		break;
+	case NFCT_T_DESTROY:
+		ret = snprintf(buf + offset, len, "-D ");
+		BUFFER_SIZE(ret, size, len, offset);
+		break;
+	default:
+		ret = 0;
+		break;
+	}
+
+	ret = ct_snprintf_opts(buf + offset, len, ct, attrs_generic);
+	BUFFER_SIZE(ret, size, len, offset);
+
+	l3proto = nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO);
+	if (!l3proto)
+		l3proto = nfct_get_attr_u8(ct, ATTR_REPL_L3PROTO);
+	switch (l3proto) {
+	case AF_INET:
+		attrs_l3 = attrs_ipv4;
+		break;
+	case AF_INET6:
+		attrs_l3 = attrs_ipv6;
+		break;
+	default:
+		fprintf(stderr,
+				"WARNING: unknown l3proto %d, skipping..\n", l3proto);
+		return 0;
+	}
+
+	ret = ct_snprintf_opts(buf + offset, len, ct, attrs_l3);
+	BUFFER_SIZE(ret, size, len, offset);
+
+	l4proto = nfct_get_attr_u8(ct, ATTR_L4PROTO);
+
+	/* is it in the list of supported protocol? */
+	list_for_each_entry(cur, &proto_list, head) {
+		if (cur->protonum != l4proto)
+			continue;
+
+		ret = snprintf(buf + offset, len, "-p %s ", cur->name);
+		BUFFER_SIZE(ret, size, len, offset);
+
+		ret = ct_snprintf_opts(buf + offset, len, ct, cur->print_opts);
+		BUFFER_SIZE(ret, size, len, offset);
+		break;
+	}
+
+	/* skip trailing space, if any */
+	for (;size && buf[size-1] == ' '; --size)
+		buf[size-1] = '\0';
+
+	return size;
+}
+
 extern struct ctproto_handler ct_proto_unknown;
 
 static struct ctproto_handler *findproto(char *name, int *pnum)
@@ -856,6 +1091,7 @@ enum {
 	_O_KTMS	= (1 << 4),
 	_O_CL	= (1 << 5),
 	_O_US	= (1 << 6),
+	_O_SAVE	= (1 << 7),
 };
 
 enum {
@@ -866,7 +1102,7 @@ enum {
 };
 
 static struct parse_parameter {
-	const char	*parameter[7];
+	const char	*parameter[8];
 	size_t  size;
 	unsigned int value[8];
 } parse_array[PARSE_MAX] = {
@@ -874,8 +1110,8 @@ static struct parse_parameter {
 	  { IPS_ASSURED, IPS_SEEN_REPLY, 0, IPS_FIXED_TIMEOUT, IPS_EXPECTED, IPS_OFFLOAD, IPS_HW_OFFLOAD} },
 	{ {"ALL", "NEW", "UPDATES", "DESTROY"}, 4,
 	  { CT_EVENT_F_ALL, CT_EVENT_F_NEW, CT_EVENT_F_UPD, CT_EVENT_F_DEL } },
-	{ {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace" }, 7,
-	  { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US },
+	{ {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace", "save"}, 8,
+	  { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US, _O_SAVE },
 	},
 };
 
@@ -1458,6 +1694,11 @@ static int event_cb(const struct nlmsghdr *nlh, void *data)
 	if (nfct_filter(obj, ct))
 		goto out;
 
+	if (output_mask & _O_SAVE) {
+		ct_save_snprintf(buf, sizeof(buf), ct, labelmap, type);
+		goto done;
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -1482,7 +1723,7 @@ static int event_cb(const struct nlmsghdr *nlh, void *data)
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf_labels(buf, sizeof(buf), ct, type, op_type, op_flags, labelmap);
-
+done:
 	if (output_mask & _O_US) {
 		if (nlh->nlmsg_pid)
 			userspace = true;
@@ -1511,6 +1752,11 @@ static int dump_cb(enum nf_conntrack_msg_type type,
 	if (nfct_filter(obj, ct))
 		return NFCT_CB_CONTINUE;
 
+	if (output_mask & _O_SAVE) {
+		ct_save_snprintf(buf, sizeof(buf), ct, labelmap, NFCT_T_NEW);
+		goto done;
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -1527,6 +1773,7 @@ static int dump_cb(enum nf_conntrack_msg_type type,
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap);
+done:
 	printf("%s\n", buf);
 
 	counter++;
@@ -1553,6 +1800,11 @@ static int delete_cb(enum nf_conntrack_msg_type type,
 			   "Operation failed: %s",
 			   err2str(errno, CT_DELETE));
 
+	if (output_mask & _O_SAVE) {
+		ct_save_snprintf(buf, sizeof(buf), ct, labelmap, NFCT_T_DESTROY);
+		goto done;
+	}
+
 	if (output_mask & _O_XML)
 		op_type = NFCT_O_XML;
 	if (output_mask & _O_EXT)
@@ -1561,6 +1813,7 @@ static int delete_cb(enum nf_conntrack_msg_type type,
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags);
+done:
 	printf("%s\n", buf);
 
 	counter++;
@@ -1576,6 +1829,11 @@ static int print_cb(enum nf_conntrack_msg_type type,
 	unsigned int op_type = NFCT_O_DEFAULT;
 	unsigned int op_flags = 0;
 
+	if (output_mask & _O_SAVE) {
+		ct_save_snprintf(buf, sizeof(buf), ct, labelmap, NFCT_T_NEW);
+		goto done;
+	}
+
 	if (output_mask & _O_XML)
 		op_type = NFCT_O_XML;
 	if (output_mask & _O_EXT)
@@ -1584,6 +1842,7 @@ static int print_cb(enum nf_conntrack_msg_type type,
 		op_flags |= NFCT_OF_ID;
 
 	nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap);
+done:
 	printf("%s\n", buf);
 
 	return NFCT_CB_CONTINUE;
@@ -1748,6 +2007,11 @@ static int dump_exp_cb(enum nf_conntrack_msg_type type,
 	unsigned int op_type = NFCT_O_DEFAULT;
 	unsigned int op_flags = 0;
 
+	if (output_mask & _O_SAVE) {
+		exit_error(PARAMETER_PROBLEM,
+		   "save output format is not supported for table of expectations");
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -1779,6 +2043,11 @@ static int event_exp_cb(enum nf_conntrack_msg_type type,
 	unsigned int op_type = NFCT_O_DEFAULT;
 	unsigned int op_flags = 0;
 
+	if (output_mask & _O_SAVE) {
+		exit_error(PARAMETER_PROBLEM,
+		   "save output format is not supported for table of expectations");
+	}
+
 	if (output_mask & _O_XML) {
 		op_type = NFCT_O_XML;
 		if (dump_xml_header_done) {
@@ -2445,6 +2714,10 @@ int main(int argc, char *argv[])
 			parse_parameter(optarg, &output_mask, PARSE_OUTPUT);
 			if (output_mask & _O_CL)
 				labelmap_init();
+			if ((output_mask & _O_SAVE) &&
+			    (output_mask & _O_XML))
+				exit_error(OTHER_PROBLEM,
+					   "cannot combine XML and save output types");
 			break;
 		case 'z':
 			options |= CT_OPT_ZERO;

^ permalink raw reply related	[flat|nested] 20+ messages in thread

* Re: [PATCH 6/8] conntrack: implement options output format
  2020-10-22 12:37     ` Pablo Neira Ayuso
@ 2020-10-23 11:07       ` Mikhail Sennikovsky
  0 siblings, 0 replies; 20+ messages in thread
From: Mikhail Sennikovsky @ 2020-10-23 11:07 UTC (permalink / raw)
  To: Pablo Neira Ayuso; +Cc: netfilter-devel

Thanks Pablo,

Do you want me to send the updated patchset with your adjustments incorporated?

Mikhail

On Thu, 22 Oct 2020 at 14:37, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
>
> On Thu, Oct 22, 2020 at 02:36:47PM +0200, Pablo Neira Ayuso wrote:
> > Hi Mikhail,
> >
> > Thanks for your patchset.
> >
> > On Fri, Sep 25, 2020 at 02:49:17PM +0200, Mikhail Sennikovsky wrote:
> > > As a counterpart to the "conntrack: accept parameters from stdin"
> > > commit, this commit allows dumping conntrack entries in the format
> > > used by the conntrack parameters.
> > > This is useful for transfering a large set of ct entries between
> > > hosts or between different ct zones in an efficient way.
> > >
> > > To enable the "options" output the "-o opts" parameter needs to be
> > > passed to the "contnrack -L" tool invocation.
> >
> > I started slightly revisiting this 6/8 patch a bit (please find it
> > enclosed to this email), I have rename -o opts to -o save, to get this
> > aligned with iptables-save.
>
> Attaching the revisited patch 6/8 to this email.
>
> > I have also added a check for -o xml,save , to reject this
> > combination.
> >
> > I have extended it to display -I, -U, -D in the conntrack events.
> >
> > I have removed several safety runtime checks, that can be done at
> > registration time (make sure the option description is well-formed
> > from there, otherwise rise an error message to spot buggy protocol
> > extensions).
> >
> > This patch should also be extended to support for other existing
> > output flags combinations. Or just bail out if they are specified.
> >
> > At this point I have concerns with NAT: I don't see how this can work
> > as is. There is also a conntrack helpers that might trigger NAT
> > sequence adjustments, this information would be lost.
> >
> > We would need to expose all these details through the -o save, see
> > below. For some of this, there is no options from command line,
> > because it made no sense to expose them.
> >
> > We have to discuss this before deciding where to go. See below for
> > details.
> >
> > > To demonstrate the overall idea of the options output format works
> > > in conjunction with the "stdin parameter"s mode,
> > > the following command will copy all ct entries from one ct zone
> > > to another.
> > >
> > > conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I -
> >
> > For zone updates in the same host, probably conntrack can be extended
> > to support for:
> >
> >         conntrack -U --zone 15 --set-zone 9915
> >
> > If --set-zone is specified, then --zone is used a filter.
> >
> > Then, for "zone transfers" *between hosts*, a different way to address
> > this is to extend conntrackd.
> >
> > The idea is:
> >
> > 1) Add new "transfer" mode which does _not_ subscribe to
> >    conntrack events, it needs to register a new struct ct_mode
> >    (currently there is "sync" and "stats" ct_modes).
> >
> > 2) Add a new message type to request a zone transfer, e.g.
> >
> >         conntrackd --from 192.168.10.20 --zone 15 --set-zone 9915
> >
> >    This will make your local daemon send a request to the conntrackd
> >    instance running on host 192.168.10.20 to retrieve zone 1200. The
> >    remote conntrackd instance dumps the existing conntrack table from
> >    kernel and sends it to you.
> >
> >    You can reuse the channel infrastructure to establish communications
> >    between conntrackd instances in the new "transfer mode". You can
> >    also reuse the sync protocol, see network.h, build.c and parse.c,
> >    which takes a conntrack object and it translates it to network
> >    message.
> >
> >    Note that the struct internal_handler actually refers to the
> >    netlink handler for this new struct ct_mode that you would be
> >    registering.
> >
> > Let me know, thanks.

^ permalink raw reply	[flat|nested] 20+ messages in thread

* Re: [PATCH 3/8] conntrack: accept parameters from stdin
  2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky
@ 2020-11-04 13:56   ` Pablo Neira Ayuso
  0 siblings, 0 replies; 20+ messages in thread
From: Pablo Neira Ayuso @ 2020-11-04 13:56 UTC (permalink / raw)
  To: Mikhail Sennikovsky; +Cc: netfilter-devel

Hi Mikhail,

On Fri, Sep 25, 2020 at 02:49:14PM +0200, Mikhail Sennikovsky wrote:
> This commit allows accepting multiple sets of ct entry-related
> parameters on stdin.
> This is useful when one needs to add/update/delete a large
> set of ct entries with a single conntrack tool invocation.
> 
> Expected syntax is "conntrack [-I|-D|-U] [table] -".
> When invoked like that, conntrack expects ct entry parameters
> to be passed to the stdin, each line presenting a separate parameter
> set.

We have to follow a slightly different approach.

For the batch mode, we have to do similar to iptables, see
do_parse() there.

This parser will create a list of command objects, something like:

struct ct_cmd {
        struct list_head        list;
        ...
        /* attributes that result from parser that describe this command */
};

Once we have the list of commands, iterate over this list of commands
and send the netlink commands.

Thanks.

^ permalink raw reply	[flat|nested] 20+ messages in thread

end of thread, other threads:[~2020-11-04 13:56 UTC | newest]

Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky
2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky
2020-10-13 18:28   ` Pablo Neira Ayuso
2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky
2020-10-13 18:29   ` Pablo Neira Ayuso
2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky
2020-11-04 13:56   ` Pablo Neira Ayuso
2020-09-25 12:49 ` [PATCH 4/8] conntrack.8: man update for stdin params support Mikhail Sennikovsky
2020-09-25 12:49 ` [PATCH 5/8] tests: conntrack parameters from stdin Mikhail Sennikovsky
2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky
2020-10-22 12:36   ` Pablo Neira Ayuso
2020-10-22 12:37     ` Pablo Neira Ayuso
2020-10-23 11:07       ` Mikhail Sennikovsky
2020-09-25 12:49 ` [PATCH 7/8] conntrack.8: man update for opts format support Mikhail Sennikovsky
2020-09-25 12:49 ` [PATCH 8/8] tests: dumping ct entries in opts format Mikhail Sennikovsky
2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal
2020-09-26 18:19   ` Pablo Neira Ayuso
2020-09-29 11:20     ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky
2020-09-29 11:20       ` [PATCH 1/2] " Mikhail Sennikovsky
2020-09-29 11:20       ` [PATCH 2/2] tests: conntrack -L/-D ip family filtering Mikhail Sennikovsky

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.