* [PATCH 0/8] Fast bulk transfers of large sets of ct entries @ 2020-09-25 12:49 Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky ` (8 more replies) 0 siblings, 9 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Hi Fellows, This is a set of patches to the conntrack tool that I think might be interesting to the community. The PATCH 1 and PATCH 2 represent a fix to the icmp ct entry creation (a test case and a fix respectively). The remaining patches represent an extension to the conntrack tool functionality that enables fast bulk transfers of large sets of ct entries, which includes creating a large set of ct entries with a single conntrack tool invocation by passing ct entry parameters on stdin and making conntrack be able to dump ct entries in a new "opts" format that could be later fed back to the conntrack. To demonstrate the overall idea, this functionality makes it possible to e.g. copy all ct entries from one ct zone (15) to another (9915) with the following command: conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I - In addition to this I have a question about the behavioural change of the "conntrack -L" done after conntrack v1.4.5. With the conntrack v1.4.5 used on Debian Buster the "conntrack -L" dumps both ipv4 and ipv6 ct entries, while with the current master, presumably starting with the commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e "conntrack -L" only dumps ipv4 entries. So is this really the desired behavior? (I see the manual page was always saying it should be like that, but since it behaved differently there might be multiple appliances out there relying on the "old" behavior). And if the "new" behavior is desired, would it make sense to add a new -f option value, e.g. "any", that would actually explicitly allow the "old" behaviour, i.e. dump both ipv4 and ipv6 entries with one go? If yes - I could create a small patch for that as well. Thanks & Regards, Mikhail Mikhail Sennikovsky (8): tests: icmp entry create/delete conntrack: fix icmp entry creation conntrack: accept parameters from stdin conntrack.8: man update for stdin params support tests: conntrack parameters from stdin conntrack: implement options output format conntrack.8: man update for opts format support tests: dumping ct entries in opts format conntrack.8 | 13 +- extensions/libct_proto_dccp.c | 24 ++ extensions/libct_proto_gre.c | 16 + extensions/libct_proto_icmp.c | 33 ++ extensions/libct_proto_icmpv6.c | 33 ++ extensions/libct_proto_sctp.c | 19 ++ extensions/libct_proto_tcp.c | 17 ++ extensions/libct_proto_udp.c | 16 + extensions/libct_proto_udplite.c | 16 + include/conntrack.h | 38 +++ src/conntrack.c | 457 +++++++++++++++++++++++++--- tests/conntrack/test-conntrack.c | 84 ++++- tests/conntrack/testsuite/00create | 4 + tests/conntrack/testsuite/08stdin | 62 ++++ tests/conntrack/testsuite/09dumpopt | 77 +++++ 15 files changed, 857 insertions(+), 52 deletions(-) create mode 100644 tests/conntrack/testsuite/08stdin create mode 100644 tests/conntrack/testsuite/09dumpopt -- 2.25.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 1/8] tests: icmp entry create/delete 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-10-13 18:28 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky ` (7 subsequent siblings) 8 siblings, 1 reply; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Add test to cover icmp entry creation/deletion with conntrack Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- tests/conntrack/testsuite/00create | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/tests/conntrack/testsuite/00create b/tests/conntrack/testsuite/00create index 4e55a7b..911e711 100644 --- a/tests/conntrack/testsuite/00create +++ b/tests/conntrack/testsuite/00create @@ -30,3 +30,7 @@ -D -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK # mismatched address family -I -s 2001:DB8::1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +# creae icmp ping request entry +-I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# delete icmp ping request entry +-D -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 1/8] tests: icmp entry create/delete 2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky @ 2020-10-13 18:28 ` Pablo Neira Ayuso 0 siblings, 0 replies; 20+ messages in thread From: Pablo Neira Ayuso @ 2020-10-13 18:28 UTC (permalink / raw) To: Mikhail Sennikovsky; +Cc: netfilter-devel On Fri, Sep 25, 2020 at 02:49:12PM +0200, Mikhail Sennikovsky wrote: > Add test to cover icmp entry creation/deletion with conntrack Applied, thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 2/8] conntrack: fix icmp entry creation 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-10-13 18:29 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky ` (6 subsequent siblings) 8 siblings, 1 reply; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Creating icmp ct entry with command like conntrack -I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 \ -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 results in nfct_query( NFCT_Q_CREATE ) request would fail because reply L4 proto is not set while having reply data specified Set reply L4 proto when reply data is given for the icmp ct entry Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- extensions/libct_proto_icmp.c | 18 ++++++++++++++++++ extensions/libct_proto_icmpv6.c | 18 ++++++++++++++++++ 2 files changed, 36 insertions(+) diff --git a/extensions/libct_proto_icmp.c b/extensions/libct_proto_icmp.c index 2ce1c65..16c2e2e 100644 --- a/extensions/libct_proto_icmp.c +++ b/extensions/libct_proto_icmp.c @@ -78,18 +78,36 @@ static int parse(char c, tmp = atoi(optarg); nfct_set_attr_u8(ct, ATTR_ICMP_TYPE, tmp); nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMP); + /* + * need to set the reply proto, otherwise the + * NFCT_Q_CREATE call would fail + */ + if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO)) + nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMP); *flags |= CT_ICMP_TYPE; break; case '2': tmp = atoi(optarg); nfct_set_attr_u8(ct, ATTR_ICMP_CODE, tmp); nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMP); + /* + * need to set the reply proto, otherwise the + * NFCT_Q_CREATE call would fail + */ + if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO)) + nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMP); *flags |= CT_ICMP_CODE; break; case '3': id = htons(atoi(optarg)); nfct_set_attr_u16(ct, ATTR_ICMP_ID, id); nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMP); + /* + * need to set the reply proto, otherwise the + * NFCT_Q_CREATE call would fail + */ + if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO)) + nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMP); *flags |= CT_ICMP_ID; break; } diff --git a/extensions/libct_proto_icmpv6.c b/extensions/libct_proto_icmpv6.c index 18dd3e5..7f5e637 100644 --- a/extensions/libct_proto_icmpv6.c +++ b/extensions/libct_proto_icmpv6.c @@ -81,18 +81,36 @@ static int parse(char c, tmp = atoi(optarg); nfct_set_attr_u8(ct, ATTR_ICMP_TYPE, tmp); nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMPV6); + /* + * need to set the reply proto, otherwise the + * NFCT_Q_CREATE call would fail + */ + if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO)) + nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMPV6); *flags |= CT_ICMP_TYPE; break; case '2': tmp = atoi(optarg); nfct_set_attr_u8(ct, ATTR_ICMP_CODE, tmp); nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMPV6); + /* + * need to set the reply proto, otherwise the + * NFCT_Q_CREATE call would fail + */ + if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO)) + nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMPV6); *flags |= CT_ICMP_CODE; break; case '3': id = htons(atoi(optarg)); nfct_set_attr_u16(ct, ATTR_ICMP_ID, id); nfct_set_attr_u8(ct, ATTR_L4PROTO, IPPROTO_ICMPV6); + /* + * need to set the reply proto, otherwise the + * NFCT_Q_CREATE call would fail + */ + if (nfct_attr_is_set(ct, ATTR_REPL_L3PROTO)) + nfct_set_attr_u8(ct, ATTR_REPL_L4PROTO, IPPROTO_ICMPV6); *flags |= CT_ICMP_ID; break; } -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 2/8] conntrack: fix icmp entry creation 2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky @ 2020-10-13 18:29 ` Pablo Neira Ayuso 0 siblings, 0 replies; 20+ messages in thread From: Pablo Neira Ayuso @ 2020-10-13 18:29 UTC (permalink / raw) To: Mikhail Sennikovsky; +Cc: netfilter-devel On Fri, Sep 25, 2020 at 02:49:13PM +0200, Mikhail Sennikovsky wrote: > Creating icmp ct entry with command like > > conntrack -I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 \ > -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 > > results in nfct_query( NFCT_Q_CREATE ) request would fail > because reply L4 proto is not set while having reply data specified > > Set reply L4 proto when reply data is given for the icmp ct entry Also applied, thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 3/8] conntrack: accept parameters from stdin 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-11-04 13:56 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 4/8] conntrack.8: man update for stdin params support Mikhail Sennikovsky ` (5 subsequent siblings) 8 siblings, 1 reply; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky This commit allows accepting multiple sets of ct entry-related parameters on stdin. This is useful when one needs to add/update/delete a large set of ct entries with a single conntrack tool invocation. Expected syntax is "conntrack [-I|-D|-U] [table] -". When invoked like that, conntrack expects ct entry parameters to be passed to the stdin, each line presenting a separate parameter set. Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- src/conntrack.c | 196 +++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 161 insertions(+), 35 deletions(-) diff --git a/src/conntrack.c b/src/conntrack.c index a26fa60..5834f2d 100644 --- a/src/conntrack.c +++ b/src/conntrack.c @@ -96,15 +96,18 @@ static struct { struct nfct_bitmask *label_modify; } tmpl; +int cur_argc; +char **cur_argv; + static int alloc_tmpl_objects(void) { + memset(&tmpl, 0, sizeof(tmpl)); + tmpl.ct = nfct_new(); tmpl.exptuple = nfct_new(); tmpl.mask = nfct_new(); tmpl.exp = nfexp_new(); - memset(&tmpl.mark, 0, sizeof(tmpl.mark)); - return tmpl.ct != NULL && tmpl.exptuple != NULL && tmpl.mask != NULL && tmpl.exp != NULL; } @@ -685,12 +688,18 @@ static void free_options(void) void __attribute__((noreturn)) exit_error(enum exittype status, const char *msg, ...) { + int i; va_list args; free_options(); va_start(args, msg); fprintf(stderr,"%s v%s (conntrack-tools): ", PROGNAME, VERSION); vfprintf(stderr, msg, args); + if (cur_argc) { + fprintf(stderr, "\nargs:"); + for (i = 1; i < cur_argc; ++i) + fprintf(stderr, " %s", cur_argv[i]); + } fprintf(stderr, "\n"); va_end(args); if (status == PARAMETER_PROBLEM) @@ -2317,23 +2326,63 @@ nfct_set_nat_details(const int opt, struct nf_conntrack *ct, nfct_set_attr_u16(ct, ATTR_DNAT_PORT, ntohs((uint16_t)atoi(port_str))); } +} + +static int line_to_argcv(char *cmd, char *line, char ***pargv, size_t *pargv_size) +{ + char *arg; + int argc; + char **argv = *pargv; + size_t argv_size = *pargv_size; + int argc_max = argv_size / sizeof(*argv); + +#define _ARG_ADD(_arg) do { \ + if (argc == argc_max) { \ + argc_max += 20; \ + argv_size = argc_max * sizeof (argv[0]); \ + argv = realloc(argv, argv_size); \ + if (!argv) \ + exit_error(OTHER_PROBLEM, "out of memory"); \ + } \ + argv[argc] = _arg; \ + ++argc; \ +} while (0) + +#define _ARG_SEP " \t\n\r" + for (argc = 0, arg = strtok (line, _ARG_SEP); + arg; + arg = strtok (NULL, _ARG_SEP)) { + /* + * getopt_long expects argv[0] to be the command name, + * and would always skip it so we need to include it here + */ + if (!argc && cmd) + _ARG_ADD(cmd); + _ARG_ADD(arg); + } + +#undef _ARG_ADD +#undef _ARG_SEP + *pargv = argv; + *pargv_size = argv_size; + + return argc; } int main(int argc, char *argv[]) { int c, cmd; - unsigned int type = 0, event_mask = 0, l4flags = 0, status = 0; + unsigned int type = 0, event_mask = 0, l4flags, status = 0; int res = 0, partial; - size_t socketbuffersize = 0; - int family = AF_UNSPEC; - int protonum = 0; + size_t socketbuffersize; + int family; + int protonum; union ct_address ad; unsigned int command = 0; - - /* we release these objects in the exit_error() path. */ - if (!alloc_tmpl_objects()) - exit_error(OTHER_PROBLEM, "out of memory"); + FILE *opts_file = NULL; + char **argv_buf = NULL, *getline_buf = NULL; + size_t argv_buf_size = 0, getline_buf_size = 0; register_tcp(); register_udp(); @@ -2348,6 +2397,23 @@ int main(int argc, char *argv[]) /* disable explicit missing arguments error output from getopt_long */ opterr = 0; +parse_opts: + + options = 0; + filter_family = 0; + memset(dir2network, 0, sizeof(dir2network)); + /* all allocate-able objects get freed zero-inited + * at the end of each iteration */ + + l4flags = 0; + family = AF_UNSPEC; + socketbuffersize = 0; + protonum = 0; + + /* we release these objects in the exit_error() path. */ + if (!alloc_tmpl_objects()) + exit_error(OTHER_PROBLEM, "out of memory"); + while ((c = getopt_long(argc, argv, getopt_str, opts, NULL)) != -1) { switch(c) { /* commands */ @@ -2585,6 +2651,26 @@ int main(int argc, char *argv[]) "`--dst-nat' with `--any-nat'"); } cmd = bit2cmd(command); + + if (!opts_file && optind == argc - 1 && !strcmp(argv[optind], "-")) { + switch (command) { + case CT_CREATE: + case EXP_CREATE: + case CT_UPDATE: + case CT_DELETE: + case EXP_DELETE: + break; + default: + exit_error(PARAMETER_PROBLEM, "stdin mode not supported " + "for this command!"); + } + if (options) + exit_error(PARAMETER_PROBLEM, "no extra options are expected " + "with stdin read mode!"); + opts_file = stdin; + goto next_opts; + } + res = generic_opt_check(options, NUMBER_OF_OPT, commands_v_options[cmd], optflags, addr_valid_flags, ADDR_VALID_FLAGS_MAX, @@ -2670,8 +2756,8 @@ int main(int argc, char *argv[]) printf("</conntrack>\n"); fflush(stdout); } - nfct_close(cth); + cth = NULL; break; case EXP_LIST: @@ -2681,12 +2767,13 @@ int main(int argc, char *argv[]) nfexp_callback_register(cth, NFCT_T_ALL, dump_exp_cb, NULL); res = nfexp_query(cth, NFCT_Q_DUMP, &family); - nfct_close(cth); if (dump_xml_header_done == 0) { printf("</expect>\n"); fflush(stdout); } + nfct_close(cth); + cth = NULL; break; case CT_CREATE: @@ -2702,14 +2789,15 @@ int main(int argc, char *argv[]) nfct_set_attr(tmpl.ct, ATTR_CONNLABELS, xnfct_bitmask_clone(tmpl.label_modify)); - cth = nfct_open(CONNTRACK, 0); - if (!cth) - exit_error(OTHER_PROBLEM, "Can't open handler"); + if (!cth) { + cth = nfct_open(CONNTRACK, 0); + if (!cth) + exit_error(OTHER_PROBLEM, "Can't open handler"); + } res = nfct_query(cth, NFCT_Q_CREATE, tmpl.ct); if (res != -1) counter++; - nfct_close(cth); break; case EXP_CREATE: @@ -2717,18 +2805,21 @@ int main(int argc, char *argv[]) nfexp_set_attr(tmpl.exp, ATTR_EXP_EXPECTED, tmpl.exptuple); nfexp_set_attr(tmpl.exp, ATTR_EXP_MASK, tmpl.mask); - cth = nfct_open(EXPECT, 0); - if (!cth) - exit_error(OTHER_PROBLEM, "Can't open handler"); + if (!cth) { + cth = nfct_open(EXPECT, 0); + if (!cth) + exit_error(OTHER_PROBLEM, "Can't open handler"); + } res = nfexp_query(cth, NFCT_Q_CREATE, tmpl.exp); - nfct_close(cth); break; case CT_UPDATE: - cth = nfct_open(CONNTRACK, 0); + if (!cth) + cth = nfct_open(CONNTRACK, 0); /* internal handler for delete_cb, otherwise we hit EILSEQ */ - ith = nfct_open(CONNTRACK, 0); + if (!ith) + ith = nfct_open(CONNTRACK, 0); if (!cth || !ith) exit_error(OTHER_PROBLEM, "Can't open handler"); @@ -2737,13 +2828,13 @@ int main(int argc, char *argv[]) nfct_callback_register(cth, NFCT_T_ALL, update_cb, tmpl.ct); res = nfct_query(cth, NFCT_Q_DUMP, &family); - nfct_close(ith); - nfct_close(cth); break; case CT_DELETE: - cth = nfct_open(CONNTRACK, 0); - ith = nfct_open(CONNTRACK, 0); + if (!cth) + cth = nfct_open(CONNTRACK, 0); + if (!ith) + ith = nfct_open(CONNTRACK, 0); if (!cth || !ith) exit_error(OTHER_PROBLEM, "Can't open handler"); @@ -2768,19 +2859,18 @@ int main(int argc, char *argv[]) nfct_filter_dump_destroy(filter_dump); - nfct_close(ith); - nfct_close(cth); break; case EXP_DELETE: nfexp_set_attr(tmpl.exp, ATTR_EXP_EXPECTED, tmpl.ct); - cth = nfct_open(EXPECT, 0); - if (!cth) - exit_error(OTHER_PROBLEM, "Can't open handler"); + if (!cth) { + cth = nfct_open(EXPECT, 0); + if (!cth) + exit_error(OTHER_PROBLEM, "Can't open handler"); + } res = nfexp_query(cth, NFCT_Q_DESTROY, tmpl.exp); - nfct_close(cth); break; case CT_GET: @@ -2791,6 +2881,7 @@ int main(int argc, char *argv[]) nfct_callback_register(cth, NFCT_T_ALL, dump_cb, tmpl.ct); res = nfct_query(cth, NFCT_Q_GET, tmpl.ct); nfct_close(cth); + cth = NULL; break; case EXP_GET: @@ -2803,6 +2894,7 @@ int main(int argc, char *argv[]) nfexp_callback_register(cth, NFCT_T_ALL, dump_exp_cb, NULL); res = nfexp_query(cth, NFCT_Q_GET, tmpl.exp); nfct_close(cth); + cth = NULL; break; case CT_FLUSH: @@ -2810,9 +2902,10 @@ int main(int argc, char *argv[]) if (!cth) exit_error(OTHER_PROBLEM, "Can't open handler"); res = nfct_query(cth, NFCT_Q_FLUSH, &family); - nfct_close(cth); fprintf(stderr, "%s v%s (conntrack-tools): ",PROGNAME,VERSION); fprintf(stderr,"connection tracking table has been emptied.\n"); + nfct_close(cth); + cth = NULL; break; case EXP_FLUSH: @@ -2820,9 +2913,10 @@ int main(int argc, char *argv[]) if (!cth) exit_error(OTHER_PROBLEM, "Can't open handler"); res = nfexp_query(cth, NFCT_Q_FLUSH, &family); - nfct_close(cth); fprintf(stderr, "%s v%s (conntrack-tools): ",PROGNAME,VERSION); fprintf(stderr,"expectation table has been emptied.\n"); + nfct_close(cth); + cth = NULL; break; case CT_EVENT: @@ -2894,6 +2988,8 @@ int main(int argc, char *argv[]) res = mnl_cb_run(buf, res, 0, 0, event_cb, tmpl.ct); } mnl_socket_close(sock.mnl); + nfct_close(cth); + cth = NULL; break; case EXP_EVENT: @@ -2922,6 +3018,7 @@ int main(int argc, char *argv[]) nfexp_callback_register(cth, NFCT_T_ALL, event_exp_cb, NULL); res = nfexp_catch(cth); nfct_close(cth); + cth = NULL; break; case CT_COUNT: /* If we fail with netlink, fall back to /proc to ensure @@ -2966,6 +3063,7 @@ try_proc_count: nfexp_callback_register(cth, NFCT_T_ALL, count_exp_cb, NULL); res = nfexp_query(cth, NFCT_Q_DUMP, &family); nfct_close(cth); + cth = NULL; printf("%d\n", counter); break; case CT_STATS: @@ -3024,10 +3122,35 @@ try_proc: exit_error(OTHER_PROBLEM, "Operation failed: %s", err2str(errno, command)); +next_opts: free_tmpl_objects(); free_options(); - if (labelmap) + if (labelmap) { nfct_labelmap_destroy(labelmap); + labelmap = NULL; + } + + if (opts_file) { + while ((res = getline(&getline_buf, &getline_buf_size, opts_file)) >= 0) { + if (!res) + continue; + argc = line_to_argcv(argv[0], getline_buf, &argv_buf, &argv_buf_size); + if (!argc) + continue; + argv = argv_buf; + + cur_argc = argc; + cur_argv = argv; + + optind = 0; + goto parse_opts; + } + } + + if (ith) + nfct_close(ith); + if (cth) + nfct_close(cth); if (command && exit_msg[cmd][0]) { fprintf(stderr, "%s v%s (conntrack-tools): ",PROGNAME,VERSION); @@ -3036,5 +3159,8 @@ try_proc: return EXIT_FAILURE; } + free(argv_buf); + free(getline_buf); + return EXIT_SUCCESS; } -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 3/8] conntrack: accept parameters from stdin 2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky @ 2020-11-04 13:56 ` Pablo Neira Ayuso 0 siblings, 0 replies; 20+ messages in thread From: Pablo Neira Ayuso @ 2020-11-04 13:56 UTC (permalink / raw) To: Mikhail Sennikovsky; +Cc: netfilter-devel Hi Mikhail, On Fri, Sep 25, 2020 at 02:49:14PM +0200, Mikhail Sennikovsky wrote: > This commit allows accepting multiple sets of ct entry-related > parameters on stdin. > This is useful when one needs to add/update/delete a large > set of ct entries with a single conntrack tool invocation. > > Expected syntax is "conntrack [-I|-D|-U] [table] -". > When invoked like that, conntrack expects ct entry parameters > to be passed to the stdin, each line presenting a separate parameter > set. We have to follow a slightly different approach. For the batch mode, we have to do similar to iptables, see do_parse() there. This parser will create a list of command objects, something like: struct ct_cmd { struct list_head list; ... /* attributes that result from parser that describe this command */ }; Once we have the list of commands, iterate over this list of commands and send the netlink commands. Thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 4/8] conntrack.8: man update for stdin params support 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky ` (2 preceding siblings ...) 2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 5/8] tests: conntrack parameters from stdin Mikhail Sennikovsky ` (4 subsequent siblings) 8 siblings, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- conntrack.8 | 8 ++++++++ 1 file changed, 8 insertions(+) diff --git a/conntrack.8 b/conntrack.8 index 1174c6c..3db4849 100644 --- a/conntrack.8 +++ b/conntrack.8 @@ -135,6 +135,14 @@ to overrun the socket buffer. Note that using a big buffer reduces the chances to hit ENOBUFS, however, this results in more memory consumption. . This option can only be used in conjunction with "\-E, \-\-event". +.TP +.BI "-" +Make conntrack accept multiple sets of ct entry-related parameters on stdin. +This option is useful for the fast bulk conntrack entries updates. +. +This option can only be used in conjunction with "\-I, \-\-create", +"\-U, \-\-update" and "\-D, \-\-delete". +No other options are allowed to be present on the command line. .SS FILTER PARAMETERS .TP -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 5/8] tests: conntrack parameters from stdin 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky ` (3 preceding siblings ...) 2020-09-25 12:49 ` [PATCH 4/8] conntrack.8: man update for stdin params support Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky ` (3 subsequent siblings) 8 siblings, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Add tests to cover handling parameters from stdin by conntrack Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- tests/conntrack/test-conntrack.c | 70 +++++++++++++++++++++++++------ tests/conntrack/testsuite/08stdin | 62 +++++++++++++++++++++++++++ 2 files changed, 120 insertions(+), 12 deletions(-) create mode 100644 tests/conntrack/testsuite/08stdin diff --git a/tests/conntrack/test-conntrack.c b/tests/conntrack/test-conntrack.c index 76ab051..90bdc5b 100644 --- a/tests/conntrack/test-conntrack.c +++ b/tests/conntrack/test-conntrack.c @@ -28,6 +28,23 @@ int main() struct dirent *dent; char file[1024]; int i,n; + char cmd_buf[1024 * 8]; + int i_cmd_buf = 0; + char cmd, cur_cmd = 0; + char *cmd_opt; + +#define cmd_strappend(_s) do { \ + char * pos = stpncpy(cmd_buf + i_cmd_buf, _s, sizeof(cmd_buf) - i_cmd_buf); \ + i_cmd_buf = pos - cmd_buf; \ + if (i_cmd_buf == sizeof(cmd_buf)) { \ + printf("buffer full!\n"); \ + exit(EXIT_FAILURE); \ + } \ +} while (0) + +#define cmd_reset() do { \ + i_cmd_buf = 0; \ +} while (0) n = scandir("testsuite", &dents, NULL, alphasort); @@ -48,9 +65,7 @@ int main() } while (fgets(buf, sizeof(buf), fp)) { - char tmp[1024] = CT_PROG, *res; - tmp[strlen(CT_PROG)] = ' '; - + char *res; line++; if (buf[0] == '#' || buf[0] == ' ') @@ -63,27 +78,58 @@ int main() exit(EXIT_FAILURE); } *res = '\0'; - res+=2; + res++; + for (; *res == ' ' || *res == '\t'; res++); + cmd = res[0]; + + if (cur_cmd && cmd != cur_cmd) { + /* complete current multi-line command */ + switch (cur_cmd) { + case '\n': + cmd_strappend("\" | "); + break; + default: + printf("Internal Error: unexpected multiline command %c", + cur_cmd); + exit(EXIT_FAILURE); + break; + } + + cur_cmd = 0; + } + + switch (cmd) { + case '\n': + if (!cur_cmd) { + cmd_strappend("echo \""); + cur_cmd = cmd; + } else + cmd_strappend("\n"); + cmd_strappend(buf); + continue; + default: + cmd_strappend(CT_PROG); + cmd_strappend(" "); + cmd_strappend(buf); + cmd_reset(); + break; + } - strcpy(tmp + strlen(CT_PROG) + 1, buf); - printf("(%d) Executing: %s\n", line, tmp); + printf("(%d) Executing: %s\n", line, cmd_buf); fflush(stdout); - ret = system(tmp); + ret = system(cmd_buf); if (WIFEXITED(ret) && WEXITSTATUS(ret) == EXIT_SUCCESS) { - if (res[0] == 'O' && - res[1] == 'K') + if (cmd == 'O') ok++; else { bad++; printf("^----- BAD\n"); } } else { - if (res[0] == 'B' && - res[1] == 'A' && - res[2] == 'D') + if (cmd == 'B') ok++; else { bad++; diff --git a/tests/conntrack/testsuite/08stdin b/tests/conntrack/testsuite/08stdin new file mode 100644 index 0000000..cf3eadd --- /dev/null +++ b/tests/conntrack/testsuite/08stdin @@ -0,0 +1,62 @@ +# create +# create a conntrack +-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +# create from reply +-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; +# create a v6 conntrack +-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +# creae icmp ping request entry +-t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; +-I - ; OK +# create again +-I -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD +# make sure create again with stdio mode fails as well +-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +-I - ; BAD +-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; +-I - ; BAD +# empty lines are ignored +; +-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +-I - ; BAD +# spaces or tabs are ignored as well + ; + ; +-t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; +-I - ; BAD +# delete +-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; +# empty lines should be just ignored +; +; +# delete reverse +-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; +# empty lines with spaces or tabs should be ignored as well + ; + ; + ; + ; + ; + ; +# delete v6 conntrack +-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; +# delete icmp ping request entry +-u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; +; +; +-D - ; OK +# create again - should succeed now +-I -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK +-I -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; OK +-I -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK +-I -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# delete again (for cleanup) +-s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; +-r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; +-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; +-u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; +; +-D - ; OK -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 6/8] conntrack: implement options output format 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky ` (4 preceding siblings ...) 2020-09-25 12:49 ` [PATCH 5/8] tests: conntrack parameters from stdin Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-10-22 12:36 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 7/8] conntrack.8: man update for opts format support Mikhail Sennikovsky ` (2 subsequent siblings) 8 siblings, 1 reply; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky As a counterpart to the "conntrack: accept parameters from stdin" commit, this commit allows dumping conntrack entries in the format used by the conntrack parameters. This is useful for transfering a large set of ct entries between hosts or between different ct zones in an efficient way. To enable the "options" output the "-o opts" parameter needs to be passed to the "contnrack -L" tool invocation. To demonstrate the overall idea of the options output format works in conjunction with the "stdin parameter"s mode, the following command will copy all ct entries from one ct zone to another. conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I - Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- extensions/libct_proto_dccp.c | 24 +++ extensions/libct_proto_gre.c | 16 ++ extensions/libct_proto_icmp.c | 15 ++ extensions/libct_proto_icmpv6.c | 15 ++ extensions/libct_proto_sctp.c | 19 +++ extensions/libct_proto_tcp.c | 17 ++ extensions/libct_proto_udp.c | 16 ++ extensions/libct_proto_udplite.c | 16 ++ include/conntrack.h | 38 +++++ src/conntrack.c | 261 ++++++++++++++++++++++++++++++- 10 files changed, 433 insertions(+), 4 deletions(-) diff --git a/extensions/libct_proto_dccp.c b/extensions/libct_proto_dccp.c index f6258ad..cfe1313 100644 --- a/extensions/libct_proto_dccp.c +++ b/extensions/libct_proto_dccp.c @@ -198,6 +198,29 @@ static int parse_options(char c, return 1; } + +static const char *dccp_roles[__DCCP_CONNTRACK_ROLE_MAX] = { + [DCCP_CONNTRACK_ROLE_CLIENT] = "client", + [DCCP_CONNTRACK_ROLE_SERVER] = "server", +}; + +static struct ctproto_attr attrs[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--state", ATTR_DCCP_STATE, CTPROTO_ATTR_U8, DCCP_CONNTRACK_MAX, dccp_states}, + {"--role", ATTR_DCCP_ROLE, CTPROTO_ATTR_U8, __DCCP_CONNTRACK_ROLE_MAX, dccp_roles}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + #define DCCP_VALID_FLAGS_MAX 2 static unsigned int dccp_valid_flags[DCCP_VALID_FLAGS_MAX] = { CT_DCCP_ORIG_SPORT | CT_DCCP_ORIG_DPORT, @@ -235,6 +258,7 @@ static struct ctproto_handler dccp = { .protonum = IPPROTO_DCCP, .parse_opts = parse_options, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_gre.c b/extensions/libct_proto_gre.c index 2dc63d1..b03887b 100644 --- a/extensions/libct_proto_gre.c +++ b/extensions/libct_proto_gre.c @@ -144,6 +144,21 @@ static int parse_options(char c, return 1; } +static struct ctproto_attr attrs[] = { + {"--srckey", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--dstkey", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-key-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-key-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + #define GRE_VALID_FLAGS_MAX 2 static unsigned int gre_valid_flags[GRE_VALID_FLAGS_MAX] = { CT_GRE_ORIG_SKEY | CT_GRE_ORIG_DKEY, @@ -181,6 +196,7 @@ static struct ctproto_handler gre = { .protonum = IPPROTO_GRE, .parse_opts = parse_options, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_icmp.c b/extensions/libct_proto_icmp.c index 16c2e2e..e1af53f 100644 --- a/extensions/libct_proto_icmp.c +++ b/extensions/libct_proto_icmp.c @@ -114,6 +114,20 @@ static int parse(char c, return 1; } +static struct ctproto_attr attrs[] = { + {"--icmp-type", ATTR_ICMP_TYPE, CTPROTO_ATTR_U8, 0, 0}, + {"--icmp-code", ATTR_ICMP_CODE, CTPROTO_ATTR_U8, 0, 0}, + {"--icmp-id", ATTR_ICMP_ID, CTPROTO_ATTR_U16_N, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + static void final_check(unsigned int flags, unsigned int cmd, struct nf_conntrack *ct) @@ -129,6 +143,7 @@ static struct ctproto_handler icmp = { .protonum = IPPROTO_ICMP, .parse_opts = parse, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_icmpv6.c b/extensions/libct_proto_icmpv6.c index 7f5e637..cb2178e 100644 --- a/extensions/libct_proto_icmpv6.c +++ b/extensions/libct_proto_icmpv6.c @@ -117,6 +117,20 @@ static int parse(char c, return 1; } +static struct ctproto_attr attrs[] = { + {"--icmpv6-type", ATTR_ICMP_TYPE, CTPROTO_ATTR_U8, 0, 0}, + {"--icmpv6-code", ATTR_ICMP_CODE, CTPROTO_ATTR_U8, 0, 0}, + {"--icmpv6-id", ATTR_ICMP_ID, CTPROTO_ATTR_U16_N, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + static void final_check(unsigned int flags, unsigned int cmd, struct nf_conntrack *ct) @@ -131,6 +145,7 @@ static struct ctproto_handler icmpv6 = { .protonum = IPPROTO_ICMPV6, .parse_opts = parse, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_sctp.c b/extensions/libct_proto_sctp.c index 04828bf..5b7a43b 100644 --- a/extensions/libct_proto_sctp.c +++ b/extensions/libct_proto_sctp.c @@ -198,6 +198,24 @@ parse_options(char c, struct nf_conntrack *ct, return 1; } +static struct ctproto_attr attrs[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--state", ATTR_SCTP_STATE, CTPROTO_ATTR_U8, SCTP_CONNTRACK_MAX, sctp_states}, + {"--orig-vtag", ATTR_SCTP_VTAG_ORIG, CTPROTO_ATTR_U32_N, 0, 0}, + {"--reply-vtag", ATTR_SCTP_VTAG_REPL, CTPROTO_ATTR_U32_N, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + #define SCTP_VALID_FLAGS_MAX 2 static unsigned int dccp_valid_flags[SCTP_VALID_FLAGS_MAX] = { CT_SCTP_ORIG_SPORT | CT_SCTP_ORIG_DPORT, @@ -235,6 +253,7 @@ static struct ctproto_handler sctp = { .protonum = IPPROTO_SCTP, .parse_opts = parse_options, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_tcp.c b/extensions/libct_proto_tcp.c index 8a37a55..90fecf6 100644 --- a/extensions/libct_proto_tcp.c +++ b/extensions/libct_proto_tcp.c @@ -177,6 +177,22 @@ static int parse_options(char c, return 1; } +static struct ctproto_attr attrs[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--state", ATTR_TCP_STATE, CTPROTO_ATTR_U8, TCP_CONNTRACK_MAX, tcp_states}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + #define TCP_VALID_FLAGS_MAX 2 static unsigned int tcp_valid_flags[TCP_VALID_FLAGS_MAX] = { CT_TCP_ORIG_SPORT | CT_TCP_ORIG_DPORT, @@ -228,6 +244,7 @@ static struct ctproto_handler tcp = { .protonum = IPPROTO_TCP, .parse_opts = parse_options, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_udp.c b/extensions/libct_proto_udp.c index e30637c..a70cee0 100644 --- a/extensions/libct_proto_udp.c +++ b/extensions/libct_proto_udp.c @@ -144,6 +144,21 @@ static int parse_options(char c, return 1; } +static struct ctproto_attr attrs[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + #define UDP_VALID_FLAGS_MAX 2 static unsigned int udp_valid_flags[UDP_VALID_FLAGS_MAX] = { CT_UDP_ORIG_SPORT | CT_UDP_ORIG_DPORT, @@ -181,6 +196,7 @@ static struct ctproto_handler udp = { .protonum = IPPROTO_UDP, .parse_opts = parse_options, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_udplite.c b/extensions/libct_proto_udplite.c index f46cef0..382d990 100644 --- a/extensions/libct_proto_udplite.c +++ b/extensions/libct_proto_udplite.c @@ -148,6 +148,21 @@ static int parse_options(char c, return 1; } +static struct ctproto_attr attrs[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CTPROTO_ATTR_U16_N, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CTPROTO_ATTR_U16_N, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int snprintf_options(char *buf, + unsigned int len, + const struct nf_conntrack *ct) +{ + return snprintf_attrs(buf, len, ct, attrs); +} + #define UDPLITE_VALID_FLAGS_MAX 2 static unsigned int udplite_valid_flags[UDPLITE_VALID_FLAGS_MAX] = { CT_UDPLITE_ORIG_SPORT | CT_UDPLITE_ORIG_DPORT, @@ -186,6 +201,7 @@ static struct ctproto_handler udplite = { .protonum = IPPROTO_UDPLITE, .parse_opts = parse_options, .final_check = final_check, + .snprintf_options = snprintf_options, .help = help, .opts = opts, .version = VERSION, diff --git a/include/conntrack.h b/include/conntrack.h index 37ccf6e..b518096 100644 --- a/include/conntrack.h +++ b/include/conntrack.h @@ -8,6 +8,9 @@ #include <netinet/in.h> +#include <linux/netfilter/nf_conntrack_common.h> +#include <libnetfilter_conntrack/libnetfilter_conntrack.h> + #define NUMBER_OF_CMD 19 #define NUMBER_OF_OPT 29 @@ -32,6 +35,10 @@ struct ctproto_handler { unsigned int command, struct nf_conntrack *ct); + int (*snprintf_options)(char *buf, + unsigned int len, + const struct nf_conntrack *ct); + void (*help)(void); struct option *opts; @@ -53,6 +60,37 @@ void exit_error(enum exittype status, const char *msg, ...); extern void register_proto(struct ctproto_handler *h); +enum ctproto_attr_value_type { + CTPROTO_ATTR_U8 = 1, + CTPROTO_ATTR_U16_N, + CTPROTO_ATTR_U16_H, + CTPROTO_ATTR_U32_N, + CTPROTO_ATTR_U32_H, + CTPROTO_ATTR_U64_H, + CTPROTO_ATTR_U32_BITMAP, + CTPROTO_ATTR_IPV4, + CTPROTO_ATTR_IPV6, +}; + +struct ctproto_attr { + const char *name; + enum nf_conntrack_attr type; + short value_type; + short val_mapping_count; + const char **val_mapping; +}; + +extern int snprintf_attr(char *buf, + unsigned int len, + const struct nf_conntrack *ct, + const struct ctproto_attr *attr); + +extern int snprintf_attrs(char *buf, + unsigned int len, + const struct nf_conntrack *ct, + const struct ctproto_attr *attrs); + + extern void register_tcp(void); extern void register_udp(void); extern void register_udplite(void); diff --git a/src/conntrack.c b/src/conntrack.c index 5834f2d..a11958b 100644 --- a/src/conntrack.c +++ b/src/conntrack.c @@ -53,6 +53,7 @@ #include <sys/socket.h> #include <sys/time.h> #include <time.h> +#include <inttypes.h> #ifdef HAVE_ARPA_INET_H #include <arpa/inet.h> #endif @@ -610,6 +611,222 @@ void register_proto(struct ctproto_handler *h) list_add(&h->head, &proto_list); } +#define BUFFER_SIZE(ret, size, len, offset) do {\ + size += ret; \ + if ((int)ret > (int)len) \ + ret = len; \ + offset += ret; \ + len -= ret; \ +} while(0) + +int snprintf_attr(char *buf, + unsigned int len, + const struct nf_conntrack *ct, + const struct ctproto_attr *attr + ) +{ + uint16_t u16; + uint32_t u32; + uint64_t u64_val = 0; + char ipstr[INET6_ADDRSTRLEN]; + + if (!nfct_attr_is_set(ct, attr->type)) + return 0; + + switch (attr->value_type) { + case CTPROTO_ATTR_U8: + u64_val = nfct_get_attr_u8(ct, attr->type); + break; + case CTPROTO_ATTR_U16_N: + u16 = nfct_get_attr_u16(ct, attr->type); + u64_val = ntohs(u16); + break; + case CTPROTO_ATTR_U16_H: + u64_val = nfct_get_attr_u16(ct, attr->type); + break; + case CTPROTO_ATTR_U32_N: + u32 = nfct_get_attr_u32(ct, attr->type); + u64_val = ntohl(u32); + break; + case CTPROTO_ATTR_U32_H: + u64_val = nfct_get_attr_u32(ct, attr->type); + break; + case CTPROTO_ATTR_U64_H: + u64_val = nfct_get_attr_u64(ct, attr->type); + break; + case CTPROTO_ATTR_IPV4: + inet_ntop(AF_INET, nfct_get_attr(ct, attr->type), ipstr, sizeof(ipstr)); + break; + case CTPROTO_ATTR_IPV6: + inet_ntop(AF_INET6, nfct_get_attr(ct, attr->type), ipstr, sizeof(ipstr)); + break; + case CTPROTO_ATTR_U32_BITMAP: + u64_val = nfct_get_attr_u32(ct, attr->type); + break; + default: + fprintf(stderr, "unsupported type %u\n", attr->value_type); + return 0; + } + + if (attr->value_type == CTPROTO_ATTR_IPV4 + || attr->value_type == CTPROTO_ATTR_IPV6) { + return snprintf(buf, len, "%s %s ", attr->name, ipstr); + } + + if (attr->val_mapping) { + if (attr->value_type == CTPROTO_ATTR_U32_BITMAP) { + unsigned int size = 0, offset = 0, ret, i, imax, found = 0; + + if (!u64_val) + return 0; + + imax = 31 < attr->val_mapping_count ? 31 : attr->val_mapping_count; + for (i = 0; i < imax; ++i) { + if (!(u64_val & (1 << i))) + continue; + if (!attr->val_mapping[i]) + continue; + + if (!found) { + ret = snprintf(buf + offset, len, "%s %s", attr->name, attr->val_mapping[i]); + found = 1; + } else + ret = snprintf(buf + offset, len, ",%s", attr->val_mapping[i]); + BUFFER_SIZE(ret, size, len, offset); + } + + if (found) { + ret = snprintf(buf + offset, len, " "); + BUFFER_SIZE(ret, size, len, offset); + } + return size; + } + + if (u64_val >= (uint64_t)attr->val_mapping_count) { + fprintf(stderr, "too big value for mapping %s %" PRIu64 "\n", + attr->name, u64_val); + return 0; + } + if (!attr->val_mapping[u64_val]) { + fprintf(stderr, "no mapping for %s %" PRIu64 "\n", + attr->name, u64_val); + return 0; + } + return snprintf(buf, len, "%s %s ", attr->name, attr->val_mapping[u64_val]); + } + + return snprintf(buf, len, "%s %" PRIu64 " ", attr->name, u64_val); +} + +int snprintf_attrs(char *buf, + unsigned int len, + const struct nf_conntrack *ct, + const struct ctproto_attr *attrs) +{ + int i; + unsigned int size = 0, offset = 0, ret; + + for (i = 0; attrs[i].name; ++i) { + ret = snprintf_attr(buf + offset, len, ct, &attrs[i]); + BUFFER_SIZE(ret, size, len, offset); + } + + return size; +} + +static struct ctproto_attr attrs_ipv4[] = { + {"-s", ATTR_ORIG_IPV4_SRC, CTPROTO_ATTR_IPV4, 0, 0}, + {"-d", ATTR_ORIG_IPV4_DST, CTPROTO_ATTR_IPV4, 0, 0}, + {"-g", ATTR_DNAT_IPV4, CTPROTO_ATTR_IPV4, 0, 0}, + {"-n", ATTR_SNAT_IPV4, CTPROTO_ATTR_IPV4, 0, 0}, + {"-r", ATTR_REPL_IPV4_SRC, CTPROTO_ATTR_IPV4, 0, 0}, + {"-q", ATTR_REPL_IPV4_DST, CTPROTO_ATTR_IPV4, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static struct ctproto_attr attrs_ipv6[] = { + {"-s", ATTR_ORIG_IPV6_SRC, CTPROTO_ATTR_IPV6, 0, 0}, + {"-d", ATTR_ORIG_IPV6_DST, CTPROTO_ATTR_IPV6, 0, 0}, + {"-g", ATTR_DNAT_IPV6, CTPROTO_ATTR_IPV6, 0, 0}, + {"-n", ATTR_SNAT_IPV6, CTPROTO_ATTR_IPV6, 0, 0}, + {"-r", ATTR_REPL_IPV6_SRC, CTPROTO_ATTR_IPV6, 0, 0}, + {"-q", ATTR_REPL_IPV6_DST, CTPROTO_ATTR_IPV6, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static const char *conntrack_status_map[] = { + [IPS_ASSURED_BIT] = "ASSURED", + [IPS_SEEN_REPLY_BIT] = "SEEN_REPLY", + [IPS_FIXED_TIMEOUT_BIT] = "FIXED_TIMEOUT", + [IPS_EXPECTED_BIT] = "EXPECTED" +}; + +static struct ctproto_attr attrs_generic[] = { + {"-t", ATTR_TIMEOUT, CTPROTO_ATTR_U32_H, 0, 0}, + {"-u", ATTR_STATUS, CTPROTO_ATTR_U32_BITMAP, + sizeof(conntrack_status_map)/sizeof(conntrack_status_map[0]), + conntrack_status_map}, + {"-c", ATTR_SECMARK, CTPROTO_ATTR_U32_H, 0, 0}, +/* {"-i", ATTR_ID, CTPROTO_ATTR_U32_H, 0, 0}, */ + {"-w", ATTR_ZONE, CTPROTO_ATTR_U16_H, 0, 0}, + {"--orig-zone", ATTR_ORIG_ZONE, CTPROTO_ATTR_U16_H, 0, 0}, + {"--reply-zone", ATTR_REPL_ZONE, CTPROTO_ATTR_U16_H, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static int nfct_snprintf_labels_opts(char *buf, + unsigned int len, + const struct nf_conntrack *ct, + struct nfct_labelmap *map) +{ + int ret = 0; + unsigned int size = 0, offset = 0; + uint8_t l3proto, l4proto; + static struct ctproto_attr *attrs_l3; + struct ctproto_handler *cur; + + ret = snprintf_attrs(buf + offset, len, ct, attrs_generic); + BUFFER_SIZE(ret, size, len, offset); + + l3proto = nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO); + if (!l3proto) + l3proto = nfct_get_attr_u8(ct, ATTR_REPL_L3PROTO); + switch (l3proto) { + case AF_INET: + attrs_l3 = attrs_ipv4; + break; + case AF_INET6: + attrs_l3 = attrs_ipv6; + break; + default: + fprintf(stderr, + "WARNING: unknown l3proto %d, skipping..\n", l3proto); + return 0; + } + + ret = snprintf_attrs(buf + offset, len, ct, attrs_l3); + BUFFER_SIZE(ret, size, len, offset); + + l4proto = nfct_get_attr_u8(ct, ATTR_L4PROTO); + + /* is it in the list of supported protocol? */ + list_for_each_entry(cur, &proto_list, head) { + if (cur->protonum == l4proto) { + ret = snprintf(buf + offset, len, "-p %s ", cur->name); + BUFFER_SIZE(ret, size, len, offset); + ret = cur->snprintf_options(buf + offset, len, ct); + BUFFER_SIZE(ret, size, len, offset); + break; + } + } + + /* skip trailing space, if any */ + for (;size && buf[size-1] == ' '; --size) + buf[size-1] = '\0'; + + return size; +} + extern struct ctproto_handler ct_proto_unknown; static struct ctproto_handler *findproto(char *name, int *pnum) @@ -865,6 +1082,7 @@ enum { _O_KTMS = (1 << 4), _O_CL = (1 << 5), _O_US = (1 << 6), + _O_OPTS = (1 << 7), }; enum { @@ -875,7 +1093,7 @@ enum { }; static struct parse_parameter { - const char *parameter[7]; + const char *parameter[8]; size_t size; unsigned int value[8]; } parse_array[PARSE_MAX] = { @@ -883,8 +1101,8 @@ static struct parse_parameter { { IPS_ASSURED, IPS_SEEN_REPLY, 0, IPS_FIXED_TIMEOUT, IPS_EXPECTED, IPS_OFFLOAD, IPS_HW_OFFLOAD} }, { {"ALL", "NEW", "UPDATES", "DESTROY"}, 4, { CT_EVENT_F_ALL, CT_EVENT_F_NEW, CT_EVENT_F_UPD, CT_EVENT_F_DEL } }, - { {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace" }, 7, - { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US }, + { {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace", "opts"}, 8, + { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US, _O_OPTS }, }, }; @@ -1467,6 +1685,11 @@ static int event_cb(const struct nlmsghdr *nlh, void *data) if (nfct_filter(obj, ct)) goto out; + if (output_mask & _O_OPTS) { + nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap); + goto done; + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -1491,7 +1714,7 @@ static int event_cb(const struct nlmsghdr *nlh, void *data) op_flags |= NFCT_OF_ID; nfct_snprintf_labels(buf, sizeof(buf), ct, type, op_type, op_flags, labelmap); - +done: if (output_mask & _O_US) { if (nlh->nlmsg_pid) userspace = true; @@ -1520,6 +1743,11 @@ static int dump_cb(enum nf_conntrack_msg_type type, if (nfct_filter(obj, ct)) return NFCT_CB_CONTINUE; + if (output_mask & _O_OPTS) { + nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap); + goto done; + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -1536,6 +1764,7 @@ static int dump_cb(enum nf_conntrack_msg_type type, op_flags |= NFCT_OF_ID; nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap); +done: printf("%s\n", buf); counter++; @@ -1562,6 +1791,11 @@ static int delete_cb(enum nf_conntrack_msg_type type, "Operation failed: %s", err2str(errno, CT_DELETE)); + if (output_mask & _O_OPTS) { + nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap); + goto done; + } + if (output_mask & _O_XML) op_type = NFCT_O_XML; if (output_mask & _O_EXT) @@ -1570,6 +1804,7 @@ static int delete_cb(enum nf_conntrack_msg_type type, op_flags |= NFCT_OF_ID; nfct_snprintf(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags); +done: printf("%s\n", buf); counter++; @@ -1585,6 +1820,11 @@ static int print_cb(enum nf_conntrack_msg_type type, unsigned int op_type = NFCT_O_DEFAULT; unsigned int op_flags = 0; + if (output_mask & _O_OPTS) { + nfct_snprintf_labels_opts(buf, sizeof(buf), ct, labelmap); + goto done; + } + if (output_mask & _O_XML) op_type = NFCT_O_XML; if (output_mask & _O_EXT) @@ -1593,6 +1833,7 @@ static int print_cb(enum nf_conntrack_msg_type type, op_flags |= NFCT_OF_ID; nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap); +done: printf("%s\n", buf); return NFCT_CB_CONTINUE; @@ -1757,6 +1998,12 @@ static int dump_exp_cb(enum nf_conntrack_msg_type type, unsigned int op_type = NFCT_O_DEFAULT; unsigned int op_flags = 0; + if (output_mask & _O_OPTS) { + /* not implemented at the moment */ + exit_error(PARAMETER_PROBLEM, + "opts output format is not supported for table of expectations"); + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -1788,6 +2035,12 @@ static int event_exp_cb(enum nf_conntrack_msg_type type, unsigned int op_type = NFCT_O_DEFAULT; unsigned int op_flags = 0; + if (output_mask & _O_OPTS) { + /* not implemented at the moment */ + exit_error(PARAMETER_PROBLEM, + "opts output format is not supported for table of expectations"); + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 6/8] conntrack: implement options output format 2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky @ 2020-10-22 12:36 ` Pablo Neira Ayuso 2020-10-22 12:37 ` Pablo Neira Ayuso 0 siblings, 1 reply; 20+ messages in thread From: Pablo Neira Ayuso @ 2020-10-22 12:36 UTC (permalink / raw) To: Mikhail Sennikovsky; +Cc: netfilter-devel Hi Mikhail, Thanks for your patchset. On Fri, Sep 25, 2020 at 02:49:17PM +0200, Mikhail Sennikovsky wrote: > As a counterpart to the "conntrack: accept parameters from stdin" > commit, this commit allows dumping conntrack entries in the format > used by the conntrack parameters. > This is useful for transfering a large set of ct entries between > hosts or between different ct zones in an efficient way. > > To enable the "options" output the "-o opts" parameter needs to be > passed to the "contnrack -L" tool invocation. I started slightly revisiting this 6/8 patch a bit (please find it enclosed to this email), I have rename -o opts to -o save, to get this aligned with iptables-save. I have also added a check for -o xml,save , to reject this combination. I have extended it to display -I, -U, -D in the conntrack events. I have removed several safety runtime checks, that can be done at registration time (make sure the option description is well-formed from there, otherwise rise an error message to spot buggy protocol extensions). This patch should also be extended to support for other existing output flags combinations. Or just bail out if they are specified. At this point I have concerns with NAT: I don't see how this can work as is. There is also a conntrack helpers that might trigger NAT sequence adjustments, this information would be lost. We would need to expose all these details through the -o save, see below. For some of this, there is no options from command line, because it made no sense to expose them. We have to discuss this before deciding where to go. See below for details. > To demonstrate the overall idea of the options output format works > in conjunction with the "stdin parameter"s mode, > the following command will copy all ct entries from one ct zone > to another. > > conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I - For zone updates in the same host, probably conntrack can be extended to support for: conntrack -U --zone 15 --set-zone 9915 If --set-zone is specified, then --zone is used a filter. Then, for "zone transfers" *between hosts*, a different way to address this is to extend conntrackd. The idea is: 1) Add new "transfer" mode which does _not_ subscribe to conntrack events, it needs to register a new struct ct_mode (currently there is "sync" and "stats" ct_modes). 2) Add a new message type to request a zone transfer, e.g. conntrackd --from 192.168.10.20 --zone 15 --set-zone 9915 This will make your local daemon send a request to the conntrackd instance running on host 192.168.10.20 to retrieve zone 1200. The remote conntrackd instance dumps the existing conntrack table from kernel and sends it to you. You can reuse the channel infrastructure to establish communications between conntrackd instances in the new "transfer mode". You can also reuse the sync protocol, see network.h, build.c and parse.c, which takes a conntrack object and it translates it to network message. Note that the struct internal_handler actually refers to the netlink handler for this new struct ct_mode that you would be registering. Let me know, thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 6/8] conntrack: implement options output format 2020-10-22 12:36 ` Pablo Neira Ayuso @ 2020-10-22 12:37 ` Pablo Neira Ayuso 2020-10-23 11:07 ` Mikhail Sennikovsky 0 siblings, 1 reply; 20+ messages in thread From: Pablo Neira Ayuso @ 2020-10-22 12:37 UTC (permalink / raw) To: Mikhail Sennikovsky; +Cc: netfilter-devel [-- Attachment #1: Type: text/plain, Size: 3489 bytes --] On Thu, Oct 22, 2020 at 02:36:47PM +0200, Pablo Neira Ayuso wrote: > Hi Mikhail, > > Thanks for your patchset. > > On Fri, Sep 25, 2020 at 02:49:17PM +0200, Mikhail Sennikovsky wrote: > > As a counterpart to the "conntrack: accept parameters from stdin" > > commit, this commit allows dumping conntrack entries in the format > > used by the conntrack parameters. > > This is useful for transfering a large set of ct entries between > > hosts or between different ct zones in an efficient way. > > > > To enable the "options" output the "-o opts" parameter needs to be > > passed to the "contnrack -L" tool invocation. > > I started slightly revisiting this 6/8 patch a bit (please find it > enclosed to this email), I have rename -o opts to -o save, to get this > aligned with iptables-save. Attaching the revisited patch 6/8 to this email. > I have also added a check for -o xml,save , to reject this > combination. > > I have extended it to display -I, -U, -D in the conntrack events. > > I have removed several safety runtime checks, that can be done at > registration time (make sure the option description is well-formed > from there, otherwise rise an error message to spot buggy protocol > extensions). > > This patch should also be extended to support for other existing > output flags combinations. Or just bail out if they are specified. > > At this point I have concerns with NAT: I don't see how this can work > as is. There is also a conntrack helpers that might trigger NAT > sequence adjustments, this information would be lost. > > We would need to expose all these details through the -o save, see > below. For some of this, there is no options from command line, > because it made no sense to expose them. > > We have to discuss this before deciding where to go. See below for > details. > > > To demonstrate the overall idea of the options output format works > > in conjunction with the "stdin parameter"s mode, > > the following command will copy all ct entries from one ct zone > > to another. > > > > conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I - > > For zone updates in the same host, probably conntrack can be extended > to support for: > > conntrack -U --zone 15 --set-zone 9915 > > If --set-zone is specified, then --zone is used a filter. > > Then, for "zone transfers" *between hosts*, a different way to address > this is to extend conntrackd. > > The idea is: > > 1) Add new "transfer" mode which does _not_ subscribe to > conntrack events, it needs to register a new struct ct_mode > (currently there is "sync" and "stats" ct_modes). > > 2) Add a new message type to request a zone transfer, e.g. > > conntrackd --from 192.168.10.20 --zone 15 --set-zone 9915 > > This will make your local daemon send a request to the conntrackd > instance running on host 192.168.10.20 to retrieve zone 1200. The > remote conntrackd instance dumps the existing conntrack table from > kernel and sends it to you. > > You can reuse the channel infrastructure to establish communications > between conntrackd instances in the new "transfer mode". You can > also reuse the sync protocol, see network.h, build.c and parse.c, > which takes a conntrack object and it translates it to network > message. > > Note that the struct internal_handler actually refers to the > netlink handler for this new struct ct_mode that you would be > registering. > > Let me know, thanks. [-- Attachment #2: x.patch --] [-- Type: text/x-diff, Size: 20886 bytes --] diff --git a/extensions/libct_proto_dccp.c b/extensions/libct_proto_dccp.c index f6258ad82a72..13cd7c53e292 100644 --- a/extensions/libct_proto_dccp.c +++ b/extensions/libct_proto_dccp.c @@ -198,6 +198,22 @@ static int parse_options(char c, return 1; } + +static const char *dccp_roles[__DCCP_CONNTRACK_ROLE_MAX] = { + [DCCP_CONNTRACK_ROLE_CLIENT] = "client", + [DCCP_CONNTRACK_ROLE_SERVER] = "server", +}; + +static struct ct_print_opts dccp_print_opts[] = { + { "--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, NULL }, + { "--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, NULL }, + { "--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, NULL }, + { "--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, NULL }, + { "--state", ATTR_DCCP_STATE, CT_ATTR_TYPE_U8, DCCP_CONNTRACK_MAX, dccp_states }, + { "--role", ATTR_DCCP_ROLE, CT_ATTR_TYPE_U8, __DCCP_CONNTRACK_ROLE_MAX, dccp_roles }, + {}, +}; + #define DCCP_VALID_FLAGS_MAX 2 static unsigned int dccp_valid_flags[DCCP_VALID_FLAGS_MAX] = { CT_DCCP_ORIG_SPORT | CT_DCCP_ORIG_DPORT, @@ -235,6 +251,7 @@ static struct ctproto_handler dccp = { .protonum = IPPROTO_DCCP, .parse_opts = parse_options, .final_check = final_check, + .print_opts = dccp_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_gre.c b/extensions/libct_proto_gre.c index 2dc63d14f498..406eb0691b1b 100644 --- a/extensions/libct_proto_gre.c +++ b/extensions/libct_proto_gre.c @@ -144,6 +144,14 @@ static int parse_options(char c, return 1; } +static struct ct_print_opts gre_print_opts[] = { + { "--srckey", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--dstkey", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--reply-key-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--reply-key-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 }, + {}, +}; + #define GRE_VALID_FLAGS_MAX 2 static unsigned int gre_valid_flags[GRE_VALID_FLAGS_MAX] = { CT_GRE_ORIG_SKEY | CT_GRE_ORIG_DKEY, @@ -181,6 +189,7 @@ static struct ctproto_handler gre = { .protonum = IPPROTO_GRE, .parse_opts = parse_options, .final_check = final_check, + .print_opts = gre_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_icmp.c b/extensions/libct_proto_icmp.c index 7fc82bdaddb6..a438d113f04a 100644 --- a/extensions/libct_proto_icmp.c +++ b/extensions/libct_proto_icmp.c @@ -102,6 +102,13 @@ static int parse(char c, return 1; } +static struct ct_print_opts icmp_print_opts[] = { + { "--icmp-type", ATTR_ICMP_TYPE, CT_ATTR_TYPE_U8, 0, 0 }, + { "--icmp-code", ATTR_ICMP_CODE, CT_ATTR_TYPE_U8, 0, 0 }, + { "--icmp-id", ATTR_ICMP_ID, CT_ATTR_TYPE_BE16, 0, 0 }, + {} +}; + static void final_check(unsigned int flags, unsigned int cmd, struct nf_conntrack *ct) @@ -117,6 +124,7 @@ static struct ctproto_handler icmp = { .protonum = IPPROTO_ICMP, .parse_opts = parse, .final_check = final_check, + .print_opts = icmp_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_icmpv6.c b/extensions/libct_proto_icmpv6.c index f872c23a1075..20b9d79aaf78 100644 --- a/extensions/libct_proto_icmpv6.c +++ b/extensions/libct_proto_icmpv6.c @@ -105,6 +105,13 @@ static int parse(char c, return 1; } +static struct ct_print_opts icmpv6_print_opts[] = { + {"--icmpv6-type", ATTR_ICMP_TYPE, CT_ATTR_TYPE_U8, 0, 0}, + {"--icmpv6-code", ATTR_ICMP_CODE, CT_ATTR_TYPE_U8, 0, 0}, + {"--icmpv6-id", ATTR_ICMP_ID, CT_ATTR_TYPE_BE16, 0, 0}, + {0, 0, 0, 0, 0}, +}; + static void final_check(unsigned int flags, unsigned int cmd, struct nf_conntrack *ct) @@ -119,6 +126,7 @@ static struct ctproto_handler icmpv6 = { .protonum = IPPROTO_ICMPV6, .parse_opts = parse, .final_check = final_check, + .print_opts = icmpv6_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_sctp.c b/extensions/libct_proto_sctp.c index 04828bf15815..e582e917d961 100644 --- a/extensions/libct_proto_sctp.c +++ b/extensions/libct_proto_sctp.c @@ -198,6 +198,17 @@ parse_options(char c, struct nf_conntrack *ct, return 1; } +static struct ct_print_opts sctp_print_opts[] = { + { "--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--state", ATTR_SCTP_STATE, CT_ATTR_TYPE_U8, SCTP_CONNTRACK_MAX, sctp_states }, + { "--orig-vtag", ATTR_SCTP_VTAG_ORIG, CT_ATTR_TYPE_BE32, 0, 0 }, + { "--reply-vtag", ATTR_SCTP_VTAG_REPL, CT_ATTR_TYPE_BE32, 0, 0 }, + {}, +}; + #define SCTP_VALID_FLAGS_MAX 2 static unsigned int dccp_valid_flags[SCTP_VALID_FLAGS_MAX] = { CT_SCTP_ORIG_SPORT | CT_SCTP_ORIG_DPORT, @@ -235,6 +246,7 @@ static struct ctproto_handler sctp = { .protonum = IPPROTO_SCTP, .parse_opts = parse_options, .final_check = final_check, + .print_opts = sctp_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_tcp.c b/extensions/libct_proto_tcp.c index 8a37a556327c..5e580e939fba 100644 --- a/extensions/libct_proto_tcp.c +++ b/extensions/libct_proto_tcp.c @@ -177,6 +177,15 @@ static int parse_options(char c, return 1; } +static struct ct_print_opts tcp_print_opts[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0}, + {"--state", ATTR_TCP_STATE, CT_ATTR_TYPE_U8, TCP_CONNTRACK_MAX, tcp_states}, + {0, 0, 0, 0, 0}, +}; + #define TCP_VALID_FLAGS_MAX 2 static unsigned int tcp_valid_flags[TCP_VALID_FLAGS_MAX] = { CT_TCP_ORIG_SPORT | CT_TCP_ORIG_DPORT, @@ -228,6 +237,7 @@ static struct ctproto_handler tcp = { .protonum = IPPROTO_TCP, .parse_opts = parse_options, .final_check = final_check, + .print_opts = tcp_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_udp.c b/extensions/libct_proto_udp.c index e30637c7bd1f..02e368b38073 100644 --- a/extensions/libct_proto_udp.c +++ b/extensions/libct_proto_udp.c @@ -144,6 +144,14 @@ static int parse_options(char c, return 1; } +static struct ct_print_opts udp_print_opts[] = { + {"--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0}, + {"--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0}, + {"--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0}, + {"--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0}, + {}, +}; + #define UDP_VALID_FLAGS_MAX 2 static unsigned int udp_valid_flags[UDP_VALID_FLAGS_MAX] = { CT_UDP_ORIG_SPORT | CT_UDP_ORIG_DPORT, @@ -181,6 +189,7 @@ static struct ctproto_handler udp = { .protonum = IPPROTO_UDP, .parse_opts = parse_options, .final_check = final_check, + .print_opts = udp_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/extensions/libct_proto_udplite.c b/extensions/libct_proto_udplite.c index f46cef0c30f5..80de70c56179 100644 --- a/extensions/libct_proto_udplite.c +++ b/extensions/libct_proto_udplite.c @@ -148,6 +148,14 @@ static int parse_options(char c, return 1; } +static struct ct_print_opts udplite_print_opts[] = { + { "--sport", ATTR_ORIG_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--dport", ATTR_ORIG_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--reply-port-src", ATTR_REPL_PORT_SRC, CT_ATTR_TYPE_BE16, 0, 0 }, + { "--reply-port-dst", ATTR_REPL_PORT_DST, CT_ATTR_TYPE_BE16, 0, 0 }, + {}, +}; + #define UDPLITE_VALID_FLAGS_MAX 2 static unsigned int udplite_valid_flags[UDPLITE_VALID_FLAGS_MAX] = { CT_UDPLITE_ORIG_SPORT | CT_UDPLITE_ORIG_DPORT, @@ -186,6 +194,7 @@ static struct ctproto_handler udplite = { .protonum = IPPROTO_UDPLITE, .parse_opts = parse_options, .final_check = final_check, + .print_opts = udplite_print_opts, .help = help, .opts = opts, .version = VERSION, diff --git a/include/conntrack.h b/include/conntrack.h index 37ccf6e9a87e..bc58d3be8f41 100644 --- a/include/conntrack.h +++ b/include/conntrack.h @@ -8,6 +8,9 @@ #include <netinet/in.h> +#include <linux/netfilter/nf_conntrack_common.h> +#include <libnetfilter_conntrack/libnetfilter_conntrack.h> + #define NUMBER_OF_CMD 19 #define NUMBER_OF_OPT 29 @@ -32,6 +35,8 @@ struct ctproto_handler { unsigned int command, struct nf_conntrack *ct); + const struct ct_print_opts *print_opts; + void (*help)(void); struct option *opts; @@ -53,6 +58,31 @@ void exit_error(enum exittype status, const char *msg, ...); extern void register_proto(struct ctproto_handler *h); +enum ct_attr_type { + CT_ATTR_TYPE_NONE = 0, + CT_ATTR_TYPE_U8, + CT_ATTR_TYPE_BE16, + CT_ATTR_TYPE_U16, + CT_ATTR_TYPE_BE32, + CT_ATTR_TYPE_U32, + CT_ATTR_TYPE_U64, + CT_ATTR_TYPE_U32_BITMAP, + CT_ATTR_TYPE_IPV4, + CT_ATTR_TYPE_IPV6, +}; + +struct ct_print_opts { + const char *name; + enum nf_conntrack_attr type; + short value_type; + short val_mapping_count; + const char **val_mapping; +}; + +extern int ct_snprintf_opts(char *buf, unsigned int len, + const struct nf_conntrack *ct, + const struct ct_print_opts *attrs); + extern void register_tcp(void); extern void register_udp(void); extern void register_udplite(void); diff --git a/src/conntrack.c b/src/conntrack.c index a26fa60bbbc9..4e1cb9fdc60c 100644 --- a/src/conntrack.c +++ b/src/conntrack.c @@ -53,6 +53,7 @@ #include <sys/socket.h> #include <sys/time.h> #include <time.h> +#include <inttypes.h> #ifdef HAVE_ARPA_INET_H #include <arpa/inet.h> #endif @@ -607,6 +608,240 @@ void register_proto(struct ctproto_handler *h) list_add(&h->head, &proto_list); } +#define BUFFER_SIZE(ret, size, len, offset) do {\ + size += ret; \ + if ((int)ret > (int)len) \ + ret = len; \ + offset += ret; \ + len -= ret; \ +} while(0) + +static int ct_snprintf_u32_bitmap(char *buf, size_t size, + const struct nf_conntrack *ct, + const struct ct_print_opts *attr) +{ + unsigned int offset = 0, ret, len = size; + bool found = false; + uint32_t val; + int i; + + val = nfct_get_attr_u32(ct, attr->type); + + for (i = 0; i < attr->val_mapping_count; i++) { + if (!(val & (1 << i))) + continue; + if (!attr->val_mapping[i]) + continue; + + ret = snprintf(buf + offset, len, "%s,", attr->val_mapping[i]); + BUFFER_SIZE(ret, size, len, offset); + found = true; + } + + if (found) { + offset--; + ret = snprintf(buf + offset, len, " "); + BUFFER_SIZE(ret, size, len, offset); + } + + return offset; +} + +static int ct_snprintf_attr(char *buf, size_t size, + const struct nf_conntrack *ct, + const struct ct_print_opts *attr) +{ + char ipstr[INET6_ADDRSTRLEN] = {}; + unsigned int offset = 0; + int type = attr->type; + int len = size; + int ret; + + ret = snprintf(buf, len, "%s ", attr->name); + BUFFER_SIZE(ret, size, len, offset); + + switch (attr->value_type) { + case CT_ATTR_TYPE_U8: + if (attr->val_mapping) + ret = snprintf(buf + offset, len, "%s ", + attr->val_mapping[nfct_get_attr_u8(ct, type)]); + else + ret = snprintf(buf + offset, len, "%u ", + nfct_get_attr_u8(ct, type)); + break; + case CT_ATTR_TYPE_BE16: + ret = snprintf(buf + offset, len, "%u ", + ntohs(nfct_get_attr_u16(ct, type))); + break; + case CT_ATTR_TYPE_U16: + ret = snprintf(buf + offset, len, "%u ", + nfct_get_attr_u16(ct, type)); + break; + case CT_ATTR_TYPE_BE32: + ret = snprintf(buf + offset, len, "%u ", + ntohl(nfct_get_attr_u32(ct, type))); + break; + case CT_ATTR_TYPE_U32: + ret = snprintf(buf + offset, len, "%u ", + nfct_get_attr_u32(ct, type)); + break; + case CT_ATTR_TYPE_U64: + ret = snprintf(buf + offset, len, "%lu ", + nfct_get_attr_u64(ct, type)); + break; + case CT_ATTR_TYPE_IPV4: + inet_ntop(AF_INET, nfct_get_attr(ct, type), + ipstr, sizeof(ipstr)); + ret = snprintf(buf + offset, len, "%s ", ipstr); + break; + case CT_ATTR_TYPE_IPV6: + inet_ntop(AF_INET6, nfct_get_attr(ct, type), + ipstr, sizeof(ipstr)); + ret = snprintf(buf + offset, len, "%s ", ipstr); + break; + case CT_ATTR_TYPE_U32_BITMAP: + ret = ct_snprintf_u32_bitmap(buf + offset, len, ct, attr); + break; + default: + /* Unsupported datatype, should not ever happen */ + ret = 0; + break; + } + BUFFER_SIZE(ret, size, len, offset); + + return offset; +} + +int ct_snprintf_opts(char *buf, unsigned int len, const struct nf_conntrack *ct, + const struct ct_print_opts *attrs) +{ + unsigned int size = 0, offset = 0, ret; + int i; + + for (i = 0; attrs[i].name; ++i) { + if (!nfct_attr_is_set(ct, attrs[i].type)) + continue; + + ret = ct_snprintf_attr(buf + offset, len, ct, &attrs[i]); + BUFFER_SIZE(ret, size, len, offset); + } + + return offset; +} + +static struct ct_print_opts attrs_ipv4[] = { + {"-s", ATTR_ORIG_IPV4_SRC, CT_ATTR_TYPE_IPV4, 0, 0}, + {"-d", ATTR_ORIG_IPV4_DST, CT_ATTR_TYPE_IPV4, 0, 0}, + {"-g", ATTR_DNAT_IPV4, CT_ATTR_TYPE_IPV4, 0, 0}, + {"-n", ATTR_SNAT_IPV4, CT_ATTR_TYPE_IPV4, 0, 0}, + {"-r", ATTR_REPL_IPV4_SRC, CT_ATTR_TYPE_IPV4, 0, 0}, + {"-q", ATTR_REPL_IPV4_DST, CT_ATTR_TYPE_IPV4, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static struct ct_print_opts attrs_ipv6[] = { + {"-s", ATTR_ORIG_IPV6_SRC, CT_ATTR_TYPE_IPV6, 0, 0}, + {"-d", ATTR_ORIG_IPV6_DST, CT_ATTR_TYPE_IPV6, 0, 0}, + {"-g", ATTR_DNAT_IPV6, CT_ATTR_TYPE_IPV6, 0, 0}, + {"-n", ATTR_SNAT_IPV6, CT_ATTR_TYPE_IPV6, 0, 0}, + {"-r", ATTR_REPL_IPV6_SRC, CT_ATTR_TYPE_IPV6, 0, 0}, + {"-q", ATTR_REPL_IPV6_DST, CT_ATTR_TYPE_IPV6, 0, 0}, + {0, 0, 0, 0, 0}, +}; + +static const char *conntrack_status_map[] = { + [IPS_ASSURED_BIT] = "ASSURED", + [IPS_SEEN_REPLY_BIT] = "SEEN_REPLY", + [IPS_FIXED_TIMEOUT_BIT] = "FIXED_TIMEOUT", + [IPS_EXPECTED_BIT] = "EXPECTED" +}; + +static struct ct_print_opts attrs_generic[] = { + {"-t", ATTR_TIMEOUT, CT_ATTR_TYPE_U32, 0, 0}, + {"-u", ATTR_STATUS, CT_ATTR_TYPE_U32_BITMAP, + sizeof(conntrack_status_map)/sizeof(conntrack_status_map[0]), + conntrack_status_map}, + {"-c", ATTR_SECMARK, CT_ATTR_TYPE_U32, 0, 0}, +/* {"-i", ATTR_ID, CT_ATTR_TYPE_U32, 0, 0}, */ + {"-w", ATTR_ZONE, CT_ATTR_TYPE_U16, 0, 0}, + {"--orig-zone", ATTR_ORIG_ZONE, CT_ATTR_TYPE_U16, 0, 0}, + {"--reply-zone", ATTR_REPL_ZONE, CT_ATTR_TYPE_U16, 0, 0}, + {}, +}; + +static int ct_save_snprintf(char *buf, size_t len, + const struct nf_conntrack *ct, + struct nfct_labelmap *map, + enum nf_conntrack_msg_type type) +{ + static struct ct_print_opts *attrs_l3; + unsigned int size = 0, offset = 0; + struct ctproto_handler *cur; + uint8_t l3proto, l4proto; + int ret; + + switch (type) { + case NFCT_T_NEW: + ret = snprintf(buf + offset, len, "-I "); + BUFFER_SIZE(ret, size, len, offset); + break; + case NFCT_T_UPDATE: + ret = snprintf(buf + offset, len, "-U "); + BUFFER_SIZE(ret, size, len, offset); + break; + case NFCT_T_DESTROY: + ret = snprintf(buf + offset, len, "-D "); + BUFFER_SIZE(ret, size, len, offset); + break; + default: + ret = 0; + break; + } + + ret = ct_snprintf_opts(buf + offset, len, ct, attrs_generic); + BUFFER_SIZE(ret, size, len, offset); + + l3proto = nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO); + if (!l3proto) + l3proto = nfct_get_attr_u8(ct, ATTR_REPL_L3PROTO); + switch (l3proto) { + case AF_INET: + attrs_l3 = attrs_ipv4; + break; + case AF_INET6: + attrs_l3 = attrs_ipv6; + break; + default: + fprintf(stderr, + "WARNING: unknown l3proto %d, skipping..\n", l3proto); + return 0; + } + + ret = ct_snprintf_opts(buf + offset, len, ct, attrs_l3); + BUFFER_SIZE(ret, size, len, offset); + + l4proto = nfct_get_attr_u8(ct, ATTR_L4PROTO); + + /* is it in the list of supported protocol? */ + list_for_each_entry(cur, &proto_list, head) { + if (cur->protonum != l4proto) + continue; + + ret = snprintf(buf + offset, len, "-p %s ", cur->name); + BUFFER_SIZE(ret, size, len, offset); + + ret = ct_snprintf_opts(buf + offset, len, ct, cur->print_opts); + BUFFER_SIZE(ret, size, len, offset); + break; + } + + /* skip trailing space, if any */ + for (;size && buf[size-1] == ' '; --size) + buf[size-1] = '\0'; + + return size; +} + extern struct ctproto_handler ct_proto_unknown; static struct ctproto_handler *findproto(char *name, int *pnum) @@ -856,6 +1091,7 @@ enum { _O_KTMS = (1 << 4), _O_CL = (1 << 5), _O_US = (1 << 6), + _O_SAVE = (1 << 7), }; enum { @@ -866,7 +1102,7 @@ enum { }; static struct parse_parameter { - const char *parameter[7]; + const char *parameter[8]; size_t size; unsigned int value[8]; } parse_array[PARSE_MAX] = { @@ -874,8 +1110,8 @@ static struct parse_parameter { { IPS_ASSURED, IPS_SEEN_REPLY, 0, IPS_FIXED_TIMEOUT, IPS_EXPECTED, IPS_OFFLOAD, IPS_HW_OFFLOAD} }, { {"ALL", "NEW", "UPDATES", "DESTROY"}, 4, { CT_EVENT_F_ALL, CT_EVENT_F_NEW, CT_EVENT_F_UPD, CT_EVENT_F_DEL } }, - { {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace" }, 7, - { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US }, + { {"xml", "extended", "timestamp", "id", "ktimestamp", "labels", "userspace", "save"}, 8, + { _O_XML, _O_EXT, _O_TMS, _O_ID, _O_KTMS, _O_CL, _O_US, _O_SAVE }, }, }; @@ -1458,6 +1694,11 @@ static int event_cb(const struct nlmsghdr *nlh, void *data) if (nfct_filter(obj, ct)) goto out; + if (output_mask & _O_SAVE) { + ct_save_snprintf(buf, sizeof(buf), ct, labelmap, type); + goto done; + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -1482,7 +1723,7 @@ static int event_cb(const struct nlmsghdr *nlh, void *data) op_flags |= NFCT_OF_ID; nfct_snprintf_labels(buf, sizeof(buf), ct, type, op_type, op_flags, labelmap); - +done: if (output_mask & _O_US) { if (nlh->nlmsg_pid) userspace = true; @@ -1511,6 +1752,11 @@ static int dump_cb(enum nf_conntrack_msg_type type, if (nfct_filter(obj, ct)) return NFCT_CB_CONTINUE; + if (output_mask & _O_SAVE) { + ct_save_snprintf(buf, sizeof(buf), ct, labelmap, NFCT_T_NEW); + goto done; + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -1527,6 +1773,7 @@ static int dump_cb(enum nf_conntrack_msg_type type, op_flags |= NFCT_OF_ID; nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap); +done: printf("%s\n", buf); counter++; @@ -1553,6 +1800,11 @@ static int delete_cb(enum nf_conntrack_msg_type type, "Operation failed: %s", err2str(errno, CT_DELETE)); + if (output_mask & _O_SAVE) { + ct_save_snprintf(buf, sizeof(buf), ct, labelmap, NFCT_T_DESTROY); + goto done; + } + if (output_mask & _O_XML) op_type = NFCT_O_XML; if (output_mask & _O_EXT) @@ -1561,6 +1813,7 @@ static int delete_cb(enum nf_conntrack_msg_type type, op_flags |= NFCT_OF_ID; nfct_snprintf(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags); +done: printf("%s\n", buf); counter++; @@ -1576,6 +1829,11 @@ static int print_cb(enum nf_conntrack_msg_type type, unsigned int op_type = NFCT_O_DEFAULT; unsigned int op_flags = 0; + if (output_mask & _O_SAVE) { + ct_save_snprintf(buf, sizeof(buf), ct, labelmap, NFCT_T_NEW); + goto done; + } + if (output_mask & _O_XML) op_type = NFCT_O_XML; if (output_mask & _O_EXT) @@ -1584,6 +1842,7 @@ static int print_cb(enum nf_conntrack_msg_type type, op_flags |= NFCT_OF_ID; nfct_snprintf_labels(buf, sizeof(buf), ct, NFCT_T_UNKNOWN, op_type, op_flags, labelmap); +done: printf("%s\n", buf); return NFCT_CB_CONTINUE; @@ -1748,6 +2007,11 @@ static int dump_exp_cb(enum nf_conntrack_msg_type type, unsigned int op_type = NFCT_O_DEFAULT; unsigned int op_flags = 0; + if (output_mask & _O_SAVE) { + exit_error(PARAMETER_PROBLEM, + "save output format is not supported for table of expectations"); + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -1779,6 +2043,11 @@ static int event_exp_cb(enum nf_conntrack_msg_type type, unsigned int op_type = NFCT_O_DEFAULT; unsigned int op_flags = 0; + if (output_mask & _O_SAVE) { + exit_error(PARAMETER_PROBLEM, + "save output format is not supported for table of expectations"); + } + if (output_mask & _O_XML) { op_type = NFCT_O_XML; if (dump_xml_header_done) { @@ -2445,6 +2714,10 @@ int main(int argc, char *argv[]) parse_parameter(optarg, &output_mask, PARSE_OUTPUT); if (output_mask & _O_CL) labelmap_init(); + if ((output_mask & _O_SAVE) && + (output_mask & _O_XML)) + exit_error(OTHER_PROBLEM, + "cannot combine XML and save output types"); break; case 'z': options |= CT_OPT_ZERO; ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 6/8] conntrack: implement options output format 2020-10-22 12:37 ` Pablo Neira Ayuso @ 2020-10-23 11:07 ` Mikhail Sennikovsky 0 siblings, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-10-23 11:07 UTC (permalink / raw) To: Pablo Neira Ayuso; +Cc: netfilter-devel Thanks Pablo, Do you want me to send the updated patchset with your adjustments incorporated? Mikhail On Thu, 22 Oct 2020 at 14:37, Pablo Neira Ayuso <pablo@netfilter.org> wrote: > > On Thu, Oct 22, 2020 at 02:36:47PM +0200, Pablo Neira Ayuso wrote: > > Hi Mikhail, > > > > Thanks for your patchset. > > > > On Fri, Sep 25, 2020 at 02:49:17PM +0200, Mikhail Sennikovsky wrote: > > > As a counterpart to the "conntrack: accept parameters from stdin" > > > commit, this commit allows dumping conntrack entries in the format > > > used by the conntrack parameters. > > > This is useful for transfering a large set of ct entries between > > > hosts or between different ct zones in an efficient way. > > > > > > To enable the "options" output the "-o opts" parameter needs to be > > > passed to the "contnrack -L" tool invocation. > > > > I started slightly revisiting this 6/8 patch a bit (please find it > > enclosed to this email), I have rename -o opts to -o save, to get this > > aligned with iptables-save. > > Attaching the revisited patch 6/8 to this email. > > > I have also added a check for -o xml,save , to reject this > > combination. > > > > I have extended it to display -I, -U, -D in the conntrack events. > > > > I have removed several safety runtime checks, that can be done at > > registration time (make sure the option description is well-formed > > from there, otherwise rise an error message to spot buggy protocol > > extensions). > > > > This patch should also be extended to support for other existing > > output flags combinations. Or just bail out if they are specified. > > > > At this point I have concerns with NAT: I don't see how this can work > > as is. There is also a conntrack helpers that might trigger NAT > > sequence adjustments, this information would be lost. > > > > We would need to expose all these details through the -o save, see > > below. For some of this, there is no options from command line, > > because it made no sense to expose them. > > > > We have to discuss this before deciding where to go. See below for > > details. > > > > > To demonstrate the overall idea of the options output format works > > > in conjunction with the "stdin parameter"s mode, > > > the following command will copy all ct entries from one ct zone > > > to another. > > > > > > conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I - > > > > For zone updates in the same host, probably conntrack can be extended > > to support for: > > > > conntrack -U --zone 15 --set-zone 9915 > > > > If --set-zone is specified, then --zone is used a filter. > > > > Then, for "zone transfers" *between hosts*, a different way to address > > this is to extend conntrackd. > > > > The idea is: > > > > 1) Add new "transfer" mode which does _not_ subscribe to > > conntrack events, it needs to register a new struct ct_mode > > (currently there is "sync" and "stats" ct_modes). > > > > 2) Add a new message type to request a zone transfer, e.g. > > > > conntrackd --from 192.168.10.20 --zone 15 --set-zone 9915 > > > > This will make your local daemon send a request to the conntrackd > > instance running on host 192.168.10.20 to retrieve zone 1200. The > > remote conntrackd instance dumps the existing conntrack table from > > kernel and sends it to you. > > > > You can reuse the channel infrastructure to establish communications > > between conntrackd instances in the new "transfer mode". You can > > also reuse the sync protocol, see network.h, build.c and parse.c, > > which takes a conntrack object and it translates it to network > > message. > > > > Note that the struct internal_handler actually refers to the > > netlink handler for this new struct ct_mode that you would be > > registering. > > > > Let me know, thanks. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 7/8] conntrack.8: man update for opts format support 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky ` (5 preceding siblings ...) 2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 8/8] tests: dumping ct entries in opts format Mikhail Sennikovsky 2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal 8 siblings, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- conntrack.8 | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-) diff --git a/conntrack.8 b/conntrack.8 index 3db4849..2d354a7 100644 --- a/conntrack.8 +++ b/conntrack.8 @@ -109,7 +109,7 @@ Show the in-kernel connection tracking system statistics. Atomically zero counters after reading them. This option is only valid in combination with the "\-L, \-\-dump" command options. .TP -.BI "-o, --output [extended,xml,timestamp,id,ktimestamp,labels,userspace] " +.BI "-o, --output [extended,xml,opts,timestamp,id,ktimestamp,labels,userspace] " Display output in a certain format. With the extended output option, this tool displays the layer 3 information. With ktimestamp, it displays the in-kernel timestamp available since 2.6.38 (you can enable it via the \fBsysctl(8)\fP @@ -399,6 +399,9 @@ Delete all flow whose source address is 1.2.3.4 .TP .B conntrack \-U \-s 1.2.3.4 \-m 1 Set connmark to 1 of all the flows whose source address is 1.2.3.4 +.TP +.B conntrack -L -w 15 -o opts | sed 's/-w 15/-w 9915/g' | conntrack -I - +Copy all ct entries from one ct zone 15 to ct zone 9915 .SH BUGS Please, report them to netfilter-devel@vger.kernel.org or file a bug in -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 8/8] tests: dumping ct entries in opts format 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky ` (6 preceding siblings ...) 2020-09-25 12:49 ` [PATCH 7/8] conntrack.8: man update for opts format support Mikhail Sennikovsky @ 2020-09-25 12:49 ` Mikhail Sennikovsky 2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal 8 siblings, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-25 12:49 UTC (permalink / raw) To: netfilter-devel; +Cc: Mikhail Sennikovsky Add tests to cover dumping ct entries in "opts" format by conntrack Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- tests/conntrack/test-conntrack.c | 14 ++++++ tests/conntrack/testsuite/09dumpopt | 77 +++++++++++++++++++++++++++++ 2 files changed, 91 insertions(+) create mode 100644 tests/conntrack/testsuite/09dumpopt diff --git a/tests/conntrack/test-conntrack.c b/tests/conntrack/test-conntrack.c index 90bdc5b..372e025 100644 --- a/tests/conntrack/test-conntrack.c +++ b/tests/conntrack/test-conntrack.c @@ -81,6 +81,11 @@ int main() res++; for (; *res == ' ' || *res == '\t'; res++); cmd = res[0]; + cmd_opt = &res[1]; + for (; *cmd_opt == ' ' || *cmd_opt == '\t'; cmd_opt++); + res = strchr(cmd_opt, '\n'); + if (res) + *res = '\0'; if (cur_cmd && cmd != cur_cmd) { /* complete current multi-line command */ @@ -111,6 +116,15 @@ int main() cmd_strappend(CT_PROG); cmd_strappend(" "); cmd_strappend(buf); + if (cmd == '|') { + cmd_strappend(" | "); + if (cmd_opt[0]) { + cmd_strappend("sed \""); + cmd_strappend(cmd_opt); + cmd_strappend("\" | "); + } + continue; + } cmd_reset(); break; } diff --git a/tests/conntrack/testsuite/09dumpopt b/tests/conntrack/testsuite/09dumpopt new file mode 100644 index 0000000..0e3c649 --- /dev/null +++ b/tests/conntrack/testsuite/09dumpopt @@ -0,0 +1,77 @@ +# test opts output for -L +# create +# create a conntrack +-w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +# create from reply +-w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; +# create a v6 conntrack +-w 10 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +# creae icmp ping request entry +-w 10 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; +-I - ; OK +# copy ipv4 bits to zone 11 +-L -w 10 -o opt -f ipv4 ; |s/-w 10/-w 11/g +-I - ; OK +# copy ipv6 bits to zone 11 +-L -w 10 -o opt -f ipv6 ; |s/-w 10/-w 11/g +-I - ; OK +# create again in zone 11 +-I -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -w 11 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD +# delete new entries +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; OK +# delete reverse +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; OK +# delete v6 conntrack +-D -w 11-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK +# delete icmp ping request entry +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# delete old entries +-D -w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; OK +# delete reverse +-D -w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; OK +# delete v6 conntrack +-D -w 10-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK +# delete icmp ping request entry +-D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# +# now test opts output for -D +# create entries again +# create a conntrack +-w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +# create from reply +-w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; +# create a v6 conntrack +-w 10 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; +# creae icmp ping request entry +-w 10 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; +-I - ; OK +# move ipv4 bits to zone 11 +-D -w 10 -o opt -f ipv4 ; |s/-w 10/-w 11/g +-I - ; OK +# move ipv6 bits to zone 11 +-D -w 10 -o opt -f ipv6 ; |s/-w 10/-w 11/g +-I - ; OK +# create again in zone 11 +-I -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; BAD +-I -w 11 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD +# delete new entries +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; OK +# delete reverse +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; OK +# delete v6 conntrack +-D -w 11-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; OK +# delete icmp ping request entry +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# delete old entries +-D -w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 ; BAD +# delete reverse +-D -w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 ; BAD +# delete v6 conntrack +-D -w 10-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; BAD +# delete icmp ping request entry +-D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD \ No newline at end of file -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* Re: [PATCH 0/8] Fast bulk transfers of large sets of ct entries 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky ` (7 preceding siblings ...) 2020-09-25 12:49 ` [PATCH 8/8] tests: dumping ct entries in opts format Mikhail Sennikovsky @ 2020-09-25 13:28 ` Florian Westphal 2020-09-26 18:19 ` Pablo Neira Ayuso 8 siblings, 1 reply; 20+ messages in thread From: Florian Westphal @ 2020-09-25 13:28 UTC (permalink / raw) To: Mikhail Sennikovsky; +Cc: netfilter-devel Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> wrote: > In addition to this I have a question about the behavioural change > of the "conntrack -L" done after conntrack v1.4.5. > With the conntrack v1.4.5 used on Debian Buster the "conntrack -L" > dumps both ipv4 and ipv6 ct entries, while with the current master, > presumably starting with the commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e > "conntrack -L" only dumps ipv4 entries. > > So is this really the desired behavior? I'd like conntrack to dump both by default. ^ permalink raw reply [flat|nested] 20+ messages in thread
* Re: [PATCH 0/8] Fast bulk transfers of large sets of ct entries 2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal @ 2020-09-26 18:19 ` Pablo Neira Ayuso 2020-09-29 11:20 ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky 0 siblings, 1 reply; 20+ messages in thread From: Pablo Neira Ayuso @ 2020-09-26 18:19 UTC (permalink / raw) To: Florian Westphal; +Cc: Mikhail Sennikovsky, netfilter-devel On Fri, Sep 25, 2020 at 03:28:34PM +0200, Florian Westphal wrote: > Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> wrote: > > In addition to this I have a question about the behavioural change > > of the "conntrack -L" done after conntrack v1.4.5. > > With the conntrack v1.4.5 used on Debian Buster the "conntrack -L" > > dumps both ipv4 and ipv6 ct entries, while with the current master, > > presumably starting with the commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e > > "conntrack -L" only dumps ipv4 entries. > > > > So is this really the desired behavior? > > I'd like conntrack to dump both by default. Dumping both with -L is fine, so -f behaves as a way to filter. ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given 2020-09-26 18:19 ` Pablo Neira Ayuso @ 2020-09-29 11:20 ` Mikhail Sennikovsky 2020-09-29 11:20 ` [PATCH 1/2] " Mikhail Sennikovsky 2020-09-29 11:20 ` [PATCH 2/2] tests: conntrack -L/-D ip family filtering Mikhail Sennikovsky 0 siblings, 2 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-29 11:20 UTC (permalink / raw) To: netfilter-devel, pablo, fw; +Cc: Mikhail Sennikovsky Hi all, As we discussed in the "Fast bulk transfers of large sets of ct entries" mailing thread, conntrack -L (and presumably conntracks -D as well) should dump/delete both IPv4 and IPv6 entries if no family is specified. As a follow-up for that here is a patch which supposed to restore this behavior and a patch with the test-cases covering both the invocation with family is specified and not. The patches are created on top of my previous set of patches, mainly for the reason that the "opts" output format and stdio input introduced there among other thing allow testing the -L functionality easily. I would really appreciate if someone could have a look into those patches btw and give some feedback on whether they make sense or not. Thanks & Regards, Mikhail Mikhail Sennikovsky (2): conntrack: -L/-D both ipv4/6 if no family is given tests: conntrack -L/-D ip family filtering src/conntrack.c | 35 +++++++++++--- tests/conntrack/testsuite/09dumpopt | 72 ++++++++++++++++++++++++++++- 2 files changed, 99 insertions(+), 8 deletions(-) -- 2.25.1 ^ permalink raw reply [flat|nested] 20+ messages in thread
* [PATCH 1/2] conntrack: -L/-D both ipv4/6 if no family is given 2020-09-29 11:20 ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky @ 2020-09-29 11:20 ` Mikhail Sennikovsky 2020-09-29 11:20 ` [PATCH 2/2] tests: conntrack -L/-D ip family filtering Mikhail Sennikovsky 1 sibling, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-29 11:20 UTC (permalink / raw) To: netfilter-devel, pablo, fw; +Cc: Mikhail Sennikovsky Starting commit 2bcbae4c14b253176d7570e6f6acc56e521ceb5e conntrack -L as well as conntrack -D list/delete IPv4 entries only if no family is specified. Restore original behavior to list/delete both IPv4 and IPv6 entries if no family is specified. Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- src/conntrack.c | 35 ++++++++++++++++++++++++++++------- 1 file changed, 28 insertions(+), 7 deletions(-) diff --git a/src/conntrack.c b/src/conntrack.c index a11958b..3f5eb37 100644 --- a/src/conntrack.c +++ b/src/conntrack.c @@ -1578,6 +1578,11 @@ nfct_filter_network_direction(const struct nf_conntrack *ct, enum ct_direction d enum nf_conntrack_attr attr; struct ct_network *net = &dir2network[dir]; + if (family == AF_UNSPEC) { + exit_error(OTHER_PROBLEM, + "Internal Error: unspecified Family!"); + } + if (nfct_get_attr_u8(ct, ATTR_ORIG_L3PROTO) != family) return 1; @@ -2433,6 +2438,10 @@ nfct_filter_init(const int family) { filter_family = family; if (options & CT_OPT_MASK_SRC) { + if (family == AF_UNSPEC) { + exit_error(OTHER_PROBLEM, + "Internal Error: unspecified Family!"); + } if (!(options & CT_OPT_ORIG_SRC)) exit_error(PARAMETER_PROBLEM, "Can't use --mask-src without --src"); @@ -2440,6 +2449,10 @@ nfct_filter_init(const int family) } if (options & CT_OPT_MASK_DST) { + if (family == AF_UNSPEC) { + exit_error(OTHER_PROBLEM, + "Internal Error: unspecified Family!"); + } if (!(options & CT_OPT_ORIG_DST)) exit_error(PARAMETER_PROBLEM, "Can't use --mask-dst without --dst"); @@ -2894,7 +2907,9 @@ parse_opts: } /* default family */ - if (family == AF_UNSPEC) + if (family == AF_UNSPEC + && command != CT_LIST + && command != CT_DELETE) family = AF_INET; /* we cannot check this combination with generic_opt_check. */ @@ -2993,9 +3008,12 @@ parse_opts: NFCT_FILTER_DUMP_MARK, &tmpl.filter_mark_kernel); } - nfct_filter_dump_set_attr_u8(filter_dump, - NFCT_FILTER_DUMP_L3NUM, - family); + + if (family != AF_UNSPEC) { + nfct_filter_dump_set_attr_u8(filter_dump, + NFCT_FILTER_DUMP_L3NUM, + family); + } if (options & CT_OPT_ZERO) res = nfct_query(cth, NFCT_Q_DUMP_FILTER_RESET, @@ -3104,9 +3122,12 @@ parse_opts: NFCT_FILTER_DUMP_MARK, &tmpl.filter_mark_kernel); } - nfct_filter_dump_set_attr_u8(filter_dump, - NFCT_FILTER_DUMP_L3NUM, - family); + + if (family != AF_UNSPEC) { + nfct_filter_dump_set_attr_u8(filter_dump, + NFCT_FILTER_DUMP_L3NUM, + family); + } res = nfct_query(cth, NFCT_Q_DUMP_FILTER, filter_dump); -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
* [PATCH 2/2] tests: conntrack -L/-D ip family filtering 2020-09-29 11:20 ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky 2020-09-29 11:20 ` [PATCH 1/2] " Mikhail Sennikovsky @ 2020-09-29 11:20 ` Mikhail Sennikovsky 1 sibling, 0 replies; 20+ messages in thread From: Mikhail Sennikovsky @ 2020-09-29 11:20 UTC (permalink / raw) To: netfilter-devel, pablo, fw; +Cc: Mikhail Sennikovsky Tests to cover conntrack -L and conntrack -D with and w/o family (-f) specfied. conntrack -L and contnrack -D shold list/delete both IPv4 and IPv6 entries if no family is specified, and should ony display the corresponding entries if the family is given. Signed-off-by: Mikhail Sennikovsky <mikhail.sennikovskii@cloud.ionos.com> --- tests/conntrack/testsuite/09dumpopt | 72 ++++++++++++++++++++++++++++- 1 file changed, 71 insertions(+), 1 deletion(-) diff --git a/tests/conntrack/testsuite/09dumpopt b/tests/conntrack/testsuite/09dumpopt index 0e3c649..9c5121d 100644 --- a/tests/conntrack/testsuite/09dumpopt +++ b/tests/conntrack/testsuite/09dumpopt @@ -74,4 +74,74 @@ # delete v6 conntrack -D -w 10-s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 ; BAD # delete icmp ping request entry --D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD \ No newline at end of file +-D -w 10 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD +# +# Additional tests to check that family attribute is treated properly +# for -L and -D commands +# namely: +# - if family (-f) is given - only entries of the given family are dumped/deleted +# - if no family is given - entries of both ipv4 and ipv6 families are dumped/deleted +# First create some ipv4 and ipv6 entries +-I -w 10 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK +-I -w 10 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY -t 50 ; OK +-I -w 10 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY -t 50 ; OK +-I -w 10 -t 29 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# dump all entries to zone 11 +-L -w 10 -o opt; |s/-w 10/-w 11/g +-I - ; OK +# ensure that both ipv4 and ipv6 entries get copied (delete for each of them should succeed) +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY ; OK +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY ; OK +-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# dump only ipv4 entries to zone 11 +-L -w 10 -o opt -f ipv4; |s/-w 10/-w 11/g +-I - ; OK +# ensure that only ipv4 entries get copied (delete only for ipv4 entries should succeed) +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# dump only ipv6 entries to zone 11 +-L -w 10 -o opt -f ipv6; |s/-w 10/-w 11/g +-I - ; OK +# ensure that only ipv6 entries get copied (delete only for ipv6 entries should succeed) +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD +# now test deleting w/ and /o family specified +# for simplicity do it by re-creating entries in zone 11 +# by copying ezisting entries from zone 10 into it +# re-create entries in ct zone 11 +-L -w 10 -o opt; |s/-w 10/-w 11/g +-I - ; OK +# delete all entries in zone 11 +-D -w 11 ; OK +# both ipv4 and ipv6 should be deleted +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD +# re-create entries in ct zone 11 +-L -w 10 -o opt; |s/-w 10/-w 11/g +-I - ; OK +# delete only ipv4 entries in zone 11 +-D -w 11 -f ipv4 ; OK +# ipv6 should remain +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; BAD + # re-create entries in ct zone 11 +-L -w 10 -o opt; |s/-w 10/-w 11/g +-I - ; OK +# delete only ipv6 entries in zone 11 +-D -w 11 -f ipv6 ; OK +# ipv4 should remain +-D -w 11 -s 1.1.1.1 -d 2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -r 2.2.2.2 -q 1.1.1.1 -p tcp --reply-port-src 11 --reply-port-dst 21 --state LISTEN -u SEEN_REPLY; OK +-D -w 11 -s 2001:DB8::1.1.1.1 -d 2001:DB8::2.2.2.2 -p tcp --sport 10 --dport 20 --state LISTEN -u SEEN_REPLY; BAD +-D -w 11 -u SEEN_REPLY -s 1.1.1.1 -d 2.2.2.2 -r 2.2.2.2 -q 1.1.1.1 -p icmp --icmp-type 8 --icmp-code 0 --icmp-id 1226 ; OK +# clean up after yourself + -D -w 10 ; OK \ No newline at end of file -- 2.25.1 ^ permalink raw reply related [flat|nested] 20+ messages in thread
end of thread, other threads:[~2020-11-04 13:56 UTC | newest] Thread overview: 20+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2020-09-25 12:49 [PATCH 0/8] Fast bulk transfers of large sets of ct entries Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 1/8] tests: icmp entry create/delete Mikhail Sennikovsky 2020-10-13 18:28 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 2/8] conntrack: fix icmp entry creation Mikhail Sennikovsky 2020-10-13 18:29 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 3/8] conntrack: accept parameters from stdin Mikhail Sennikovsky 2020-11-04 13:56 ` Pablo Neira Ayuso 2020-09-25 12:49 ` [PATCH 4/8] conntrack.8: man update for stdin params support Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 5/8] tests: conntrack parameters from stdin Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 6/8] conntrack: implement options output format Mikhail Sennikovsky 2020-10-22 12:36 ` Pablo Neira Ayuso 2020-10-22 12:37 ` Pablo Neira Ayuso 2020-10-23 11:07 ` Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 7/8] conntrack.8: man update for opts format support Mikhail Sennikovsky 2020-09-25 12:49 ` [PATCH 8/8] tests: dumping ct entries in opts format Mikhail Sennikovsky 2020-09-25 13:28 ` [PATCH 0/8] Fast bulk transfers of large sets of ct entries Florian Westphal 2020-09-26 18:19 ` Pablo Neira Ayuso 2020-09-29 11:20 ` [PATCH 0/2] conntrack: -L/-D both ipv4/6 if no family is given Mikhail Sennikovsky 2020-09-29 11:20 ` [PATCH 1/2] " Mikhail Sennikovsky 2020-09-29 11:20 ` [PATCH 2/2] tests: conntrack -L/-D ip family filtering Mikhail Sennikovsky
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).