* Implementing Deletion of Set Elements in Rulesets @ 2019-03-21 6:27 Karuna Grewal 2019-03-21 8:23 ` Phil Sutter 2019-03-21 11:08 ` Phil Sutter 0 siblings, 2 replies; 6+ messages in thread From: Karuna Grewal @ 2019-03-21 6:27 UTC (permalink / raw) To: Pablo Neira Ayuso, Florian Westphal; +Cc: netfilter-devel Hello, I'm trying to implement "deletion of set elements in ruleset". For which I wanted to understand the way existing set operations are implemented. While grepping through the code I have noticed that the implementation has some parts in the kernel, libnftnl 's dynset and the userspace's netlink_(de)linearize . I'm unable to get a clear view of how the control flow goes from the userspace's `evaluate` to the kernel's `nft_dynset.c` in case of the set operations. Can someone please share some pointers in this direction? Also how does the `set_stmt_alloc` in nftables's statement.c relate to the `set_evaluate` in evaluate.c ? Best Regards, Karuna ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Implementing Deletion of Set Elements in Rulesets 2019-03-21 6:27 Implementing Deletion of Set Elements in Rulesets Karuna Grewal @ 2019-03-21 8:23 ` Phil Sutter 2019-03-21 8:45 ` Florian Westphal 2019-03-21 11:08 ` Phil Sutter 1 sibling, 1 reply; 6+ messages in thread From: Phil Sutter @ 2019-03-21 8:23 UTC (permalink / raw) To: Karuna Grewal; +Cc: Pablo Neira Ayuso, Florian Westphal, netfilter-devel Hi Karuna, On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote: > I'm trying to implement "deletion of set elements in ruleset". For > which I wanted to understand the way existing set operations are > implemented. What are you trying to achieve? Anonymous sets are immutable by design. If you want to add/delete set elements, you can create a named set and reference that from rules. See nftables wiki[1] for examples of usage. Cheers, Phil [1] https://wiki.nftables.org/wiki-nftables/index.php/Sets#Named_sets ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Implementing Deletion of Set Elements in Rulesets 2019-03-21 8:23 ` Phil Sutter @ 2019-03-21 8:45 ` Florian Westphal 0 siblings, 0 replies; 6+ messages in thread From: Florian Westphal @ 2019-03-21 8:45 UTC (permalink / raw) To: Phil Sutter, Karuna Grewal, Pablo Neira Ayuso, Florian Westphal, netfilter-devel Phil Sutter <phil@nwl.cc> wrote: > Hi Karuna, > > On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote: > > I'm trying to implement "deletion of set elements in ruleset". For > > which I wanted to understand the way existing set operations are > > implemented. > > What are you trying to achieve? Anonymous sets are immutable by design. > If you want to add/delete set elements, you can create a named set and > reference that from rules. See nftables wiki[1] for examples of usage. This is about deletion of elements from the packet path in dynamic sets, see https://people.netfilter.org/pablo/nf-ideas-2019.txt, 1.4 . ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Implementing Deletion of Set Elements in Rulesets 2019-03-21 6:27 Implementing Deletion of Set Elements in Rulesets Karuna Grewal 2019-03-21 8:23 ` Phil Sutter @ 2019-03-21 11:08 ` Phil Sutter 2019-03-24 4:33 ` Karuna Grewal [not found] ` <CAHRz_yazsKDFYfsVemDLd4av3M+3k6MJnctvFYYMjDD7q2zSMA@mail.gmail.com> 1 sibling, 2 replies; 6+ messages in thread From: Phil Sutter @ 2019-03-21 11:08 UTC (permalink / raw) To: Karuna Grewal, Florian Westphal; +Cc: Pablo Neira Ayuso, netfilter-devel On Thu, Mar 21, 2019 at 09:45:16AM +0100, Florian Westphal wrote: > This is about deletion of elements from the packet path in dynamic > sets, see https://people.netfilter.org/pablo/nf-ideas-2019.txt, 1.4 . Ah, thanks for the pointer! Obviously I confused dynamic with anonymous in Karuna's mail. On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote: > I'm trying to implement "deletion of set elements in ruleset". For > which I wanted to understand the way existing set operations are > implemented. > While grepping through the code I have noticed that the implementation > has some parts in the kernel, libnftnl 's dynset and the userspace's > netlink_(de)linearize . > I'm unable to get a clear view of how the control flow goes from the > userspace's `evaluate` to the kernel's `nft_dynset.c` in case of the > set operations. > Can someone please share some pointers in this direction? > Also how does the `set_stmt_alloc` in nftables's statement.c relate to > the `set_evaluate` in evaluate.c ? I don't quite see where you're stuck. So here's a bit of generic code-flow explanation, maybe it helps: - User calls 'nft' with some command - Arguments are parsed in scanner.l/parser_bison.y, resulting in a struct cmd instance - Last step of parsing is to call cmd_evaluate() (see parser_bison.y:799) - Assuming the command was: 'nft add rule ip test testchain update @testset { ip saddr timeout 1m }' code flows like this: - cmd_evaluate_add() - case CMD_OBJ_RULE - rule_evaluate() - stmt_evaluate() - case STMT_SET - stmt_evaluate_set() - ... - rule_postprocess() - payload_try_merge() (probably noop in this case) - If evaluation succeeds (most of it is sanitization checking), command is appended to list in state->cmds - After parsing has finished, code continues in nft_run_cmd_from_buffer() of libnftables.c - nft_netlink() - do_command() - do_command_add() - case CMD_OBJ_RULE - mnl_nft_rule_add() this converts the rule into a netlink message which is appended to batch buffer - mnl_batch_talk() this submits the batch to kernel My guess is that you over-estimate evaluation stage. The real work is done by do_command() as this turns parser output into netlink messages. I'll skip kernel side for now, hopefully user space is more clear now. Feel free to follow-up with further questions. Cheers, Phil ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: Implementing Deletion of Set Elements in Rulesets 2019-03-21 11:08 ` Phil Sutter @ 2019-03-24 4:33 ` Karuna Grewal [not found] ` <CAHRz_yazsKDFYfsVemDLd4av3M+3k6MJnctvFYYMjDD7q2zSMA@mail.gmail.com> 1 sibling, 0 replies; 6+ messages in thread From: Karuna Grewal @ 2019-03-24 4:33 UTC (permalink / raw) To: netfilter-devel Thanks a lot Phil. This was lucid indeed. I still have a few more doubts: * what's the purpose of a context (netlink or eval)? * What is the purpose of the cache being used after the netlink message has been already sent to the kernel? * Could you please explain a bit about the kernel interaction once the netlink message is sent esp. which structures store the data which was carried by the message from userpace. I'm aware of the concept of hooks being registered and thereon the processing is handled by the netfilter code but I'm not completely clear about how the netlink message gets handled internally. Regards, Karuna On Thu, Mar 21, 2019 at 4:38 PM Phil Sutter <phil@nwl.cc> wrote: > > On Thu, Mar 21, 2019 at 09:45:16AM +0100, Florian Westphal wrote: > > This is about deletion of elements from the packet path in dynamic > > sets, see https://people.netfilter.org/pablo/nf-ideas-2019.txt, 1.4 . > > Ah, thanks for the pointer! Obviously I confused dynamic with anonymous > in Karuna's mail. > > On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote: > > I'm trying to implement "deletion of set elements in ruleset". For > > which I wanted to understand the way existing set operations are > > implemented. > > While grepping through the code I have noticed that the implementation > > has some parts in the kernel, libnftnl 's dynset and the userspace's > > netlink_(de)linearize . > > I'm unable to get a clear view of how the control flow goes from the > > userspace's `evaluate` to the kernel's `nft_dynset.c` in case of the > > set operations. > > Can someone please share some pointers in this direction? > > Also how does the `set_stmt_alloc` in nftables's statement.c relate to > > the `set_evaluate` in evaluate.c ? > > I don't quite see where you're stuck. So here's a bit of generic > code-flow explanation, maybe it helps: > > - User calls 'nft' with some command > - Arguments are parsed in scanner.l/parser_bison.y, resulting in a > struct cmd instance > - Last step of parsing is to call cmd_evaluate() (see > parser_bison.y:799) > - Assuming the command was: > 'nft add rule ip test testchain update @testset { ip saddr timeout 1m }' > code flows like this: > - cmd_evaluate_add() > - case CMD_OBJ_RULE > - rule_evaluate() > - stmt_evaluate() > - case STMT_SET > - stmt_evaluate_set() > - ... > - rule_postprocess() > - payload_try_merge() (probably noop in this case) > - If evaluation succeeds (most of it is sanitization checking), command > is appended to list in state->cmds > - After parsing has finished, code continues in > nft_run_cmd_from_buffer() of libnftables.c > - nft_netlink() > - do_command() > - do_command_add() > - case CMD_OBJ_RULE > - mnl_nft_rule_add() this converts the rule into a netlink > message which is appended to batch buffer > - mnl_batch_talk() this submits the batch to kernel > > My guess is that you over-estimate evaluation stage. The real work is > done by do_command() as this turns parser output into netlink messages. > > I'll skip kernel side for now, hopefully user space is more clear now. > Feel free to follow-up with further questions. > > Cheers, Phil ^ permalink raw reply [flat|nested] 6+ messages in thread
[parent not found: <CAHRz_yazsKDFYfsVemDLd4av3M+3k6MJnctvFYYMjDD7q2zSMA@mail.gmail.com>]
* Re: Implementing Deletion of Set Elements in Rulesets [not found] ` <CAHRz_yazsKDFYfsVemDLd4av3M+3k6MJnctvFYYMjDD7q2zSMA@mail.gmail.com> @ 2019-03-25 10:44 ` Phil Sutter 0 siblings, 0 replies; 6+ messages in thread From: Phil Sutter @ 2019-03-25 10:44 UTC (permalink / raw) To: Karuna Grewal; +Cc: Florian Westphal, Pablo Neira Ayuso, netfilter-devel On Sat, Mar 23, 2019 at 11:10:24PM +0530, Karuna Grewal wrote: > Thanks a lot Phil. This was lucid indeed. I still have a few more doubts: > * what's the purpose of a context (netlink or eval)? Contexts provide context. ;) E.g. nft_ctx holds application information like configured output/error file descriptors. In general, these context structures are used to provide data to functions further down in the call stack without the need for excessive amounts of function parameters or use of global variables (which should be avoided for obvious reasons). If you want to know more about where and how they are used, look up what fields are contained in each context struct and where they are instantiated. > * What is the purpose of the cache being used after the netlink message has > been already sent to the kernel? Cache belongs to nft_ctx, which is created by the application. The application may run multiple commands so keeping the cache after a commit to kernel is useful. I think it is used from echo callback as well, although I can't find an example right now. > * Could you please explain a bit about the kernel interaction once the > netlink message is sent esp. which structures store the data which was > carried by the message from userpace. I'm aware of the concept of hooks > being registered and thereon the processing is handled by the netfilter > code but I'm not completely clear about how the netlink message gets > handled internally. Libnftables objects (struct rule, struct chain, etc.) are converted into libnftnl objects (struct nftnl_rule, struct nftnl_chain, etc.) within libnftables. In libnftnl, there are *_build_payload() functions which serialize libnftnl objects into a netlink message identified by an instance of struct nlmsghdr. Netlink messages contain a static header (see struct nlmsghdr) and an arbitrary amount of attributes of the form [len, type, data]. Libnftnl uses libmnl to append those attributes to a message. In order to find out where and how a given netlink message is handled in the kernel, the quickest way is often to grep for some attribute type definition. One caveat with nlmsg attributes is that libnftnl and kernel have distinct ones. E.g. NFTNL_TABLE_NAME in libnftnl corresponds with NFTA_TABLE_NAME in kernel. See *_build_payload() functions in libnftnl for details, but in general NFTNL_FOO corresponds with NFTA_FOO in kernel. Cheers, Phil ^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-03-25 10:44 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-03-21 6:27 Implementing Deletion of Set Elements in Rulesets Karuna Grewal 2019-03-21 8:23 ` Phil Sutter 2019-03-21 8:45 ` Florian Westphal 2019-03-21 11:08 ` Phil Sutter 2019-03-24 4:33 ` Karuna Grewal [not found] ` <CAHRz_yazsKDFYfsVemDLd4av3M+3k6MJnctvFYYMjDD7q2zSMA@mail.gmail.com> 2019-03-25 10:44 ` Phil Sutter
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).