Implementing Deletion of Set Elements in Rulesets

Implementing Deletion of Set Elements in Rulesets

[Karuna Grewal]

Hello,
I'm trying to implement  "deletion of set elements in ruleset". For
which I wanted to understand the way existing set operations are
implemented.
While grepping through the code I have noticed that the implementation
has some parts in the kernel, libnftnl 's dynset and the userspace's
netlink_(de)linearize .
I'm unable to get  a clear view of how the control flow goes from the
userspace's `evaluate` to the kernel's `nft_dynset.c`  in case of the
set operations.
Can someone please share some pointers in this direction?
Also how does the `set_stmt_alloc` in nftables's statement.c relate to
the `set_evaluate` in evaluate.c ?
Best Regards,
Karuna

Re: Implementing Deletion of Set Elements in Rulesets

[Phil Sutter]

Hi Karuna,

On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote:
> I'm trying to implement  "deletion of set elements in ruleset". For
> which I wanted to understand the way existing set operations are
> implemented.

What are you trying to achieve? Anonymous sets are immutable by design.
If you want to add/delete set elements, you can create a named set and
reference that from rules. See nftables wiki[1] for examples of usage.

Cheers, Phil

[1] https://wiki.nftables.org/wiki-nftables/index.php/Sets#Named_sets

Re: Implementing Deletion of Set Elements in Rulesets

[Florian Westphal]

Phil Sutter <phil@nwl.cc> wrote:
> Hi Karuna,
> 
> On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote:
> > I'm trying to implement  "deletion of set elements in ruleset". For
> > which I wanted to understand the way existing set operations are
> > implemented.
> 
> What are you trying to achieve? Anonymous sets are immutable by design.
> If you want to add/delete set elements, you can create a named set and
> reference that from rules. See nftables wiki[1] for examples of usage.

This is about deletion of elements from the packet path in dynamic
sets, see https://people.netfilter.org/pablo/nf-ideas-2019.txt, 1.4 .

Re: Implementing Deletion of Set Elements in Rulesets

[Phil Sutter]

On Thu, Mar 21, 2019 at 09:45:16AM +0100, Florian Westphal wrote:
> This is about deletion of elements from the packet path in dynamic
> sets, see https://people.netfilter.org/pablo/nf-ideas-2019.txt, 1.4 .

Ah, thanks for the pointer! Obviously I confused dynamic with anonymous
in Karuna's mail.

On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote:
> I'm trying to implement  "deletion of set elements in ruleset". For
> which I wanted to understand the way existing set operations are
> implemented.
> While grepping through the code I have noticed that the implementation
> has some parts in the kernel, libnftnl 's dynset and the userspace's
> netlink_(de)linearize .
> I'm unable to get  a clear view of how the control flow goes from the
> userspace's `evaluate` to the kernel's `nft_dynset.c`  in case of the
> set operations.
> Can someone please share some pointers in this direction?
> Also how does the `set_stmt_alloc` in nftables's statement.c relate to
> the `set_evaluate` in evaluate.c ?

I don't quite see where you're stuck. So here's a bit of generic
code-flow explanation, maybe it helps:

- User calls 'nft' with some command
- Arguments are parsed in scanner.l/parser_bison.y, resulting in a
  struct cmd instance
- Last step of parsing is to call cmd_evaluate() (see
  parser_bison.y:799)
- Assuming the command was:
  'nft add rule ip test testchain update @testset { ip saddr timeout 1m }'
  code flows like this:
  - cmd_evaluate_add()
    - case CMD_OBJ_RULE
      - rule_evaluate()
        - stmt_evaluate()
          - case STMT_SET
            - stmt_evaluate_set()
            - ...
      - rule_postprocess()
	- payload_try_merge() (probably noop in this case)
- If evaluation succeeds (most of it is sanitization checking), command
  is appended to list in state->cmds
- After parsing has finished, code continues in
  nft_run_cmd_from_buffer() of libnftables.c
  - nft_netlink()
    - do_command()
      - do_command_add()
        - case CMD_OBJ_RULE
          - mnl_nft_rule_add() this converts the rule into a netlink
            message which is appended to batch buffer
    - mnl_batch_talk() this submits the batch to kernel

My guess is that you over-estimate evaluation stage. The real work is
done by do_command() as this turns parser output into netlink messages.

I'll skip kernel side for now, hopefully user space is more clear now.
Feel free to follow-up with further questions.

Cheers, Phil

Re: Implementing Deletion of Set Elements in Rulesets

[Karuna Grewal]

Thanks a lot Phil. This was lucid indeed. I still have a few more doubts:
* what's the purpose of  a context (netlink or eval)?
* What is the purpose of the cache being used after the netlink
message has been already sent to the kernel?
* Could you please explain a bit about the kernel interaction once the
netlink message is sent esp. which structures store the data which was
carried by the message from userpace. I'm aware of the concept of
hooks being registered and thereon the processing is handled by the
netfilter code but I'm not completely clear about how the netlink
message gets handled internally.

Regards,
Karuna

On Thu, Mar 21, 2019 at 4:38 PM Phil Sutter <phil@nwl.cc> wrote:
>
> On Thu, Mar 21, 2019 at 09:45:16AM +0100, Florian Westphal wrote:
> > This is about deletion of elements from the packet path in dynamic
> > sets, see https://people.netfilter.org/pablo/nf-ideas-2019.txt, 1.4 .
>
> Ah, thanks for the pointer! Obviously I confused dynamic with anonymous
> in Karuna's mail.
>
> On Thu, Mar 21, 2019 at 11:57:15AM +0530, Karuna Grewal wrote:
> > I'm trying to implement  "deletion of set elements in ruleset". For
> > which I wanted to understand the way existing set operations are
> > implemented.
> > While grepping through the code I have noticed that the implementation
> > has some parts in the kernel, libnftnl 's dynset and the userspace's
> > netlink_(de)linearize .
> > I'm unable to get  a clear view of how the control flow goes from the
> > userspace's `evaluate` to the kernel's `nft_dynset.c`  in case of the
> > set operations.
> > Can someone please share some pointers in this direction?
> > Also how does the `set_stmt_alloc` in nftables's statement.c relate to
> > the `set_evaluate` in evaluate.c ?
>
> I don't quite see where you're stuck. So here's a bit of generic
> code-flow explanation, maybe it helps:
>
> - User calls 'nft' with some command
> - Arguments are parsed in scanner.l/parser_bison.y, resulting in a
>   struct cmd instance
> - Last step of parsing is to call cmd_evaluate() (see
>   parser_bison.y:799)
> - Assuming the command was:
>   'nft add rule ip test testchain update @testset { ip saddr timeout 1m }'
>   code flows like this:
>   - cmd_evaluate_add()
>     - case CMD_OBJ_RULE
>       - rule_evaluate()
>         - stmt_evaluate()
>           - case STMT_SET
>             - stmt_evaluate_set()
>             - ...
>       - rule_postprocess()
>         - payload_try_merge() (probably noop in this case)
> - If evaluation succeeds (most of it is sanitization checking), command
>   is appended to list in state->cmds
> - After parsing has finished, code continues in
>   nft_run_cmd_from_buffer() of libnftables.c
>   - nft_netlink()
>     - do_command()
>       - do_command_add()
>         - case CMD_OBJ_RULE
>           - mnl_nft_rule_add() this converts the rule into a netlink
>             message which is appended to batch buffer
>     - mnl_batch_talk() this submits the batch to kernel
>
> My guess is that you over-estimate evaluation stage. The real work is
> done by do_command() as this turns parser output into netlink messages.
>
> I'll skip kernel side for now, hopefully user space is more clear now.
> Feel free to follow-up with further questions.
>
> Cheers, Phil

Re: Implementing Deletion of Set Elements in Rulesets

[Phil Sutter]

On Sat, Mar 23, 2019 at 11:10:24PM +0530, Karuna Grewal wrote:
> Thanks a lot Phil. This was lucid indeed. I still have a few more doubts:
> * what's the purpose of  a context (netlink or eval)?

Contexts provide context. ;)
E.g. nft_ctx holds application information like configured output/error
file descriptors.
In general, these context structures are used to provide data to
functions further down in the call stack without the need for excessive
amounts of function parameters or use of global variables (which should
be avoided for obvious reasons).
If you want to know more about where and how they are used, look up what
fields are contained in each context struct and where they are
instantiated.

> * What is the purpose of the cache being used after the netlink message has
> been already sent to the kernel?

Cache belongs to nft_ctx, which is created by the application. The
application may run multiple commands so keeping the cache after a
commit to kernel is useful. I think it is used from echo callback as
well, although I can't find an example right now.

> * Could you please explain a bit about the kernel interaction once the
> netlink message is sent esp. which structures store the data which was
> carried by the message from userpace. I'm aware of the concept of hooks
> being registered and thereon the processing is handled by the netfilter
> code but I'm not completely clear about how the netlink message gets
> handled internally.

Libnftables objects (struct rule, struct chain, etc.) are converted into
libnftnl objects (struct nftnl_rule, struct nftnl_chain, etc.) within
libnftables. In libnftnl, there are *_build_payload() functions which
serialize libnftnl objects into a netlink message identified by an
instance of struct nlmsghdr.

Netlink messages contain a static header (see struct nlmsghdr) and an
arbitrary amount of attributes of the form [len, type, data]. Libnftnl
uses libmnl to append those attributes to a message. In order to find
out where and how a given netlink message is handled in the kernel, the
quickest way is often to grep for some attribute type definition.

One caveat with nlmsg attributes is that libnftnl and kernel have
distinct ones. E.g. NFTNL_TABLE_NAME in libnftnl corresponds with
NFTA_TABLE_NAME in kernel. See *_build_payload() functions in libnftnl
for details, but in general NFTNL_FOO corresponds with NFTA_FOO in
kernel.

Cheers, Phil