using nft & iptables nat in parallel

using nft & iptables nat in parallel

[Florian Westphal]

Hi.

As you might know its currently not possible to use iptables
and nft nat at the same time.

I had a look at this and think we should fix this as follows:

1. decouple nat rewrite from nat chain/iptable_nat iteration.

Currently we do this from nf_nat_ipv4/6_fn, so first call (be it nft nat
or ip(6)table_nat) attaches nat translation or a null binding which makes
2nd invocation a no-op.

2. place a l3 protocol independent function that does null binding and
rewrite in nf_nat_core.  It can be independent because nf_nat_info has
the l3 information and we don't need to deal with anything special
except hooking at either NFPROTO_IPV4 or V6.

3. make all the nat expressions/targets call a helper function at
init/destroy time so we can register the new rewrite hooks.
This is easy, all these currently call nf_ct_netns_get/put so we
only need to add nf_nat_netns_get/put which calls nf_ct_netns_get
and registers a nat null-bind/rewrite hook.

The downside is that we need one more hook function at priority
NF_IP_PRI_NAT_SRC/DST + 1 in all places except forward.

Another side effect is that this avoids the need to add (in nft case)
the 'empty' nat base chains to take care of reply translation.

Thoughts?

Otherwise I'll have a go at this.

This would also possibly allow us to add nat hooks in the INET family.

Re: using nft & iptables nat in parallel

[Arturo Borrero Gonzalez]

On 14 June 2017 at 11:24, Florian Westphal <fw@strlen.de> wrote:
>
> Another side effect is that this avoids the need to add (in nft case)
> the 'empty' nat base chains to take care of reply translation.
>

good!

> Thoughts?
>

I'm curious, What is the use case of using both nftables and iptables
at the same time?
Some missing functionality in nft?
Perhaps some ipt->nft partial migration procedure?

>
> This would also possibly allow us to add nat hooks in the INET family.

good!

Re: using nft & iptables nat in parallel

[Florian Westphal]

Arturo Borrero Gonzalez <arturo@debian.org> wrote:
> I'm curious, What is the use case of using both nftables and iptables
> at the same time?
> Some missing functionality in nft?
> Perhaps some ipt->nft partial migration procedure?

Yes, partial migration.

Right now there are an awful lot of tools out there (docker, libvirt,
kubernetes, ..) that call iptables(-restore) directly (or inject them via
firewalld).

And unfortunately I don't see how we can magically move all of this
to nftables.

So allowing to do a step-by-step migration seems the only viable
option.

Re: using nft & iptables nat in parallel

[Pablo Neira Ayuso]

On Wed, Jun 14, 2017 at 11:58:03AM +0200, Florian Westphal wrote:
> Arturo Borrero Gonzalez <arturo@debian.org> wrote:
> > I'm curious, What is the use case of using both nftables and iptables
> > at the same time?
> > Some missing functionality in nft?
> > Perhaps some ipt->nft partial migration procedure?
> 
> Yes, partial migration.
> 
> Right now there are an awful lot of tools out there (docker, libvirt,
> kubernetes, ..) that call iptables(-restore) directly (or inject them via
> firewalld).
> 
> And unfortunately I don't see how we can magically move all of this
> to nftables.
>
> So allowing to do a step-by-step migration seems the only viable
> option.

We have iptables-compat-restore, this uses the nf_tables netlink
frontend and packet classification code, plus x_tables extensions to
run iptables code. It should allow us to schedule the x_tables core
for removal at some point.

In Montpellier, we mentioned that distro could add a iptables-restore
symlink to iptables-compat-restore so it becomes visible that we're
relying on the compat infrastructure.

This iptables-compat-restore interacts with nft via translate layer,
so if you just 'nft list ruleset' after using iptables-compat-restore,
you will get a translation (if available).

Re: using nft & iptables nat in parallel

[Florian Westphal]

Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Wed, Jun 14, 2017 at 11:58:03AM +0200, Florian Westphal wrote:
> > Arturo Borrero Gonzalez <arturo@debian.org> wrote:
> > > I'm curious, What is the use case of using both nftables and iptables
> > > at the same time?
> > > Some missing functionality in nft?
> > > Perhaps some ipt->nft partial migration procedure?
> > 
> > Yes, partial migration.
> > 
> > Right now there are an awful lot of tools out there (docker, libvirt,
> > kubernetes, ..) that call iptables(-restore) directly (or inject them via
> > firewalld).
> > 
> > And unfortunately I don't see how we can magically move all of this
> > to nftables.
> >
> > So allowing to do a step-by-step migration seems the only viable
> > option.
> 
> We have iptables-compat-restore, this uses the nf_tables netlink
> frontend and packet classification code, plus x_tables extensions to
> run iptables code. It should allow us to schedule the x_tables core
> for removal at some point.

Yes, perhaps.

> In Montpellier, we mentioned that distro could add a iptables-restore
> symlink to iptables-compat-restore so it becomes visible that we're
> relying on the compat infrastructure.
> 
> This iptables-compat-restore interacts with nft via translate layer,
> so if you just 'nft list ruleset' after using iptables-compat-restore,
> you will get a translation (if available).

That still means drastic change, swapping out xt_core for nftables
rather than using "old" iptables is still a big difference...

I also think that ability to use both iptables or nftables nat
is a good thing, since it lowers barrier to 'just try it' without
having to unload iptable_nat and friends.

Also forgot to mention: this will allow to use nat base chains
in multiple tables (even though it is probably not a good idea to do
that due to possible problems because of unexpected/different evaluation
order).

Re: using nft & iptables nat in parallel

[Pablo Neira Ayuso]

On Wed, Jun 14, 2017 at 01:19:34PM +0200, Florian Westphal wrote:
> Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > On Wed, Jun 14, 2017 at 11:58:03AM +0200, Florian Westphal wrote:
> > > Arturo Borrero Gonzalez <arturo@debian.org> wrote:
> > > > I'm curious, What is the use case of using both nftables and iptables
> > > > at the same time?
> > > > Some missing functionality in nft?
> > > > Perhaps some ipt->nft partial migration procedure?
> > > 
> > > Yes, partial migration.
> > > 
> > > Right now there are an awful lot of tools out there (docker, libvirt,
> > > kubernetes, ..) that call iptables(-restore) directly (or inject them via
> > > firewalld).
> > > 
> > > And unfortunately I don't see how we can magically move all of this
> > > to nftables.
> > >
> > > So allowing to do a step-by-step migration seems the only viable
> > > option.
> > 
> > We have iptables-compat-restore, this uses the nf_tables netlink
> > frontend and packet classification code, plus x_tables extensions to
> > run iptables code. It should allow us to schedule the x_tables core
> > for removal at some point.
> 
> Yes, perhaps.
> 
> > In Montpellier, we mentioned that distro could add a iptables-restore
> > symlink to iptables-compat-restore so it becomes visible that we're
> > relying on the compat infrastructure.
> > 
> > This iptables-compat-restore interacts with nft via translate layer,
> > so if you just 'nft list ruleset' after using iptables-compat-restore,
> > you will get a translation (if available).
> 
> That still means drastic change, swapping out xt_core for nftables
> rather than using "old" iptables is still a big difference...

Not drastic. The idea is that compat provides same semantics. Did you
give it a try to evaluate the state of this? I would be glad to see
feedback on this. We can probably get some GSoC people helping us
polish this...

> I also think that ability to use both iptables or nftables nat
> is a good thing, since it lowers barrier to 'just try it' without
> having to unload iptable_nat and friends.

Don't get me wrong, I'm not opposing to your idea of get both working
if you want to work on this if you think this is useful.

The extra hook has a performance impact though, is it something that
would just go away one x_tables is gone? What is your plan on this?

On a different front (just an idea), I would really like to provide an
alternative to setting conntrack templates, so we can get rid of
having a chain (hence another hook) just to set the zone. It's just
more cycles on a hook to do something very simple.

> Also forgot to mention: this will allow to use nat base chains
> in multiple tables (even though it is probably not a good idea to do
> that due to possible problems because of unexpected/different evaluation
> order).

nftables NAT configuration needs a closer look indeed, we should
probably autoregister the hook that are needed without having to
require the user to register explicitly an empty chain as the
documentation suggest now.

Re: using nft & iptables nat in parallel

[Florian Westphal]

Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > That still means drastic change, swapping out xt_core for nftables
> > rather than using "old" iptables is still a big difference...
> 
> Not drastic. The idea is that compat provides same semantics. Did you
> give it a try to evaluate the state of this?

Not yet.

> The extra hook has a performance impact though, is it something that
> would just go away one x_tables is gone? What is your plan on this?

Once we do it we can't remove it again, because you can have multiple
nat base chains after this change, and removing hook and merging it back
into the l3 nat code means first chain attaches a null binding again.

> On a different front (just an idea), I would really like to provide an
> alternative to setting conntrack templates, so we can get rid of
> having a chain (hence another hook) just to set the zone. It's just
> more cycles on a hook to do something very simple.

I don't think it can be done, the zone id has to be set before conntrack
lookup, I see no way to avoid this?

So even if we find a way to transport zone id from one hook to the
next (sans skb->nfct template) we still need one extra hook to
configure/set the zone id to use.

Re: using nft & iptables nat in parallel

[Arturo Borrero Gonzalez]

On 14 June 2017 at 11:58, Florian Westphal <fw@strlen.de> wrote:
> Arturo Borrero Gonzalez <arturo@debian.org> wrote:
>> I'm curious, What is the use case of using both nftables and iptables
>> at the same time?
>> Some missing functionality in nft?
>> Perhaps some ipt->nft partial migration procedure?
>
> Yes, partial migration.
>
> Right now there are an awful lot of tools out there (docker, libvirt,
> kubernetes, ..) that call iptables(-restore) directly (or inject them via
> firewalld).
>

Well, just quickly checked firewalld code [0].
It relies so massively in ipXtables/ipset that I bet a migration to
nftables would require a major rewrite.

Not sure if iptables-compat address this case, or even a high level libnftables.

[0] https://github.com/t-woerner/firewalld/blob/master/src/firewall/core/ipXtables.py

Re: using nft & iptables nat in parallel

[Pablo Neira Ayuso]

On Wed, Jun 14, 2017 at 01:53:38PM +0200, Florian Westphal wrote:
> Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> 
> > The extra hook has a performance impact though, is it something that
> > would just go away one x_tables is gone? What is your plan on this?
> 
> Once we do it we can't remove it again, because you can have multiple
> nat base chains after this change, and removing hook and merging it back
> into the l3 nat code means first chain attaches a null binding again.

With multiple nat chains, in case of overlap, we would just take the
last coming in the pipeline. Just like several chains several times
the same packet from a filter chain, right?

> > On a different front (just an idea), I would really like to provide an
> > alternative to setting conntrack templates, so we can get rid of
> > having a chain (hence another hook) just to set the zone. It's just
> > more cycles on a hook to do something very simple.
> 
> I don't think it can be done, the zone id has to be set before conntrack
> lookup, I see no way to avoid this?
>
> So even if we find a way to transport zone id from one hook to the
> next (sans skb->nfct template) we still need one extra hook to
> configure/set the zone id to use.

Can we do this from rule? So we run sort of nf_conntrack_in() from a
expression that enables tracking? I remember you mentioned about ICMP
and related traffic, we just have to document how to configure this so
people don't shoot on the feet, ie. track both tcp and icmp if you
want to do this right with this selective tracking expressed in
positive logic (contrary to the notrack, negative, logic).

I guess you would need something like this for bridge conntrack
support, right?

Re: using nft & iptables nat in parallel

[Pablo Neira Ayuso]

On Wed, Jun 14, 2017 at 07:13:12PM +0200, Pablo Neira Ayuso wrote:
> On Wed, Jun 14, 2017 at 01:53:38PM +0200, Florian Westphal wrote:
> > Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > 
> > > The extra hook has a performance impact though, is it something that
> > > would just go away one x_tables is gone? What is your plan on this?
> > 
> > Once we do it we can't remove it again, because you can have multiple
> > nat base chains after this change, and removing hook and merging it back
> > into the l3 nat code means first chain attaches a null binding again.
> 
> With multiple nat chains, in case of overlap, we would just take the
> last coming in the pipeline. Just like several chains several times
> the same packet from a filter chain, right?

I meant: just like marking the packet several times from different
chains.

Re: using nft & iptables nat in parallel

[Florian Westphal]

Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Wed, Jun 14, 2017 at 01:53:38PM +0200, Florian Westphal wrote:
> > Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> > 
> > > The extra hook has a performance impact though, is it something that
> > > would just go away one x_tables is gone? What is your plan on this?
> > 
> > Once we do it we can't remove it again, because you can have multiple
> > nat base chains after this change, and removing hook and merging it back
> > into the l3 nat code means first chain attaches a null binding again.
> 
> With multiple nat chains, in case of overlap, we would just take the
> last coming in the pipeline. Just like several chains several times
> the same packet from a filter chain, right?

First match would win because nf_nat_initialized() is true, just like:

-j MASQUERADE
-p tcp --dport 42 -j SNAT ... # no-op

> > > On a different front (just an idea), I would really like to provide an
> > > alternative to setting conntrack templates, so we can get rid of
> > > having a chain (hence another hook) just to set the zone. It's just
> > > more cycles on a hook to do something very simple.
> > 
> > I don't think it can be done, the zone id has to be set before conntrack
> > lookup, I see no way to avoid this?
> >
> > So even if we find a way to transport zone id from one hook to the
> > next (sans skb->nfct template) we still need one extra hook to
> > configure/set the zone id to use.
> 
> Can we do this from rule? So we run sort of nf_conntrack_in() from a
> expression that enables tracking? I remember you mentioned about ICMP
> and related traffic, we just have to document how to configure this so
> people don't shoot on the feet, ie. track both tcp and icmp if you
> want to do this right with this selective tracking expressed in
> positive logic (contrary to the notrack, negative, logic).

Hmm, we can discuss this again during nfws, but I abandoned this
idea (it provides too much rope for ppl to hang themselvers with :-) )

> I guess you would need something like this for bridge conntrack
> support, right?

The (inclomplete, sigh) patches have a hybrid solution which is
worst-of-both-worlds hooking-wise, i.e. you do explicit tracking
request but there is still an implicit hook to catch related and
reply traffic.