Overlapping IP networks no longer allowed?

Overlapping IP networks no longer allowed?

[Mantas Mikulėnas]

Hello,

As of nftables 0.8.1, it seems I can no longer write anonymous sets
which contain overlapping networks (CIDR masks).

For example, I want to write the following ruleset:

#!/usr/bin/nft -f
define users = { 10.0.0.0/8, 193.219.181.192/26 }
define admins = { 10.123.0.0/24, 31.220.42.129 }
define allowed = { $users, $admins }
table inet filter {
        chain foobar {
                ip saddr $allowed accept
        }
}

results in an error message:

    Error: interval overlaps with previous one

I noticed a few nftables.git commits related to disabling auto-merge
for interval sets... but mine don't have the 'interval' flag, and
there doesn't seem to be any way to specify 'auto-merge' for anonymous
sets, either.

-- 
Mantas Mikulėnas

Re: Overlapping IP networks no longer allowed?

[Pablo Neira Ayuso]

On Wed, Feb 14, 2018 at 07:02:32PM +0200, Mantas Mikulėnas wrote:
> Hello,
> 
> As of nftables 0.8.1, it seems I can no longer write anonymous sets
> which contain overlapping networks (CIDR masks).
> 
> For example, I want to write the following ruleset:
> 
> #!/usr/bin/nft -f
> define users = { 10.0.0.0/8, 193.219.181.192/26 }
> define admins = { 10.123.0.0/24, 31.220.42.129 }
> define allowed = { $users, $admins }
> table inet filter {
>         chain foobar {
>                 ip saddr $allowed accept
>         }
> }
> 
> results in an error message:
> 
>     Error: interval overlaps with previous one
> 
> I noticed a few nftables.git commits related to disabling auto-merge
> for interval sets... but mine don't have the 'interval' flag, and
> there doesn't seem to be any way to specify 'auto-merge' for anonymous
> sets, either.

I would like not to enable this by default since typo in rulesets
could go through unnoticed.

So the two alternatives I see are:

1) add per-table configuration options, this would allow us to
   enable auto-merge explicitly for all anonymous sets. This is also
   required if we want to allow user to select "policy memory;" for
   anonymous sets. Only problem with this approach is that this needs
   a kernel patch, so it will take a while to restore the behaviour you
   want since we need a new NFTA_TABLE_USERDATA attribute to store user
   preferences on this.

2) We add a -m option that we can combine with -f for this, which
   globally enables auto-merge for every set, including anonymous and
   named sets.

Let me know.

Re: Overlapping IP networks no longer allowed?

[Florian Westphal]

Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Wed, Feb 14, 2018 at 07:02:32PM +0200, Mantas Mikulėnas wrote:
> > Hello,
> > 
> > As of nftables 0.8.1, it seems I can no longer write anonymous sets
> > which contain overlapping networks (CIDR masks).
> > 
> > For example, I want to write the following ruleset:
> > 
> > #!/usr/bin/nft -f
> > define users = { 10.0.0.0/8, 193.219.181.192/26 }
> > define admins = { 10.123.0.0/24, 31.220.42.129 }
> > define allowed = { $users, $admins }
> > table inet filter {
> >         chain foobar {
> >                 ip saddr $allowed accept
> >         }
> > }
> > 
> > results in an error message:
> > 
> >     Error: interval overlaps with previous one
> > 
> > I noticed a few nftables.git commits related to disabling auto-merge
> > for interval sets... but mine don't have the 'interval' flag, and
> > there doesn't seem to be any way to specify 'auto-merge' for anonymous
> > sets, either.
> 
> I would like not to enable this by default since typo in rulesets
> could go through unnoticed.

nft add rule filter input ip protocol '{6 ,tcp }'
works just fine, eliminating duplicate set elements.
So I don't see how that is different from removing the redundant parts
of an anon set.

Especially with 'define' things I believe that automerge by default
is desireable.

> So the two alternatives I see are:
> 
> 1) add per-table configuration options, this would allow us to
>    enable auto-merge explicitly for all anonymous sets. This is also
>    required if we want to allow user to select "policy memory;" for
>    anonymous sets. Only problem with this approach is that this needs
>    a kernel patch, so it will take a while to restore the behaviour you
>    want since we need a new NFTA_TABLE_USERDATA attribute to store user
>    preferences on this.

Right.

> 2) We add a -m option that we can combine with -f for this, which
>    globally enables auto-merge for every set, including anonymous and
>    named sets.

What about doing automerge by default again for anon sets?

I know you don't like it but it restores old behaviour.
We could have a debug option that tells users which addresse(s) were
autoremoved.

The typo argument -- not sure its a valid:
Consider '10.0.0.01' (instead of .10), we don't try to be 'smart'
and thats a good thing.

For named sets, the no automerge makes sense because it seems like
we can't make any reasonable default choice when users try to delete
a no-longer existing (i.e. merged) element.

But that problem doesn't exist with constant (anon or not) sets.

Re: Overlapping IP networks no longer allowed?

[Pablo Neira Ayuso]

On Wed, Feb 14, 2018 at 11:32:18PM +0100, Florian Westphal wrote:
> For named sets, the no automerge makes sense because it seems like
> we can't make any reasonable default choice when users try to delete
> a no-longer existing (i.e. merged) element.
> 
> But that problem doesn't exist with constant (anon or not) sets.

OK, then this will be our default behaviour in the upcoming 0.8.3 wrt.
auto-merge:

                   simple key     mapping       timeouts
anonymous set      enabled       disabled       disabled
named set          disabled [*]  disabled       disabled

[*] can be enabled on-demand via auto-merge option.

OK?

Re: Overlapping IP networks no longer allowed?

[Mantas Mikulėnas]

On Wed, Feb 14, 2018 at 8:22 PM, Pablo Neira Ayuso <pablo@netfilter.org> wrote:
> On Wed, Feb 14, 2018 at 07:02:32PM +0200, Mantas Mikulėnas wrote:
>> Hello,
>>
>> As of nftables 0.8.1, it seems I can no longer write anonymous sets
>> which contain overlapping networks (CIDR masks).
>>
>> For example, I want to write the following ruleset:
>>
>> #!/usr/bin/nft -f
>> define users = { 10.0.0.0/8, 193.219.181.192/26 }
>> define admins = { 10.123.0.0/24, 31.220.42.129 }
>> define allowed = { $users, $admins }
>> table inet filter {
>>         chain foobar {
>>                 ip saddr $allowed accept
>>         }
>> }
>>
>> results in an error message:
>>
>>     Error: interval overlaps with previous one
>>
>> I noticed a few nftables.git commits related to disabling auto-merge
>> for interval sets... but mine don't have the 'interval' flag, and
>> there doesn't seem to be any way to specify 'auto-merge' for anonymous
>> sets, either.
>
> I would like not to enable this by default since typo in rulesets
> could go through unnoticed.
>
> So the two alternatives I see are:
>
> 1) add per-table configuration options, this would allow us to
>    enable auto-merge explicitly for all anonymous sets. This is also
>    required if we want to allow user to select "policy memory;" for
>    anonymous sets. Only problem with this approach is that this needs
>    a kernel patch, so it will take a while to restore the behaviour you
>    want since we need a new NFTA_TABLE_USERDATA attribute to store user
>    preferences on this.
>
> 2) We add a -m option that we can combine with -f for this, which
>    globally enables auto-merge for every set, including anonymous and
>    named sets.

For anonymous sets, 2) seems to make more sense – though maybe it
should be settable from within the ruleset itself (a line like "option
auto-merge;"), rather than via command line. [Just like I currently
use "flush ruleset;" rather than `nft --flush`. Also similar to how
perl prefers "use warnings;" over `perl -w`.]

But I agree with Florian's comments, and I think some of the arguments
in commit log don't make as much sense for anonymous sets as they
would for named ones; e.g. "problematic because it introduces an
inconsistency between what we add and what we later on get. This is
going to get worse with the upcoming timeout support for intervals" –
afaik, anonymous inline sets cannot be added to nor removed from (so
no timeouts).

(A fourth option would be to support a new syntax for set options, for
example "{ [automerge] $foo, $bar }" and "{ [interval, automerge]
1-10, 2, 4, 6 }"... if that makes sense internally?)

-- 
Mantas Mikulėnas