iptables-nft-restore issue

iptables-nft-restore issue

[Arturo Borrero Gonzalez]

Hi Phil,

(CC'ing netfilter-devel)

I discovered my openstack neutron linuxbridge-agent malfunctioning when using
iptables-nft and it seems this ruleset is causing the issue:

=== 8< ===
*raw
:OUTPUT - [0:0]
:PREROUTING - [0:0]
:neutron-linuxbri-OUTPUT - [0:0]
:neutron-linuxbri-PREROUTING - [0:0]
-I OUTPUT 1 -j neutron-linuxbri-OUTPUT
-I PREROUTING 1 -j neutron-linuxbri-PREROUTING
-I neutron-linuxbri-PREROUTING 1 -m physdev --physdev-in brq7425e328-56 -m
comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
-I neutron-linuxbri-PREROUTING 2 -i brq7425e328-56 -m comment --comment "Set
zone for f101a28-1d" -j CT --zone 4097
-I neutron-linuxbri-PREROUTING 3 -m physdev --physdev-in tap7f101a28-1d -m
comment --comment "Set zone for f101a28-1d" -j CT --zone 4097

COMMIT
# Completed by iptables_manager
=== 8< ===

I'm testing current iptables git HEAD (f75750ff) and this is the diff between
iptables-nft and iptables-legacy:

=== 8< ===
arturo@endurance:~/git/netfilter/iptables master ± sudo
iptables/xtables-legacy-multi iptables-restore --verbose ~/t

Flushing chain `PREROUTING'
Flushing chain `OUTPUT'
Flushing chain `neutron-linuxbri-OUTPUT'
Flushing chain `neutron-linuxbri-PREROUTING'
Deleting chain `neutron-linuxbri-OUTPUT'
Deleting chain `neutron-linuxbri-PREROUTING'
# Completed by iptables_manager

arturo@endurance:~/git/netfilter/iptables master ± sudo
iptables/xtables-nft-multi iptables-restore --verbose ~/t

Flushing chain `PREROUTING'
Flushing chain `OUTPUT'
iptables-restore: line 12 failed
=== 8< ===

In case it helps, this is linux kernel 5.8.10 here, but I can reproduce the
issue in older kernels (4.19.132 in the case of my neutron server).

Let me know if I should open a ticket in netfilter's bugzilla, or this is
something you are already working on.

regards.

Re: iptables-nft-restore issue

[Florian Westphal]

Arturo Borrero Gonzalez <arturo@netfilter.org> wrote:
> Hi Phil,
> 
> (CC'ing netfilter-devel)
> 
> I discovered my openstack neutron linuxbridge-agent malfunctioning when using
> iptables-nft and it seems this ruleset is causing the issue:
 
> === 8< ===
> *raw
> :OUTPUT - [0:0]
> :PREROUTING - [0:0]
> :neutron-linuxbri-OUTPUT - [0:0]
> :neutron-linuxbri-PREROUTING - [0:0]
> -I OUTPUT 1 -j neutron-linuxbri-OUTPUT
> -I PREROUTING 1 -j neutron-linuxbri-PREROUTING
> -I neutron-linuxbri-PREROUTING 1 -m physdev --physdev-in brq7425e328-56 -m
> comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
> -I neutron-linuxbri-PREROUTING 2 -i brq7425e328-56 -m comment --comment "Set
> zone for f101a28-1d" -j CT --zone 4097
> -I neutron-linuxbri-PREROUTING 3 -m physdev --physdev-in tap7f101a28-1d -m
> comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
> 
> COMMIT

git bisect start
# good: [bba6bc692b0e6137e13881a1f398c134822e9f83] configure: bump
# versions for 1.8.2 release
git bisect good bba6bc692b0e6137e13881a1f398c134822e9f83
# bad: [72ed608bf1ea550ac13b5b880afc7ad3ffa0afd0] nft: Fix for broken
# address mask match detection
git bisect bad 72ed608bf1ea550ac13b5b880afc7ad3ffa0afd0
# good: [4e9782cae29034c4eefd31703ba77aee7eca2233] nft: Pass nft_handle
# to flush_cache()
git bisect good 4e9782cae29034c4eefd31703ba77aee7eca2233
# good: [f56d91bd80f0e86aaad56a32ddc84f373bb80745] connlabel: Allow
# numeric labels even if connlabel.conf exists
git bisect good f56d91bd80f0e86aaad56a32ddc84f373bb80745
# bad: [869e38fcdecda3de35d999b75fbaacc750fe3aaa] ebtables: Free
# statically loaded extensions again
git bisect bad 869e38fcdecda3de35d999b75fbaacc750fe3aaa
# good: [72470c66326d9b5186dd4614bc2d18269324e54b] nft: cache: Eliminate
# init_chain_cache()
git bisect good 72470c66326d9b5186dd4614bc2d18269324e54b
# bad: [6d1d5aa5c93eca890e28b508ef426b7844caf2b7] nft: cache: Introduce
# struct nft_cache_req
git bisect bad 6d1d5aa5c93eca890e28b508ef426b7844caf2b7
# bad: [9d07514ac5c7a27ec72df5a81bf067073d63bd99] nft: calculate cache
# requirements from list of commands
git bisect bad 9d07514ac5c7a27ec72df5a81bf067073d63bd99
# good: [accaecdf5889911e6a1ca4737c6f6599a77afe24] nft: cache: Fetch
# sets per table
git bisect good accaecdf5889911e6a1ca4737c6f6599a77afe24
# bad: [a7f1e208cdf9c6392c99d3c52764701d004bdde7] nft: split parsing
# from netlink commands
git bisect bad a7f1e208cdf9c6392c99d3c52764701d004bdde7
# good: [70a3c1a07585de64b5780a415dc157079c34911b] ebtables-restore:
# Table line to trigger implicit commit
git bisect good 70a3c1a07585de64b5780a415dc157079c34911b
# first bad commit: [a7f1e208cdf9c6392c99d3c52764701d004bdde7] nft:
# split parsing from netlink commands

Can't look at it further ATM, I double-checked that the commit
preceeding
a7f1e208cdf9c6392c99d3c52764701d004bdde7 works.

Re: iptables-nft-restore issue

[Pablo Neira Ayuso]

On Wed, Sep 30, 2020 at 01:59:22PM +0200, Florian Westphal wrote:
> Arturo Borrero Gonzalez <arturo@netfilter.org> wrote:
> > Hi Phil,
> > 
> > (CC'ing netfilter-devel)
> > 
> > I discovered my openstack neutron linuxbridge-agent malfunctioning when using
> > iptables-nft and it seems this ruleset is causing the issue:
>  
> > === 8< ===
> > *raw
> > :OUTPUT - [0:0]
> > :PREROUTING - [0:0]

If I replace these two by:

:OUTPUT ACCEPT [0:0]
:PREROUTING ACCEPT [0:0]

it works. Looks like some issue with the basechain policy?

> > :neutron-linuxbri-OUTPUT - [0:0]
> > :neutron-linuxbri-PREROUTING - [0:0]
> > -I OUTPUT 1 -j neutron-linuxbri-OUTPUT
> > -I PREROUTING 1 -j neutron-linuxbri-PREROUTING
> > -I neutron-linuxbri-PREROUTING 1 -m physdev --physdev-in brq7425e328-56 -m
> > comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
> > -I neutron-linuxbri-PREROUTING 2 -i brq7425e328-56 -m comment --comment "Set
> > zone for f101a28-1d" -j CT --zone 4097
> > -I neutron-linuxbri-PREROUTING 3 -m physdev --physdev-in tap7f101a28-1d -m
> > comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
> > 
> > COMMIT

Re: iptables-nft-restore issue

[Pablo Neira Ayuso]

[-- Attachment #1: Type: text/plain, Size: 1284 bytes --]

On Wed, Sep 30, 2020 at 02:13:14PM +0200, Pablo Neira Ayuso wrote:
> On Wed, Sep 30, 2020 at 01:59:22PM +0200, Florian Westphal wrote:
> > Arturo Borrero Gonzalez <arturo@netfilter.org> wrote:
> > > Hi Phil,
> > > 
> > > (CC'ing netfilter-devel)
> > > 
> > > I discovered my openstack neutron linuxbridge-agent malfunctioning when using
> > > iptables-nft and it seems this ruleset is causing the issue:
> >  
> > > === 8< ===
> > > *raw
> > > :OUTPUT - [0:0]
> > > :PREROUTING - [0:0]
> 
> If I replace these two by:
> 
> :OUTPUT ACCEPT [0:0]
> :PREROUTING ACCEPT [0:0]
> 
> it works. Looks like some issue with the basechain policy?

Probably this patch?

> > > :neutron-linuxbri-OUTPUT - [0:0]
> > > :neutron-linuxbri-PREROUTING - [0:0]
> > > -I OUTPUT 1 -j neutron-linuxbri-OUTPUT
> > > -I PREROUTING 1 -j neutron-linuxbri-PREROUTING
> > > -I neutron-linuxbri-PREROUTING 1 -m physdev --physdev-in brq7425e328-56 -m
> > > comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
> > > -I neutron-linuxbri-PREROUTING 2 -i brq7425e328-56 -m comment --comment "Set
> > > zone for f101a28-1d" -j CT --zone 4097
> > > -I neutron-linuxbri-PREROUTING 3 -m physdev --physdev-in tap7f101a28-1d -m
> > > comment --comment "Set zone for f101a28-1d" -j CT --zone 4097
> > > 
> > > COMMIT

[-- Attachment #2: x.patch --]
[-- Type: text/x-diff, Size: 938 bytes --]

diff --git a/iptables/nft.c b/iptables/nft.c
index 27bb98d184c7..f29fe5b47575 100644
--- a/iptables/nft.c
+++ b/iptables/nft.c
@@ -678,7 +678,9 @@ nft_chain_builtin_alloc(const struct builtin_table *table,
 	nftnl_chain_set_str(c, NFTNL_CHAIN_NAME, chain->name);
 	nftnl_chain_set_u32(c, NFTNL_CHAIN_HOOKNUM, chain->hook);
 	nftnl_chain_set_u32(c, NFTNL_CHAIN_PRIO, chain->prio);
-	nftnl_chain_set_u32(c, NFTNL_CHAIN_POLICY, policy);
+	if (policy >= 0)
+		nftnl_chain_set_u32(c, NFTNL_CHAIN_POLICY, policy);
+
 	nftnl_chain_set_str(c, NFTNL_CHAIN_TYPE, chain->type);
 
 	return c;
@@ -911,6 +913,8 @@ int nft_chain_set(struct nft_handle *h, const char *table,
 		c = nft_chain_new(h, table, chain, NF_DROP, counters);
 	else if (strcmp(policy, "ACCEPT") == 0)
 		c = nft_chain_new(h, table, chain, NF_ACCEPT, counters);
+	else if (strcmp(policy, "-") == 0)
+		c = nft_chain_new(h, table, chain, -1, counters);
 	else
 		errno = EINVAL;
 

Re: iptables-nft-restore issue

[Phil Sutter]

Hi Arturo,

On Wed, Sep 30, 2020 at 11:58:52AM +0200, Arturo Borrero Gonzalez wrote:
> I discovered my openstack neutron linuxbridge-agent malfunctioning when using
> iptables-nft and it seems this ruleset is causing the issue:

The problem is the '-' policy in builtin chains. Maybe I broke that a
while ago. I tried to come up with a fix, but it seems
iptables-legacy-restore is a bit quirky: it leaves the chain's policy
untouched, although --noflush was not given. Implementing this is a bit
problematic with how iptables-nft does the caching.

Cheers, Phil

Re: iptables-nft-restore issue

[Arturo Borrero Gonzalez]

On 2020-09-30 14:26, Pablo Neira Ayuso wrote:
> 
> Probably this patch?
> 

The patch seems to work. Let me submit it with a testcase.

Re: iptables-nft-restore issue

[Pablo Neira Ayuso]

On Fri, Oct 02, 2020 at 01:30:31PM +0200, Arturo Borrero Gonzalez wrote:
> On 2020-09-30 14:26, Pablo Neira Ayuso wrote:
> > 
> > Probably this patch?
> > 
> 
> The patch seems to work. Let me submit it with a testcase.

Go ahead.