Rocky Linux 9 with firewalld and nftables always tracks connections

Rocky Linux 9 with firewalld and nftables always tracks connections

[Blaine Elzey]

I enabled the public zone and the dns service (udp/tcp port 53).  Firewalld service is running and nftables service is not.  The connection tracking table gets full on busy servers, and drops packets.  In prior Linux (RHEL7) I set iptables rules via firewalld direct configuration file.  The same rules are not working now.  I set dns to listen on a 2nd port (5353), and tried 2 nft commands:

  # nft insert rule inet firewalld mangle_PREROUTING udp dport 5353 notrack meta mark set 53
  # nft insert rule inet firewalld filter_IN_public_allow mark 5353 accept

All the rules are automatically created by default firewalld config, based on the specified zone and services, except the 2 rules added with nft, above.

The mark is passed through the chains, as the port is allowed based on the packet mark, but the conntrack output shows "mark=0", even though I don't think the packets should even be in the conntrack table.

This does allow the dns client to get a response, but the dns connections are still tracked (included in /proc/net/nfconntrack).

Spent hours searching online with many hits, but none of them work.  It's like the connections are always added to the conntrack table, no matter what.  Hopefully someone has used a similar config, or can provide how to set specific packets to successfully notrack in a similar environment (detailed below).

# uname -a
Linux qip1250re-2 5.14.0-284.30.1.el9_2.x86_64 #1 SMP PREEMPT_DYNAMIC Sat Sep 16 09:55:41 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
# cat /etc/rocky-release
Rocky Linux release 9.2 (Blue Onyx)
# rpm -qa|grep firewalld
firewalld-filesystem-1.2.1-1.el9.noarch
firewalld-1.2.1-1.el9.noarch
# rpm -qa|grep nftables
nftables-1.0.4-10.el9_1.x86_64
python3-nftables-1.0.4-10.el9_1.x86_64
# cat /proc/net/nf_conntrack
ipv4     2 udp      17 28 src=192.168.1.2 dst=192.168.1.1 sport=43160 dport=5353 src=192.168.1.1 dst=135.114.104.251 sport=5353 dport=43160 mark=0 zone=0 use=2
# cat /etc/firewalld/zones/public.xml
<?xml version="1.0" encoding="utf-8"?>
<zone>
  <short>Public</short>
  <description>For use in public areas. You do not trust the other computers on networks to not harm your computer. Only selected incoming connections are accepted.</description>
  <service name="ssh"/>
  <service name="DNS"/>
  <forward/>
</zone>
# cat /etc/firewalld/services/dns.xml
<?xml version="1.0" encoding="utf-8"?>
<service>
  <short>DNS</short>
  <description>The Domain Name System (DNS) is used to provide and request host and domain names. Enable this option, if you plan to provide a domain name service (e.g. with bind).</description>
  <port protocol="tcp" port="53"/>
  <port protocol="udp" port="53"/>
</service>
# nft list ruleset
table inet firewalld {
        chain mangle_PREROUTING {
                type filter hook prerouting priority mangle + 10; policy accept;
                udp dport 5353 notrack meta mark set 0x00000035
                jump mangle_PREROUTING_ZONES
        }

        chain mangle_PREROUTING_POLICIES_pre {
                jump mangle_PRE_policy_allow-host-ipv6
        }

        chain mangle_PREROUTING_ZONES {
                iifname "eth0" goto mangle_PRE_public
                goto mangle_PRE_public
        }

        chain mangle_PREROUTING_POLICIES_post {
        }

        chain nat_PREROUTING {
                type nat hook prerouting priority dstnat + 10; policy accept;
                jump nat_PREROUTING_ZONES
        }

        chain nat_PREROUTING_POLICIES_pre {
                jump nat_PRE_policy_allow-host-ipv6
        }

        chain nat_PREROUTING_ZONES {
                iifname "eth0" goto nat_PRE_public
                goto nat_PRE_public
        }

        chain nat_PREROUTING_POLICIES_post {
        }

        chain nat_POSTROUTING {
                type nat hook postrouting priority srcnat + 10; policy accept;
                jump nat_POSTROUTING_ZONES
        }

        chain nat_POSTROUTING_POLICIES_pre {
        }

        chain nat_POSTROUTING_ZONES {
                oifname "eth0" goto nat_POST_public
                goto nat_POST_public
        }

        chain nat_POSTROUTING_POLICIES_post {
        }

        chain nat_OUTPUT {
                type nat hook output priority -90; policy accept;
                jump nat_OUTPUT_POLICIES_pre
                jump nat_OUTPUT_POLICIES_post
        }

        chain nat_OUTPUT_POLICIES_pre {
        }

        chain nat_OUTPUT_POLICIES_post {
        }

        chain filter_PREROUTING {
                type filter hook prerouting priority filter + 10; policy accept;
                icmpv6 type { nd-router-advert, nd-neighbor-solicit } accept
                meta nfproto ipv6 fib saddr . mark . iif oif missing log prefix "rpfilter_DROP: " drop
        }

        chain filter_INPUT {
                type filter hook input priority filter + 10; policy accept;
                ct state { established, related } accept
                ct status dnat accept
                ct state invalid log prefix "STATE_INVALID_DROP: "
                ct state invalid drop
                iifname "lo" accept
                jump filter_INPUT_ZONES
                log prefix "FINAL_REJECT: "
                reject with icmpx admin-prohibited
        }

        chain filter_FORWARD {
                type filter hook forward priority filter + 10; policy accept;
                ct state { established, related } accept
                ct status dnat accept
                ct state invalid log prefix "STATE_INVALID_DROP: "
                ct state invalid drop
                iifname "lo" accept
                ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } log prefix "RFC3964_IPv4_REJECT: " reject with icmpv6 addr-unreachable
                jump filter_FORWARD_ZONES
                log prefix "FINAL_REJECT: "
                reject with icmpx admin-prohibited
        }

        chain filter_OUTPUT {
                type filter hook output priority filter + 10; policy accept;
                ct state { established, related } accept
                oifname "lo" accept
                ip6 daddr { ::/96, ::ffff:0.0.0.0/96, 2002::/24, 2002:a00::/24, 2002:7f00::/24, 2002:a9fe::/32, 2002:ac10::/28, 2002:c0a8::/32, 2002:e000::/19 } log prefix "RFC3964_IPv4_REJECT: " reject with icmpv6 addr-unreachable
                jump filter_OUTPUT_POLICIES_pre
                jump filter_OUTPUT_POLICIES_post
        }

        chain filter_INPUT_POLICIES_pre {
                jump filter_IN_policy_allow-host-ipv6
        }

        chain filter_INPUT_ZONES {
                iifname "eth0" goto filter_IN_public
                goto filter_IN_public
        }

        chain filter_INPUT_POLICIES_post {
        }

        chain filter_FORWARD_POLICIES_pre {
        }

        chain filter_FORWARD_ZONES {
                iifname "eth0" goto filter_FWD_public
                goto filter_FWD_public
        }

        chain filter_FORWARD_POLICIES_post {
        }

        chain filter_OUTPUT_POLICIES_pre {
        }

        chain filter_OUTPUT_POLICIES_post {
        }

        chain filter_IN_public {
                jump filter_INPUT_POLICIES_pre
                jump filter_IN_public_pre
                jump filter_IN_public_log
                jump filter_IN_public_deny
                jump filter_IN_public_allow
                jump filter_IN_public_post
                jump filter_INPUT_POLICIES_post
                meta l4proto { icmp, ipv6-icmp } accept
                log prefix "filter_IN_public_REJECT: "
                reject with icmpx admin-prohibited
        }

        chain filter_IN_public_pre {
        }

        chain filter_IN_public_log {
        }

        chain filter_IN_public_deny {
        }

        chain filter_IN_public_allow {
                meta mark 0x00000035 accept
                tcp dport 22 ct state { new, untracked } accept
                udp dport 53 ct state { new, untracked } accept
                tcp dport 53 ct state { new, untracked } accept
        }

        chain filter_IN_public_post {
        }

        chain nat_POST_public {
                jump nat_POSTROUTING_POLICIES_pre
                jump nat_POST_public_pre
                jump nat_POST_public_log
                jump nat_POST_public_deny
                jump nat_POST_public_allow
                jump nat_POST_public_post
                jump nat_POSTROUTING_POLICIES_post
        }

        chain nat_POST_public_pre {
        }

        chain nat_POST_public_log {
        }

        chain nat_POST_public_deny {
        }

        chain nat_POST_public_allow {
        }

        chain nat_POST_public_post {
        }

        chain filter_FWD_public {
                jump filter_FORWARD_POLICIES_pre
                jump filter_FWD_public_pre
                jump filter_FWD_public_log
                jump filter_FWD_public_deny
                jump filter_FWD_public_allow
                jump filter_FWD_public_post
                jump filter_FORWARD_POLICIES_post
                log prefix "filter_FWD_public_REJECT: "
                reject with icmpx admin-prohibited
        }

        chain filter_FWD_public_pre {
        }

        chain filter_FWD_public_log {
        }

        chain filter_FWD_public_deny {
        }

        chain filter_FWD_public_allow {
                oifname "eth0" accept
        }

        chain filter_FWD_public_post {
        }

        chain nat_PRE_public {
                jump nat_PREROUTING_POLICIES_pre
                jump nat_PRE_public_pre
                jump nat_PRE_public_log
                jump nat_PRE_public_deny
                jump nat_PRE_public_allow
                jump nat_PRE_public_post
                jump nat_PREROUTING_POLICIES_post
        }

        chain nat_PRE_public_pre {
        }

        chain nat_PRE_public_log {
        }

        chain nat_PRE_public_deny {
        }

        chain nat_PRE_public_allow {
        }

        chain nat_PRE_public_post {
        }

        chain mangle_PRE_public {
                jump mangle_PREROUTING_POLICIES_pre
                jump mangle_PRE_public_pre
                jump mangle_PRE_public_log
                jump mangle_PRE_public_deny
                jump mangle_PRE_public_allow
                jump mangle_PRE_public_post
                jump mangle_PREROUTING_POLICIES_post
        }

        chain mangle_PRE_public_pre {
        }

        chain mangle_PRE_public_log {
        }

        chain mangle_PRE_public_deny {
        }

        chain mangle_PRE_public_allow {
        }

        chain mangle_PRE_public_post {
        }

        chain filter_IN_policy_allow-host-ipv6 {
                jump filter_IN_policy_allow-host-ipv6_pre
                jump filter_IN_policy_allow-host-ipv6_log
                jump filter_IN_policy_allow-host-ipv6_deny
                jump filter_IN_policy_allow-host-ipv6_allow
                jump filter_IN_policy_allow-host-ipv6_post
        }

        chain filter_IN_policy_allow-host-ipv6_pre {
        }

        chain filter_IN_policy_allow-host-ipv6_log {
        }

        chain filter_IN_policy_allow-host-ipv6_deny {
        }

        chain filter_IN_policy_allow-host-ipv6_allow {
                icmpv6 type nd-neighbor-advert accept
                icmpv6 type nd-neighbor-solicit accept
                icmpv6 type nd-router-advert accept
                icmpv6 type nd-redirect accept
        }

        chain filter_IN_policy_allow-host-ipv6_post {
        }

        chain nat_PRE_policy_allow-host-ipv6 {
                jump nat_PRE_policy_allow-host-ipv6_pre
                jump nat_PRE_policy_allow-host-ipv6_log
                jump nat_PRE_policy_allow-host-ipv6_deny
                jump nat_PRE_policy_allow-host-ipv6_allow
                jump nat_PRE_policy_allow-host-ipv6_post
        }

        chain nat_PRE_policy_allow-host-ipv6_pre {
        }

        chain nat_PRE_policy_allow-host-ipv6_log {
        }

        chain nat_PRE_policy_allow-host-ipv6_deny {
        }

        chain nat_PRE_policy_allow-host-ipv6_allow {
        }

        chain nat_PRE_policy_allow-host-ipv6_post {
        }

        chain mangle_PRE_policy_allow-host-ipv6 {
                jump mangle_PRE_policy_allow-host-ipv6_pre
                jump mangle_PRE_policy_allow-host-ipv6_log
                jump mangle_PRE_policy_allow-host-ipv6_deny
                jump mangle_PRE_policy_allow-host-ipv6_allow
                jump mangle_PRE_policy_allow-host-ipv6_post
        }

        chain mangle_PRE_policy_allow-host-ipv6_pre {
        }

        chain mangle_PRE_policy_allow-host-ipv6_log {
        }

        chain mangle_PRE_policy_allow-host-ipv6_deny {
        }

        chain mangle_PRE_policy_allow-host-ipv6_allow {
        }

        chain mangle_PRE_policy_allow-host-ipv6_post {
        }
}
#

TIA,
Blaine

Re: Rocky Linux 9 with firewalld and nftables always tracks connections

[Kerin Millar]

Hi,

On Sat, 30 Mar 2024 05:08:04 +0000
Blaine Elzey <Blaine.Elzey@cygnalabs.com> wrote:

> I enabled the public zone and the dns service (udp/tcp port 53).  Firewalld service is running and nftables service is not.  The connection tracking table gets full on busy servers, and drops packets.  In prior Linux (RHEL7) I set iptables rules via firewalld direct configuration file.  The same rules are not working now.  I set dns to listen on a 2nd port (5353), and tried 2 nft commands:
> 
>   # nft insert rule inet firewalld mangle_PREROUTING udp dport 5353 notrack meta mark set 53
>   # nft insert rule inet firewalld filter_IN_public_allow mark 5353 accept

You go on to show that the mangle_PREROUTING table is defined in the following manner.

chain mangle_PREROUTING {
	type filter hook prerouting priority mangle + 10; policy accept;
}

As concerns the use of notrack, let's consider what the latest nft(8) manual has to say.

"Note that for this statement to be effective, it has to be applied to packets before a conntrack lookup happens. Therefore, it needs to sit in a chain with either prerouting or output hook and a hook priority of -300 (raw) or less."

You have not fulfilled this requirement because your priority level is define as -140 (-150 + 10). Below is an example of how one might correctly employ notrack. It presumes that the DNS server is running at the same host as bears the ruleset.

table inet raw {
	chain PREROUTING {
		type filter hook prerouting priority raw; policy accept;
		# Match packets destined to the DNS server (excluding packets that are to be forwarded).
		fib daddr type local udp dport 5353 notrack
	}

	chain OUTPUT {
		type filter hook output priority raw; policy accept;
		# Match packets locally generated by the DNS server.
		udp sport 5353 notrack
		# Not keeping state for "lo" can also be an effective way of conserving resources.
		oifname "lo" notrack
	}
}

Should you wish to constrain your use of the conntrack table further still, you could potentially go as far as to refrain from using conntrack at all for packets generated by - and destined to - the firewall, instead reserving the use of conntrack for forwarding duty.

-- 
Kerin Millar

RE: Rocky Linux 9 with firewalld and nftables always tracks connections

[Blaine Elzey]

From: Kerin Millar <kfm@plushkava.net> 
Sent: Saturday, March 30, 2024 2:28 AM

Hi,

> You have not fulfilled this requirement because your priority level is define as -140 (-150 + 10).  Below is an
> example of how one might correctly employ notrack. It presumes that the DNS server is running at the same
> host as bears the ruleset.

I had read that and tried this and many variations that did not work.  Below I tried to create your suggested chains, but also failed. Some good news is that this led me to a working solution.

Suggested solution denied traffic:

# nft add table inet raw
# nft add chain inet raw PREROUTING { type filter hook prerouting priority -300 \;policy accept\;}
# nft add rule inet raw PREROUTING fib daddr type local udp dport 5353 notrack
# nft add chain inet raw OUTPUT { type filter hook output priority -300 \;policy accept\;}
# nft add rule inet raw OUTPUT udp sport 5353 notrack
# nft add rule inet raw OUTPUT oifname "lo" notrack

The resulting new table (just the new table shown, the firewalld table is the same, except without the 2 mark rules I added in my initial post):

table inet raw {
        chain PREROUTING {
                type filter hook prerouting priority raw; policy accept;
                fib daddr type local udp dport 5353 notrack
        }

        chain OUTPUT {
                type filter hook output priority raw; policy accept;
                udp sport 5353 notrack
                oifname "lo" notrack
        }
}

The chains in the firewalld table that are different from my original post (no " udp dport 5353 notrack meta mark set 0x00000035" in mangle_PREROUTING or "meta mark 0x00000035 accept" in filter_IN_public_allow):
table inet firewalld {
        chain mangle_PREROUTING {
                type filter hook prerouting priority mangle + 10; policy accept;
                jump mangle_PREROUTING_ZONES
        }
        chain filter_IN_public_allow {
                tcp dport 22 ct state { new, untracked } accept
                udp dport 53 ct state { new, untracked } accept
                tcp dport 53 ct state { new, untracked } accept

Testing results:

From client (192.168.1.2):

# dig @192.168.1.1 +timeout=1 +retries=1 1.0.0.127.in-addr.arpa ptr +short -p 553

; <<>> DiG 9.16.23-RH <<>> @192.168.1.1 +timeout +retries 1.0.0.127.in-addr.arpa ptr +short -p 53
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

# dig @192.168.1.1 +timeout=1 +retries=1 1.0.0.127.in-addr.arpa ptr +short -p 553

; <<>> DiG 9.16.23-RH <<>> @192.168.1.1 +timeout +retries 1.0.0.127.in-addr.arpa ptr +short -p 53
; (1 server found)
;; global options: +cmd
;; connection timed out; no servers could be reached

On system with DNS server and nftables config, there were no connections:
# cat /proc/net/nf_conntrack|grep 192\.168\.1\.2

But the log_denied messages on the DNS server:

Mar 30 14:26:55 qip1250re-2 kernel: filter_IN_public_REJECT: IN=eth0 OUT= MAC=c4:5a:b1:b2:95:a4:c4:5a:b1:a5:ea:1d:08:00 SRC=192.168.1.2 DST=192.168.1.1 LEN=91 TOS=0x00 PREC=0x00 TTL=64 ID=26795 PROTO=UDP SPT=53369 DPT=53 LEN=71
Mar 30 14:26:56 qip1250re-2 kernel: filter_IN_public_REJECT: IN=eth0 OUT= MAC=c4:5a:b1:b2:95:a4:c4:5a:b1:a5:ea:1d:08:00 SRC=192.168.1.2 DST=192.168.1.1 LEN=91 TOS=0x00 PREC=0x00 TTL=64 ID=27658 PROTO=UDP SPT=57512 DPT=53 LEN=71

Looks like the filter_IN_public is still denying the traffic (so I don't see the connections like in my initial post where traffic is allowed):

        chain filter_IN_public {
                jump filter_INPUT_POLICIES_pre
                jump filter_IN_public_pre
                jump filter_IN_public_log
                jump filter_IN_public_deny
                jump filter_IN_public_allow
                jump filter_IN_public_post
                jump filter_INPUT_POLICIES_post
                meta l4proto { icmp, ipv6-icmp } accept
                log prefix "filter_IN_public_REJECT: "
                reject with icmpx admin-prohibited
        }

To get the traffic accepted, but not tracked:

Flush the ruleset and restart firewalld
# nft flush ruleset
# systemctl restart firewalld

Run nft commands to setup the suggested new raw table and add the extra input chain accept:
# nft add table inet raw
# nft add chain inet raw PREROUTING { type filter hook prerouting priority -300 \;policy accept\;}
# nft add rule inet raw PREROUTING fib daddr type local udp dport 5353 notrack meta mark set 53
# nft add chain inet raw OUTPUT { type filter hook output priority -300 \;policy accept\;}
# nft add rule inet raw OUTPUT udp sport 5353 notrack
# nft add rule inet raw OUTPUT oifname "lo" notrack
# nft insert rule inet firewalld filter_IN_public_allow mark 53 accept

Resulting new table and updated chain:
table inet raw {
        chain PREROUTING {
                type filter hook prerouting priority raw; policy accept;
                fib daddr type local udp dport 5353 notrack meta mark set 0x00000035
        }

        chain OUTPUT {
                type filter hook output priority raw; policy accept;
                udp sport 5353 notrack
                oifname "lo" notrack
        }
}
table inet firewalld {
        chain filter_IN_public_allow {
                meta mark 0x00000035 accept
                tcp dport 22 ct state { new, untracked } accept
                udp dport 53 ct state { new, untracked } accept
                tcp dport 53 ct state { new, untracked } accept
        }

On the Client, both authoritative and recursive queries are answered:
# dig @192.168.1.1 +timeout=1 +retries=1 1.0.0.127.in-addr.arpa ptr +nocmd +short -p 53
localhost.
# dig @192.168.1.1 +short www.google.com
142.250.80.36

On system with DNS server and nftables config, there were no connections for DNS:
# cat /proc/net/nf_conntrack|grep 192\.168\.1\.2

There are no logged reject/denied messages for DNS packets on the DNS server either

I thank you very much for the pointers to successfully configuring notrack of the DNS packets.

Now that I know the underlying nftables configuration, I need to incorporate this configuration into the firewalld configuration.  I understand if this is off-topic and I may need to post in a different forum.

I see that firewalld says that direct rules is deprecated, and to use policies.

Unfortunately policies do not support notrack, and the necessary arguments that I have tried in direct rules are invalid.  I have also come to find that the firewall direct rules only support iptables, while my firewalld.conf is set to use nftables; thus, unable to see nftables tables and chains setup by firewalld.  Maybe the nftable rules from firewalld override the iptables rules from firewalld direct?

At the moment, I created some workaround scripts from the information in this post, and call the in the systemd  ExecStartPre= for DNS service start.  I remove them in ExecStopPost=.

Cheers,

Blaine

Re: Rocky Linux 9 with firewalld and nftables always tracks connections

[Kerin Millar]

On Sun, 31 Mar 2024, at 12:22 AM, Blaine Elzey wrote:

[...]

> Now that I know the underlying nftables configuration, I need to 
> incorporate this configuration into the firewalld configuration.  I 
> understand if this is off-topic and I may need to post in a different 
> forum.
>
> I see that firewalld says that direct rules is deprecated, and to use policies.

How very helpful of it.

>
> Unfortunately policies do not support notrack, and the necessary 
> arguments that I have tried in direct rules are invalid.  I have also 
> come to find that the firewall direct rules only support iptables, 
> while my firewalld.conf is set to use nftables; thus, unable to see 
> nftables tables and chains setup by firewalld.  Maybe the nftable rules 
> from firewalld override the iptables rules from firewalld direct?

Rocky 9 offers iptables-nft, which uses nftables as a backend while continuing to support xtables extensions. Unfortunately, it cannot be relied upon to translate the syntax of a given iptables(8) rule to a native nft(8) rule, even for those rules where it is technically possible. Consider the following.

# nft flush ruleset
# iptables -V
iptables v1.8.10 (nf_tables)
# iptables -t raw -A PREROUTING -j CT --notrack
# nft list ruleset
# Warning: table ip raw is managed by iptables-nft, do not touch!
table ip raw {
	chain PREROUTING {
		type filter hook prerouting priority raw; policy accept;
		counter packets 0 bytes 0 xt target "CT"
	}
}

This shows iptables electing to use the CT extension from xtables, rather than produce the native, equivalent nftables rule. Consequently, one ends up with a hybrid ruleset that can no longer be safely managed with nft alone.

So, should you wish to try the firewall.direct(5) syntax, be sure to define "FirewallBackend=iptables". Further, in each case that the backend is switched, be sure to stop the service and run "nft flush ruleset" before starting it again.

>
> At the moment, I created some workaround scripts from the information 
> in this post, and call the in the systemd  ExecStartPre= for DNS 
> service start.  I remove them in ExecStopPost=.

I fear that this might prove brittle. Presumably, firewalld can be instructed to interact with the ruleset in such a way that the necessary rules are lost, well after the systemd service has been started. Still, I can think of nothing else, other than to forgo the use of firewalld altogether.

-- 
Kerin Millar

RE: Rocky Linux 9 with firewalld and nftables always tracks connections

[Blaine Elzey]

Thanks for your time and suggestions.

--
Blaine
This message is confidential.  The information is intended for use only by the individual(s) or entity named above. If you are not the intended recipient, please notify the sender immediately by e-mail and be aware that any disclosure, copying, distribution or use of the contents of this e-mail is strictly prohibited. Cygna Labs monitors its e-mail system and may record your e-mails. Cygna Labs Corp, 1111 Lincoln Rd., Suite 760, Miami Beach, FL 33139


-----Original Message-----
From: Kerin Millar <kfm@plushkava.net> 
Sent: Sunday, March 31, 2024 1:40 AM
To: Blaine Elzey <Blaine.Elzey@cygnalabs.com>; netfilter@vger.kernel.org
Subject: Re: Rocky Linux 9 with firewalld and nftables always tracks connections

On Sun, 31 Mar 2024, at 12:22 AM, Blaine Elzey wrote:

[...]

> Now that I know the underlying nftables configuration, I need to 
> incorporate this configuration into the firewalld configuration.  I 
> understand if this is off-topic and I may need to post in a different 
> forum.
>
> I see that firewalld says that direct rules is deprecated, and to use policies.

How very helpful of it.

>
> Unfortunately policies do not support notrack, and the necessary 
> arguments that I have tried in direct rules are invalid.  I have also 
> come to find that the firewall direct rules only support iptables, 
> while my firewalld.conf is set to use nftables; thus, unable to see 
> nftables tables and chains setup by firewalld.  Maybe the nftable 
> rules from firewalld override the iptables rules from firewalld direct?

Rocky 9 offers iptables-nft, which uses nftables as a backend while continuing to support xtables extensions. Unfortunately, it cannot be relied upon to translate the syntax of a given iptables(8) rule to a native nft(8) rule, even for those rules where it is technically possible. Consider the following.

# nft flush ruleset
# iptables -V
iptables v1.8.10 (nf_tables)
# iptables -t raw -A PREROUTING -j CT --notrack # nft list ruleset # Warning: table ip raw is managed by iptables-nft, do not touch!
table ip raw {
	chain PREROUTING {
		type filter hook prerouting priority raw; policy accept;
		counter packets 0 bytes 0 xt target "CT"
	}
}

This shows iptables electing to use the CT extension from xtables, rather than produce the native, equivalent nftables rule. Consequently, one ends up with a hybrid ruleset that can no longer be safely managed with nft alone.

So, should you wish to try the firewall.direct(5) syntax, be sure to define "FirewallBackend=iptables". Further, in each case that the backend is switched, be sure to stop the service and run "nft flush ruleset" before starting it again.

>
> At the moment, I created some workaround scripts from the information 
> in this post, and call the in the systemd  ExecStartPre= for DNS 
> service start.  I remove them in ExecStopPost=.

I fear that this might prove brittle. Presumably, firewalld can be instructed to interact with the ruleset in such a way that the necessary rules are lost, well after the systemd service has been started. Still, I can think of nothing else, other than to forgo the use of firewalld altogether.

--
Kerin Millar