conntrack_ftp and DNAT

conntrack_ftp and DNAT

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hello,

allow me to ask a question about conntrack and nf_conntrack_ftp and
nf_nat_ftp and DNAT.

I have a host where I do DNAT from the main IPv4 address to the backend
ftp server. Currently I have the server data connections limited to a
small port range and have a hard wired DNAT of that port range to the
ftp server too. But this is not optimal as that port range is not open
for other connections.

I wanted to replace that with conntrack but failed.

Here are the relevant entries in iptables:
iptables -t raw -A PREROUTING -p tcp -m tcp --dport 21 -j CT --helper ftp
iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp -m tcp --dport 20:21 -j DNAT --to-destination 10.0.0.3
iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp -m tcp --dport 52100:52199 -j DNAT --to-destination 10.0.0.3
iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m tcp --dport 52100:52199 -m helper --helper ftp -j ACCEPT
iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m tcp --dport 52100:52199 -j ACCEPT

The second last rule never gets any hit (That is the reason for the last
rule).

I inserted several logging rules to see if a package gets conntrack
marked by the ftp helper but for me it seems that this is never the
case.

I have a similar setup on a client firewall where conntrack works well.
(Using MASQUERADE in nat table POSTROUTING.) I checked that and the
packages gets really handled by the ftp helper.

On both systems I loaded nf_conntrack_ftp and nf_nat_ftp kernel module.

I would really like to drop the last rule above and use bigger port
range for the ftp server. But I have no idea what to do to get that
working.

Allow be also to say that this helper stuff is somehow underdocumented
and could be improved a bit more. ;-)

Regards
   Klaus

Ps. On both systems, I use kernel 4.8.15 from devuan.
PPs. Please include me in responses as I am not subscribed to the list.
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlifiq4ACgkQpnwKsYAZ
9qxwUQwAp5K/ln1ErgEIAocb0gyiwlURXFoMJPpmzuJUUn3PDez8AK05uXMLxAVY
zUTRZ6vN3qfWoIeG8pEjGtX9eecKgysBm2Xmt5k9TdkjhuV8Vp3wnZyOiuXgV0NH
B5/FhmLYsAjpXxzwD9EwyLO39+N2qR5ykxlj6aQc282m9fT/Te58A/VrPht2sgiy
lx4bWTVw8SL9Ba+eQRKDyICSD+Dl7kIhJJuU+4vVev0Gx+H3/HyisKeNU15fX+Ux
asgRI0bMVUo+YFZz8uMOhnu8jY5IxGtHXxajivM3ekWlGTS/9y4JBntTfJEi8C5i
8HOvAT0mQRN9EPwSiVANWMU62C21lCc77nE43p4JmrY7ME0iXl7FuXFVqQxETEwR
pRVPOgXr9+KcMXXMHVCZKE6NR00tmQ6xhK4EyS3OmmbTyp900okZC9kBr+Kdzx0s
Ry2PKCA8Ne6KBB77ZIY33TcIhuqn7nNfs/DN59Um39Wp3/rcedfbGNEEd4yb2axD
rZPAaXt7
=N5Ju
-----END PGP SIGNATURE-----

Re: conntrack_ftp and DNAT

[Florian Westphal]

Klaus Ethgen <Klaus+lkml@ethgen.de> wrote:
> allow me to ask a question about conntrack and nf_conntrack_ftp and
> nf_nat_ftp and DNAT.
> 
> I have a host where I do DNAT from the main IPv4 address to the backend
> ftp server. Currently I have the server data connections limited to a
> small port range and have a hard wired DNAT of that port range to the
> ftp server too. But this is not optimal as that port range is not open
> for other connections.
> 
> I wanted to replace that with conntrack but failed.
> 
> Here are the relevant entries in iptables:
> iptables -t raw -A PREROUTING -p tcp -m tcp --dport 21 -j CT --helper ftp
> iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp -m tcp --dport 20:21 -j DNAT --to-destination 10.0.0.3
> iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp -m tcp --dport 52100:52199 -j DNAT --to-destination 10.0.0.3
> iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m tcp --dport 52100:52199 -m helper --helper ftp -j ACCEPT
> iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m tcp --dport 52100:52199 -j ACCEPT
> 
> The second last rule never gets any hit (That is the reason for the last
> rule).
>
> I inserted several logging rules to see if a package gets conntrack
> marked by the ftp helper but for me it seems that this is never the
> case.

Strange, and I can't tell you why its not working.
This should be enough:

# make sure ftp helper checks standard control connection
iptables -t raw -A PREROUTING -p tcp --dport 21 -j CT --helper ftp

# nat control connection
iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp --dport 21 -j DNAT --to-destination 10.0.0.3

# accept all traffic handled by ftp helper)
iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m helper --helper ftp -j ACCEPT

The common 'not working' reasons are:
1. control traffic gets encrypted (i.e. tls extension is used)
2. ftp server uses foreign (non-local) ip addresses in PORT command
   (this needs fixing of ftp server or use of 'loose' mode, see modinfo
    nf_conntrack_ftp)

you seem to be using a bridge, maybe there is some bug w.
call-iptables...  I can have a look next week.

> Allow be also to say that this helper stuff is somehow underdocumented
> and could be improved a bit more. ;-)

Yes :-(

Re: conntrack_ftp and DNAT

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi Florian,

Am Do den 16. Feb 2017 um  1:17 schrieb Florian Westphal:
> Klaus Ethgen <Klaus+lkml@ethgen.de> wrote:
[Contrack and DNAT]
> > Here are the relevant entries in iptables:
> > iptables -t raw -A PREROUTING -p tcp -m tcp --dport 21 -j CT --helper ftp
> > iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp -m tcp --dport 20:21 -j DNAT --to-destination 10.0.0.3
> > iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp -m tcp --dport 52100:52199 -j DNAT --to-destination 10.0.0.3
> > iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m tcp --dport 52100:52199 -m helper --helper ftp -j ACCEPT
> > iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m tcp --dport 52100:52199 -j ACCEPT
[...]
> This should be enough:
> 
> # make sure ftp helper checks standard control connection
> iptables -t raw -A PREROUTING -p tcp --dport 21 -j CT --helper ftp
> 
> # nat control connection
> iptables -t nat -A PREROUTING -d X.X.X.X/32 -p tcp --dport 21 -j DNAT --to-destination 10.0.0.3
> 
> # accept all traffic handled by ftp helper)
> iptables -A FORWARD -d 10.0.0.3/32 -o br0 -p tcp -m helper --helper ftp -j ACCEPT

As I understood it, that is also my opinion. And on my client box, this
works this way.

> The common 'not working' reasons are:
> 1. control traffic gets encrypted (i.e. tls extension is used)

I do not see why. Although I have configured TLS, there are little
clients out there that uses this.

And my tls log is empty at all.

Even more, if I do ftp from my client box, where I have a similar setup,
just with MASQUERADE (SNAT), there the helper work but in the same time
not on the server where I do DNAT.

> 2. ftp server uses foreign (non-local) ip addresses in PORT command
>    (this needs fixing of ftp server or use of 'loose' mode, see modinfo
>     nf_conntrack_ftp)

It is a standard proftpd with the following relevant settings:
   PassivePorts 52100 52199
   MasqueradeAddress X.X.X.X

Where X.X.X.X is the outside IP.

> you seem to be using a bridge, maybe there is some bug w.
> call-iptables...  I can have a look next week.

Yes. This is using a bridge that bridges between the master and its KVM
hosts.

How can I check or rule out this?

Regards
   Klaus
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlilYIYACgkQpnwKsYAZ
9qz5fgv/f62dxwSxMz9O8Df8Xlvmy/HDUP0HHRjJeEjoZ4WhbEMaAfiiiESk4zuK
793OjGV/vipmcICqQc0uwCeovlmu5TtpklTjhzY441TghBjlm8rawpYCuhp6X1m2
atRXTou94N6Brx2nsOHXwNODp+Ys2AMfapXBCRckBg9G8EjqAYil5V+JWAqiG5sQ
eOwNnZmJ3k+3oCOirXDZUwE9w5uoENBmlUx4cIAks01dtTXUtbaB+Dwr7S7pwm85
mBn7u6F0XG1jN9ppRKriMnDsAYPTridcDRQwPtPR2hAdLkb9Mm8dXVCxNrqB1AiE
906UvD8G3bAOBUz3/rOJlEDWAlu7LKZP/Yu/bHO0ylflZHKU5tnR3tWYER5Aj/hm
snZEgLlykrk08GxmEZtkOgLHd35INajek0rNTOmgBqpvUMYerXGSLjV3qDd1oboF
R8/RixKe1+iYHje5pio+1R1OLVuJ641tW/TNOuouyAAiMK0pZM4wjVuYnllEfS08
hk5+dvtp
=tkLj
-----END PGP SIGNATURE-----

Re: conntrack_ftp and DNAT

[Florian Westphal]

Klaus Ethgen <Klaus+lkml@ethgen.de> wrote:
> > 2. ftp server uses foreign (non-local) ip addresses in PORT command
> >    (this needs fixing of ftp server or use of 'loose' mode, see modinfo
> >     nf_conntrack_ftp)
> 
> It is a standard proftpd with the following relevant settings:
>    PassivePorts 52100 52199
>    MasqueradeAddress X.X.X.X
> Where X.X.X.X is the outside IP.

Try without the "MasqueradeAddress".

The netfilter conntrack helper by default only
installs expectations for PORT addresses that match the originating
ip of the control connection.

If the server already uses the to-be-natted-to address in the
control connection the helper will ignore them unless you enable
"loose" tracking (see modinfo above).

Doing that has security implications (which can be worked around
by restricting related matches like this:

-s (address) -d (address) -m conntrack --ctstate RELATED -m helper
--helper ftp -j ACCEPT
-p tcp -m conntrack --ctstate RELATED -j DROP

> > you seem to be using a bridge, maybe there is some bug w.
> > call-iptables...  I can have a look next week.
> 
> Yes. This is using a bridge that bridges between the master and its KVM
> hosts.
> 
> How can I check or rule out this?

I guess its caused by ProFtp masquerade setting.

Re: conntrack_ftp and DNAT

[Klaus Ethgen]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi Florian,

Am Do den 16. Feb 2017 um  9:41 schrieb Florian Westphal:
> Klaus Ethgen <Klaus+lkml@ethgen.de> wrote:
> > > 2. ftp server uses foreign (non-local) ip addresses in PORT command
> > >    (this needs fixing of ftp server or use of 'loose' mode, see modinfo
> > >     nf_conntrack_ftp)
> > 
> > It is a standard proftpd with the following relevant settings:
> >    PassivePorts 52100 52199
> >    MasqueradeAddress X.X.X.X
> > Where X.X.X.X is the outside IP.
> 
> Try without the "MasqueradeAddress".
> 
> The netfilter conntrack helper by default only
> installs expectations for PORT addresses that match the originating
> ip of the control connection.

That did it.

Thanks for the help and clarifying.

> If the server already uses the to-be-natted-to address in the
> control connection the helper will ignore them unless you enable
> "loose" tracking (see modinfo above).
> 
> Doing that has security implications (which can be worked around
> by restricting related matches like this:
> 
> -s (address) -d (address) -m conntrack --ctstate RELATED -m helper
> --helper ftp -j ACCEPT
> -p tcp -m conntrack --ctstate RELATED -j DROP

I do not need it right now but will have that in mind when I need it,
thanks.

By the way, the best documentation for that is [0]. Maybe that can be
consolidated to a more visible place.

Regards
   Klaus

[0] https://home.regit.org/netfilter-en/secure-use-of-helpers/
- -- 
Klaus Ethgen                                       http://www.ethgen.ch/
pub  4096R/4E20AF1C 2011-05-16            Klaus Ethgen <Klaus@Ethgen.ch>
Fingerprint: 85D4 CA42 952C 949B 1753  62B3 79D0 B06F 4E20 AF1C
-----BEGIN PGP SIGNATURE-----
Comment: Charset: ISO-8859-1

iQGzBAEBCgAdFiEEMWF28vh4/UMJJLQEpnwKsYAZ9qwFAlillhEACgkQpnwKsYAZ
9qxixQv9Emi/i5dcOZCN31JL204hOp2iWkqVJ1VGfGlyHu7nElu1djSfpUv++AHG
fUbU6sxjJsDQqVJzCSLPDTnDy+3foj4MGX8g9Rqe1ZkKovWoUZxO57fvZZXNz413
qYjsw0ZuefmD9QOqRFoxhcrdHAdxuZ1s5G8WQjoGsmPdfh8q1fW963Ksu0x0HYOB
0o16hTedhtJBIBflAGQUbrCleXJy43WMgV6UY7bzwLe8Mo0Kax9us8EkdKuJcJQI
OEIuHAxTdk6fLkOpEsYORGRUSP20kbH6CZB5D2SFC/QLIz1PmkOF6XJfkUepWgE+
sEFwFZ1eHs+7C2ddY1QXNC8VZpfK71HwGAG6I38ikEug2zW5f03wJeGk/k8eW+3b
r9llQDFCef1skPlKUM9t8I2WLuik6CbvntQSgwLQ1EaJgeocCU9i/WFpVYXGd+gw
XOcrk1Ulu+m9JozMWulOTJoTUCTQ9pCJXV8/ojv9BuvogvTTczwAneruBH5ZPgWz
8TNAjItX
=UW70
-----END PGP SIGNATURE-----