Dropped fin acks (iptables + lvs)

Dropped fin acks (iptables + lvs)

[Patrik Karén]

[-- Attachment #1: Type: text/plain, Size: 2143 bytes --]

Hi!

I am running iptables and lvs on two boxes loadbalancing http[s] and ssh traffic to two real servers.
Everything is working just fine from the users point of view. However, I keep seeing a lot of dropped packets of type ack/fin and ack/rst in my iptables log. Seems like the connection tracking isn't working the way I expect it to. The iptables config in short is:

$IPTABLES -P INPUT DROP
$IPTABLES -P OUTPUT ACCEPT
$IPTABLES -P FORWARD DROP
$IPTABLES -N Firewall-INPUT
$IPTABLES -A INPUT -j Firewall-INPUT
$IPTABLES -A FORWARD -j Firewall-INPUT
#This is the rule that should allow established connections, right?
$IPTABLES -A Firewall-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
#The next rule allows everything from the inside. Since the above rule doesn't seem to work
#all replies from the webservers to the clients will be dropped if this rule is not in place.
$IPTABLES -A Firewall-INPUT -i eth1 -j ACCEPT
$IPTABLES -A Firewall-INPUT -d $VIP1_e -p tcp -m state --state NEW -m tcp --dport 80 -j ACCEPT
$IPTABLES -A Firewall-INPUT -d $VIP1_e -p tcp -m state --state NEW -m tcp --dport 443 -j ACCEPT
$IPTABLES -A Firewall-INPUT -m limit --limit $LOGLIMIT --limit-burst $LOGLIMITBURST -j LOG --log-level debug --log-prefix "drop: "
$IPTABLES -A Firewall-INPUT -j DROP

And in the log I get lots this for each user session: 
Jan 24 16:46:11 10.0.1.107 kernel: drop: IN=eth0 OUT= MAC=00:15:c5:ee:48:a7:00:04:de:18:18:00:08:00 SRC=<CLIENTIP> DST=<$VIP1_e> LEN=52 TOS=0x00 PREC=0x00 TTL=49 ID=28407 PROTO=TCP SPT=48404 DPT=443 WINDOW=65535 RES=0x00 ACK FIN URGP=0

Why? Is there something about the connection tracking I'm not understanding?
If I do a 'cat /proc/net/ip_conntrack' on the director/fw, shouldn't I see connections between my external VIP and the clients IP? All I see there are connections between the director/fw and my webservers.

Any help is would be much appreciated.

Regards,
Patrik

Om du är singel och vill träffa någon, besök då Spray Date! På Spray Date finns det 500 000 glada singlar som bara längtar efter att träffa någon alldeles speciell. http://spraydate.spray.se/

Re: Dropped fin acks (iptables + lvs)

[Jan Engelhardt]

>I am running iptables and lvs on two boxes loadbalancing http[s] and ssh traffic to two real servers.
>Everything is working just fine from the users point of view. However, 
>I keep seeing a lot of dropped packets of type ack/fin and ack/rst in 
>my iptables log. Seems like the connection tracking isn't working the 
>way I expect it to. The iptables config in short is:

RST-ACK is received as a response to SYN to a closed port, and hence, is 
not part of a connection.

>#This is the rule that should allow established connections, right?
>$IPTABLES -A Firewall-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT

>Jan 24 16:46:11 10.0.1.107 kernel: drop: IN=eth0 OUT= 
>MAC=00:15:c5:ee:48:a7:00:04:de:18:18:00:08:00 SRC=<CLIENTIP> 
>DST=<$VIP1_e> LEN=52 TOS=0x00 PREC=0x00 TTL=49 ID=28407 PROTO=TCP 
>SPT=48404 DPT=443 WINDOW=65535 RES=0x00 ACK FIN URGP=0

The FIN-ACK case however looks worth looking into. I'd say do it without 
-m limit and see if _every_ connection ends up that way. Also use 
tcpdump to match sessions.


	-`J'
-- 

Re: Dropped fin acks (iptables + lvs)

[Patrik Karén]

Jan Engelhardt skrev:
>> I am running iptables and lvs on two boxes loadbalancing http[s] and ssh traffic to two real servers.
>> Everything is working just fine from the users point of view. However, 
>> I keep seeing a lot of dropped packets of type ack/fin and ack/rst in 
>> my iptables log. Seems like the connection tracking isn't working the 
>> way I expect it to. The iptables config in short is:
>>     
>
> RST-ACK is received as a response to SYN to a closed port, and hence, is 
> not part of a connection.
>
>   
>> #This is the rule that should allow established connections, right?
>> $IPTABLES -A Firewall-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
>>     
>
>   
>> Jan 24 16:46:11 10.0.1.107 kernel: drop: IN=eth0 OUT= 
>> MAC=00:15:c5:ee:48:a7:00:04:de:18:18:00:08:00 SRC=<CLIENTIP> 
>> DST=<$VIP1_e> LEN=52 TOS=0x00 PREC=0x00 TTL=49 ID=28407 PROTO=TCP 
>> SPT=48404 DPT=443 WINDOW=65535 RES=0x00 ACK FIN URGP=0
>>     
>
> The FIN-ACK case however looks worth looking into. I'd say do it without 
> -m limit and see if _every_ connection ends up that way. Also use 
> tcpdump to match sessions.
>
>
> 	-`J'
>   
Yes, the FIN-ACKs are the ones that bother me. There are lots of them, 
but I don't know if they occur for every single tcp session.
I'm going to do some tcpdumping tomorrow on different interfaces to see 
if I can find a pattern.

//Patrik

Re: Dropped fin acks (iptables + lvs)

[Pascal Hambourg]

Hello,

Jan Engelhardt a écrit :
>>I am running iptables and lvs on two boxes loadbalancing http[s] and ssh traffic to two real servers.
>>Everything is working just fine from the users point of view. However, 
>>I keep seeing a lot of dropped packets of type ack/fin and ack/rst in 
>>my iptables log. Seems like the connection tracking isn't working the 
>>way I expect it to.
> 
> RST-ACK is received as a response to SYN to a closed port, and hence, is 
> not part of a connection.

At Netfilter connection tracking level, ACK/RST in response to SYN is 
part of a connection and is supposed to be in the ESTABLISHED state, 
even though at TCP level the connection is not established.

Re: Dropped fin acks (iptables + lvs)

[Klaas Jan Wierenga]

Hi Patrik,

Have you gotten any further with the "Dropped fin acks (iptables +  
lvs)" problem?I'm experiencing similar problems, but even worse, some  
of my long HTTP streaming sessions are disconnected because LVS or  
Netfilter decides to send "ICMP host unreachable" on an existing and  
active connection.

See: http://archive.linuxvirtualserver.org/html/lvs-users/2007-03/ 
msg00057.html

Could my problem be related to your problem?

Regards,

Klaas Jan Wierenga