ip_conntrack & timing out of connections

ip_conntrack & timing out of connections

[Lehmann]

linux-2.4.13-ac5 (other versions untested) has this peculiar behaviour: If I
"killall -STOP thttpd", I, of course, still get connection requests which
usually time out:

tcp      238      0 217.227.148.85:80       213.76.191.129:3120 CLOSE_WAIT  
tcp      162      0 217.227.148.85:80       213.76.191.129:3128 CLOSE_WAIT  
tcp      238      0 217.227.148.85:80       213.76.191.129:3136 CLOSE_WAIT  
tcp      162      0 217.227.148.85:80       213.76.191.129:3152 CLOSE_WAIT  
tcp      134      0 217.227.148.85:80       66.42.121.15:3305 CLOSE_WAIT  
tcp      162      0 217.227.148.85:80       213.76.191.129:3160 CLOSE_WAIT  
tcp      279      0 217.227.148.85:80       62.83.11.19:2742 CLOSE_WAIT  

however, after some time, I get many of these messages:

Nov  6 02:39:55 doom kernel: ip_conntrack: table full, dropping packet. 

/proc/net/ip_conntrack has lots of connections like these:

tcp      6 430665 ESTABLISHED src=213.76.191.129 dst=217.227.148.85 sport=3881 dport=80 src=217.227.148.85 dst=213.76.191.129 sport=80 dport=388 1 [ASSURED] use=1 

that is, connections to port 80. a grep dport=80 in ip_conntrack gives me
3768 lines, where netstat -t only shows 159 connections, so it seems that
conntrack has a problems with time-outs (or something similar).

-- 
      -----==-                                             |
      ----==-- _                                           |
      ---==---(_)__  __ ____  __       Marc Lehmann      +--
      --==---/ / _ \/ // /\ \/ /       pcg@goof.com      |e|
      -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
    The choice of a GNU generation                       |
                                                         |

Re: ip_conntrack & timing out of connections

[Rasmus Bøg Hansen]

On Tue, 6 Nov 2001 pcg@goof.com wrote:

> Nov  6 02:39:55 doom kernel: ip_conntrack: table full, dropping packet. 

You probably need to do something like:

# We need a lot of concurrent connections
echo 65536 > /proc/sys/net/ipv4/ip_conntrack_max

(or how many you will need). Be aware  that it will use up more memory - 
the netfilter docs can tell you how much.

Rasmus

-- 
-- [ Rasmus 'Møffe' Bøg Hansen ] ---------------------------------------
If you only have a hammer
everything looks like a nail
--------------------------------- [ moffe at amagerkollegiet dot dk ] --

Re: ip_conntrack & timing out of connections

[David Lang]

the tcp dimeout is 60 seconds and the ip_conntrack timeout is 120 seconds.

I ran into this myself a couple days ago and have sent the info to Rusty
and to the netfilter mailing list (although only to the list this morning
so I don't know if there is a response there yet)

David Lang

On Tue, 6 Nov 2001 pcg@goof.com wrote:

> Date: Tue, 6 Nov 2001 12:19:47 +0100
> From: pcg@goof.com
> To: linux-kernel@vger.kernel.org
> Subject: ip_conntrack & timing out of connections
>
> linux-2.4.13-ac5 (other versions untested) has this peculiar behaviour: If I
> "killall -STOP thttpd", I, of course, still get connection requests which
> usually time out:
>
> tcp      238      0 217.227.148.85:80       213.76.191.129:3120 CLOSE_WAIT
> tcp      162      0 217.227.148.85:80       213.76.191.129:3128 CLOSE_WAIT
> tcp      238      0 217.227.148.85:80       213.76.191.129:3136 CLOSE_WAIT
> tcp      162      0 217.227.148.85:80       213.76.191.129:3152 CLOSE_WAIT
> tcp      134      0 217.227.148.85:80       66.42.121.15:3305 CLOSE_WAIT
> tcp      162      0 217.227.148.85:80       213.76.191.129:3160 CLOSE_WAIT
> tcp      279      0 217.227.148.85:80       62.83.11.19:2742 CLOSE_WAIT
>
> however, after some time, I get many of these messages:
>
> Nov  6 02:39:55 doom kernel: ip_conntrack: table full, dropping packet.
>
> /proc/net/ip_conntrack has lots of connections like these:
>
> tcp      6 430665 ESTABLISHED src=213.76.191.129 dst=217.227.148.85 sport=3881 dport=80 src=217.227.148.85 dst=213.76.191.129 sport=80 dport=388 1 [ASSURED] use=1
>
> that is, connections to port 80. a grep dport=80 in ip_conntrack gives me
> 3768 lines, where netstat -t only shows 159 connections, so it seems that
> conntrack has a problems with time-outs (or something similar).
>
> --
>       -----==-                                             |
>       ----==-- _                                           |
>       ---==---(_)__  __ ____  __       Marc Lehmann      +--
>       --==---/ / _ \/ // /\ \/ /       pcg@goof.com      |e|
>       -=====/_/_//_/\_,_/ /_/\_\       XX11-RIPE         --+
>     The choice of a GNU generation                       |
>                                                          |
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
>

Re: ip_conntrack & timing out of connections

[kuznet]

Hello!

> the tcp dimeout is 60 seconds and the ip_conntrack timeout is 120 seconds.

This is absolutely different case.


> > From: pcg@goof.com
...
> > linux-2.4.13-ac5 (other versions untested) has this peculiar behaviour: If I
> > "killall -STOP thttpd", I, of course, still get connection requests which
> > usually time out:
> >
> > tcp      238      0 217.227.148.85:80       213.76.191.129:3120 CLOSE_WAIT

Blatant lie. Such connections cannot timeout. If they do, kernel really
have fatal bug.


> > Nov  6 02:39:55 doom kernel: ip_conntrack: table full, dropping packet.
> >
> > /proc/net/ip_conntrack has lots of connections like these:
> >
> > tcp      6 430665 ESTABLISHED src=213.76.191.129 dst=217.227.148.85 sport=3881 dport=80 src=217.227.148.85 dst=213.76.191.129 sport=80 dport=388 1 [ASSURED] use=1

It is absolutely right. Established connections must not timeout.

Alexey

Re: ip_conntrack & timing out of connections

[Trever L. Adams]

On Wed, 2001-11-07 at 13:55, kuznet@ms2.inr.ac.ru wrote:
> > > From: pcg@goof.com
> ...
> > > linux-2.4.13-ac5 (other versions untested) has this peculiar behaviour: If I
> > > "killall -STOP thttpd", I, of course, still get connection requests which
> > > usually time out:
> > >
> > > tcp      238      0 217.227.148.85:80       213.76.191.129:3120 CLOSE_WAIT
> 
> Blatant lie. Such connections cannot timeout. If they do, kernel really
> have fatal bug.
> 

Then the kernel (iptables or what not) has a huge fatal bug.  I have
seen this happen as well.  The firewall then catches all of these 'ACK
FIN' etc.  This is getting more rare for me and usually takes a moderate
to heavy load (for link capacity) before it starts happening, but it
does happen.

Trever

Re: ip_conntrack & timing out of connections

[Harald Welte]

On Tue, Nov 06, 2001 at 12:19:47PM +0100,  Marc A. Lehmann  wrote:

> however, after some time, I get many of these messages:
> 
> Nov  6 02:39:55 doom kernel: ip_conntrack: table full, dropping packet. 
> 
> /proc/net/ip_conntrack has lots of connections like these:
> 
> tcp      6 430665 ESTABLISHED src=213.76.191.129 dst=217.227.148.85 sport=3881 dport=80 src=217.227.148.85 dst=213.76.191.129 sport=80 dport=388 1 [ASSURED] use=1 
> 
> that is, connections to port 80. a grep dport=80 in ip_conntrack gives me
> 3768 lines, where netstat -t only shows 159 connections, so it seems that
> conntrack has a problems with time-outs (or something similar).

connection tracking keeps all TCP conntrack entries for 120 seconds after
completion of FIN <-> FIN closedown.  This is the TIME_WAIT state of the
tcp protocol.

Maybe the linux tcp stack doesn't wait for 120 seconds, or some other 
condition in the tcp stack makes the sockets disappear from the netstat -t
list.

-- 
Live long and prosper
- Harald Welte / laforge@gnumonks.org               http://www.gnumonks.org/
============================================================================
GCS/E/IT d- s-: a-- C+++ UL++++$ P+++ L++++$ E--- W- N++ o? K- w--- O- M- 
V-- PS+ PE-- Y+ PGP++ t++ 5-- !X !R tv-- b+++ DI? !D G+ e* h+ r% y+(*)