Libnetfilter_conntrack problems

Libnetfilter_conntrack problems

[Sébastien LAVEZE]

Hi,
I'm developping an application using the libnetfilter_conntrack
I'm using a 2.16.1 kernel and all libraries compiled with SVN sources.

1) I'm trying to get the events relatives to the destruction of conntracks
It works fine for update events ==>nfct_open(CONNTRACK,
NF_NETLINK_CONNTRACK_UPDATE)
but not for destroy ==> nfct_open(CONNTRACK,
NF_NETLINK_CONNTRACK_DESTROY), i never get any event

2) I also would like to work with ids, when i get a conntrack via a
callback the id field is always set to zero, is it normal ?

Thanks
Sebastien

Re: Libnetfilter_conntrack problems

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 926 bytes --]

On Thu, Apr 27, 2006 at 01:55:13PM +0200, Sébastien LAVEZE wrote:
> but not for destroy ==> nfct_open(CONNTRACK,
> NF_NETLINK_CONNTRACK_DESTROY), i never get any event

are you sure?  did you wait until the respective entries really timed
out from the conntrack table?

what does 'conntrack -E -e DESTROY' say?  do you then see the events?

> 2) I also would like to work with ids, when i get a conntrack via a
> callback the id field is always set to zero, is it normal ?

IIRC, we dropped ID's recently. I'm a bit lagging behind, though.

-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Libnetfilter_conntrack problems

[Sébastien LAVEZE]

OK
I was trying to have multiple handles in different threads and it was
working only for the first created thread. I change my application in
a multi-process one and all seems to work fine
However the DESTROY events come a long time after the actual end of a
connection by timeout...am I right ?

Thanks
Sebastian

On 5/1/06, Harald Welte <laforge@netfilter.org> wrote:
> On Thu, Apr 27, 2006 at 01:55:13PM +0200, Sébastien LAVEZE wrote:
> > but not for destroy ==> nfct_open(CONNTRACK,
> > NF_NETLINK_CONNTRACK_DESTROY), i never get any event
>
> are you sure?  did you wait until the respective entries really timed
> out from the conntrack table?
>
> what does 'conntrack -E -e DESTROY' say?  do you then see the events?
>
> > 2) I also would like to work with ids, when i get a conntrack via a
> > callback the id field is always set to zero, is it normal ?
>
> IIRC, we dropped ID's recently. I'm a bit lagging behind, though.
>
> --
> - Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
> ============================================================================
>   "Fragmentation is like classful addressing -- an interesting early
>    architectural error that shows how much experimentation was going
>    on while IP was being designed."                    -- Paul Vixie
>
>
>

Re: Libnetfilter_conntrack problems

[Harald Welte]

[-- Attachment #1: Type: text/plain, Size: 1184 bytes --]

On Wed, May 03, 2006 at 11:49:04AM +0200, Sébastien LAVEZE wrote:
> OK
> I was trying to have multiple handles in different threads and it was
> working only for the first created thread. I change my application in
> a multi-process one and all seems to work fine

The libraries are not thread safe at all, and there was never any work
to make them so.  I personally (like many other netfilter developers, I
believe) never use threads in my programs, so it's unlikely that we'd be
working on that.  patches are welcome, though.

> However the DESTROY events come a long time after the actual end of a
> connection by timeout...am I right ?

not really.  please be more specific.  what is 'end of a connection by
timeout'? which specific timeout of which particular state are you
talking about?

-- 
- Harald Welte <laforge@netfilter.org>                 http://netfilter.org/
============================================================================
  "Fragmentation is like classful addressing -- an interesting early
   architectural error that shows how much experimentation was going
   on while IP was being designed."                    -- Paul Vixie

[-- Attachment #2: Type: application/pgp-signature, Size: 189 bytes --]

Re: Libnetfilter_conntrack problems

[Sébastien LAVEZE]

Hi,
> > However the DESTROY events come a long time after the actual end of a
> > connection by timeout...am I right ?
>
> not really.  please be more specific.  what is 'end of a connection by
> timeout'? which specific timeout of which particular state are you
> talking about?

My problem is pretty strange...
I got two processes, one has a callback registered to UPDATE events,
the other one has a callback registered to DESTROY event

I do get destroy events but not all the time, when i have tcp
connections that go from ESTABLISHED state to FIN_WAIT and finally to
TIME_WAIT ( i see the progression of these states with the process
linked to  UPDATE events ) But when the TIME_WAIT timer is over i do
not get the DESTROY event...

Thanks
Sebastien
These connections are forwarded connections and have NAT process
Otherwise it works great for UDP connections

Re: Libnetfilter_conntrack problems

[Pablo Neira Ayuso]

Hi,

Sébastien LAVEZE wrote:
>> > However the DESTROY events come a long time after the actual end of a
>> > connection by timeout...am I right ?
>>
>> not really.  please be more specific.  what is 'end of a connection by
>> timeout'? which specific timeout of which particular state are you
>> talking about?
> 
> 
> My problem is pretty strange...
> I got two processes, one has a callback registered to UPDATE events,
> the other one has a callback registered to DESTROY event
> 
> I do get destroy events but not all the time, when i have tcp
> connections that go from ESTABLISHED state to FIN_WAIT and finally to
> TIME_WAIT ( i see the progression of these states with the process
> linked to  UPDATE events ) But when the TIME_WAIT timer is over i do
> not get the DESTROY event...

Could you try to do the same thing with the conntrack tool and let us 
know if the problem still persists?

to catch update events
# conntrack -E -e UPDATE

to catch destroy events
# conntrack -E -e DESTROY

-- 
Pablo

Re: Libnetfilter_conntrack problems

[Sébastien LAVEZE]

Hi,
I have problem using the conntrack utility...
It always display the help message and i get the message
conntrack v1.00beta2: Missing arguments...
I'm using the latest SVN version of libnetfilet_conntrack,
libnfnetlink and conntrack and I'm using a 2.6.16.1 kernel on an ARM
platform

Sebastien

On 5/9/06, Pablo Neira Ayuso <pablo@eurodev.net> wrote:
> Hi,
>
> Sébastien LAVEZE wrote:
> >> > However the DESTROY events come a long time after the actual end of a
> >> > connection by timeout...am I right ?
> >>
> >> not really.  please be more specific.  what is 'end of a connection by
> >> timeout'? which specific timeout of which particular state are you
> >> talking about?
> >
> >
> > My problem is pretty strange...
> > I got two processes, one has a callback registered to UPDATE events,
> > the other one has a callback registered to DESTROY event
> >
> > I do get destroy events but not all the time, when i have tcp
> > connections that go from ESTABLISHED state to FIN_WAIT and finally to
> > TIME_WAIT ( i see the progression of these states with the process
> > linked to  UPDATE events ) But when the TIME_WAIT timer is over i do
> > not get the DESTROY event...
>
> Could you try to do the same thing with the conntrack tool and let us
> know if the problem still persists?
>
> to catch update events
> # conntrack -E -e UPDATE
>
> to catch destroy events
> # conntrack -E -e DESTROY
>
> --
> Pablo
>

Re: Libnetfilter_conntrack problems

[Sébastien LAVEZE]

I guess the problem is having two event handles on two different
processes and this might cause some strange behaviour. (I also have
destroy events that come 5 mn after the expiration of the TIME_WAIT
timer...)
Has someone ever tried that ??
Maybe it should be safer to have only one process listening to ALL
events and then to dispatch them..
Is there a way to make the difference between UPDATE and DESTROY
events with a single callback listening to ALL different ?? (type
field maybe ?)

Thanks
Sebastian

[PATCH] conntrack: Fix option parsing for ARM

[Philip Craig]

[-- Attachment #1: Type: text/plain, Size: 340 bytes --]

On 05/10/2006 06:18 PM, Sébastien LAVEZE wrote:
> I have problem using the conntrack utility...
> It always display the help message and i get the message
> conntrack v1.00beta2: Missing arguments...

The result of getopt_long() was assigned to a char,
which defaults to unsigned on ARM, so the comparison
against -1 always failed.

[-- Attachment #2: conntrack-getopt.patch --]
[-- Type: text/plain, Size: 408 bytes --]

Index: conntrack/src/conntrack.c
===================================================================
--- conntrack/src/conntrack.c	(revision 6601)
+++ conntrack/src/conntrack.c	(working copy)
@@ -602,7 +602,7 @@
 
 int main(int argc, char *argv[])
 {
-	char c;
+	int c;
 	unsigned int command = 0, options = 0;
 	unsigned int type = 0, event_mask = 0;
 	unsigned int l3flags = 0, l4flags = 0, metaflags = 0;

Re: Libnetfilter_conntrack problems

[Sébastien LAVEZE]

Hi,
I applied the patch that fixes ARM parsing problem
Now the conntrack utility works !
I'm tracking forwarded connection (my system does nat and forwarding)
Here are the results i get  with the conntrack utility

conntrack -E -e UPDATE (it works fine)

[UPDATE] tcp      6 60 SYN_RECV src=192.168.32.42 dst=172.16.16.16 sport=48004
dport=21 src=172.16.16.16 dst=192.168.31.39 sport=21 dport=48004
 [UPDATE] tcp      6 432000 ESTABLISHED src=192.168.32.42 dst=172.16.16.16 sport
=48004 dport=21 src=172.16.16.16 dst=192.168.31.39 sport=21
dport=48004 [ASSURED]
 [UPDATE] tcp      6 120 FIN_WAIT src=192.168.32.42 dst=172.16.16.16 sport=48004
 dport=21 src=172.16.16.16 dst=192.168.31.39 sport=21 dport=48004
 [UPDATE] tcp      6 60 CLOSE_WAIT src=192.168.32.42 dst=172.16.16.16 sport=4800
4 dport=21 src=172.16.16.16 dst=192.168.31.39 sport=21 dport=48004
 [UPDATE] tcp      6 30 LAST_ACK src=192.168.32.42 dst=172.16.16.16 sport=48004
dport=21 src=172.16.16.16 dst=192.168.31.39 sport=21 dport=48004
 [UPDATE] tcp      6 120 TIME_WAIT src=192.168.32.42 dst=172.16.16.16 sport=4800
4 dport=21 src=172.16.16.16 dst=192.168.31.39 sport=21 dport=48004

But if i do conntrack -E -e DESTROY
I never get any events for the same connection as i did with the UPDATE events

So apparently there is no problem to use multiple callbacks in
different processes but there is a problem with DESTROY events for TCP
connections

For my application it's fine as now i'm working with  connections
states instead of waiting for  destroy events

If you want me to do more tests I'm ok with that.

Thanks

Sebastien


On 5/10/06, Sébastien LAVEZE <slaveze@gmail.com> wrote:
> I guess the problem is having two event handles on two different
> processes and this might cause some strange behaviour. (I also have
> destroy events that come 5 mn after the expiration of the TIME_WAIT
> timer...)
> Has someone ever tried that ??
> Maybe it should be safer to have only one process listening to ALL
> events and then to dispatch them..
> Is there a way to make the difference between UPDATE and DESTROY
> events with a single callback listening to ALL different ?? (type
> field maybe ?)
>
> Thanks
> Sebastian
>

Re: Libnetfilter_conntrack problems

[Philip Craig]

On 05/12/2006 06:34 PM, Sébastien LAVEZE wrote:
> But if i do conntrack -E -e DESTROY
> I never get any events for the same connection as i did with the UPDATE events

It's working for me using 2.6.16 on ARM and userspace from SVN.
Are you sure you are waiting 2 minutes for the timeout?
How about if you try pings?

Re: Libnetfilter_conntrack problems

[Sébastien LAVEZE]

>It's working for me using 2.6.16 on ARM and userspace from SVN.
>Are you sure you are waiting 2 minutes for the timeout?
> How about if you try pings?

Ok
I'm also using 2.6.16 kernel with latest SVN
It works fine for LOCAL connection but not for FORWARDED connections
I do get UPDATE events for forwarded connection but there are no
DESTROY events for forwarded connections
Same stuff for pings (OK for local, NO for forward)

Re: [PATCH] conntrack: Fix option parsing for ARM

[Patrick McHardy]

Philip Craig wrote:
> On 05/10/2006 06:18 PM, Sébastien LAVEZE wrote:
> 
>>I have problem using the conntrack utility...
>>It always display the help message and i get the message
>>conntrack v1.00beta2: Missing arguments...
> 
> 
> The result of getopt_long() was assigned to a char,
> which defaults to unsigned on ARM, so the comparison
> against -1 always failed.


Thanks for tracking this down Philip. I'm having trouble using
SVN again .. Pablo, can you try to commit this please?

Re: [PATCH] conntrack: Fix option parsing for ARM

[Pablo Neira Ayuso]

Patrick McHardy wrote:
> Philip Craig wrote:
> 
>>On 05/10/2006 06:18 PM, Sébastien LAVEZE wrote:
>>
>>
>>>I have problem using the conntrack utility...
>>>It always display the help message and i get the message
>>>conntrack v1.00beta2: Missing arguments...
>>
>>
>>The result of getopt_long() was assigned to a char,
>>which defaults to unsigned on ARM, so the comparison
>>against -1 always failed.
> 
> Thanks for tracking this down Philip. I'm having trouble using
> SVN again .. Pablo, can you try to commit this please?

I'd like to but I can't, it seems that my x509 certificate expired on 
Apr 16:

         Issuer: C=DE, ST=Bayern, L=Erlangen, O=gnumonks.org, CN=Harald 
Welte/emailAddress=hostmaster@gnumonks.org
         Validity
             Not Before: Apr 16 16:55:23 2005 GMT
             Not After : Apr 16 16:55:23 2006 GMT
         Subject: C=DE, ST=Berlin, L=Berlin, O=Netfilter Project, 
OU=Development, CN=pablo/emailAddress=pablo@netfilter.org

I need a renewal, could you tell to Harald about this?

-- 
Pablo

Re: [PATCH] conntrack: Fix option parsing for ARM

[Patrick McHardy]

Pablo Neira Ayuso wrote:
>> Thanks for tracking this down Philip. I'm having trouble using
>> SVN again .. Pablo, can you try to commit this please?
> 
> 
> I'd like to but I can't, it seems that my x509 certificate expired on
> Apr 16:
> 
>         Issuer: C=DE, ST=Bayern, L=Erlangen, O=gnumonks.org, CN=Harald
> Welte/emailAddress=hostmaster@gnumonks.org
>         Validity
>             Not Before: Apr 16 16:55:23 2005 GMT
>             Not After : Apr 16 16:55:23 2006 GMT
>         Subject: C=DE, ST=Berlin, L=Berlin, O=Netfilter Project,
> OU=Development, CN=pablo/emailAddress=pablo@netfilter.org
> 
> I need a renewal, could you tell to Harald about this?


I've CCed him. My certificate is still valid, but I'm getting this
error:

svn: Commit failed (details follow):
svn: The CHECKOUT response did not contain a 'Location:' header
svn: Your commit message was left in a temporary file:

Re: Libnetfilter_conntrack problems

[Pablo Neira Ayuso]

Sébastien LAVEZE wrote:
>> It's working for me using 2.6.16 on ARM and userspace from SVN.
>> Are you sure you are waiting 2 minutes for the timeout?
>> How about if you try pings?
> 
> Ok
> I'm also using 2.6.16 kernel with latest SVN
> It works fine for LOCAL connection but not for FORWARDED connections
> I do get UPDATE events for forwarded connection but there are no
> DESTROY events for forwarded connections
> Same stuff for pings (OK for local, NO for forward)

Weird. I had problems with hardware lately, I had some time to set up
some UML stuff last night, I'll be having a look at this.

Please, could you pass me as much information as you can in order to
reproduce the problem? Thanks.

-- 
Pablo