per flow stats collection using libnetfilter_conntrack

per flow stats collection using libnetfilter_conntrack

[Tarun Khanna]

Is it possible to register for callbacks so that a user application is 
updated on a regular basis when the packet or bytes counts change for 
flows? I can poll the stats using something similar to "conntrack -L". 
However, I was wondering if it's possible to be notified regularly on 
updates.

Re: per flow stats collection using libnetfilter_conntrack

[Florian Westphal]

Tarun Khanna <tkhanna@akamai.com> wrote:
> Is it possible to register for callbacks so that a user application is
> updated on a regular basis when the packet or bytes counts change for flows?

No, update to packet/byte count doesn't trigger an event (it would
trigger too often).

> I can poll the stats using something similar to "conntrack -L". However, I
> was wondering if it's possible to be notified regularly on updates.

No.  What semantics would you be interested in?

Dumping is quite fast, whats the problem you need to solve?

Re: per flow stats collection using libnetfilter_conntrack

[Tarun Khanna]

On 02/27/2017 05:12 PM, Florian Westphal wrote:
> Tarun Khanna <tkhanna@akamai.com> wrote:
>> Is it possible to register for callbacks so that a user application is
>> updated on a regular basis when the packet or bytes counts change for flows?
> No, update to packet/byte count doesn't trigger an event (it would
> trigger too often).

I was hoping if there was an option to set a timeout, so that an update 
would trigger every 5 seconds for flows where the counters have changed.

>
>> I can poll the stats using something similar to "conntrack -L". However, I
>> was wondering if it's possible to be notified regularly on updates.
> No.  What semantics would you be interested in?
>
> Dumping is quite fast, whats the problem you need to solve?
I have an application that needs to keep track of per flow stats for all 
tcp flows destined to port 80. The best way I have found so far is to 
call nfct_query regularly and to go through each flow returned to 
retrieve the counters. Would that be the best way to do it?

Thank you for your assistance.

Re: per flow stats collection using libnetfilter_conntrack

[Florian Westphal]

Tarun Khanna <tkhanna@akamai.com> wrote:
> On 02/27/2017 05:12 PM, Florian Westphal wrote:
> >Tarun Khanna <tkhanna@akamai.com> wrote:
> >>Is it possible to register for callbacks so that a user application is
> >>updated on a regular basis when the packet or bytes counts change for flows?
> >No, update to packet/byte count doesn't trigger an event (it would
> >trigger too often).
> 
> I was hoping if there was an option to set a timeout, so that an update
> would trigger every 5 seconds for flows where the counters have changed.

No, there is no such feature at the moment.

> I have an application that needs to keep track of per flow stats for all tcp
> flows destined to port 80. The best way I have found so far is to call
> nfct_query regularly and to go through each flow returned to retrieve the
> counters. Would that be the best way to do it?

Its your only option right now.  If you need a periodic update feature
we'll need to think about how to best make this.

We now have a periodic gc worker in conntrack so we could probably do
this without re-adding a per-conntrack timer.

Re: per flow stats collection using libnetfilter_conntrack

[Tarun Khanna]

> Tarun Khanna <tkhanna@akamai.com> wrote:
>> On 02/27/2017 05:12 PM, Florian Westphal wrote:
>>> Tarun Khanna <tkhanna@akamai.com> wrote:
>>>> Is it possible to register for callbacks so that a user application is
>>>> updated on a regular basis when the packet or bytes counts change for flows?
>>> No, update to packet/byte count doesn't trigger an event (it would
>>> trigger too often).
>> I was hoping if there was an option to set a timeout, so that an update
>> would trigger every 5 seconds for flows where the counters have changed.
> No, there is no such feature at the moment.
>
>> I have an application that needs to keep track of per flow stats for all tcp
>> flows destined to port 80. The best way I have found so far is to call
>> nfct_query regularly and to go through each flow returned to retrieve the
>> counters. Would that be the best way to do it?
> Its your only option right now.  If you need a periodic update feature
> we'll need to think about how to best make this.
>
> We now have a periodic gc worker in conntrack so we could probably do
> this without re-adding a per-conntrack timer.

Thank you. The current way of polling works for me. I was thinking about 
it and it doesn't really make a difference if the library sends me 
updates periodically or if I check for them because in both cases I'll 
need to setup some event on epoll (either a timer or the netfilter fd). 
So, functionally both would be the same. Thanks again for your help.