nf_conntrack: falling back to vmalloc.

nf_conntrack: falling back to vmalloc.

[Toralf Förster]

I do run a server with a 64 bit hardened Gentoo Linux (kernel currently 4.0.8).
Around 12th of July it started to spew those messages into kern.log :

/var/log/kern.log:Jul 12 15:26:07 tor-relay kernel: [538360.650490] nf_conntrack: falling back to vmalloc.
/var/log/kern.log:Jul 12 15:26:07 tor-relay kernel: [538360.650615] nf_conntrack: falling back to vmalloc.
/var/log/kern.log:Jul 12 15:26:08 tor-relay kernel: [538361.673649] nf_conntrack: falling back to vmalloc.
/var/log/kern.log:Jul 12 15:26:08 tor-relay kernel: [538361.673786] nf_conntrack: falling back to vmalloc.

I read https://bugzilla.openvz.org/show_bug.cgi?id=3092 but a reboot did not help.
I got 1-2 thousends of those lines per day.

Any hints ?

-- 
Toralf, pgp key: 872AE508 0076E94E

Re: nf_conntrack: falling back to vmalloc.

[Florian Westphal]

Toralf Förster <toralf.foerster@gmx.de> wrote:
> I do run a server with a 64 bit hardened Gentoo Linux (kernel currently 4.0.8).
> Around 12th of July it started to spew those messages into kern.log :
> 
> /var/log/kern.log:Jul 12 15:26:07 tor-relay kernel: [538360.650490] nf_conntrack: falling back to vmalloc.
> /var/log/kern.log:Jul 12 15:26:07 tor-relay kernel: [538360.650615] nf_conntrack: falling back to vmalloc.
> /var/log/kern.log:Jul 12 15:26:08 tor-relay kernel: [538361.673649] nf_conntrack: falling back to vmalloc.
> /var/log/kern.log:Jul 12 15:26:08 tor-relay kernel: [538361.673786] nf_conntrack: falling back to vmalloc.
> 
> I read https://bugzilla.openvz.org/show_bug.cgi?id=3092 but a reboot did not help.
> I got 1-2 thousends of those lines per day.

Most likely result of 88eab472ec21f01d3e36ff ("netfilter: conntrack:
adjust nf_conntrack_buckets default value".

Do you run containers?

This message can only be printed when a new network namespace is created
(or something is rmmod/modprobing nf_conntrack module all the time).

I wonder if this is caused by some program creating netns for
sandboxing?

Pablo, Patrick -- any idea on how to stop conntrack from becoming active
in a newly created netns automatically without breaking anything?

With upcoming per netns hooks, we might be able to delay registering
conntrack, defrag etc.  until after a -m conntrack rule has been added.
Dou you think that could work?

For nft we could create an expression to configure conntrack explicitly
(inverse NOTRACK).

[ obviously we can also add that for xtables but that would break
  setups if we suddently move to "you must ask for conntrack via
  ruleset" model. ]
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack: falling back to vmalloc.

[Toralf Förster]

On 07/17/2015 12:10 PM, Florian Westphal wrote:
> Do you run containers?

No, I just run 4 Gentoo chroot images in parallel - but I do that since autumn last year.

Unfortunately I cannot see any correlation to an os/sw upgrade or a config change which could cause these messages nowadays to be appear.
Therefore I do wonder if some network related "traffic" causes this, b/c that box acts as a Tor exit relay too.


-- 
Toralf, pgp key: 872AE508 0076E94E

Re: nf_conntrack: falling back to vmalloc.

[Florian Westphal]

Toralf Förster <toralf.foerster@gmx.de> wrote:
> On 07/17/2015 12:10 PM, Florian Westphal wrote:
> > Do you run containers?
> 
> No, I just run 4 Gentoo chroot images in parallel - but I do that since autumn last year.
> 
> Unfortunately I cannot see any correlation to an os/sw upgrade or a config change which could cause these messages nowadays to be appear.
> Therefore I do wonder if some network related "traffic" causes this, b/c that box acts as a Tor exit relay too.

No, I see no way how this would happen in response to network traffic.

A conntrack hash table is only allocated on network namespace creation, or when
a resize happens (someone changing /sys/module/nf_conntrack/parameters/hashsize).
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: nf_conntrack: falling back to vmalloc.

[Pablo Neira Ayuso]

[-- Attachment #1: Type: text/plain, Size: 1893 bytes --]

On Fri, Jul 17, 2015 at 12:10:09PM +0200, Florian Westphal wrote:
> Toralf Förster <toralf.foerster@gmx.de> wrote:
> > I do run a server with a 64 bit hardened Gentoo Linux (kernel currently 4.0.8).
> > Around 12th of July it started to spew those messages into kern.log :
> > 
> > /var/log/kern.log:Jul 12 15:26:07 tor-relay kernel: [538360.650490] nf_conntrack: falling back to vmalloc.
> > /var/log/kern.log:Jul 12 15:26:07 tor-relay kernel: [538360.650615] nf_conntrack: falling back to vmalloc.
> > /var/log/kern.log:Jul 12 15:26:08 tor-relay kernel: [538361.673649] nf_conntrack: falling back to vmalloc.
> > /var/log/kern.log:Jul 12 15:26:08 tor-relay kernel: [538361.673786] nf_conntrack: falling back to vmalloc.
> > 
> > I read https://bugzilla.openvz.org/show_bug.cgi?id=3092 but a reboot did not help.
> > I got 1-2 thousends of those lines per day.
> 
> Most likely result of 88eab472ec21f01d3e36ff ("netfilter: conntrack:
> adjust nf_conntrack_buckets default value".

I think we can get rid of that warning, it will most likely shown
every time after that patch.

[...]
> Pablo, Patrick -- any idea on how to stop conntrack from becoming active
> in a newly created netns automatically without breaking anything?
>
> With upcoming per netns hooks, we might be able to delay registering
> conntrack, defrag etc.  until after a -m conntrack rule has been added.
> Dou you think that could work?
>
> For nft we could create an expression to configure conntrack explicitly
> (inverse NOTRACK).
> 
> [ obviously we can also add that for xtables but that would break
>   setups if we suddently move to "you must ask for conntrack via
>   ruleset" model. ]

With netns in place, I think it's now clear that we need that this is
configurable at ruleset level that we've been discussing for a while.
But users will have to request this new behaviour to avoid breaking
existing setups...

[-- Attachment #2: x.patch --]
[-- Type: text/x-diff, Size: 603 bytes --]

diff --git a/net/netfilter/nf_conntrack_core.c b/net/netfilter/nf_conntrack_core.c
index 13fad86..9b7dc11 100644
--- a/net/netfilter/nf_conntrack_core.c
+++ b/net/netfilter/nf_conntrack_core.c
@@ -1522,10 +1522,8 @@ void *nf_ct_alloc_hashtable(unsigned int *sizep, int nulls)
 	sz = nr_slots * sizeof(struct hlist_nulls_head);
 	hash = (void *)__get_free_pages(GFP_KERNEL | __GFP_NOWARN | __GFP_ZERO,
 					get_order(sz));
-	if (!hash) {
-		printk(KERN_WARNING "nf_conntrack: falling back to vmalloc.\n");
+	if (!hash)
 		hash = vzalloc(sz);
-	}
 
 	if (hash && nulls)
 		for (i = 0; i < nr_slots; i++)