[PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[Octavian Purdila]

Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
---
 net/ipv4/fib_frontend.c |   11 ++++++-----
 1 files changed, 6 insertions(+), 5 deletions(-)

diff --git a/net/ipv4/fib_frontend.c b/net/ipv4/fib_frontend.c
index 816e218..6c1e56a 100644
--- a/net/ipv4/fib_frontend.c
+++ b/net/ipv4/fib_frontend.c
@@ -895,11 +895,11 @@ static void nl_fib_lookup_exit(struct net *net)
 	net->ipv4.fibnl = NULL;
 }
 
-static void fib_disable_ip(struct net_device *dev, int force)
+static void fib_disable_ip(struct net_device *dev, int force, int delay)
 {
 	if (fib_sync_down_dev(dev, force))
 		fib_flush(dev_net(dev));
-	rt_cache_flush(dev_net(dev), 0);
+	rt_cache_flush(dev_net(dev), delay);
 	arp_ifdown(dev);
 }
 
@@ -922,7 +922,7 @@ static int fib_inetaddr_event(struct notifier_block *this, unsigned long event,
 			/* Last address was deleted from this interface.
 			   Disable IP.
 			 */
-			fib_disable_ip(dev, 1);
+			fib_disable_ip(dev, 1, 0);
 		} else {
 			rt_cache_flush(dev_net(dev), -1);
 		}
@@ -937,7 +937,7 @@ static int fib_netdev_event(struct notifier_block *this, unsigned long event, vo
 	struct in_device *in_dev = __in_dev_get_rtnl(dev);
 
 	if (event == NETDEV_UNREGISTER) {
-		fib_disable_ip(dev, 2);
+		fib_disable_ip(dev, 2, -1);
 		return NOTIFY_DONE;
 	}
 
@@ -955,10 +955,11 @@ static int fib_netdev_event(struct notifier_block *this, unsigned long event, vo
 		rt_cache_flush(dev_net(dev), -1);
 		break;
 	case NETDEV_DOWN:
-		fib_disable_ip(dev, 0);
+		fib_disable_ip(dev, 0, 0);
 		break;
 	case NETDEV_CHANGEMTU:
 	case NETDEV_CHANGE:
+	case NETDEV_UNREGISTER_PERNET:
 		rt_cache_flush(dev_net(dev), 0);
 		break;
 	}
-- 
1.5.6.5

Re: [PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[Stephen Hemminger]

On Tue, 17 Nov 2009 01:49:49 +0200
Octavian Purdila <opurdila@ixiacom.com> wrote:

> Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
> ---
>  net/ipv4/fib_frontend.c |   11 ++++++-----
>  1 files changed, 6 insertions(+), 5 deletions(-)

And what about IPV6?


-- 

Re: [PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[Benjamin LaHaise]

On Mon, Nov 16, 2009 at 04:06:46PM -0800, Stephen Hemminger wrote:
> And what about IPV6?

It's not possible to create more than a few thousand IPv6 interfaces in 
my tests with a 4GB vm.  The culprit seems to be the per-cpu arrays for 
statistics using vmalloc() and wasting gobs of memory as a result (memory 
usage seems to be 10x per interface vs what IPv4 is using), but I haven't 
dug too deeply yet.

		-ben

Re: [PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[Octavian Purdila]

On Tuesday 17 November 2009 02:06:46 you wrote:
> On Tue, 17 Nov 2009 01:49:49 +0200
> 
> Octavian Purdila <opurdila@ixiacom.com> wrote:
> > Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>
> > ---
> >  net/ipv4/fib_frontend.c |   11 ++++++-----
> >  1 files changed, 6 insertions(+), 5 deletions(-)
> 
> And what about IPV6?
> 

IPv6 did not appear during profiling my simple dummy test and also a quick scan 
through the code did not reveal anything obvious on the unregister path which 
could be factorized.

BTW, it appears we don't have a route cache for IPv6. Right?

I known we will run into other scalability issues when we  will set IPv4/IPv6 
addresses, and that is one of the next items on my list to profile :)

Re: [PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[David Miller]

From: Octavian Purdila <opurdila@ixiacom.com>
Date: Tue, 17 Nov 2009 09:22:08 +0200

> BTW, it appears we don't have a route cache for IPv6. Right?

That's right, no IPV6 route cache.

But it doesn't seem to matter so much, as routing tables are
still small in that realm.  Although I heard the trend is for
that to change.

I also suspect that with things like policy routing we could run into
problems, but we have a multi-way tree for the IPV6 routing table
datastructure explicitly to handle those kinds of things.

Re: [PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[David Miller]

From: Benjamin LaHaise <bcrl@lhnet.ca>
Date: Mon, 16 Nov 2009 19:22:38 -0500

> It's not possible to create more than a few thousand IPv6 interfaces in 
> my tests with a 4GB vm.  The culprit seems to be the per-cpu arrays for 
> statistics using vmalloc() and wasting gobs of memory as a result (memory 
> usage seems to be 10x per interface vs what IPv4 is using), but I haven't 
> dug too deeply yet.

That definitely should be fixed.

Re: [PATCH 2/2] ipv4: factorize cache clearing for batched unregister operations

[David Miller]

From: Octavian Purdila <opurdila@ixiacom.com>
Date: Tue, 17 Nov 2009 01:49:49 +0200

> Signed-off-by: Octavian Purdila <opurdila@ixiacom.com>

Applied to net-next-2.6