[PATCH net-next-2.6] macvlan: remove one synchronize_rcu() call

[PATCH net-next-2.6] macvlan: remove one synchronize_rcu() call

[Eric Dumazet]

When one macvlan device is dismantled, we can avoid one
synchronize_rcu() call done after deletion from hash list, since caller
will perform a synchronize_net() call after its ndo_stop() call.

Add a new netdev->dismantle field to signal this dismantle intent.

Reduces RTNL hold time.

Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>
CC: Patrick McHardy <kaber@trash.net>
CC: Ben Greear <greearb@candelatech.com>
---
 drivers/net/macvlan.c     |    9 +++++----
 include/linux/netdevice.h |    4 +++-
 net/core/dev.c            |    2 +-
 3 files changed, 9 insertions(+), 6 deletions(-)

diff --git a/drivers/net/macvlan.c b/drivers/net/macvlan.c
index d7c0bc62..07bcb80 100644
--- a/drivers/net/macvlan.c
+++ b/drivers/net/macvlan.c
@@ -70,16 +70,17 @@ static void macvlan_hash_add(struct macvlan_dev *vlan)
 	hlist_add_head_rcu(&vlan->hlist, &port->vlan_hash[addr[5]]);
 }
 
-static void macvlan_hash_del(struct macvlan_dev *vlan)
+static void macvlan_hash_del(struct macvlan_dev *vlan, bool sync)
 {
 	hlist_del_rcu(&vlan->hlist);
-	synchronize_rcu();
+	if (sync)
+		synchronize_rcu();
 }
 
 static void macvlan_hash_change_addr(struct macvlan_dev *vlan,
 					const unsigned char *addr)
 {
-	macvlan_hash_del(vlan);
+	macvlan_hash_del(vlan, true);
 	/* Now that we are unhashed it is safe to change the device
 	 * address without confusing packet delivery.
 	 */
@@ -345,7 +346,7 @@ static int macvlan_stop(struct net_device *dev)
 	dev_uc_del(lowerdev, dev->dev_addr);
 
 hash_del:
-	macvlan_hash_del(vlan);
+	macvlan_hash_del(vlan, !dev->dismantle);
 	return 0;
 }
 
diff --git a/include/linux/netdevice.h b/include/linux/netdevice.h
index a134d80..ca333e7 100644
--- a/include/linux/netdevice.h
+++ b/include/linux/netdevice.h
@@ -1293,7 +1293,9 @@ struct net_device {
 	       NETREG_UNREGISTERED,	/* completed unregister todo */
 	       NETREG_RELEASED,		/* called free_netdev */
 	       NETREG_DUMMY,		/* dummy device for NAPI poll */
-	} reg_state:16;
+	} reg_state:8;
+
+	bool dismantle; /* device is going do be freed */
 
 	enum {
 		RTNL_LINK_INITIALIZED,
diff --git a/net/core/dev.c b/net/core/dev.c
index 155de20..d945379 100644
--- a/net/core/dev.c
+++ b/net/core/dev.c
@@ -5126,7 +5126,7 @@ static void rollback_registered_many(struct list_head *head)
 			list_del(&dev->unreg_list);
 			continue;
 		}
-
+		dev->dismantle = true;
 		BUG_ON(dev->reg_state != NETREG_REGISTERED);
 	}
 

Re: [PATCH net-next-2.6] macvlan: remove one synchronize_rcu() call

[Ben Greear]

On 05/19/2011 03:24 PM, Eric Dumazet wrote:
> When one macvlan device is dismantled, we can avoid one
> synchronize_rcu() call done after deletion from hash list, since caller
> will perform a synchronize_net() call after its ndo_stop() call.
>
> Add a new netdev->dismantle field to signal this dismantle intent.

I applied this to today's wireless-testing kernel.  There is a consistent
speedup in deleting mac-vlans!  I wouldn't read much into changes in
creating macvlans or adding IPs..those numbers just jump around a bit
from run to run.

Before the patch:

[root@lec2010-ath9k-1 lanforge]# /mnt/b32/greearb/tmp/test_macvlans.pl 500 macvlan
Creating 500 macvlan.
Created 500 macvlan in 12.662865 seconds (0.02532573 per interface).
Added IP addresses in 9.104435 seconds (0.01820887 per addr).
Deleted 500 macvlan in 25.424282 seconds. (0.050848564 per interface)


After the patch:

[root@lec2010-ath9k-1 lanforge]# /mnt/b32/greearb/tmp/test_macvlans.pl 500 macvlan
Creating 500 macvlan.
Created 500 macvlan in 12.461308 seconds (0.024922616 per interface).
Added IP addresses in 8.787694 seconds (0.017575388 per addr).
Deleted 500 macvlan in 21.831413 seconds. (0.043662826 per interface)

Thanks,
Ben

-- 
Ben Greear <greearb@candelatech.com>
Candela Technologies Inc  http://www.candelatech.com

Re: [PATCH net-next-2.6] macvlan: remove one synchronize_rcu() call

[Eric Dumazet]

Le jeudi 19 mai 2011 à 17:18 -0700, Ben Greear a écrit :

> I applied this to today's wireless-testing kernel.  There is a consistent
> speedup in deleting mac-vlans!  I wouldn't read much into changes in
> creating macvlans or adding IPs..those numbers just jump around a bit
> from run to run.
> 
> Before the patch:
> Deleted 500 macvlan in 25.424282 seconds. (0.050848564 per interface)
> 
> 
> After the patch:
> 
> Deleted 500 macvlan in 21.831413 seconds. (0.043662826 per interface)
> 

Thanks for testing !

My ultimate goal would be to reduce vlan/macvlan delete latency from 50
ms to less than 5 ms. Step by step...

(But this will wait after this merge window, since next changes are a
bit more complex)

Stay tuned ;)

Re: [PATCH net-next-2.6] macvlan: remove one synchronize_rcu() call

[David Miller]

From: Eric Dumazet <eric.dumazet@gmail.com>
Date: Fri, 20 May 2011 00:24:16 +0200

> When one macvlan device is dismantled, we can avoid one
> synchronize_rcu() call done after deletion from hash list, since caller
> will perform a synchronize_net() call after its ndo_stop() call.
> 
> Add a new netdev->dismantle field to signal this dismantle intent.
> 
> Reduces RTNL hold time.
> 
> Signed-off-by: Eric Dumazet <eric.dumazet@gmail.com>

Applied.