* [net-next 0/2] ipvs: speedup ipvs netns dismantle @ 2019-07-13 15:19 Haishuang Yan 2019-07-13 15:19 ` [net-next 1/2] ipvs: batch __ip_vs_cleanup Haishuang Yan 2019-07-13 15:19 ` [net-next 2/2] ipvs: batch __ip_vs_dev_cleanup Haishuang Yan 0 siblings, 2 replies; 6+ messages in thread From: Haishuang Yan @ 2019-07-13 15:19 UTC (permalink / raw) To: David S. Miller, Pablo Neira Ayuso, Simon Horman Cc: netdev, lvs-devel, linux-kernel, netfilter-devel, Haishuang Yan Implement exit_batch() method to dismantle more ipvs netns per round. Haishuang Yan (2): ipvs: batch __ip_vs_cleanup ipvs: batch __ip_vs_dev_cleanup include/net/ip_vs.h | 2 +- net/netfilter/ipvs/ip_vs_core.c | 49 +++++++++++++++++++++++++---------------- net/netfilter/ipvs/ip_vs_ctl.c | 13 ++++++++--- 3 files changed, 41 insertions(+), 23 deletions(-) -- 1.8.3.1 ^ permalink raw reply [flat|nested] 6+ messages in thread
* [net-next 1/2] ipvs: batch __ip_vs_cleanup 2019-07-13 15:19 [net-next 0/2] ipvs: speedup ipvs netns dismantle Haishuang Yan @ 2019-07-13 15:19 ` Haishuang Yan 2019-07-15 20:39 ` Julian Anastasov 2019-07-13 15:19 ` [net-next 2/2] ipvs: batch __ip_vs_dev_cleanup Haishuang Yan 1 sibling, 1 reply; 6+ messages in thread From: Haishuang Yan @ 2019-07-13 15:19 UTC (permalink / raw) To: David S. Miller, Pablo Neira Ayuso, Simon Horman Cc: netdev, lvs-devel, linux-kernel, netfilter-devel, Haishuang Yan It's better to batch __ip_vs_cleanup to speedup ipvs connections dismantle. Signed-off-by: Haishuang Yan <yanhaishuang@cmss.chinamobile.com> --- include/net/ip_vs.h | 2 +- net/netfilter/ipvs/ip_vs_core.c | 29 +++++++++++++++++------------ net/netfilter/ipvs/ip_vs_ctl.c | 13 ++++++++++--- 3 files changed, 28 insertions(+), 16 deletions(-) diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h index 3759167..93e7a25 100644 --- a/include/net/ip_vs.h +++ b/include/net/ip_vs.h @@ -1324,7 +1324,7 @@ static inline void ip_vs_control_del(struct ip_vs_conn *cp) void ip_vs_control_net_cleanup(struct netns_ipvs *ipvs); void ip_vs_estimator_net_cleanup(struct netns_ipvs *ipvs); void ip_vs_sync_net_cleanup(struct netns_ipvs *ipvs); -void ip_vs_service_net_cleanup(struct netns_ipvs *ipvs); +void ip_vs_service_nets_cleanup(struct list_head *net_list); /* IPVS application functions * (from ip_vs_app.c) diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c index 46f06f9..b4d79b7 100644 --- a/net/netfilter/ipvs/ip_vs_core.c +++ b/net/netfilter/ipvs/ip_vs_core.c @@ -2402,18 +2402,23 @@ static int __net_init __ip_vs_init(struct net *net) return -ENOMEM; } -static void __net_exit __ip_vs_cleanup(struct net *net) +static void __net_exit __ip_vs_cleanup_batch(struct list_head *net_list) { - struct netns_ipvs *ipvs = net_ipvs(net); - - ip_vs_service_net_cleanup(ipvs); /* ip_vs_flush() with locks */ - ip_vs_conn_net_cleanup(ipvs); - ip_vs_app_net_cleanup(ipvs); - ip_vs_protocol_net_cleanup(ipvs); - ip_vs_control_net_cleanup(ipvs); - ip_vs_estimator_net_cleanup(ipvs); - IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); - net->ipvs = NULL; + struct netns_ipvs *ipvs; + struct net *net; + LIST_HEAD(list); + + ip_vs_service_nets_cleanup(net_list); /* ip_vs_flush() with locks */ + list_for_each_entry(net, net_list, exit_list) { + ipvs = net_ipvs(net); + ip_vs_conn_net_cleanup(ipvs); + ip_vs_app_net_cleanup(ipvs); + ip_vs_protocol_net_cleanup(ipvs); + ip_vs_control_net_cleanup(ipvs); + ip_vs_estimator_net_cleanup(ipvs); + IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); + net->ipvs = NULL; + } } static int __net_init __ip_vs_dev_init(struct net *net) @@ -2442,7 +2447,7 @@ static void __net_exit __ip_vs_dev_cleanup(struct net *net) static struct pernet_operations ipvs_core_ops = { .init = __ip_vs_init, - .exit = __ip_vs_cleanup, + .exit_batch = __ip_vs_cleanup_batch, .id = &ip_vs_net_id, .size = sizeof(struct netns_ipvs), }; diff --git a/net/netfilter/ipvs/ip_vs_ctl.c b/net/netfilter/ipvs/ip_vs_ctl.c index 07e0967..c8e652b 100644 --- a/net/netfilter/ipvs/ip_vs_ctl.c +++ b/net/netfilter/ipvs/ip_vs_ctl.c @@ -1607,14 +1607,21 @@ static int ip_vs_flush(struct netns_ipvs *ipvs, bool cleanup) /* * Delete service by {netns} in the service table. - * Called by __ip_vs_cleanup() + * Called by __ip_vs_batch_cleanup() */ -void ip_vs_service_net_cleanup(struct netns_ipvs *ipvs) +void ip_vs_service_nets_cleanup(struct list_head *net_list) { + struct netns_ipvs *ipvs; + struct net *net; + LIST_HEAD(list); + EnterFunction(2); /* Check for "full" addressed entries */ mutex_lock(&__ip_vs_mutex); - ip_vs_flush(ipvs, true); + list_for_each_entry(net, net_list, exit_list) { + ipvs = net_ipvs(net); + ip_vs_flush(ipvs, true); + } mutex_unlock(&__ip_vs_mutex); LeaveFunction(2); } -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [net-next 1/2] ipvs: batch __ip_vs_cleanup 2019-07-13 15:19 ` [net-next 1/2] ipvs: batch __ip_vs_cleanup Haishuang Yan @ 2019-07-15 20:39 ` Julian Anastasov 2019-07-18 14:16 ` Haishuang Yan 0 siblings, 1 reply; 6+ messages in thread From: Julian Anastasov @ 2019-07-15 20:39 UTC (permalink / raw) To: Haishuang Yan Cc: David S. Miller, Pablo Neira Ayuso, Simon Horman, netdev, lvs-devel, linux-kernel, netfilter-devel Hello, On Sat, 13 Jul 2019, Haishuang Yan wrote: > It's better to batch __ip_vs_cleanup to speedup ipvs > connections dismantle. > > Signed-off-by: Haishuang Yan <yanhaishuang@cmss.chinamobile.com> > --- > include/net/ip_vs.h | 2 +- > net/netfilter/ipvs/ip_vs_core.c | 29 +++++++++++++++++------------ > net/netfilter/ipvs/ip_vs_ctl.c | 13 ++++++++++--- > 3 files changed, 28 insertions(+), 16 deletions(-) > > diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h > index 3759167..93e7a25 100644 > --- a/include/net/ip_vs.h > +++ b/include/net/ip_vs.h > @@ -1324,7 +1324,7 @@ static inline void ip_vs_control_del(struct ip_vs_conn *cp) > void ip_vs_control_net_cleanup(struct netns_ipvs *ipvs); > void ip_vs_estimator_net_cleanup(struct netns_ipvs *ipvs); > void ip_vs_sync_net_cleanup(struct netns_ipvs *ipvs); > -void ip_vs_service_net_cleanup(struct netns_ipvs *ipvs); > +void ip_vs_service_nets_cleanup(struct list_head *net_list); > > /* IPVS application functions > * (from ip_vs_app.c) > diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c > index 46f06f9..b4d79b7 100644 > --- a/net/netfilter/ipvs/ip_vs_core.c > +++ b/net/netfilter/ipvs/ip_vs_core.c > @@ -2402,18 +2402,23 @@ static int __net_init __ip_vs_init(struct net *net) > return -ENOMEM; > } > > -static void __net_exit __ip_vs_cleanup(struct net *net) > +static void __net_exit __ip_vs_cleanup_batch(struct list_head *net_list) > { > - struct netns_ipvs *ipvs = net_ipvs(net); > - > - ip_vs_service_net_cleanup(ipvs); /* ip_vs_flush() with locks */ > - ip_vs_conn_net_cleanup(ipvs); > - ip_vs_app_net_cleanup(ipvs); > - ip_vs_protocol_net_cleanup(ipvs); > - ip_vs_control_net_cleanup(ipvs); > - ip_vs_estimator_net_cleanup(ipvs); > - IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); > - net->ipvs = NULL; > + struct netns_ipvs *ipvs; > + struct net *net; > + LIST_HEAD(list); > + > + ip_vs_service_nets_cleanup(net_list); /* ip_vs_flush() with locks */ > + list_for_each_entry(net, net_list, exit_list) { How much faster is to replace list_for_each_entry in ops_exit_list() with this one. IPVS can waste time in calls such as kthread_stop() and del_timer_sync() but I'm not sure we can solve it easily. What gain do you see in benchmarks? > + ipvs = net_ipvs(net); > + ip_vs_conn_net_cleanup(ipvs); > + ip_vs_app_net_cleanup(ipvs); > + ip_vs_protocol_net_cleanup(ipvs); > + ip_vs_control_net_cleanup(ipvs); > + ip_vs_estimator_net_cleanup(ipvs); > + IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); > + net->ipvs = NULL; > + } > } Regards -- Julian Anastasov <ja@ssi.bg> ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [net-next 1/2] ipvs: batch __ip_vs_cleanup 2019-07-15 20:39 ` Julian Anastasov @ 2019-07-18 14:16 ` Haishuang Yan 2019-07-29 20:03 ` Julian Anastasov 0 siblings, 1 reply; 6+ messages in thread From: Haishuang Yan @ 2019-07-18 14:16 UTC (permalink / raw) To: Julian Anastasov Cc: David S. Miller, Pablo Neira Ayuso, Simon Horman, netdev, lvs-devel, linux-kernel, netfilter-devel > On 2019年7月16日, at 上午4:39, Julian Anastasov <ja@ssi.bg> wrote: > > > Hello, > > On Sat, 13 Jul 2019, Haishuang Yan wrote: > >> It's better to batch __ip_vs_cleanup to speedup ipvs >> connections dismantle. >> >> Signed-off-by: Haishuang Yan <yanhaishuang@cmss.chinamobile.com> >> --- >> include/net/ip_vs.h | 2 +- >> net/netfilter/ipvs/ip_vs_core.c | 29 +++++++++++++++++------------ >> net/netfilter/ipvs/ip_vs_ctl.c | 13 ++++++++++--- >> 3 files changed, 28 insertions(+), 16 deletions(-) >> >> diff --git a/include/net/ip_vs.h b/include/net/ip_vs.h >> index 3759167..93e7a25 100644 >> --- a/include/net/ip_vs.h >> +++ b/include/net/ip_vs.h >> @@ -1324,7 +1324,7 @@ static inline void ip_vs_control_del(struct ip_vs_conn *cp) >> void ip_vs_control_net_cleanup(struct netns_ipvs *ipvs); >> void ip_vs_estimator_net_cleanup(struct netns_ipvs *ipvs); >> void ip_vs_sync_net_cleanup(struct netns_ipvs *ipvs); >> -void ip_vs_service_net_cleanup(struct netns_ipvs *ipvs); >> +void ip_vs_service_nets_cleanup(struct list_head *net_list); >> >> /* IPVS application functions >> * (from ip_vs_app.c) >> diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c >> index 46f06f9..b4d79b7 100644 >> --- a/net/netfilter/ipvs/ip_vs_core.c >> +++ b/net/netfilter/ipvs/ip_vs_core.c >> @@ -2402,18 +2402,23 @@ static int __net_init __ip_vs_init(struct net *net) >> return -ENOMEM; >> } >> >> -static void __net_exit __ip_vs_cleanup(struct net *net) >> +static void __net_exit __ip_vs_cleanup_batch(struct list_head *net_list) >> { >> - struct netns_ipvs *ipvs = net_ipvs(net); >> - >> - ip_vs_service_net_cleanup(ipvs); /* ip_vs_flush() with locks */ >> - ip_vs_conn_net_cleanup(ipvs); >> - ip_vs_app_net_cleanup(ipvs); >> - ip_vs_protocol_net_cleanup(ipvs); >> - ip_vs_control_net_cleanup(ipvs); >> - ip_vs_estimator_net_cleanup(ipvs); >> - IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); >> - net->ipvs = NULL; >> + struct netns_ipvs *ipvs; >> + struct net *net; >> + LIST_HEAD(list); >> + >> + ip_vs_service_nets_cleanup(net_list); /* ip_vs_flush() with locks */ >> + list_for_each_entry(net, net_list, exit_list) { > > How much faster is to replace list_for_each_entry in > ops_exit_list() with this one. IPVS can waste time in calls > such as kthread_stop() and del_timer_sync() but I'm not sure > we can solve it easily. What gain do you see in benchmarks? Hi, As the following benchmark testing results show, there is a little performance improvement: $ cat add_del_unshare.sh #!/bin/bash for i in `seq 1 100` do (for j in `seq 1 40` ; do unshare -n ipvsadm -A -t 172.16.$i.$j:80 >/dev/null ; done) & done wait; grep net_namespace /proc/slabinfo Befor patch: $ time sh add_del_unshare.sh net_namespace 4020 4020 4736 6 8 : tunables 0 0 0 : slabdata 670 670 0 real 0m8.086s user 0m2.025s sys 0m36.956s After patch: $ time sh add_del_unshare.sh net_namespace 4020 4020 4736 6 8 : tunables 0 0 0 : slabdata 670 670 0 real 0m7.623s user 0m2.003s sys 0m32.935s > >> + ipvs = net_ipvs(net); >> + ip_vs_conn_net_cleanup(ipvs); >> + ip_vs_app_net_cleanup(ipvs); >> + ip_vs_protocol_net_cleanup(ipvs); >> + ip_vs_control_net_cleanup(ipvs); >> + ip_vs_estimator_net_cleanup(ipvs); >> + IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); >> + net->ipvs = NULL; >> + } >> } > > Regards > > -- > Julian Anastasov <ja@ssi.bg> > ^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [net-next 1/2] ipvs: batch __ip_vs_cleanup 2019-07-18 14:16 ` Haishuang Yan @ 2019-07-29 20:03 ` Julian Anastasov 0 siblings, 0 replies; 6+ messages in thread From: Julian Anastasov @ 2019-07-29 20:03 UTC (permalink / raw) To: Haishuang Yan Cc: David S. Miller, Pablo Neira Ayuso, Simon Horman, netdev, lvs-devel, linux-kernel, netfilter-devel Hello, On Thu, 18 Jul 2019, Haishuang Yan wrote: > As the following benchmark testing results show, there is a little performance improvement: OK, can you send v2 after removing the LIST_HEAD(list) from both patches, I guess, it is not needed. If you prefer, you can include these benchmark results too. > $ cat add_del_unshare.sh > #!/bin/bash > > for i in `seq 1 100` > do > (for j in `seq 1 40` ; do unshare -n ipvsadm -A -t 172.16.$i.$j:80 >/dev/null ; done) & > done > wait; grep net_namespace /proc/slabinfo > > Befor patch: > $ time sh add_del_unshare.sh > net_namespace 4020 4020 4736 6 8 : tunables 0 0 0 : slabdata 670 670 0 > > real 0m8.086s > user 0m2.025s > sys 0m36.956s > > After patch: > $ time sh add_del_unshare.sh > net_namespace 4020 4020 4736 6 8 : tunables 0 0 0 : slabdata 670 670 0 > > real 0m7.623s > user 0m2.003s > sys 0m32.935s > > > > > >> + ipvs = net_ipvs(net); > >> + ip_vs_conn_net_cleanup(ipvs); > >> + ip_vs_app_net_cleanup(ipvs); > >> + ip_vs_protocol_net_cleanup(ipvs); > >> + ip_vs_control_net_cleanup(ipvs); > >> + ip_vs_estimator_net_cleanup(ipvs); > >> + IP_VS_DBG(2, "ipvs netns %d released\n", ipvs->gen); > >> + net->ipvs = NULL; Regards -- Julian Anastasov <ja@ssi.bg> ^ permalink raw reply [flat|nested] 6+ messages in thread
* [net-next 2/2] ipvs: batch __ip_vs_dev_cleanup 2019-07-13 15:19 [net-next 0/2] ipvs: speedup ipvs netns dismantle Haishuang Yan 2019-07-13 15:19 ` [net-next 1/2] ipvs: batch __ip_vs_cleanup Haishuang Yan @ 2019-07-13 15:19 ` Haishuang Yan 1 sibling, 0 replies; 6+ messages in thread From: Haishuang Yan @ 2019-07-13 15:19 UTC (permalink / raw) To: David S. Miller, Pablo Neira Ayuso, Simon Horman Cc: netdev, lvs-devel, linux-kernel, netfilter-devel, Haishuang Yan It's better to batch __ip_vs_cleanup to speedup ipvs devices dismantle. Signed-off-by: Haishuang Yan <yanhaishuang@cmss.chinamobile.com> --- net/netfilter/ipvs/ip_vs_core.c | 20 +++++++++++++------- 1 file changed, 13 insertions(+), 7 deletions(-) diff --git a/net/netfilter/ipvs/ip_vs_core.c b/net/netfilter/ipvs/ip_vs_core.c index b4d79b7..58af24a 100644 --- a/net/netfilter/ipvs/ip_vs_core.c +++ b/net/netfilter/ipvs/ip_vs_core.c @@ -2434,14 +2434,20 @@ static int __net_init __ip_vs_dev_init(struct net *net) return ret; } -static void __net_exit __ip_vs_dev_cleanup(struct net *net) +static void __net_exit __ip_vs_dev_cleanup_batch(struct list_head *net_list) { - struct netns_ipvs *ipvs = net_ipvs(net); + struct netns_ipvs *ipvs; + struct net *net; + LIST_HEAD(list); + EnterFunction(2); - nf_unregister_net_hooks(net, ip_vs_ops, ARRAY_SIZE(ip_vs_ops)); - ipvs->enable = 0; /* Disable packet reception */ - smp_wmb(); - ip_vs_sync_net_cleanup(ipvs); + list_for_each_entry(net, net_list, exit_list) { + ipvs = net_ipvs(net); + nf_unregister_net_hooks(net, ip_vs_ops, ARRAY_SIZE(ip_vs_ops)); + ipvs->enable = 0; /* Disable packet reception */ + smp_wmb(); + ip_vs_sync_net_cleanup(ipvs); + } LeaveFunction(2); } @@ -2454,7 +2460,7 @@ static void __net_exit __ip_vs_dev_cleanup(struct net *net) static struct pernet_operations ipvs_core_dev_ops = { .init = __ip_vs_dev_init, - .exit = __ip_vs_dev_cleanup, + .exit_batch = __ip_vs_dev_cleanup_batch, }; /* -- 1.8.3.1 ^ permalink raw reply related [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-07-29 20:03 UTC | newest] Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2019-07-13 15:19 [net-next 0/2] ipvs: speedup ipvs netns dismantle Haishuang Yan 2019-07-13 15:19 ` [net-next 1/2] ipvs: batch __ip_vs_cleanup Haishuang Yan 2019-07-15 20:39 ` Julian Anastasov 2019-07-18 14:16 ` Haishuang Yan 2019-07-29 20:03 ` Julian Anastasov 2019-07-13 15:19 ` [net-next 2/2] ipvs: batch __ip_vs_dev_cleanup Haishuang Yan
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).