[PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[Simon Horman]

Add a helper that for use in loops which read data protected by RCU and
may have a large number of iterations.  Such an example is dumping the list
of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().

This series also updates IPVS to use the helper.

Simon Horman (2):
  sched: add cond_resched_rcu() helper
  ipvs: use cond_resched_rcu() helper when walking connections

 include/linux/sched.h           |  9 +++++++++
 net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
 2 files changed, 17 insertions(+), 15 deletions(-)

-- 
1.8.2.1

[PATCH ipvs-next v3 1/2] sched: add cond_resched_rcu() helper

[Simon Horman]

This is intended for use in loops which read data protected by RCU and may
have a large number of iterations.  Such an example is dumping the list of
connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().

The benefits are for CONFIG_PREEMPT_RCU=y where we save CPU cycles
by moving rcu_read_lock and rcu_read_unlock out of large loops
but still allowing the current task to be preempted after every
loop iteration for the CONFIG_PREEMPT_RCU=n case.

The call to cond_resched() is not needed when CONFIG_PREEMPT_RCU=y.
Thanks to Paul E. McKenney for explaining this and for the
final version that checks the context with CONFIG_DEBUG_ATOMIC_SLEEP=y
for all possible configurations.

The function can be empty in the CONFIG_PREEMPT_RCU case,
rcu_read_lock and rcu_read_unlock are not needed in this case
because the task can be preempted on indication from scheduler.
Thanks to Peter Zijlstra for catching this and for his help
in trying a solution that changes __might_sleep.

Initial cond_resched_rcu_lock() function suggested by Eric Dumazet.

Tested-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
 include/linux/sched.h | 9 +++++++++
 1 file changed, 9 insertions(+)

diff --git a/include/linux/sched.h b/include/linux/sched.h
index e692a02..2080446 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -2608,6 +2608,15 @@ extern int __cond_resched_softirq(void);
 	__cond_resched_softirq();					\
 })
 
+static inline void cond_resched_rcu(void)
+{
+#if defined(CONFIG_DEBUG_ATOMIC_SLEEP) || !defined(CONFIG_PREEMPT_RCU)
+	rcu_read_unlock();
+	cond_resched();
+	rcu_read_lock();
+#endif
+}
+
 /*
  * Does a critical section need to be broken due to another
  * task waiting?: (technically does not depend on CONFIG_PREEMPT,
-- 
1.8.2.1

[PATCH ipvs-next v3 2/2] ipvs: use cond_resched_rcu() helper when walking connections

[Simon Horman]

This avoids the situation where walking of a large number of connections
may prevent scheduling for a long time while also avoiding excessive
calls to rcu_read_unlock() and rcu_read_lock().

Note that in the case of !CONFIG_PREEMPT_RCU this will
add a call to cond_resched().

[ja@ssi.bg: add cond_resched_rcu to more places]
Signed-off-by: Julian Anastasov <ja@ssi.bg>
Signed-off-by: Simon Horman <horms@verge.net.au>
---
 net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
 1 file changed, 8 insertions(+), 15 deletions(-)

diff --git a/net/netfilter/ipvs/ip_vs_conn.c b/net/netfilter/ipvs/ip_vs_conn.c
index a083bda..c8c52a9 100644
--- a/net/netfilter/ipvs/ip_vs_conn.c
+++ b/net/netfilter/ipvs/ip_vs_conn.c
@@ -975,8 +975,7 @@ static void *ip_vs_conn_array(struct seq_file *seq, loff_t pos)
 				return cp;
 			}
 		}
-		rcu_read_unlock();
-		rcu_read_lock();
+		cond_resched_rcu();
 	}
 
 	return NULL;
@@ -1015,8 +1014,7 @@ static void *ip_vs_conn_seq_next(struct seq_file *seq, void *v, loff_t *pos)
 			iter->l = &ip_vs_conn_tab[idx];
 			return cp;
 		}
-		rcu_read_unlock();
-		rcu_read_lock();
+		cond_resched_rcu();
 	}
 	iter->l = NULL;
 	return NULL;
@@ -1206,17 +1204,13 @@ void ip_vs_random_dropentry(struct net *net)
 	int idx;
 	struct ip_vs_conn *cp, *cp_c;
 
+	rcu_read_lock();
 	/*
 	 * Randomly scan 1/32 of the whole table every second
 	 */
 	for (idx = 0; idx < (ip_vs_conn_tab_size>>5); idx++) {
 		unsigned int hash = net_random() & ip_vs_conn_tab_mask;
 
-		/*
-		 *  Lock is actually needed in this loop.
-		 */
-		rcu_read_lock();
-
 		hlist_for_each_entry_rcu(cp, &ip_vs_conn_tab[hash], c_list) {
 			if (cp->flags & IP_VS_CONN_F_TEMPLATE)
 				/* connection template */
@@ -1252,8 +1246,9 @@ void ip_vs_random_dropentry(struct net *net)
 				__ip_vs_conn_put(cp);
 			}
 		}
-		rcu_read_unlock();
+		cond_resched_rcu();
 	}
+	rcu_read_unlock();
 }
 
 
@@ -1267,11 +1262,8 @@ static void ip_vs_conn_flush(struct net *net)
 	struct netns_ipvs *ipvs = net_ipvs(net);
 
 flush_again:
+	rcu_read_lock();
 	for (idx = 0; idx < ip_vs_conn_tab_size; idx++) {
-		/*
-		 *  Lock is actually needed in this loop.
-		 */
-		rcu_read_lock();
 
 		hlist_for_each_entry_rcu(cp, &ip_vs_conn_tab[idx], c_list) {
 			if (!ip_vs_conn_net_eq(cp, net))
@@ -1286,8 +1278,9 @@ flush_again:
 				__ip_vs_conn_put(cp);
 			}
 		}
-		rcu_read_unlock();
+		cond_resched_rcu();
 	}
+	rcu_read_unlock();
 
 	/* the counter may be not NULL, because maybe some conn entries
 	   are run by slow timer handler or unhashed but still referred */
-- 
1.8.2.1

Re: [PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[Peter Zijlstra]

On Wed, May 22, 2013 at 02:50:30PM +0900, Simon Horman wrote:
> Add a helper that for use in loops which read data protected by RCU and
> may have a large number of iterations.  Such an example is dumping the list
> of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
> 
> This series also updates IPVS to use the helper.
> 
> Simon Horman (2):
>   sched: add cond_resched_rcu() helper
>   ipvs: use cond_resched_rcu() helper when walking connections
> 
>  include/linux/sched.h           |  9 +++++++++
>  net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
>  2 files changed, 17 insertions(+), 15 deletions(-)

Acked-by: Peter Zijlstra <peterz@infradead.org>

How do you want to go about merging this? Do you want to route it
through the net tree or through -tip?

Re: [PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[David Miller]

From: Peter Zijlstra <peterz@infradead.org>
Date: Wed, 22 May 2013 09:54:38 +0200

> On Wed, May 22, 2013 at 02:50:30PM +0900, Simon Horman wrote:
>> Add a helper that for use in loops which read data protected by RCU and
>> may have a large number of iterations.  Such an example is dumping the list
>> of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
>> 
>> This series also updates IPVS to use the helper.
>> 
>> Simon Horman (2):
>>   sched: add cond_resched_rcu() helper
>>   ipvs: use cond_resched_rcu() helper when walking connections
>> 
>>  include/linux/sched.h           |  9 +++++++++
>>  net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
>>  2 files changed, 17 insertions(+), 15 deletions(-)
> 
> Acked-by: Peter Zijlstra <peterz@infradead.org>
> 
> How do you want to go about merging this? Do you want to route it
> through the net tree or through -tip?

I think we can have Pablo merge it into his netfilter tree.

Re: [PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[Peter Zijlstra]

On Wed, May 22, 2013 at 01:31:08AM -0700, David Miller wrote:
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Wed, 22 May 2013 09:54:38 +0200
> 
> > On Wed, May 22, 2013 at 02:50:30PM +0900, Simon Horman wrote:
> >> Add a helper that for use in loops which read data protected by RCU and
> >> may have a large number of iterations.  Such an example is dumping the list
> >> of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
> >> 
> >> This series also updates IPVS to use the helper.
> >> 
> >> Simon Horman (2):
> >>   sched: add cond_resched_rcu() helper
> >>   ipvs: use cond_resched_rcu() helper when walking connections
> >> 
> >>  include/linux/sched.h           |  9 +++++++++
> >>  net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
> >>  2 files changed, 17 insertions(+), 15 deletions(-)
> > 
> > Acked-by: Peter Zijlstra <peterz@infradead.org>
> > 
> > How do you want to go about merging this? Do you want to route it
> > through the net tree or through -tip?
> 
> I think we can have Pablo merge it into his netfilter tree.

Works for me. Thanks!

Re: [PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[Simon Horman]

On Wed, May 22, 2013 at 11:00:51AM +0200, Peter Zijlstra wrote:
> On Wed, May 22, 2013 at 01:31:08AM -0700, David Miller wrote:
> > From: Peter Zijlstra <peterz@infradead.org>
> > Date: Wed, 22 May 2013 09:54:38 +0200
> > 
> > > On Wed, May 22, 2013 at 02:50:30PM +0900, Simon Horman wrote:
> > >> Add a helper that for use in loops which read data protected by RCU and
> > >> may have a large number of iterations.  Such an example is dumping the list
> > >> of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
> > >> 
> > >> This series also updates IPVS to use the helper.
> > >> 
> > >> Simon Horman (2):
> > >>   sched: add cond_resched_rcu() helper
> > >>   ipvs: use cond_resched_rcu() helper when walking connections
> > >> 
> > >>  include/linux/sched.h           |  9 +++++++++
> > >>  net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
> > >>  2 files changed, 17 insertions(+), 15 deletions(-)
> > > 
> > > Acked-by: Peter Zijlstra <peterz@infradead.org>
> > > 
> > > How do you want to go about merging this? Do you want to route it
> > > through the net tree or through -tip?
> > 
> > I think we can have Pablo merge it into his netfilter tree.
> 
> Works for me. Thanks!

Thanks, I will queue it up in ipvs-next and pass it to Pablo from there.

Re: [PATCH ipvs-next v3 1/2] sched: add cond_resched_rcu() helper

[Pablo Neira Ayuso]

On Wed, May 22, 2013 at 02:50:31PM +0900, Simon Horman wrote:
> This is intended for use in loops which read data protected by RCU and may
> have a large number of iterations.  Such an example is dumping the list of
> connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
> 
> The benefits are for CONFIG_PREEMPT_RCU=y where we save CPU cycles
> by moving rcu_read_lock and rcu_read_unlock out of large loops
> but still allowing the current task to be preempted after every
> loop iteration for the CONFIG_PREEMPT_RCU=n case.
> 
> The call to cond_resched() is not needed when CONFIG_PREEMPT_RCU=y.
> Thanks to Paul E. McKenney for explaining this and for the
> final version that checks the context with CONFIG_DEBUG_ATOMIC_SLEEP=y
> for all possible configurations.
> 
> The function can be empty in the CONFIG_PREEMPT_RCU case,
> rcu_read_lock and rcu_read_unlock are not needed in this case
> because the task can be preempted on indication from scheduler.
> Thanks to Peter Zijlstra for catching this and for his help
> in trying a solution that changes __might_sleep.
> 
> Initial cond_resched_rcu_lock() function suggested by Eric Dumazet.

Applied, thanks.

Re: [PATCH ipvs-next v3 2/2] ipvs: use cond_resched_rcu() helper when walking connections

[Pablo Neira Ayuso]

On Wed, May 22, 2013 at 02:50:32PM +0900, Simon Horman wrote:
> This avoids the situation where walking of a large number of connections
> may prevent scheduling for a long time while also avoiding excessive
> calls to rcu_read_unlock() and rcu_read_lock().
> 
> Note that in the case of !CONFIG_PREEMPT_RCU this will
> add a call to cond_resched().

Applied, thanks.

Re: [PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[Pablo Neira Ayuso]

On Wed, May 22, 2013 at 01:31:08AM -0700, David Miller wrote:
> From: Peter Zijlstra <peterz@infradead.org>
> Date: Wed, 22 May 2013 09:54:38 +0200
> 
> > On Wed, May 22, 2013 at 02:50:30PM +0900, Simon Horman wrote:
> >> Add a helper that for use in loops which read data protected by RCU and
> >> may have a large number of iterations.  Such an example is dumping the list
> >> of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
> >> 
> >> This series also updates IPVS to use the helper.
> >> 
> >> Simon Horman (2):
> >>   sched: add cond_resched_rcu() helper
> >>   ipvs: use cond_resched_rcu() helper when walking connections
> >> 
> >>  include/linux/sched.h           |  9 +++++++++
> >>  net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
> >>  2 files changed, 17 insertions(+), 15 deletions(-)
> > 
> > Acked-by: Peter Zijlstra <peterz@infradead.org>
> > 
> > How do you want to go about merging this? Do you want to route it
> > through the net tree or through -tip?
> 
> I think we can have Pablo merge it into his netfilter tree.

I did so. Thanks David.

Re: [PATCH ipvs-next v3 0/2] sched: Add cond_resched_rcu_lock() helper

[Ingo Molnar]

* David Miller <davem@davemloft.net> wrote:

> From: Peter Zijlstra <peterz@infradead.org>
> Date: Wed, 22 May 2013 09:54:38 +0200
> 
> > On Wed, May 22, 2013 at 02:50:30PM +0900, Simon Horman wrote:
> >> Add a helper that for use in loops which read data protected by RCU and
> >> may have a large number of iterations.  Such an example is dumping the list
> >> of connections known to IPVS: ip_vs_conn_array() and ip_vs_conn_seq_next().
> >> 
> >> This series also updates IPVS to use the helper.
> >> 
> >> Simon Horman (2):
> >>   sched: add cond_resched_rcu() helper
> >>   ipvs: use cond_resched_rcu() helper when walking connections
> >> 
> >>  include/linux/sched.h           |  9 +++++++++
> >>  net/netfilter/ipvs/ip_vs_conn.c | 23 ++++++++---------------
> >>  2 files changed, 17 insertions(+), 15 deletions(-)
> > 
> > Acked-by: Peter Zijlstra <peterz@infradead.org>
> > 
> > How do you want to go about merging this? Do you want to route it
> > through the net tree or through -tip?
> 
> I think we can have Pablo merge it into his netfilter tree.

Yeah, that's what we agreed upon earlier during the review of these 
patches.

Thanks,

	Ingo