Re: [PATCH v3 tip/core/rcu 1/9] rcu: Add call_rcu_tasks()

[Lai Jiangshan <laijs@cn.fujitsu.com>]

On 08/04/2014 03:46 PM, Peter Zijlstra wrote:
> On Mon, Aug 04, 2014 at 09:28:45AM +0800, Lai Jiangshan wrote:
>> On 08/01/2014 05:55 AM, Paul E. McKenney wrote:
>>> +		rcu_read_lock();
>>> +		for_each_process_thread(g, t) {
>>> +			if (t != current && ACCESS_ONCE(t->on_rq) &&
>>> +			    !is_idle_task(t)) {
>>> +				get_task_struct(t);
>>> +				t->rcu_tasks_nvcsw = ACCESS_ONCE(t->nvcsw);
>>> +				ACCESS_ONCE(t->rcu_tasks_holdout) = 1;
>>> +				list_add(&t->rcu_tasks_holdout_list,
>>> +					 &rcu_tasks_holdouts);
>>
>> This loop will collect all the runnable tasks.  It is too much tasks.
>> Is it possible to collect only on_cpu tasks or PREEMPT_ACTIVE tasks?
>> It seems hard to achieve it.
> 
> Without taking the rq->lock you cannot do that race-free. And we're not
> going to be taking rq->lock here.

It is because we can't fetch task->on_cpu and preempt_count atomically
so that rq->lock is required.

3 bleeding solutions:

1) Allocate one bit in preempt_count to stand for not_on_cpu ( = !task->on_cpu)
2) allocate one bit in nvcsw to stand for on_scheduled (or not_on_scheduled, see next)
3) introduce task->on_scheduled whose semantics is between on_cpu and on_rq,
   on_scheduled = scheduled on cpu or preempted, (not voluntary scheduled out)

But the scheduler doesn't need neither of such things.  So these is still no hope.