[Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Rakib Mullick]

Hello all,

In ttwu_do_activate(), we're decrementing nr_uninterruptible if
p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
nr_uninterruptible from activate_task at the same path. Why we're
doing it twice for a single task activation path?


Thanks,
Rakib

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Peter Zijlstra]

On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
> Hello all,
> 
> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
> nr_uninterruptible from activate_task at the same path. Why we're
> doing it twice for a single task activation path?

activate_task() does:

 if (task_contributes_to_load(p))
   rq->nr_uninterruptible--;

Now task_contributes_to_load() reads like:

#define task_contributes_to_load(task)	\
				((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
				 (task->flags & PF_FREEZING) == 0)

which will be false, since we've set TASK_WAKING.

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Rakib Mullick]

On Wed, Jan 11, 2012 at 11:29 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
>> Hello all,
>>
>> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
>> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
>> nr_uninterruptible from activate_task at the same path. Why we're
>> doing it twice for a single task activation path?
>
> activate_task() does:
>
>  if (task_contributes_to_load(p))
>   rq->nr_uninterruptible--;
>
> Now task_contributes_to_load() reads like:
>
> #define task_contributes_to_load(task)  \
>                                ((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
>                                 (task->flags & PF_FREEZING) == 0)
>
> which will be false, since we've set TASK_WAKING.

Enough confusing. TASK_WAKING will be set when called from
try_to_wake_up(). ttwu_do_activate() gets called from other places:
scheduler_ipi() and sched_ttwu_pending() (at the time of cpu goes
down). TASK_WAKING will be not set at that time, moreover it is
possible that, task has p->sched_contributes_to_load is set and latter
on gets wake up by sched_ttwu_pending/scheduler_ipi() call.

Thanks,
Rakib

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Peter Zijlstra]

On Thu, 2012-01-12 at 12:09 +0600, Rakib Mullick wrote:
> On Wed, Jan 11, 2012 at 11:29 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> > On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
> >> Hello all,
> >>
> >> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
> >> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
> >> nr_uninterruptible from activate_task at the same path. Why we're
> >> doing it twice for a single task activation path?
> >
> > activate_task() does:
> >
> >  if (task_contributes_to_load(p))
> >   rq->nr_uninterruptible--;
> >
> > Now task_contributes_to_load() reads like:
> >
> > #define task_contributes_to_load(task)  \
> >                                ((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
> >                                 (task->flags & PF_FREEZING) == 0)
> >
> > which will be false, since we've set TASK_WAKING.
> 
> Enough confusing. TASK_WAKING will be set when called from
> try_to_wake_up(). ttwu_do_activate() gets called from other places:
> scheduler_ipi() and sched_ttwu_pending() (at the time of cpu goes
> down). TASK_WAKING will be not set at that time,

Yes it will be, the only way to get on that list is throught
ttwu_queue_remote() at which point tasks are TASK_WAKING.

>  moreover it is
> possible that, task has p->sched_contributes_to_load is set and latter
> on gets wake up by sched_ttwu_pending/scheduler_ipi() call.

That's the entire point. But all ways to ttwu_queue_remote() explicitly
set ->sched_contributes_to_load.

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Rakib Mullick]

On Thu, Jan 12, 2012 at 1:25 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
> On Thu, 2012-01-12 at 12:09 +0600, Rakib Mullick wrote:
>> On Wed, Jan 11, 2012 at 11:29 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>> > On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
>> >> Hello all,
>> >>
>> >> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
>> >> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
>> >> nr_uninterruptible from activate_task at the same path. Why we're
>> >> doing it twice for a single task activation path?
>> >
>> > activate_task() does:
>> >
>> >  if (task_contributes_to_load(p))
>> >   rq->nr_uninterruptible--;
>> >
>> > Now task_contributes_to_load() reads like:
>> >
>> > #define task_contributes_to_load(task)  \
>> >                                ((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
>> >                                 (task->flags & PF_FREEZING) == 0)
>> >
>> > which will be false, since we've set TASK_WAKING.
>>
>> Enough confusing. TASK_WAKING will be set when called from
>> try_to_wake_up(). ttwu_do_activate() gets called from other places:
>> scheduler_ipi() and sched_ttwu_pending() (at the time of cpu goes
>> down). TASK_WAKING will be not set at that time,
>
> Yes it will be, the only way to get on that list is throught
> ttwu_queue_remote() at which point tasks are TASK_WAKING.
>
>>  moreover it is
>> possible that, task has p->sched_contributes_to_load is set and latter
>> on gets wake up by sched_ttwu_pending/scheduler_ipi() call.
>
> That's the entire point. But all ways to ttwu_queue_remote() explicitly
> set ->sched_contributes_to_load.

That might be the case for scheduler_ipi(), but when
sched_ttwu_pending() gets called when a cpu goes down, all tasks from
wake_list of that cpu has been moved without TASK_WAKING is set. For a
particular task it might be possible that when it ran previously it
had p->sched_contributes_to_load is set. Latter, this task's cpu has
been put down and calls sched_ttwu_pending(), then for that task
p->sched_contributes_to_load is set and TASK_WAKING is not set.
Couldn't be happen?

Thanks,
Rakib

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Peter Zijlstra]

On Thu, 2012-01-12 at 23:08 +0600, Rakib Mullick wrote:

> That might be the case for scheduler_ipi(), but when
> sched_ttwu_pending() gets called when a cpu goes down, all tasks from
> wake_list of that cpu has been moved without TASK_WAKING is set. For a
> particular task it might be possible that when it ran previously it
> had p->sched_contributes_to_load is set. Latter, this task's cpu has
> been put down and calls sched_ttwu_pending(), then for that task
> p->sched_contributes_to_load is set and TASK_WAKING is not set.
> Couldn't be happen?

No, look again, its impossible to be on that list and not be
TASK_WAKING.

The only way onto the list is through ttwu_queue_remote(), the only way
off the list is through sched_ttwu_pending(). 

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Michael Wang]

On 01/13/2012 01:08 AM, Rakib Mullick wrote:

> On Thu, Jan 12, 2012 at 1:25 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>> On Thu, 2012-01-12 at 12:09 +0600, Rakib Mullick wrote:
>>> On Wed, Jan 11, 2012 at 11:29 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>>>> On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
>>>>> Hello all,
>>>>>
>>>>> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
>>>>> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
>>>>> nr_uninterruptible from activate_task at the same path. Why we're
>>>>> doing it twice for a single task activation path?
>>>>
>>>> activate_task() does:
>>>>
>>>>  if (task_contributes_to_load(p))
>>>>   rq->nr_uninterruptible--;
>>>>
>>>> Now task_contributes_to_load() reads like:
>>>>
>>>> #define task_contributes_to_load(task)  \
>>>>                                ((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
>>>>                                 (task->flags & PF_FREEZING) == 0)
>>>>
>>>> which will be false, since we've set TASK_WAKING.
>>>
>>> Enough confusing. TASK_WAKING will be set when called from
>>> try_to_wake_up(). ttwu_do_activate() gets called from other places:
>>> scheduler_ipi() and sched_ttwu_pending() (at the time of cpu goes
>>> down). TASK_WAKING will be not set at that time,
>>
>> Yes it will be, the only way to get on that list is throught
>> ttwu_queue_remote() at which point tasks are TASK_WAKING.
>>
>>>  moreover it is
>>> possible that, task has p->sched_contributes_to_load is set and latter
>>> on gets wake up by sched_ttwu_pending/scheduler_ipi() call.
>>
>> That's the entire point. But all ways to ttwu_queue_remote() explicitly
>> set ->sched_contributes_to_load.
> 
> That might be the case for scheduler_ipi(), but when
> sched_ttwu_pending() gets called when a cpu goes down, all tasks from
> wake_list of that cpu has been moved without TASK_WAKING is set. For a


I think the task in rq->wake_list should already have state:TASK_WAKING,
because it's a wake list.

> particular task it might be possible that when it ran previously it
> had p->sched_contributes_to_load is set. Latter, this task's cpu has
> been put down and calls sched_ttwu_pending(), then for that task
> p->sched_contributes_to_load is set and TASK_WAKING is not set.
> Couldn't be happen?
> 
> Thanks,
> Rakib
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/
> 

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Rakib Mullick]

On Mon, Jan 16, 2012 at 1:53 PM, Michael Wang
<wangyun@linux.vnet.ibm.com> wrote:
> On 01/13/2012 01:08 AM, Rakib Mullick wrote:
>
>> On Thu, Jan 12, 2012 at 1:25 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>>> On Thu, 2012-01-12 at 12:09 +0600, Rakib Mullick wrote:
>>>> On Wed, Jan 11, 2012 at 11:29 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>>>>> On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
>>>>>> Hello all,
>>>>>>
>>>>>> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
>>>>>> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
>>>>>> nr_uninterruptible from activate_task at the same path. Why we're
>>>>>> doing it twice for a single task activation path?
>>>>>
>>>>> activate_task() does:
>>>>>
>>>>>  if (task_contributes_to_load(p))
>>>>>   rq->nr_uninterruptible--;
>>>>>
>>>>> Now task_contributes_to_load() reads like:
>>>>>
>>>>> #define task_contributes_to_load(task)  \
>>>>>                                ((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
>>>>>                                 (task->flags & PF_FREEZING) == 0)
>>>>>
>>>>> which will be false, since we've set TASK_WAKING.
>>>>
>>>> Enough confusing. TASK_WAKING will be set when called from
>>>> try_to_wake_up(). ttwu_do_activate() gets called from other places:
>>>> scheduler_ipi() and sched_ttwu_pending() (at the time of cpu goes
>>>> down). TASK_WAKING will be not set at that time,
>>>
>>> Yes it will be, the only way to get on that list is throught
>>> ttwu_queue_remote() at which point tasks are TASK_WAKING.
>>>
>>>>  moreover it is
>>>> possible that, task has p->sched_contributes_to_load is set and latter
>>>> on gets wake up by sched_ttwu_pending/scheduler_ipi() call.
>>>
>>> That's the entire point. But all ways to ttwu_queue_remote() explicitly
>>> set ->sched_contributes_to_load.
>>
>> That might be the case for scheduler_ipi(), but when
>> sched_ttwu_pending() gets called when a cpu goes down, all tasks from
>> wake_list of that cpu has been moved without TASK_WAKING is set. For a
>
>
> I think the task in rq->wake_list should already have state:TASK_WAKING,
> because it's a wake list.
>
But, what I got by means of TASK_WAKING is this task is about to RUN,
very soon it'll have TASK_RUNNING state. And, if I hadn't miss any
portion of code, then rq->wake_list doesn't have TASK_WAKING state.

Thanks,
Rakib

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Michael Wang]

On 01/16/2012 04:27 PM, Rakib Mullick wrote:

> On Mon, Jan 16, 2012 at 1:53 PM, Michael Wang
> <wangyun@linux.vnet.ibm.com> wrote:
>> On 01/13/2012 01:08 AM, Rakib Mullick wrote:
>>
>>> On Thu, Jan 12, 2012 at 1:25 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>>>> On Thu, 2012-01-12 at 12:09 +0600, Rakib Mullick wrote:
>>>>> On Wed, Jan 11, 2012 at 11:29 PM, Peter Zijlstra <a.p.zijlstra@chello.nl> wrote:
>>>>>> On Wed, 2012-01-11 at 23:22 +0600, Rakib Mullick wrote:
>>>>>>> Hello all,
>>>>>>>
>>>>>>> In ttwu_do_activate(), we're decrementing nr_uninterruptible if
>>>>>>> p->sched_contributes_to_load (for SMP=y). But, we're also decrementing
>>>>>>> nr_uninterruptible from activate_task at the same path. Why we're
>>>>>>> doing it twice for a single task activation path?
>>>>>>
>>>>>> activate_task() does:
>>>>>>
>>>>>>  if (task_contributes_to_load(p))
>>>>>>   rq->nr_uninterruptible--;
>>>>>>
>>>>>> Now task_contributes_to_load() reads like:
>>>>>>
>>>>>> #define task_contributes_to_load(task)  \
>>>>>>                                ((task->state & TASK_UNINTERRUPTIBLE) != 0 && \
>>>>>>                                 (task->flags & PF_FREEZING) == 0)
>>>>>>
>>>>>> which will be false, since we've set TASK_WAKING.
>>>>>
>>>>> Enough confusing. TASK_WAKING will be set when called from
>>>>> try_to_wake_up(). ttwu_do_activate() gets called from other places:
>>>>> scheduler_ipi() and sched_ttwu_pending() (at the time of cpu goes
>>>>> down). TASK_WAKING will be not set at that time,
>>>>
>>>> Yes it will be, the only way to get on that list is throught
>>>> ttwu_queue_remote() at which point tasks are TASK_WAKING.
>>>>
>>>>>  moreover it is
>>>>> possible that, task has p->sched_contributes_to_load is set and latter
>>>>> on gets wake up by sched_ttwu_pending/scheduler_ipi() call.
>>>>
>>>> That's the entire point. But all ways to ttwu_queue_remote() explicitly
>>>> set ->sched_contributes_to_load.
>>>
>>> That might be the case for scheduler_ipi(), but when
>>> sched_ttwu_pending() gets called when a cpu goes down, all tasks from
>>> wake_list of that cpu has been moved without TASK_WAKING is set. For a
>>
>>
>> I think the task in rq->wake_list should already have state:TASK_WAKING,
>> because it's a wake list.
>>
> But, what I got by means of TASK_WAKING is this task is about to RUN,
> very soon it'll have TASK_RUNNING state. And, if I hadn't miss any
> portion of code, then rq->wake_list doesn't have TASK_WAKING state.
> 

I saw this is the way to enqueue wake_list:

try_to_wake_up --> p->state = TASK_WAKING; --> ttwu_queue -->
ttwu_queue_remote --> llist_add(&p->wake_entry, &cpu_rq(cpu)->wake_list)

BTW, I'm just start to learn scheduler, may be I'm wrong, let's find out
the right answer :)

Thanks,
Michael Wang

> Thanks,
> Rakib
> 

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Rakib Mullick]

On Mon, Jan 16, 2012 at 3:22 PM, Michael Wang
<wangyun@linux.vnet.ibm.com> wrote:
> On 01/16/2012 04:27 PM, Rakib Mullick wrote:
>
>> On Mon, Jan 16, 2012 at 1:53 PM, Michael Wang
>
> I saw this is the way to enqueue wake_list:
>
> try_to_wake_up --> p->state = TASK_WAKING; --> ttwu_queue -->
> ttwu_queue_remote --> llist_add(&p->wake_entry, &cpu_rq(cpu)->wake_list)
>
Yes, right and if the task needs to wakeup on a remote CPU, then it
follows an reschedule IPI. But, what I was missing that, I wasn't sure
whether task gets added to wake_list only from ttwu_queue_remote() or
not. Now, I've found that - it is and you are right rq->wake_list only
have TASK_WAKING state.

> BTW, I'm just start to learn scheduler, may be I'm wrong, let's find out
> the right answer :)
>
I think it's quite clear to me now. And you are learning scheduler
well at-least better than me :)

Thanks,
Rakib

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Hillf Danton]

On Mon, Jan 16, 2012 at 4:27 PM, Rakib Mullick <rakib.mullick@gmail.com> wrote:
> On Mon, Jan 16, 2012 at 1:53 PM, Michael Wang
> <wangyun@linux.vnet.ibm.com> wrote:
>> I think the task in rq->wake_list should already have state:TASK_WAKING,
>> because it's a wake list.
>>
> But, what I got by means of TASK_WAKING is this task is about to RUN,
> very soon it'll have TASK_RUNNING state. And, if I hadn't miss any
> portion of code, then rq->wake_list doesn't have TASK_WAKING state.
>
Hi Rakib

The question maybe settled down by adding a BUG_ON or similar to
capture what you concern, and wait results while drinking tea or
coffee.

Best regards
Hillf

Re: [Question] sched: Should nr_uninterruptible be decremented in ttwu_do_activate()?

[Rakib Mullick]

On Mon, Jan 16, 2012 at 7:00 PM, Hillf Danton <dhillf@gmail.com> wrote:
> On Mon, Jan 16, 2012 at 4:27 PM, Rakib Mullick <rakib.mullick@gmail.com> wrote:
>> On Mon, Jan 16, 2012 at 1:53 PM, Michael Wang
>> <wangyun@linux.vnet.ibm.com> wrote:
>>> I think the task in rq->wake_list should already have state:TASK_WAKING,
>>> because it's a wake list.
>>>
>> But, what I got by means of TASK_WAKING is this task is about to RUN,
>> very soon it'll have TASK_RUNNING state. And, if I hadn't miss any
>> portion of code, then rq->wake_list doesn't have TASK_WAKING state.
>>
> Hi Rakib
>
> The question maybe settled down by adding a BUG_ON or similar to
> capture what you concern, and wait results while drinking tea or
> coffee.
>
Hi Hillf,

Thanks for your suggestions, will apply this technique when some new
confusion arises. Now, it's quite clear :)

Thanks,
Rakib