From: chengchao <chengchao@kedacom.com>
To: Oleg Nesterov <oleg@redhat.com>
Cc: mingo@kernel.org, peterz@infradead.org, tj@kernel.org,
akpm@linux-foundation.org, chris@chris-wilson.co.uk,
linux-kernel@vger.kernel.org
Subject: Re: [PATCH] sched/core: simpler function for sched_exec migration
Date: Tue, 6 Sep 2016 10:11:59 +0800 [thread overview]
Message-ID: <db5c6fcd-ae5d-0f41-2d45-d161421cf9c4@kedacom.com> (raw)
In-Reply-To: <20160905131147.GA8552@redhat.com>
Oleg, thank you.
the key point is for CONFIG_PREEMPT_NONE=y, if sched_exec needs migrate the current,
migration_cpu_stop doesn't migrate the task(current) at all, it means that the stopper thread does some
unuseful works in this scenario.
finally,the stopper thread calls cpu_stop_signal_done() to wake up this task, it calls select_task_rq() again,
maybe select another different cpu. totally calls select_task_rq() two times(first at sched_exec())
plus one time(wake_up_new_task() also calls select_task_rq()).
it is too much overhead for one task(fork()+exec()), isn't it?
1.
sched_exec()
->stop_one_cpu()
->wait_for_completion().
wait_for_completion() makes the current TASK_UNINTERRUPTIBLE and call schedule_timeout()
schedule_timeout(timeout) timeout is MAX_SCHEDULE_TIMEOUT.
->schedule()
deactivate_task(rq, current, DEQUEUE_SLEEP);
current->on_rq = 0;
2.
migration_cpu_stop() checks the task_on_rq_queued(p), but the task p->on_rq is 0.
#define TASK_ON_RQ_QUEUED 1
static inline int task_on_rq_queued(struct task_struct *p)
{
return p->on_rq == TASK_ON_RQ_QUEUED;
}
migration_cpu_stop()
...
if (task_rq(p) == rq && task_on_rq_queued(p))
rq = __migrate_task(rq, p, arg->dest_cpu);
...
thanks again, any suggestions and more reviews are welcome.
on 09/05/2016 09:11 PM, Oleg Nesterov wrote:
> On 09/05, cheng chao wrote:
>>
>> @@ -2958,7 +2958,7 @@ void sched_exec(void)
>> struct migration_arg arg = { p, dest_cpu };
>>
>> raw_spin_unlock_irqrestore(&p->pi_lock, flags);
>> - stop_one_cpu(task_cpu(p), migration_cpu_stop, &arg);
>> + stop_one_cpu_sync(task_cpu(p), migration_cpu_stop, &arg);
>> return;
>> }
>> unlock:
>> diff --git a/kernel/stop_machine.c b/kernel/stop_machine.c
>> index 4a1ca5f..24f8637 100644
>> --- a/kernel/stop_machine.c
>> +++ b/kernel/stop_machine.c
>> @@ -130,6 +130,27 @@ int stop_one_cpu(unsigned int cpu, cpu_stop_fn_t fn, void *arg)
>> return done.ret;
>> }
>>
>> +/**
>> + * the caller keeps task_on_rq_queued, so it's more suitable for
>> + * sched_exec on the case when needs migration
>> + */
>> +void stop_one_cpu_sync(unsigned int cpu, cpu_stop_fn_t fn, void *arg)
>> +{
>> + struct cpu_stop_work work = { .fn = fn, .arg = arg, .done = NULL };
>> +
>> + if (!cpu_stop_queue_work(cpu, &work))
>> + return;
>> +
>> +#if defined(CONFIG_PREEMPT_NONE) || defined(CONFIG_PREEMPT_VOLUNTARY)
>> + /*
>> + * CONFIG_PREEMPT doesn't need call schedule here, because
>> + * preempt_enable already does the similar thing when call
>> + * cpu_stop_queue_work
>> + */
>> + schedule();
>> +#endif
>> +}
>
> Honestly, I don't really understand the changelog, but this looks wrong.
>
> stop_one_cpu_sync() assumes that cpu == smp_processor_id/task_cpu(current),
> and thus the stopper thread should preempt us at least after schedule()
> (if CONFIG_PREEMPT_NONE), so we do not need to synchronize.
>
yes. the stop_one_cpu_sync is not a good name, stop_one_cpu_schedule is better?
there is nothing about synchronization.
> But this is not necessarily true? This task can migrate to another CPU
> before cpu_stop_queue_work() ?
>
before sched_exec() calls stop_one_cpu()/cpu_stop_queue_work(), this task(current) cannot migrate
to another cpu,because this task is running on the cpu.
> Oleg.
>
>
next prev parent reply other threads:[~2016-09-06 2:12 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-09-05 6:20 [PATCH] sched/core: simpler function for sched_exec migration cheng chao
2016-09-05 13:11 ` Oleg Nesterov
2016-09-06 2:11 ` chengchao [this message]
2016-09-06 15:22 ` Oleg Nesterov
2016-09-07 3:22 ` chengchao
2016-09-07 12:35 ` Oleg Nesterov
2016-09-08 2:17 ` chengchao
2016-09-09 10:05 ` Peter Zijlstra
2016-09-09 1:39 ` [lkp] [sched/core] 3d26b7622f: BUG: unable to handle kernel NULL pointer dereference at 00000001 kernel test robot
2016-09-09 2:04 ` chengchao
2016-09-09 2:26 ` Ye Xiaolong
2016-09-09 2:36 ` chengchao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=db5c6fcd-ae5d-0f41-2d45-d161421cf9c4@kedacom.com \
--to=chengchao@kedacom.com \
--cc=akpm@linux-foundation.org \
--cc=chris@chris-wilson.co.uk \
--cc=linux-kernel@vger.kernel.org \
--cc=mingo@kernel.org \
--cc=oleg@redhat.com \
--cc=peterz@infradead.org \
--cc=tj@kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).