Re: [PATCH] sched/core: simpler function for sched_exec migration

[Oleg Nesterov <oleg@redhat.com>]

On 09/06, chengchao wrote:
>
> the key point is for CONFIG_PREEMPT_NONE=y,
> ...
> it is too much overhead for one task(fork()+exec()), isn't it?

Yes, yes, I see, this is suboptimal. Not sure we actually do care,
but yes, perhaps another helper which migrates the current task makes
sense, I dunno.

But,

> > stop_one_cpu_sync() assumes that cpu == smp_processor_id/task_cpu(current),
> > and thus the stopper thread should preempt us at least after schedule()
> > (if CONFIG_PREEMPT_NONE), so we do not need to synchronize.
> >
>    yes. the stop_one_cpu_sync is not a good name, stop_one_cpu_schedule is better?  
> there is nothing about synchronization.

We need to synchronize with the stopper to ensure it can't touch
cpu_stop_work on stack after stop_one_cpu_sync() returns, and

> > But this is not necessarily true? This task can migrate to another CPU
> > before cpu_stop_queue_work() ?
> >
>   before sched_exec() calls stop_one_cpu()/cpu_stop_queue_work(), this
> task(current) cannot migrate  to another cpu,because this task is running
> on the cpu.

Why? The running task can migrate to another CPU at any moment. Unless it
runs with preemption disabled or CONFIG_PREEMPT_NONE=y.

And this means that cpu_stop_queue_work() can queue the work on another
CPU != smp_processor_id(), and in this case the kernel can crash because
the pending cpu_stop_work can be overwritten right after return.

So you need something like

	void stop_one_cpu_sync(cpu_stop_fn_t fn, void *arg)
	{
		struct cpu_stop_work work = { .fn = fn, .arg = arg, .done = NULL };

		preempt_disable();
		cpu_stop_queue_work(raw_smp_processor_id(), &work);
		preempt_enable_no_resched();
		schedule();
	}

or I am totally confused. Note that it doesn't (and shouldn't) have
the "int cpu" argument.

Oleg.