Re: [PATCH] flush_cpu_workqueue: don't flush an empty ->worklist

[Oleg Nesterov <oleg@tv-sign.ru>]

On 01/15, Srivatsa Vaddagiri wrote:
>
> On Mon, Jan 15, 2007 at 03:54:01PM +0300, Oleg Nesterov wrote:
> > > - singlethread_cpu needs to be hotplug safe (broken currently)
> > 
> > Why? Could you explain?
> 
> What if 'singlethread_cpu' dies?

Still can't understand you. Probably you missed what singlethread_cpu is.

singlethread_cpu is just a "random" bit from cpu_possible_map. Single
threaded workqueue is not bound to any cpu. We only need it to be sure that
percpu_data.ptrs[singlethread_cpu] is populated by __percpu_alloc_mask().

> > > - Any reason why cpu_populated_map is not modified on CPU_DEAD?
> > 
> > Because CPU_DEAD/CPU_UP_CANCELED doesn't wait for cwq->thread to exit.
> > cpu_populated_map never shrinks, it only grows on CPU_UP_PREPARE.
> > 
> > We can change this, but it needs some more code, and I am not sure
> > we need it. Note that a "false" bit in cpu_populated_map only means
> > that flush_work/flush_workqueue/destroy_workqueu will do lock/unlock
> > of cwq->lock, nothing more.
> 
> What abt __create_workqueue/schedule_on_each_cpu?

As I said already __create_workqueue() needs a fix, schedule_on_each_cpu()
is already broken, and should be fixed as well.

> > > - I feel more comfortable if workqueue_cpu_callback were to take
> > >   workqueue_mutex in LOCK_ACQ and release it in LOCK_RELEASE
> > >   notifications.
> > 
> > The whole purpose of this change to avoid this!
> 
> I guess it depends on how __create_workqueue/schedule_on_each_cpu is
> modified (whether we take/release lock upon LOCK_ACQ/LOCK_RELEASE)

Sorry, can't understand this...

> > > Finally, I wonder if these changes will be unnecessary if we move to
> > > process freezer based hotplug locking ...
> > 
> > This change ir not strictly necessary but imho make the code better and
> > shrinks .text by 379 bytes.
> > 
> > But I believe that freezer will change nothing for workqueue. We still
> > need take_over_work(), and hacks like migrate_sequence. And no, CPU_DEAD
> > can't just thaw cwq->thread which was bound to the dead CPU to complete
> > kthread_stop(), we should thaw all processes.
> 
> What abt stopping that thread in CPU_DOWN_PREPARE (before freezing
> processes)? I understand that it may add to the latency, but compared to
> the overall latency of process freezer, I suspect it may not be much.

Srivatsa, why do you think this would be better?

It add to the complexity! What do you mean by "stopping that thread" ?
Kill it? - this is wrong. Make it TASK_STOPPED? - very untrivial and I
can't see how this helps.

Oleg.