Re: [PATCH] cpuset: fix allocating page cache/slab object on the unallowed node when memory spread is set

[Paul Menage <menage@google.com>]

On Tue, Feb 3, 2009 at 2:16 PM, Andrew Morton <akpm@linux-foundation.org> wrote:
> On Tue, 03 Feb 2009 11:25:25 +0800
> Miao Xie <miaox@cn.fujitsu.com> wrote:
>
>> on 2009-1-28 6:42 Andrew Morton wrote:
>> > On Wed, 21 Jan 2009 16:06:20 +0800
>> > Miao Xie <miaox@cn.fujitsu.com> wrote:
>> >
>> >> The task still allocated the page caches on old node after modifying its
>> >> cpuset's mems when 'memory_spread_page' was set, it is caused by the old
>> >> mem_allowed_list of the task, the current kernel doesn't updates it unless some
>> >> function invokes cpuset_update_task_memory_state(), it is too late sometimes.
>> >> We must update the mem_allowed_list of the tasks in time.
>> >>
>> >> Slab has the same problem.
>> >>
>> >> We fixes the bug by updating tasks' mem_allowed_list and spread flag after
>> >> its cpuset's mems or spread flag is changed.
>> >>
>> >>
>> >> ...
>> >>
>> >> --- a/kernel/kthread.c
>> >> +++ b/kernel/kthread.c
>> >> @@ -242,6 +242,7 @@ int kthreadd(void *unused)
>> >>    set_user_nice(tsk, KTHREAD_NICE_LEVEL);
>> >>    set_cpus_allowed_ptr(tsk, CPU_MASK_ALL_PTR);
>> >>
>> >> +  current->mems_allowed = node_possible_map;
>> >>    current->flags |= PF_NOFREEZE | PF_FREEZER_NOSIG;
>> >
>> > Why this change?  kthreadd() is called from rest_init(), before anyone
>> > has had a chance to alter ->mems_allowed?
>>
>> I found that after mems_allowed of kthreadd was not initialized applying this patch,
>> every bit of it is 1, so...
>> Maybe it is redundant.
>
> I think it is redundant.  kthreadd's mems_allowed _should_ be all-ones.
> Or at least, all-nodes-allowed.
>
> I wasn't able to find out where the setting of init'smems_allowed
> happens, after a bit of grepping and hunting.  It should be done within
> INIT_TASK, but isn't.
>
> Still, kthreadd is reliably parented by swapper, and there shold be no
> need to alter its mems_allowed.
>
> Similarly, what was the reason for setting current->mems_allowed in
> kernel_init()?  That also should be unneeded.
>
> Finally, I've somewhat lost track of where we are with this patch.
> Paul, do you see any other remaining issues?

AFAICS this patch still has a race between a thread reading its
mems_allowed, and another thread updating it. The current architecture
of having task->mems_allowed be only updatable by current was PaulJ's
code originally, and I'm a bit loathe to touch it. But if we're going
to, we'll need at the minimum to add a lock for any code that touches
current->mems_allowed.

Paul