Re: page migration patchset

[Ray Bryant <raybry@sgi.com>]

Andi and Steve,

Steve Longerbeam wrote:
<snip>

>>
>> My personal preference would be to keep as much of this as possible
>> under user space control; that is, rather than having a big autonomous
>> system call that migrates pages and then updates policy information,
>> I'd prefer to split the work into several smaller system calls that
>> are issued by a user space program responsible for coordinating the
>> process migration as a series of steps, e. g.:
>>
>> (1)  suspend the process via SIGSTOP
>> (2)  update the mempolicy information
>> (3)  migrate the process's pages
>> (4)  migrate the process to the new cpu via set_schedaffinity()
>> (5)  resume the process via SIGCONT
>>
> 
> steps 2 and 3 can be accomplished by a call to mbind() and
> specifying MPOL_MF_MOVE. And since mbind() takes an
> address range, you could probably migrate pages and change
> the policies for all of the process' mappings in a single mbind()
> call.

OK, I just got around to looking into this suggestion.  Unfortunately,
it doesn't look as if this will do what I want.  I need to be able to
conserve the topology of the application when it is migrated (required
to give the application the same performance in its new location that
it got in its old location).  So, I need to be able to say "take the
pages on this node and move them to that node".  The sys_mbind() call
doesn't have the necessry arguments to do this.  I'm thinking of
something like:

migrate_process_pages(pid, numnodes, oldnodelist, newnodelist);

This would scan the address space of process pid, and each page that
is found on oldnodelist[i] would be moved to node newnodelist[i].

Pages that are found to be swapped out would be handled as follows:
Add the original node id to either the swap pte or the swp_entry_t.
Swap in will be modified to allocate the page on the same node it
came from.  Then, as part of migrate_process_pages, all that would
be done for swapped out pages would be to change the "original node"
field to point at the new node.

However, I could probably do both steps (2) and (3) as part of the
migrate_process_pages() call.

Does this all seem reasonable?

-- 
Best Regards,
Ray
-----------------------------------------------
                   Ray Bryant
512-453-9679 (work)         512-507-7807 (cell)
raybry@sgi.com             raybry@austin.rr.com
The box said: "Requires Windows 98 or better",
            so I installed Linux.
-----------------------------------------------