Re: [WiP]: aio support for migrating pages (Re: [PATCH V2 1/2] mm: hotplug: implement non-movable version of get_user_pages() called get_user_pages_non_movable())

[Benjamin LaHaise <bcrl@kvack.org>]

On Fri, May 17, 2013 at 11:28:52AM +0800, Tang Chen wrote:
> Hi Benjamin,
> 
> Thank you very much for your idea. :)
> 
> I have no objection to your idea, but seeing from your patch, this only
> works for aio subsystem because you changed the way to allocate the aio
> ring pages, with a file mapping.

That is correct.  There is no way you're going to be able to solve this 
problem without dealing with the issue on a subsystem by subsystem basis.

> So far as I know, not only aio, but also other subsystems, such CMA, will
> also have problem like this. The page cannot be migrated because it is
> pinned in memory. So I think we should work out a common way to solve how
> to migrate pinned pages.

A generic approach would require hardware support, but I doubt that is 
going to happen.

> I'm working in the way Mel has said, migrate_unpin() and migrate_pin()
> callbacks. But as you saw, I met some problems, like I don't where to put
> these two callbacks. And discussed with you guys, I want to try this:
> 
> 1. Add a new member to struct page, used to remember the pin holders of
>    this page, including the pin and unpin callbacks and the necessary data.
>    This is more like a callback chain.
>    (I'm worry about this step, I'm not sure if it is good enough. After 
> all,
>     we need a good place to put the callbacks.)

Putting function pointers into struct page is not going to happen.  You'd 
be adding a significant amount of memory overhead for something that is 
never going to be used on the vast majority of systems (2 function pointers 
would be 16 bytes per page on a 64 bit system).  Keep in mind that distro 
kernels tend to enable almost all config options on their kernels, so the 
overhead of any approach has to make sense for the users of the kernel that 
will never make use of this kind of migration.

> And then, like Mel said,
> 
> 2. Implement the callbacks in the subsystems, and register them to the
>    new member in struct page.

No, the hook should be in the address_space_operations.  We already have 
a pointer to an address space in struct page.  This avoids adding more 
overhead to struct page.

> 3. Call these callbacks before and after migration.

How is that better than using the existing hook in address_space_operations?

> I think I'll send a RFC patch next week when I finished the outline. I'm
> just thinking of finding a common way to solve this problem that all the
> other subsystems will benefit.

Before pursuing this approach, make sure you've got buy-in for all of the 
overhead you're adding to the system.  I don't think that growing struct 
page is going to be an acceptable design choice given the amount of 
overhead it will incur.

> Thanks. :)

Cheers,

		-ben
-- 
"Thought is the essence of where you are now."