[PATCH] migration: only migrate_prep() once per move_pages()

[PATCH] migration: only migrate_prep() once per move_pages()

[Brice Goglin]

migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
throughput by breaking it into chunks, but it also made migrate_prep()
be called once per chunk (every 128pages or so) instead of once per
move_pages().

This patch reverts to calling migrate_prep() only once per chunk
as we did before 2.6.29.
It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
    mm: move migrate_prep out from under mmap_sem

This improves migration throughput on the above machine from 600MB/s
to 750MB/s.

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>

diff --git a/mm/migrate.c b/mm/migrate.c
index 068655d..a2d3e83 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 	struct page_to_node *pp;
 	LIST_HEAD(pagelist);
 
-	migrate_prep();
 	down_read(&mm->mmap_sem);
 
 	/*
@@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task,
 	pm = (struct page_to_node *)__get_free_page(GFP_KERNEL);
 	if (!pm)
 		goto out;
+
+	migrate_prep();
+
 	/*
 	 * Store a chunk of page_to_node array in a page,
 	 * but keep the last one as a marker

[PATCH] migration: only migrate_prep() once per move_pages()

[Brice Goglin]

migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
throughput by breaking it into chunks, but it also made migrate_prep()
be called once per chunk (every 128pages or so) instead of once per
move_pages().

This patch reverts to calling migrate_prep() only once per chunk
as we did before 2.6.29.
It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
    mm: move migrate_prep out from under mmap_sem

This improves migration throughput on the above machine from 600MB/s
to 750MB/s.

Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>

diff --git a/mm/migrate.c b/mm/migrate.c
index 068655d..a2d3e83 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 	struct page_to_node *pp;
 	LIST_HEAD(pagelist);
 
-	migrate_prep();
 	down_read(&mm->mmap_sem);
 
 	/*
@@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task,
 	pm = (struct page_to_node *)__get_free_page(GFP_KERNEL);
 	if (!pm)
 		goto out;
+
+	migrate_prep();
+
 	/*
 	 * Store a chunk of page_to_node array in a page,
 	 * but keep the last one as a marker


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[KAMEZAWA Hiroyuki]

On Wed, 15 Apr 2009 09:32:10 +0200
Brice Goglin <Brice.Goglin@ens-lyon.org> wrote:

> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
> throughput by breaking it into chunks, but it also made migrate_prep()
> be called once per chunk (every 128pages or so) instead of once per
> move_pages().
> 
> This patch reverts to calling migrate_prep() only once per chunk
> as we did before 2.6.29.
> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
>     mm: move migrate_prep out from under mmap_sem
> 
> This improves migration throughput on the above machine from 600MB/s
> to 750MB/s.
> 
> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
> 
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>

I think this patch is good. page migration is best-effort syscall ;)

BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?

Thanks,
-Kame


> diff --git a/mm/migrate.c b/mm/migrate.c
> index 068655d..a2d3e83 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
>  	struct page_to_node *pp;
>  	LIST_HEAD(pagelist);
>  
> -	migrate_prep();
>  	down_read(&mm->mmap_sem);
>  
>  	/*
> @@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task,
>  	pm = (struct page_to_node *)__get_free_page(GFP_KERNEL);
>  	if (!pm)
>  		goto out;
> +
> +	migrate_prep();
> +
>  	/*
>  	 * Store a chunk of page_to_node array in a page,
>  	 * but keep the last one as a marker
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[KAMEZAWA Hiroyuki]

On Wed, 15 Apr 2009 09:32:10 +0200
Brice Goglin <Brice.Goglin@ens-lyon.org> wrote:

> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
> throughput by breaking it into chunks, but it also made migrate_prep()
> be called once per chunk (every 128pages or so) instead of once per
> move_pages().
> 
> This patch reverts to calling migrate_prep() only once per chunk
> as we did before 2.6.29.
> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
>     mm: move migrate_prep out from under mmap_sem
> 
> This improves migration throughput on the above machine from 600MB/s
> to 750MB/s.
> 
> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
> 
Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>

I think this patch is good. page migration is best-effort syscall ;)

BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?

Thanks,
-Kame


> diff --git a/mm/migrate.c b/mm/migrate.c
> index 068655d..a2d3e83 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -820,7 +820,6 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
>  	struct page_to_node *pp;
>  	LIST_HEAD(pagelist);
>  
> -	migrate_prep();
>  	down_read(&mm->mmap_sem);
>  
>  	/*
> @@ -907,6 +906,9 @@ static int do_pages_move(struct mm_struct *mm, struct task_struct *task,
>  	pm = (struct page_to_node *)__get_free_page(GFP_KERNEL);
>  	if (!pm)
>  		goto out;
> +
> +	migrate_prep();
> +
>  	/*
>  	 * Store a chunk of page_to_node array in a page,
>  	 * but keep the last one as a marker
> 
> 
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
> 

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Brice Goglin]

KAMEZAWA Hiroyuki wrote:
> On Wed, 15 Apr 2009 09:32:10 +0200
> Brice Goglin <Brice.Goglin@ens-lyon.org> wrote:
>
>   
>> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
>> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
>> throughput by breaking it into chunks, but it also made migrate_prep()
>> be called once per chunk (every 128pages or so) instead of once per
>> move_pages().
>>
>> This patch reverts to calling migrate_prep() only once per chunk
>> as we did before 2.6.29.
>> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
>>     mm: move migrate_prep out from under mmap_sem
>>
>> This improves migration throughput on the above machine from 600MB/s
>> to 750MB/s.
>>
>> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
>>
>>     
> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>
> I think this patch is good. page migration is best-effort syscall ;)
>   

My next feeling now is about improving migrate_prep() itself. It makes
the move_pages() startup overhead very high.

But lru_add_drain_all() touches some code that I am far from
understanding :/ Can we imagine using IPI instead of a deferred
work_struct for this kind of things? Or maybe, for each processor, check
whether drain_cpu_pagevecs() would have something to do before actually
scheduling the local work_struct? It's racy, but migrate_prep() doesn't
guarantee anyway that pages won't be moved out of the LRU before the
actual migration, so...

Also I don't see why the cost of lru_add_drain_all() seems to increase
linearly with the number of cores in the machine. There may be some lock
contention, but it should scale better when there's pretty-much nothing
in the CPU lists...

> BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?
>   

I'd say they ignore it since it doesn't happen often :)

Brice

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Brice Goglin]

KAMEZAWA Hiroyuki wrote:
> On Wed, 15 Apr 2009 09:32:10 +0200
> Brice Goglin <Brice.Goglin@ens-lyon.org> wrote:
>
>   
>> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
>> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
>> throughput by breaking it into chunks, but it also made migrate_prep()
>> be called once per chunk (every 128pages or so) instead of once per
>> move_pages().
>>
>> This patch reverts to calling migrate_prep() only once per chunk
>> as we did before 2.6.29.
>> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
>>     mm: move migrate_prep out from under mmap_sem
>>
>> This improves migration throughput on the above machine from 600MB/s
>> to 750MB/s.
>>
>> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
>>
>>     
> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>
> I think this patch is good. page migration is best-effort syscall ;)
>   

My next feeling now is about improving migrate_prep() itself. It makes
the move_pages() startup overhead very high.

But lru_add_drain_all() touches some code that I am far from
understanding :/ Can we imagine using IPI instead of a deferred
work_struct for this kind of things? Or maybe, for each processor, check
whether drain_cpu_pagevecs() would have something to do before actually
scheduling the local work_struct? It's racy, but migrate_prep() doesn't
guarantee anyway that pages won't be moved out of the LRU before the
actual migration, so...

Also I don't see why the cost of lru_add_drain_all() seems to increase
linearly with the number of cores in the machine. There may be some lock
contention, but it should scale better when there's pretty-much nothing
in the CPU lists...

> BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?
>   

I'd say they ignore it since it doesn't happen often :)

Brice

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Christoph Lameter]

On Wed, 15 Apr 2009, Brice Goglin wrote:

> But lru_add_drain_all() touches some code that I am far from
> understanding :/ Can we imagine using IPI instead of a deferred
> work_struct for this kind of things? Or maybe, for each processor, check
> whether drain_cpu_pagevecs() would have something to do before actually
> scheduling the local work_struct? It's racy, but migrate_prep() doesn't
> guarantee anyway that pages won't be moved out of the LRU before the
> actual migration, so...

IPI means that code must run with interrupts disabled.

> > BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?
> >
>
> I'd say they ignore it since it doesn't happen often :)

Right.

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Christoph Lameter]

On Wed, 15 Apr 2009, Brice Goglin wrote:

> But lru_add_drain_all() touches some code that I am far from
> understanding :/ Can we imagine using IPI instead of a deferred
> work_struct for this kind of things? Or maybe, for each processor, check
> whether drain_cpu_pagevecs() would have something to do before actually
> scheduling the local work_struct? It's racy, but migrate_prep() doesn't
> guarantee anyway that pages won't be moved out of the LRU before the
> actual migration, so...

IPI means that code must run with interrupts disabled.

> > BTW, current users of sys_move_pages() does retry when it gets -EBUSY ?
> >
>
> I'd say they ignore it since it doesn't happen often :)

Right.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Brice Goglin]

KAMEZAWA Hiroyuki wrote:
> On Wed, 15 Apr 2009 09:32:10 +0200
> Brice Goglin <Brice.Goglin@ens-lyon.org> wrote:
>
>   
>> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
>> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
>> throughput by breaking it into chunks, but it also made migrate_prep()
>> be called once per chunk (every 128pages or so) instead of once per
>> move_pages().
>>
>> This patch reverts to calling migrate_prep() only once per chunk
>> as we did before 2.6.29.
>> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
>>     mm: move migrate_prep out from under mmap_sem
>>
>> This improves migration throughput on the above machine from 600MB/s
>> to 750MB/s.
>>
>> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
>>
>>     
> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>
> I think this patch is good. page migration is best-effort syscall ;)
>   

Since nobody complained about this patch, may I get a Ack and get the
patch merged for 2.6.31?

thanks,
Brice

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Brice Goglin]

KAMEZAWA Hiroyuki wrote:
> On Wed, 15 Apr 2009 09:32:10 +0200
> Brice Goglin <Brice.Goglin@ens-lyon.org> wrote:
>
>   
>> migrate_prep() is fairly expensive (72us on 16-core barcelona 1.9GHz).
>> Commit 3140a2273009c01c27d316f35ab76a37e105fdd8 improved move_pages()
>> throughput by breaking it into chunks, but it also made migrate_prep()
>> be called once per chunk (every 128pages or so) instead of once per
>> move_pages().
>>
>> This patch reverts to calling migrate_prep() only once per chunk
>> as we did before 2.6.29.
>> It is also a followup to commit 0aedadf91a70a11c4a3e7c7d99b21e5528af8d5d
>>     mm: move migrate_prep out from under mmap_sem
>>
>> This improves migration throughput on the above machine from 600MB/s
>> to 750MB/s.
>>
>> Signed-off-by: Brice Goglin <Brice.Goglin@inria.fr>
>>
>>     
> Reviewed-by: KAMEZAWA Hiroyuki <kamezawa.hiroyu@jp.fujitsu.com>
>
> I think this patch is good. page migration is best-effort syscall ;)
>   

Since nobody complained about this patch, may I get a Ack and get the
patch merged for 2.6.31?

thanks,
Brice

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Christoph Lameter]

I acked it.

Re: [PATCH] migration: only migrate_prep() once per move_pages()

[Christoph Lameter]

I acked it.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>