All of lore.kernel.org
 help / color / mirror / Atom feed
* [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 18:14 Larry Woodman
  2012-03-22 18:22   ` Rik van Riel
  2012-03-22 18:45   ` KOSAKI Motohiro
  0 siblings, 2 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 18:14 UTC (permalink / raw)
  To: linux-mm
  Cc: Linux Kernel Mailing List, Andrew Morton, Rik van Riel, Motohiro Kosaki

[-- Attachment #1: Type: text/plain, Size: 1233 bytes --]

While moving tasks between cpusets I noticed some strange behavior.  Specifically if the nodes of the destination
cpuset are a subset of the nodes of the source cpuset do_migrate_pages() will move pages that are already on a node
in the destination cpuset.  The reason for this is do_migrate_pages() does not check whether each node in the source
nodemask is in the destination nodemask before calling migrate_to_node().  If we simply do this check and skip them
when the source is in the destination moving we wont move nodes that dont need to be moved.
  
Adding a little debug printk to migrate_to_node():

Without this change migrating tasks from a cpuset containing nodes 0-7 to a cpuset containing nodes 3-4, we migrate
from ALL the nodes even if they are in the both the source and destination nodesets:

   Migrating 7 to 4
   Migrating 6 to 3
   Migrating 5 to 4
   Migrating 4 to 3
   Migrating 1 to 4
   Migrating 3 to 4
   Migrating 0 to 3
   Migrating 2 to 3


With this change we only migrate from nodes that are not in the destination nodesets:

   Migrating 7 to 4
   Migrating 6 to 3
   Migrating 5 to 4
   Migrating 2 to 3
   Migrating 1 to 4
   Migrating 0 to 3

Signed-off-by: Larry Woodman<lwoodman@redhat.com>


[-- Attachment #2: upstream-do_migrate_pages.patch --]
[-- Type: text/plain, Size: 407 bytes --]

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 47296fe..2bd13e9 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1012,6 +1012,9 @@ int do_migrate_pages(struct mm_struct *mm,
 		int dest = 0;
 
 		for_each_node_mask(s, tmp) {
+			/* no need to move if its already there */
+			if (node_isset(s, *to_nodes))
+				continue;
 			d = node_remap(s, *from_nodes, *to_nodes);
 			if (s == d)
 				continue;

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:14 [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node Larry Woodman
@ 2012-03-22 18:22   ` Rik van Riel
  2012-03-22 18:45   ` KOSAKI Motohiro
  1 sibling, 0 replies; 34+ messages in thread
From: Rik van Riel @ 2012-03-22 18:22 UTC (permalink / raw)
  To: lwoodman
  Cc: linux-mm, Linux Kernel Mailing List, Andrew Morton, Motohiro Kosaki

On 03/22/2012 02:14 PM, Larry Woodman wrote:

> With this change we only migrate from nodes that are not in the
> destination nodesets:

That's a pretty obvious improvement :)

> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 2 to 3
> Migrating 1 to 4
> Migrating 0 to 3
>
> Signed-off-by: Larry Woodman<lwoodman@redhat.com>

Acked-by: Rik van Riel <riel@redhat.com>


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 18:22   ` Rik van Riel
  0 siblings, 0 replies; 34+ messages in thread
From: Rik van Riel @ 2012-03-22 18:22 UTC (permalink / raw)
  To: lwoodman
  Cc: linux-mm, Linux Kernel Mailing List, Andrew Morton, Motohiro Kosaki

On 03/22/2012 02:14 PM, Larry Woodman wrote:

> With this change we only migrate from nodes that are not in the
> destination nodesets:

That's a pretty obvious improvement :)

> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 2 to 3
> Migrating 1 to 4
> Migrating 0 to 3
>
> Signed-off-by: Larry Woodman<lwoodman@redhat.com>

Acked-by: Rik van Riel <riel@redhat.com>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:14 [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node Larry Woodman
@ 2012-03-22 18:45   ` KOSAKI Motohiro
  2012-03-22 18:45   ` KOSAKI Motohiro
  1 sibling, 0 replies; 34+ messages in thread
From: KOSAKI Motohiro @ 2012-03-22 18:45 UTC (permalink / raw)
  To: lwoodman, Christoph Lameter
  Cc: linux-mm, Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki, kosaki.motohiro

CC to Christoph.

> While moving tasks between cpusets I noticed some strange behavior. Specifically if the nodes of the destination
> cpuset are a subset of the nodes of the source cpuset do_migrate_pages() will move pages that are already on a node
> in the destination cpuset. The reason for this is do_migrate_pages() does not check whether each node in the source
> nodemask is in the destination nodemask before calling migrate_to_node(). If we simply do this check and skip them
> when the source is in the destination moving we wont move nodes that dont need to be moved.
>
> Adding a little debug printk to migrate_to_node():
>
> Without this change migrating tasks from a cpuset containing nodes 0-7 to a cpuset containing nodes 3-4, we migrate
> from ALL the nodes even if they are in the both the source and destination nodesets:
>
> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 4 to 3
> Migrating 1 to 4
> Migrating 3 to 4
> Migrating 0 to 3
> Migrating 2 to 3

Wait.

This may be non-optimal for cpusets, but maybe optimal migrate_pages, especially
the usecase is HPC. I guess this is intended behavior. I think we need to hear
Christoph's intention.

But, I'm not against this if he has no objection.




>
>
> With this change we only migrate from nodes that are not in the destination nodesets:
>
> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 2 to 3
> Migrating 1 to 4
> Migrating 0 to 3
>
> Signed-off-by: Larry Woodman<lwoodman@redhat.com>
>


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 18:45   ` KOSAKI Motohiro
  0 siblings, 0 replies; 34+ messages in thread
From: KOSAKI Motohiro @ 2012-03-22 18:45 UTC (permalink / raw)
  To: lwoodman, Christoph Lameter
  Cc: linux-mm, Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki, kosaki.motohiro

CC to Christoph.

> While moving tasks between cpusets I noticed some strange behavior. Specifically if the nodes of the destination
> cpuset are a subset of the nodes of the source cpuset do_migrate_pages() will move pages that are already on a node
> in the destination cpuset. The reason for this is do_migrate_pages() does not check whether each node in the source
> nodemask is in the destination nodemask before calling migrate_to_node(). If we simply do this check and skip them
> when the source is in the destination moving we wont move nodes that dont need to be moved.
>
> Adding a little debug printk to migrate_to_node():
>
> Without this change migrating tasks from a cpuset containing nodes 0-7 to a cpuset containing nodes 3-4, we migrate
> from ALL the nodes even if they are in the both the source and destination nodesets:
>
> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 4 to 3
> Migrating 1 to 4
> Migrating 3 to 4
> Migrating 0 to 3
> Migrating 2 to 3

Wait.

This may be non-optimal for cpusets, but maybe optimal migrate_pages, especially
the usecase is HPC. I guess this is intended behavior. I think we need to hear
Christoph's intention.

But, I'm not against this if he has no objection.




>
>
> With this change we only migrate from nodes that are not in the destination nodesets:
>
> Migrating 7 to 4
> Migrating 6 to 3
> Migrating 5 to 4
> Migrating 2 to 3
> Migrating 1 to 4
> Migrating 0 to 3
>
> Signed-off-by: Larry Woodman<lwoodman@redhat.com>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:45   ` KOSAKI Motohiro
@ 2012-03-22 18:47     ` Rik van Riel
  -1 siblings, 0 replies; 34+ messages in thread
From: Rik van Riel @ 2012-03-22 18:47 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: lwoodman, Christoph Lameter, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Motohiro Kosaki

On 03/22/2012 02:45 PM, KOSAKI Motohiro wrote:
> CC to Christoph.
>
>> While moving tasks between cpusets I noticed some strange behavior.
>> Specifically if the nodes of the destination
>> cpuset are a subset of the nodes of the source cpuset
>> do_migrate_pages() will move pages that are already on a node
>> in the destination cpuset. The reason for this is do_migrate_pages()
>> does not check whether each node in the source
>> nodemask is in the destination nodemask before calling
>> migrate_to_node(). If we simply do this check and skip them
>> when the source is in the destination moving we wont move nodes that
>> dont need to be moved.
>>
>> Adding a little debug printk to migrate_to_node():
>>
>> Without this change migrating tasks from a cpuset containing nodes 0-7
>> to a cpuset containing nodes 3-4, we migrate
>> from ALL the nodes even if they are in the both the source and
>> destination nodesets:
>>
>> Migrating 7 to 4
>> Migrating 6 to 3
>> Migrating 5 to 4
>> Migrating 4 to 3
>> Migrating 1 to 4
>> Migrating 3 to 4
>> Migrating 0 to 3
>> Migrating 2 to 3
>
> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need
> to hear
> Christoph's intention.

How is the moving of pages that are already in the
intended cpuset better than just leaving them alone?

> But, I'm not against this if he has no objection.

Sure. Christoph?

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 18:47     ` Rik van Riel
  0 siblings, 0 replies; 34+ messages in thread
From: Rik van Riel @ 2012-03-22 18:47 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: lwoodman, Christoph Lameter, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Motohiro Kosaki

On 03/22/2012 02:45 PM, KOSAKI Motohiro wrote:
> CC to Christoph.
>
>> While moving tasks between cpusets I noticed some strange behavior.
>> Specifically if the nodes of the destination
>> cpuset are a subset of the nodes of the source cpuset
>> do_migrate_pages() will move pages that are already on a node
>> in the destination cpuset. The reason for this is do_migrate_pages()
>> does not check whether each node in the source
>> nodemask is in the destination nodemask before calling
>> migrate_to_node(). If we simply do this check and skip them
>> when the source is in the destination moving we wont move nodes that
>> dont need to be moved.
>>
>> Adding a little debug printk to migrate_to_node():
>>
>> Without this change migrating tasks from a cpuset containing nodes 0-7
>> to a cpuset containing nodes 3-4, we migrate
>> from ALL the nodes even if they are in the both the source and
>> destination nodesets:
>>
>> Migrating 7 to 4
>> Migrating 6 to 3
>> Migrating 5 to 4
>> Migrating 4 to 3
>> Migrating 1 to 4
>> Migrating 3 to 4
>> Migrating 0 to 3
>> Migrating 2 to 3
>
> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need
> to hear
> Christoph's intention.

How is the moving of pages that are already in the
intended cpuset better than just leaving them alone?

> But, I'm not against this if he has no objection.

Sure. Christoph?

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:45   ` KOSAKI Motohiro
@ 2012-03-22 18:49     ` Larry Woodman
  -1 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 18:49 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: Christoph Lameter, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/22/2012 02:45 PM, KOSAKI Motohiro wrote:
> CC to Christoph.
>
>
Sorry Christoph I meant to cc you on the original message!

Larry

> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages, 
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need 
> to hear
> Christoph's intention.
>
> But, I'm not against this if he has no objection.
>
>


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 18:49     ` Larry Woodman
  0 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 18:49 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: Christoph Lameter, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/22/2012 02:45 PM, KOSAKI Motohiro wrote:
> CC to Christoph.
>
>
Sorry Christoph I meant to cc you on the original message!

Larry

> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages, 
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need 
> to hear
> Christoph's intention.
>
> But, I'm not against this if he has no objection.
>
>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:45   ` KOSAKI Motohiro
@ 2012-03-22 18:51     ` Christoph Lameter
  -1 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-22 18:51 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: lwoodman, linux-mm, Linux Kernel Mailing List, Andrew Morton,
	Rik van Riel, Motohiro Kosaki

On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:

> CC to Christoph.
>
> > While moving tasks between cpusets I noticed some strange behavior.
> > Specifically if the nodes of the destination
> > cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
> > will move pages that are already on a node
> > in the destination cpuset. The reason for this is do_migrate_pages() does
> > not check whether each node in the source
> > nodemask is in the destination nodemask before calling migrate_to_node(). If
> > we simply do this check and skip them
> > when the source is in the destination moving we wont move nodes that dont
> > need to be moved.
> >
> > Adding a little debug printk to migrate_to_node():
> >
> > Without this change migrating tasks from a cpuset containing nodes 0-7 to a
> > cpuset containing nodes 3-4, we migrate
> > from ALL the nodes even if they are in the both the source and destination
> > nodesets:
> >
> > Migrating 7 to 4
> > Migrating 6 to 3
> > Migrating 5 to 4
> > Migrating 4 to 3
> > Migrating 1 to 4
> > Migrating 3 to 4
> > Migrating 0 to 3
> > Migrating 2 to 3
>
> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need to hear
> Christoph's intention.
>
> But, I'm not against this if he has no objection.

The use case for this is if you have an app running on nodes 3,4,5 on your
machine and now you want to shift it to 4,5,6. The expectation is that the
location of the pages relative to the first node stay the same.
Application may manage their locality given a range of nodes and each of
the x .. x+n nodes has their particular purpose.

If you justd copy 3 to 6 then the app may get confused when doing
additional allocations since different types of information is now stored
on the "first" node (which is now 4).




^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 18:51     ` Christoph Lameter
  0 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-22 18:51 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: lwoodman, linux-mm, Linux Kernel Mailing List, Andrew Morton,
	Rik van Riel, Motohiro Kosaki

On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:

> CC to Christoph.
>
> > While moving tasks between cpusets I noticed some strange behavior.
> > Specifically if the nodes of the destination
> > cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
> > will move pages that are already on a node
> > in the destination cpuset. The reason for this is do_migrate_pages() does
> > not check whether each node in the source
> > nodemask is in the destination nodemask before calling migrate_to_node(). If
> > we simply do this check and skip them
> > when the source is in the destination moving we wont move nodes that dont
> > need to be moved.
> >
> > Adding a little debug printk to migrate_to_node():
> >
> > Without this change migrating tasks from a cpuset containing nodes 0-7 to a
> > cpuset containing nodes 3-4, we migrate
> > from ALL the nodes even if they are in the both the source and destination
> > nodesets:
> >
> > Migrating 7 to 4
> > Migrating 6 to 3
> > Migrating 5 to 4
> > Migrating 4 to 3
> > Migrating 1 to 4
> > Migrating 3 to 4
> > Migrating 0 to 3
> > Migrating 2 to 3
>
> Wait.
>
> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
> especially
> the usecase is HPC. I guess this is intended behavior. I think we need to hear
> Christoph's intention.
>
> But, I'm not against this if he has no objection.

The use case for this is if you have an app running on nodes 3,4,5 on your
machine and now you want to shift it to 4,5,6. The expectation is that the
location of the pages relative to the first node stay the same.
Application may manage their locality given a range of nodes and each of
the x .. x+n nodes has their particular purpose.

If you justd copy 3 to 6 then the app may get confused when doing
additional allocations since different types of information is now stored
on the "first" node (which is now 4).



--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:51     ` Christoph Lameter
@ 2012-03-22 19:07       ` Larry Woodman
  -1 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 19:07 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/22/2012 02:51 PM, Christoph Lameter wrote:
> On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:
>
>> CC to Christoph.
>>
>>> While moving tasks between cpusets I noticed some strange behavior.
>>> Specifically if the nodes of the destination
>>> cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
>>> will move pages that are already on a node
>>> in the destination cpuset. The reason for this is do_migrate_pages() does
>>> not check whether each node in the source
>>> nodemask is in the destination nodemask before calling migrate_to_node(). If
>>> we simply do this check and skip them
>>> when the source is in the destination moving we wont move nodes that dont
>>> need to be moved.
>>>
>>> Adding a little debug printk to migrate_to_node():
>>>
>>> Without this change migrating tasks from a cpuset containing nodes 0-7 to a
>>> cpuset containing nodes 3-4, we migrate
>>> from ALL the nodes even if they are in the both the source and destination
>>> nodesets:
>>>
>>> Migrating 7 to 4
>>> Migrating 6 to 3
>>> Migrating 5 to 4
>>> Migrating 4 to 3
>>> Migrating 1 to 4
>>> Migrating 3 to 4
>>> Migrating 0 to 3
>>> Migrating 2 to 3
>> Wait.
>>
>> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
>> especially
>> the usecase is HPC. I guess this is intended behavior. I think we need to hear
>> Christoph's intention.
>>
>> But, I'm not against this if he has no objection.
> The use case for this is if you have an app running on nodes 3,4,5 on your
> machine and now you want to shift it to 4,5,6. The expectation is that the
> location of the pages relative to the first node stay the same.
> Application may manage their locality given a range of nodes and each of
> the x .. x+n nodes has their particular purpose.
So to be clear on this, in that case the intention would be move 3 to 4, 
4 to 5 and 5 to 6
to keep the node ordering the same?

Larry
> If you justd copy 3 to 6 then the app may get confused when doing
> additional allocations since different types of information is now stored
> on the "first" node (which is now 4).
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email:<a href=mailto:"dont@kvack.org">  email@kvack.org</a>


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 19:07       ` Larry Woodman
  0 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 19:07 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/22/2012 02:51 PM, Christoph Lameter wrote:
> On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:
>
>> CC to Christoph.
>>
>>> While moving tasks between cpusets I noticed some strange behavior.
>>> Specifically if the nodes of the destination
>>> cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
>>> will move pages that are already on a node
>>> in the destination cpuset. The reason for this is do_migrate_pages() does
>>> not check whether each node in the source
>>> nodemask is in the destination nodemask before calling migrate_to_node(). If
>>> we simply do this check and skip them
>>> when the source is in the destination moving we wont move nodes that dont
>>> need to be moved.
>>>
>>> Adding a little debug printk to migrate_to_node():
>>>
>>> Without this change migrating tasks from a cpuset containing nodes 0-7 to a
>>> cpuset containing nodes 3-4, we migrate
>>> from ALL the nodes even if they are in the both the source and destination
>>> nodesets:
>>>
>>> Migrating 7 to 4
>>> Migrating 6 to 3
>>> Migrating 5 to 4
>>> Migrating 4 to 3
>>> Migrating 1 to 4
>>> Migrating 3 to 4
>>> Migrating 0 to 3
>>> Migrating 2 to 3
>> Wait.
>>
>> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
>> especially
>> the usecase is HPC. I guess this is intended behavior. I think we need to hear
>> Christoph's intention.
>>
>> But, I'm not against this if he has no objection.
> The use case for this is if you have an app running on nodes 3,4,5 on your
> machine and now you want to shift it to 4,5,6. The expectation is that the
> location of the pages relative to the first node stay the same.
> Application may manage their locality given a range of nodes and each of
> the x .. x+n nodes has their particular purpose.
So to be clear on this, in that case the intention would be move 3 to 4, 
4 to 5 and 5 to 6
to keep the node ordering the same?

Larry
> If you justd copy 3 to 6 then the app may get confused when doing
> additional allocations since different types of information is now stored
> on the "first" node (which is now 4).
>
>
>
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
> Don't email:<a href=mailto:"dont@kvack.org">  email@kvack.org</a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 18:51     ` Christoph Lameter
@ 2012-03-22 19:07       ` KOSAKI Motohiro
  -1 siblings, 0 replies; 34+ messages in thread
From: KOSAKI Motohiro @ 2012-03-22 19:07 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, lwoodman, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

(3/22/12 2:51 PM), Christoph Lameter wrote:
> On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:
>
>> CC to Christoph.
>>
>>> While moving tasks between cpusets I noticed some strange behavior.
>>> Specifically if the nodes of the destination
>>> cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
>>> will move pages that are already on a node
>>> in the destination cpuset. The reason for this is do_migrate_pages() does
>>> not check whether each node in the source
>>> nodemask is in the destination nodemask before calling migrate_to_node(). If
>>> we simply do this check and skip them
>>> when the source is in the destination moving we wont move nodes that dont
>>> need to be moved.
>>>
>>> Adding a little debug printk to migrate_to_node():
>>>
>>> Without this change migrating tasks from a cpuset containing nodes 0-7 to a
>>> cpuset containing nodes 3-4, we migrate
>>> from ALL the nodes even if they are in the both the source and destination
>>> nodesets:
>>>
>>> Migrating 7 to 4
>>> Migrating 6 to 3
>>> Migrating 5 to 4
>>> Migrating 4 to 3
>>> Migrating 1 to 4
>>> Migrating 3 to 4
>>> Migrating 0 to 3
>>> Migrating 2 to 3
>>
>> Wait.
>>
>> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
>> especially
>> the usecase is HPC. I guess this is intended behavior. I think we need to hear
>> Christoph's intention.
>>
>> But, I'm not against this if he has no objection.
>
> The use case for this is if you have an app running on nodes 3,4,5 on your
> machine and now you want to shift it to 4,5,6. The expectation is that the
> location of the pages relative to the first node stay the same.
> Application may manage their locality given a range of nodes and each of
> the x .. x+n nodes has their particular purpose.
>
> If you justd copy 3 to 6 then the app may get confused when doing
> additional allocations since different types of information is now stored
> on the "first" node (which is now 4).

MPOL_INTERLEAVE is more simple situaltion. applications naturally assume the
memory is mapped intealeaving and application threads optimize for it. if we
broke intereaving, the applications may slow down.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 19:07       ` KOSAKI Motohiro
  0 siblings, 0 replies; 34+ messages in thread
From: KOSAKI Motohiro @ 2012-03-22 19:07 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, lwoodman, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

(3/22/12 2:51 PM), Christoph Lameter wrote:
> On Thu, 22 Mar 2012, KOSAKI Motohiro wrote:
>
>> CC to Christoph.
>>
>>> While moving tasks between cpusets I noticed some strange behavior.
>>> Specifically if the nodes of the destination
>>> cpuset are a subset of the nodes of the source cpuset do_migrate_pages()
>>> will move pages that are already on a node
>>> in the destination cpuset. The reason for this is do_migrate_pages() does
>>> not check whether each node in the source
>>> nodemask is in the destination nodemask before calling migrate_to_node(). If
>>> we simply do this check and skip them
>>> when the source is in the destination moving we wont move nodes that dont
>>> need to be moved.
>>>
>>> Adding a little debug printk to migrate_to_node():
>>>
>>> Without this change migrating tasks from a cpuset containing nodes 0-7 to a
>>> cpuset containing nodes 3-4, we migrate
>>> from ALL the nodes even if they are in the both the source and destination
>>> nodesets:
>>>
>>> Migrating 7 to 4
>>> Migrating 6 to 3
>>> Migrating 5 to 4
>>> Migrating 4 to 3
>>> Migrating 1 to 4
>>> Migrating 3 to 4
>>> Migrating 0 to 3
>>> Migrating 2 to 3
>>
>> Wait.
>>
>> This may be non-optimal for cpusets, but maybe optimal migrate_pages,
>> especially
>> the usecase is HPC. I guess this is intended behavior. I think we need to hear
>> Christoph's intention.
>>
>> But, I'm not against this if he has no objection.
>
> The use case for this is if you have an app running on nodes 3,4,5 on your
> machine and now you want to shift it to 4,5,6. The expectation is that the
> location of the pages relative to the first node stay the same.
> Application may manage their locality given a range of nodes and each of
> the x .. x+n nodes has their particular purpose.
>
> If you justd copy 3 to 6 then the app may get confused when doing
> additional allocations since different types of information is now stored
> on the "first" node (which is now 4).

MPOL_INTERLEAVE is more simple situaltion. applications naturally assume the
memory is mapped intealeaving and application threads optimize for it. if we
broke intereaving, the applications may slow down.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 19:07       ` Larry Woodman
@ 2012-03-22 19:30         ` Christoph Lameter
  -1 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-22 19:30 UTC (permalink / raw)
  To: Larry Woodman
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On Thu, 22 Mar 2012, Larry Woodman wrote:

> > Application may manage their locality given a range of nodes and each of
> > the x .. x+n nodes has their particular purpose.
> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
> 5 and 5 to 6
> to keep the node ordering the same?

Yup. Have a look at do_migrate_pages and the descrition in the comment by
there by Paul Jackson.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 19:30         ` Christoph Lameter
  0 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-22 19:30 UTC (permalink / raw)
  To: Larry Woodman
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On Thu, 22 Mar 2012, Larry Woodman wrote:

> > Application may manage their locality given a range of nodes and each of
> > the x .. x+n nodes has their particular purpose.
> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
> 5 and 5 to 6
> to keep the node ordering the same?

Yup. Have a look at do_migrate_pages and the descrition in the comment by
there by Paul Jackson.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 19:07       ` Larry Woodman
  (?)
  (?)
@ 2012-03-22 19:36       ` Valdis.Kletnieks
  2012-03-22 19:41           ` Christoph Lameter
  2012-03-22 20:14           ` Larry Woodman
  -1 siblings, 2 replies; 34+ messages in thread
From: Valdis.Kletnieks @ 2012-03-22 19:36 UTC (permalink / raw)
  To: lwoodman
  Cc: Christoph Lameter, KOSAKI Motohiro, linux-mm,
	Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki

[-- Attachment #1: Type: text/plain, Size: 330 bytes --]

On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:

> So to be clear on this, in that case the intention would be move 3 to 4,
> 4 to 5 and 5 to 6
> to keep the node ordering the same?

Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
from 3 to 4 before clearing the old 4 stuff out, it might get crowded?


[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 19:36       ` Valdis.Kletnieks
@ 2012-03-22 19:41           ` Christoph Lameter
  2012-03-22 20:14           ` Larry Woodman
  1 sibling, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-22 19:41 UTC (permalink / raw)
  To: Valdis.Kletnieks
  Cc: lwoodman, KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On Thu, 22 Mar 2012, Valdis.Kletnieks@vt.edu wrote:

> > So to be clear on this, in that case the intention would be move 3 to 4,
> > 4 to 5 and 5 to 6
> > to keep the node ordering the same?
>
> Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> from 3 to 4 before clearing the old 4 stuff out, it might get crowded?

Right. I thought Paul did take care of that way back when it was written?



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 19:41           ` Christoph Lameter
  0 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-22 19:41 UTC (permalink / raw)
  To: Valdis.Kletnieks
  Cc: lwoodman, KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On Thu, 22 Mar 2012, Valdis.Kletnieks@vt.edu wrote:

> > So to be clear on this, in that case the intention would be move 3 to 4,
> > 4 to 5 and 5 to 6
> > to keep the node ordering the same?
>
> Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> from 3 to 4 before clearing the old 4 stuff out, it might get crowded?

Right. I thought Paul did take care of that way back when it was written?


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 19:36       ` Valdis.Kletnieks
@ 2012-03-22 20:14           ` Larry Woodman
  2012-03-22 20:14           ` Larry Woodman
  1 sibling, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 20:14 UTC (permalink / raw)
  To: Valdis.Kletnieks
  Cc: Christoph Lameter, KOSAKI Motohiro, linux-mm,
	Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki

On 03/22/2012 03:36 PM, Valdis.Kletnieks@vt.edu wrote:
> On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:
>
>> So to be clear on this, in that case the intention would be move 3 to 4,
>> 4 to 5 and 5 to 6
>> to keep the node ordering the same?
> Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> from 3 to 4 before clearing the old 4 stuff out, it might get crowded?
>
Yes, I didnt try to imply the order in which pages were moved just
the additional moving necessary.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-22 20:14           ` Larry Woodman
  0 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-22 20:14 UTC (permalink / raw)
  To: Valdis.Kletnieks
  Cc: Christoph Lameter, KOSAKI Motohiro, linux-mm,
	Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki

On 03/22/2012 03:36 PM, Valdis.Kletnieks@vt.edu wrote:
> On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:
>
>> So to be clear on this, in that case the intention would be move 3 to 4,
>> 4 to 5 and 5 to 6
>> to keep the node ordering the same?
> Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> from 3 to 4 before clearing the old 4 stuff out, it might get crowded?
>
Yes, I didnt try to imply the order in which pages were moved just
the additional moving necessary.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 20:14           ` Larry Woodman
  (?)
@ 2012-03-23  1:21           ` Valdis.Kletnieks
  -1 siblings, 0 replies; 34+ messages in thread
From: Valdis.Kletnieks @ 2012-03-23  1:21 UTC (permalink / raw)
  To: lwoodman
  Cc: Christoph Lameter, KOSAKI Motohiro, linux-mm,
	Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki

[-- Attachment #1: Type: text/plain, Size: 587 bytes --]

On Thu, 22 Mar 2012 16:14:04 -0400, Larry Woodman said:
> On 03/22/2012 03:36 PM, Valdis.Kletnieks@vt.edu wrote:
> > On Thu, 22 Mar 2012 15:07:00 -0400, Larry Woodman said:
> >
> >> So to be clear on this, in that case the intention would be move 3 to 4,
> >> 4 to 5 and 5 to 6
> >> to keep the node ordering the same?
> > Would it make more sense to do 5->6, 4->5, 3->4?  If we move stuff
> > from 3 to 4 before clearing the old 4 stuff out, it might get crowded?
> >
> Yes, I didnt try to imply the order in which pages were moved just
> the additional moving necessary.

Oh, OK.. :)


[-- Attachment #2: Type: application/pgp-signature, Size: 865 bytes --]

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-22 19:30         ` Christoph Lameter
  (?)
@ 2012-03-29 18:00         ` Larry Woodman
  2012-03-29 19:43             ` KOSAKI Motohiro
  -1 siblings, 1 reply; 34+ messages in thread
From: Larry Woodman @ 2012-03-29 18:00 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

[-- Attachment #1: Type: text/plain, Size: 521 bytes --]

On 03/22/2012 03:30 PM, Christoph Lameter wrote:
> On Thu, 22 Mar 2012, Larry Woodman wrote:
>
>>> Application may manage their locality given a range of nodes and each of
>>> the x .. x+n nodes has their particular purpose.
>> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
>> 5 and 5 to 6
>> to keep the node ordering the same?
> Yup. Have a look at do_migrate_pages and the descrition in the comment by
> there by Paul Jackson.
>
>
Christoph and others what do you think about this???



[-- Attachment #2: upstream-do_migrate_pages.patch --]
[-- Type: text/plain, Size: 702 bytes --]

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 47296fe..6c189fa 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -1012,6 +1012,16 @@ int do_migrate_pages(struct mm_struct *mm,
 		int dest = 0;
 
 		for_each_node_mask(s, tmp) {
+
+			/* IFF there is an equal number of source and
+			 * destination nodes, maintain relative node distance
+			 * even when source and destination nodes overlap.
+			 * However, when the node weight is unequal, never move
+			 * memory out of any destination nodes */
+			if ((nodes_weight(*from_nodes) != nodes_weight(*to_nodes)) && 
+						(node_isset(s, *to_nodes)))
+				continue;
+
 			d = node_remap(s, *from_nodes, *to_nodes);
 			if (s == d)
 				continue;

^ permalink raw reply related	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-29 18:00         ` Larry Woodman
@ 2012-03-29 19:43             ` KOSAKI Motohiro
  0 siblings, 0 replies; 34+ messages in thread
From: KOSAKI Motohiro @ 2012-03-29 19:43 UTC (permalink / raw)
  To: lwoodman
  Cc: Christoph Lameter, KOSAKI Motohiro, linux-mm,
	Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki, kosaki.motohiro

(3/29/12 2:00 PM), Larry Woodman wrote:
> On 03/22/2012 03:30 PM, Christoph Lameter wrote:
>> On Thu, 22 Mar 2012, Larry Woodman wrote:
>>
>>>> Application may manage their locality given a range of nodes and each of
>>>> the x .. x+n nodes has their particular purpose.
>>> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
>>> 5 and 5 to 6
>>> to keep the node ordering the same?
>> Yup. Have a look at do_migrate_pages and the descrition in the comment by
>> there by Paul Jackson.
>>
>>
> Christoph and others what do you think about this???
>
>
> 		for_each_node_mask(s, tmp) {
>+
>+			/* IFF there is an equal number of source and
>+			 * destination nodes, maintain relative node distance
>+			 * even when source and destination nodes overlap.
>+			 * However, when the node weight is unequal, never move
>+			 * memory out of any destination nodes */
>+			if ((nodes_weight(*from_nodes) != nodes_weight(*to_nodes)) &&
>+						(node_isset(s, *to_nodes)))
>+				continue;
>+
> 			d = node_remap(s, *from_nodes, *to_nodes);
> 			if (s == d)
> 				continue;

I'm confused. Could you please explain why you choose nodes_weight()? On my first impression,
it seems almostly unrelated factor.



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-29 19:43             ` KOSAKI Motohiro
  0 siblings, 0 replies; 34+ messages in thread
From: KOSAKI Motohiro @ 2012-03-29 19:43 UTC (permalink / raw)
  To: lwoodman
  Cc: Christoph Lameter, KOSAKI Motohiro, linux-mm,
	Linux Kernel Mailing List, Andrew Morton, Rik van Riel,
	Motohiro Kosaki

(3/29/12 2:00 PM), Larry Woodman wrote:
> On 03/22/2012 03:30 PM, Christoph Lameter wrote:
>> On Thu, 22 Mar 2012, Larry Woodman wrote:
>>
>>>> Application may manage their locality given a range of nodes and each of
>>>> the x .. x+n nodes has their particular purpose.
>>> So to be clear on this, in that case the intention would be move 3 to 4, 4 to
>>> 5 and 5 to 6
>>> to keep the node ordering the same?
>> Yup. Have a look at do_migrate_pages and the descrition in the comment by
>> there by Paul Jackson.
>>
>>
> Christoph and others what do you think about this???
>
>
> 		for_each_node_mask(s, tmp) {
>+
>+			/* IFF there is an equal number of source and
>+			 * destination nodes, maintain relative node distance
>+			 * even when source and destination nodes overlap.
>+			 * However, when the node weight is unequal, never move
>+			 * memory out of any destination nodes */
>+			if ((nodes_weight(*from_nodes) != nodes_weight(*to_nodes)) &&
>+						(node_isset(s, *to_nodes)))
>+				continue;
>+
> 			d = node_remap(s, *from_nodes, *to_nodes);
> 			if (s == d)
> 				continue;

I'm confused. Could you please explain why you choose nodes_weight()? On my first impression,
it seems almostly unrelated factor.


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-29 19:43             ` KOSAKI Motohiro
@ 2012-03-29 20:01               ` Larry Woodman
  -1 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-29 20:01 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: Christoph Lameter, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/29/2012 03:43 PM, KOSAKI Motohiro wrote:
> (3/29/12 2:00 PM), Larry Woodman wrote:
>> On 03/22/2012 03:30 PM, Christoph Lameter wrote:
>>> On Thu, 22 Mar 2012, Larry Woodman wrote:
>>>
>>>>> Application may manage their locality given a range of nodes and 
>>>>> each of
>>>>> the x .. x+n nodes has their particular purpose.
>>>> So to be clear on this, in that case the intention would be move 3 
>>>> to 4, 4 to
>>>> 5 and 5 to 6
>>>> to keep the node ordering the same?
>>> Yup. Have a look at do_migrate_pages and the descrition in the 
>>> comment by
>>> there by Paul Jackson.
>>>
>>>
>> Christoph and others what do you think about this???
>>
>>
>>         for_each_node_mask(s, tmp) {
>> +
>> +            /* IFF there is an equal number of source and
>> +             * destination nodes, maintain relative node distance
>> +             * even when source and destination nodes overlap.
>> +             * However, when the node weight is unequal, never move
>> +             * memory out of any destination nodes */
>> +            if ((nodes_weight(*from_nodes) != 
>> nodes_weight(*to_nodes)) &&
>> +                        (node_isset(s, *to_nodes)))
>> +                continue;
>> +
>>             d = node_remap(s, *from_nodes, *to_nodes);
>>             if (s == d)
>>                 continue;
>
> I'm confused. Could you please explain why you choose nodes_weight()? 
> On my first impression,
> it seems almostly unrelated factor.

nodes_weight() tells us the number of nodes in the cpuset so if you are 
migrating
from say 2, 3 &4 to 3, 4 &5 we wont go from 2 to 5 and call it done like 
the original
patch did.  With this patch we will preserve the migrating of 2, 3 &4 to 
3, 4 &5  yet
if we are migrating from 0-7 to 3-4 we wont do this:

Migrating 7 to 4
Migrating 6 to 3
Migrating 5 to 4
Migrating 4 to 3
Migrating 1 to 4
Migrating 3 to 4
Migrating 0 to 3
Migrating 2 to 3

Instead, will do this:

Migrating 7 to 4
Migrating 6 to 3
Migrating 5 to 4
Migrating 1 to 4
Migrating 0 to 3
Migrating 2 to 3

Larry

>
>
> -- 
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign 
> http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-29 20:01               ` Larry Woodman
  0 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-29 20:01 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: Christoph Lameter, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/29/2012 03:43 PM, KOSAKI Motohiro wrote:
> (3/29/12 2:00 PM), Larry Woodman wrote:
>> On 03/22/2012 03:30 PM, Christoph Lameter wrote:
>>> On Thu, 22 Mar 2012, Larry Woodman wrote:
>>>
>>>>> Application may manage their locality given a range of nodes and 
>>>>> each of
>>>>> the x .. x+n nodes has their particular purpose.
>>>> So to be clear on this, in that case the intention would be move 3 
>>>> to 4, 4 to
>>>> 5 and 5 to 6
>>>> to keep the node ordering the same?
>>> Yup. Have a look at do_migrate_pages and the descrition in the 
>>> comment by
>>> there by Paul Jackson.
>>>
>>>
>> Christoph and others what do you think about this???
>>
>>
>>         for_each_node_mask(s, tmp) {
>> +
>> +            /* IFF there is an equal number of source and
>> +             * destination nodes, maintain relative node distance
>> +             * even when source and destination nodes overlap.
>> +             * However, when the node weight is unequal, never move
>> +             * memory out of any destination nodes */
>> +            if ((nodes_weight(*from_nodes) != 
>> nodes_weight(*to_nodes)) &&
>> +                        (node_isset(s, *to_nodes)))
>> +                continue;
>> +
>>             d = node_remap(s, *from_nodes, *to_nodes);
>>             if (s == d)
>>                 continue;
>
> I'm confused. Could you please explain why you choose nodes_weight()? 
> On my first impression,
> it seems almostly unrelated factor.

nodes_weight() tells us the number of nodes in the cpuset so if you are 
migrating
from say 2, 3 &4 to 3, 4 &5 we wont go from 2 to 5 and call it done like 
the original
patch did.  With this patch we will preserve the migrating of 2, 3 &4 to 
3, 4 &5  yet
if we are migrating from 0-7 to 3-4 we wont do this:

Migrating 7 to 4
Migrating 6 to 3
Migrating 5 to 4
Migrating 4 to 3
Migrating 1 to 4
Migrating 3 to 4
Migrating 0 to 3
Migrating 2 to 3

Instead, will do this:

Migrating 7 to 4
Migrating 6 to 3
Migrating 5 to 4
Migrating 1 to 4
Migrating 0 to 3
Migrating 2 to 3

Larry

>
>
> -- 
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo@kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Fight unfair telecom internet charges in Canada: sign 
> http://stopthemeter.ca/
> Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-29 19:43             ` KOSAKI Motohiro
@ 2012-03-30 16:15               ` Christoph Lameter
  -1 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-30 16:15 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: lwoodman, linux-mm, Linux Kernel Mailing List, Andrew Morton,
	Rik van Riel, Motohiro Kosaki

On Thu, 29 Mar 2012, KOSAKI Motohiro wrote:

> >
> > 		for_each_node_mask(s, tmp) {
> > +
> > +			/* IFF there is an equal number of source and
> > +			 * destination nodes, maintain relative node distance
> > +			 * even when source and destination nodes overlap.
> > +			 * However, when the node weight is unequal, never
> > move
> > +			 * memory out of any destination nodes */
> > +			if ((nodes_weight(*from_nodes) !=
> > nodes_weight(*to_nodes)) &&
> > +						(node_isset(s, *to_nodes)))
> > +				continue;
> > +
> > 			d = node_remap(s, *from_nodes, *to_nodes);
> > 			if (s == d)
> > 				continue;
>
> I'm confused. Could you please explain why you choose nodes_weight()? On my
> first impression,
> it seems almostly unrelated factor.

Isnt this the original code by Paul? I would think that the 1-1 movement
is only useful to do if the number of nodes in both the destination and
the source is the same.

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-30 16:15               ` Christoph Lameter
  0 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-30 16:15 UTC (permalink / raw)
  To: KOSAKI Motohiro
  Cc: lwoodman, linux-mm, Linux Kernel Mailing List, Andrew Morton,
	Rik van Riel, Motohiro Kosaki

On Thu, 29 Mar 2012, KOSAKI Motohiro wrote:

> >
> > 		for_each_node_mask(s, tmp) {
> > +
> > +			/* IFF there is an equal number of source and
> > +			 * destination nodes, maintain relative node distance
> > +			 * even when source and destination nodes overlap.
> > +			 * However, when the node weight is unequal, never
> > move
> > +			 * memory out of any destination nodes */
> > +			if ((nodes_weight(*from_nodes) !=
> > nodes_weight(*to_nodes)) &&
> > +						(node_isset(s, *to_nodes)))
> > +				continue;
> > +
> > 			d = node_remap(s, *from_nodes, *to_nodes);
> > 			if (s == d)
> > 				continue;
>
> I'm confused. Could you please explain why you choose nodes_weight()? On my
> first impression,
> it seems almostly unrelated factor.

Isnt this the original code by Paul? I would think that the 1-1 movement
is only useful to do if the number of nodes in both the destination and
the source is the same.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-30 16:15               ` Christoph Lameter
@ 2012-03-30 17:30                 ` Larry Woodman
  -1 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-30 17:30 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/30/2012 12:15 PM, Christoph Lameter wrote:
> On Thu, 29 Mar 2012, KOSAKI Motohiro wrote:
>
>>> 		for_each_node_mask(s, tmp) {
>>> +
>>> +			/* IFF there is an equal number of source and
>>> +			 * destination nodes, maintain relative node distance
>>> +			 * even when source and destination nodes overlap.
>>> +			 * However, when the node weight is unequal, never
>>> move
>>> +			 * memory out of any destination nodes */
>>> +			if ((nodes_weight(*from_nodes) !=
>>> nodes_weight(*to_nodes))&&
>>> +						(node_isset(s, *to_nodes)))
>>> +				continue;
>>> +
>>> 			d = node_remap(s, *from_nodes, *to_nodes);
>>> 			if (s == d)
>>> 				continue;
>> I'm confused. Could you please explain why you choose nodes_weight()? On my
>> first impression,
>> it seems almostly unrelated factor.
> Isnt this the original code by Paul?
No, I added the test to see if the source and destination has the same 
number of nodes.
>   I would think that the 1-1 movement
> is only useful to do if the number of nodes in both the destination and
> the source is the same.
Agreed, thats exactly what this patch does.  are you OK with this change 
then???

Larry



^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-30 17:30                 ` Larry Woodman
  0 siblings, 0 replies; 34+ messages in thread
From: Larry Woodman @ 2012-03-30 17:30 UTC (permalink / raw)
  To: Christoph Lameter
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On 03/30/2012 12:15 PM, Christoph Lameter wrote:
> On Thu, 29 Mar 2012, KOSAKI Motohiro wrote:
>
>>> 		for_each_node_mask(s, tmp) {
>>> +
>>> +			/* IFF there is an equal number of source and
>>> +			 * destination nodes, maintain relative node distance
>>> +			 * even when source and destination nodes overlap.
>>> +			 * However, when the node weight is unequal, never
>>> move
>>> +			 * memory out of any destination nodes */
>>> +			if ((nodes_weight(*from_nodes) !=
>>> nodes_weight(*to_nodes))&&
>>> +						(node_isset(s, *to_nodes)))
>>> +				continue;
>>> +
>>> 			d = node_remap(s, *from_nodes, *to_nodes);
>>> 			if (s == d)
>>> 				continue;
>> I'm confused. Could you please explain why you choose nodes_weight()? On my
>> first impression,
>> it seems almostly unrelated factor.
> Isnt this the original code by Paul?
No, I added the test to see if the source and destination has the same 
number of nodes.
>   I would think that the 1-1 movement
> is only useful to do if the number of nodes in both the destination and
> the source is the same.
Agreed, thats exactly what this patch does.  are you OK with this change 
then???

Larry


--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
  2012-03-30 17:30                 ` Larry Woodman
@ 2012-03-30 20:49                   ` Christoph Lameter
  -1 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-30 20:49 UTC (permalink / raw)
  To: Larry Woodman
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On Fri, 30 Mar 2012, Larry Woodman wrote:

> of nodes.
> >   I would think that the 1-1 movement
> > is only useful to do if the number of nodes in both the destination and
> > the source is the same.
> Agreed, thats exactly what this patch does.  are you OK with this change
> then???

Please add to the patch description some explanation how this patch
changes the way page migration does things.


^ permalink raw reply	[flat|nested] 34+ messages in thread

* Re: [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node
@ 2012-03-30 20:49                   ` Christoph Lameter
  0 siblings, 0 replies; 34+ messages in thread
From: Christoph Lameter @ 2012-03-30 20:49 UTC (permalink / raw)
  To: Larry Woodman
  Cc: KOSAKI Motohiro, linux-mm, Linux Kernel Mailing List,
	Andrew Morton, Rik van Riel, Motohiro Kosaki

On Fri, 30 Mar 2012, Larry Woodman wrote:

> of nodes.
> >   I would think that the 1-1 movement
> > is only useful to do if the number of nodes in both the destination and
> > the source is the same.
> Agreed, thats exactly what this patch does.  are you OK with this change
> then???

Please add to the patch description some explanation how this patch
changes the way page migration does things.

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Fight unfair telecom internet charges in Canada: sign http://stopthemeter.ca/
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

^ permalink raw reply	[flat|nested] 34+ messages in thread

end of thread, other threads:[~2012-03-30 20:49 UTC | newest]

Thread overview: 34+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2012-03-22 18:14 [PATCH -mm] do_migrate_pages() calls migrate_to_node() even if task is already on a correct node Larry Woodman
2012-03-22 18:22 ` Rik van Riel
2012-03-22 18:22   ` Rik van Riel
2012-03-22 18:45 ` KOSAKI Motohiro
2012-03-22 18:45   ` KOSAKI Motohiro
2012-03-22 18:47   ` Rik van Riel
2012-03-22 18:47     ` Rik van Riel
2012-03-22 18:49   ` Larry Woodman
2012-03-22 18:49     ` Larry Woodman
2012-03-22 18:51   ` Christoph Lameter
2012-03-22 18:51     ` Christoph Lameter
2012-03-22 19:07     ` Larry Woodman
2012-03-22 19:07       ` Larry Woodman
2012-03-22 19:30       ` Christoph Lameter
2012-03-22 19:30         ` Christoph Lameter
2012-03-29 18:00         ` Larry Woodman
2012-03-29 19:43           ` KOSAKI Motohiro
2012-03-29 19:43             ` KOSAKI Motohiro
2012-03-29 20:01             ` Larry Woodman
2012-03-29 20:01               ` Larry Woodman
2012-03-30 16:15             ` Christoph Lameter
2012-03-30 16:15               ` Christoph Lameter
2012-03-30 17:30               ` Larry Woodman
2012-03-30 17:30                 ` Larry Woodman
2012-03-30 20:49                 ` Christoph Lameter
2012-03-30 20:49                   ` Christoph Lameter
2012-03-22 19:36       ` Valdis.Kletnieks
2012-03-22 19:41         ` Christoph Lameter
2012-03-22 19:41           ` Christoph Lameter
2012-03-22 20:14         ` Larry Woodman
2012-03-22 20:14           ` Larry Woodman
2012-03-23  1:21           ` Valdis.Kletnieks
2012-03-22 19:07     ` KOSAKI Motohiro
2012-03-22 19:07       ` KOSAKI Motohiro

This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.