[PATCH -V3 0/2] autonuma: Migrate on fault among multiple bound nodes

[PATCH -V3 0/2] autonuma: Migrate on fault among multiple bound nodes

[Huang Ying]

To make it possible to optimize cross-socket memory accessing with
AutoNUMA even if the memory of the application is bound to multiple
NUMA nodes.

Changes:

v3:

- Rebased on latest upstream (v5.10-rc3)

- Revised the change log.

v2:

- Rebased on latest upstream (v5.10-rc1)

Huang Ying (2):
  mempolicy: Rename MPOL_F_MORON to MPOL_F_MOPRON
  autonuma: Migrate on fault among multiple bound nodes

 include/uapi/linux/mempolicy.h |  2 +-
 kernel/sched/debug.c           |  2 +-
 mm/mempolicy.c                 | 23 ++++++++++++++---------
 3 files changed, 16 insertions(+), 11 deletions(-)

-- 
2.28.0

[PATCH -V3 1/2] mempolicy: Rename MPOL_F_MORON to MPOL_F_MOPRON

[Huang Ying]

To follow code-of-conduct better.  Although the patch changes a user
visible flag definition in uapi, the flag isn't used by the user space
at all.  The flag is only used internally by kernel in struct
mempolicy flags field.  It is defined in uapi just to avoid to
conflict with MPOL_MODE_FLAGS.  The flag is upper case with prefix, so
it looks generally OK by itself.  But in the following patch, we will
introduce a label named after the flag, which is lower case and
without prefix, so it's better to rename it.

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Suggested-by: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Acked-by: Rafael Aquini <aquini@redhat.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Mel Gorman <mgorman@suse.de>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: David Rientjes <rientjes@google.com>
---
 include/uapi/linux/mempolicy.h | 2 +-
 kernel/sched/debug.c           | 2 +-
 mm/mempolicy.c                 | 6 +++---
 3 files changed, 5 insertions(+), 5 deletions(-)

diff --git a/include/uapi/linux/mempolicy.h b/include/uapi/linux/mempolicy.h
index 3354774af61e..3c3666d017e6 100644
--- a/include/uapi/linux/mempolicy.h
+++ b/include/uapi/linux/mempolicy.h
@@ -60,7 +60,7 @@ enum {
 #define MPOL_F_SHARED  (1 << 0)	/* identify shared policies */
 #define MPOL_F_LOCAL   (1 << 1)	/* preferred local allocation */
 #define MPOL_F_MOF	(1 << 3) /* this policy wants migrate on fault */
-#define MPOL_F_MORON	(1 << 4) /* Migrate On protnone Reference On Node */
+#define MPOL_F_MOPRON	(1 << 4) /* Migrate On Protnone Reference On Node */
 
 
 #endif /* _UAPI_LINUX_MEMPOLICY_H */
diff --git a/kernel/sched/debug.c b/kernel/sched/debug.c
index 0655524700d2..8bfb6adb3f31 100644
--- a/kernel/sched/debug.c
+++ b/kernel/sched/debug.c
@@ -898,7 +898,7 @@ static void sched_show_numa(struct task_struct *p, struct seq_file *m)
 
 	task_lock(p);
 	pol = p->mempolicy;
-	if (pol && !(pol->flags & MPOL_F_MORON))
+	if (pol && !(pol->flags & MPOL_F_MOPRON))
 		pol = NULL;
 	mpol_get(pol);
 	task_unlock(p);
diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 3ca4898f3f24..63d91fbd3ce6 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2511,7 +2511,7 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long
 	}
 
 	/* Migrate the page towards the node whose CPU is referencing it */
-	if (pol->flags & MPOL_F_MORON) {
+	if (pol->flags & MPOL_F_MOPRON) {
 		polnid = thisnid;
 
 		if (!should_numa_migrate_memory(current, page, curnid, thiscpu))
@@ -2802,7 +2802,7 @@ void __init numa_policy_init(void)
 		preferred_node_policy[nid] = (struct mempolicy) {
 			.refcnt = ATOMIC_INIT(1),
 			.mode = MPOL_PREFERRED,
-			.flags = MPOL_F_MOF | MPOL_F_MORON,
+			.flags = MPOL_F_MOF | MPOL_F_MOPRON,
 			.v = { .preferred_node = nid, },
 		};
 	}
@@ -3010,7 +3010,7 @@ void mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol)
 	unsigned short mode = MPOL_DEFAULT;
 	unsigned short flags = 0;
 
-	if (pol && pol != &default_policy && !(pol->flags & MPOL_F_MORON)) {
+	if (pol && pol != &default_policy && !(pol->flags & MPOL_F_MOPRON)) {
 		mode = pol->mode;
 		flags = pol->flags;
 	}
-- 
2.28.0

[PATCH -V3 2/2] autonuma: Migrate on fault among multiple bound nodes

[Huang Ying]

Now, AutoNUMA can only optimize the page placement among the NUMA nodes if the
default memory policy is used.  Because the memory policy specified explicitly
should take precedence.  But this seems too strict in some situations.  For
example, on a system with 4 NUMA nodes, if the memory of an application is bound
to the node 0 and 1, AutoNUMA can potentially migrate the pages between the node
0 and 1 to reduce cross-node accessing without breaking the explicit memory
binding policy.

So in this patch, if mbind(.mode=MPOL_BIND, .flags=MPOL_MF_LAZY) is used to bind
the memory of the application to multiple nodes, and in the hint page fault
handler both the faulting page node and the accessing node are in the policy
nodemask, the page will be tried to be migrated to the accessing node to reduce
the cross-node accessing.

[Peter Zijlstra: provided the simplified implementation method.]

Signed-off-by: "Huang, Ying" <ying.huang@intel.com>
Acked-by: Mel Gorman <mgorman@suse.de>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Ingo Molnar <mingo@redhat.com>
Cc: Rik van Riel <riel@redhat.com>
Cc: Johannes Weiner <hannes@cmpxchg.org>
Cc: "Matthew Wilcox (Oracle)" <willy@infradead.org>
Cc: Dave Hansen <dave.hansen@intel.com>
Cc: Andi Kleen <ak@linux.intel.com>
Cc: Michal Hocko <mhocko@suse.com>
Cc: David Rientjes <rientjes@google.com>
---
 mm/mempolicy.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 63d91fbd3ce6..40f2ff2607b3 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -2490,15 +2490,19 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long
 		break;
 
 	case MPOL_BIND:
-
 		/*
-		 * allows binding to multiple nodes.
-		 * use current page if in policy nodemask,
-		 * else select nearest allowed node, if any.
-		 * If no allowed nodes, use current [!misplaced].
+		 * Allows binding to multiple nodes.  If both current and
+		 * accessing nodes are in policy nodemask, migrate to
+		 * accessing node to optimize page placement. Otherwise,
+		 * use current page if in policy nodemask, else select
+		 * nearest allowed node, if any.  If no allowed nodes, use
+		 * current [!misplaced].
 		 */
-		if (node_isset(curnid, pol->v.nodes))
+		if (node_isset(curnid, pol->v.nodes)) {
+			if (node_isset(thisnid, pol->v.nodes))
+				goto mopron;
 			goto out;
+		}
 		z = first_zones_zonelist(
 				node_zonelist(numa_node_id(), GFP_HIGHUSER),
 				gfp_zone(GFP_HIGHUSER),
@@ -2512,6 +2516,7 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long
 
 	/* Migrate the page towards the node whose CPU is referencing it */
 	if (pol->flags & MPOL_F_MOPRON) {
+mopron:
 		polnid = thisnid;
 
 		if (!should_numa_migrate_memory(current, page, curnid, thiscpu))
-- 
2.28.0