[PATCH v5 0/9] extend hugepage migration

[PATCH v5 0/9] extend hugepage migration

[Naoya Horiguchi]

Here is the 5th version of hugepage migration patchset.
Changes in this version are as follows:
 - removed putback_active_hugepages() as a cleanup (1/9)
 - added code to check movability of a given hugepage (8/9)
 - set GFP MOVABLE flag depending on the movability of hugepage (9/9).

I feel that 8/9 and 9/9 contain some new things, so need reviews on them.

TODOs: (likely to be done after this work)
 - split page table lock for pmd/pud based hugepage (maybe applicable to thp)
 - improve alloc_migrate_target (especially in node choice)
 - using page walker in check_range

Thanks,
Naoya Horiguchi
---
GitHub:
  git://github.com/Naoya-Horiguchi/linux.git extend_hugepage_migration.v5

Test code:
  git://github.com/Naoya-Horiguchi/test_hugepage_migration_extension.git

Naoya Horiguchi (9):
      migrate: make core migration code aware of hugepage
      soft-offline: use migrate_pages() instead of migrate_huge_page()
      migrate: add hugepage migration code to migrate_pages()
      migrate: add hugepage migration code to move_pages()
      mbind: add hugepage migration code to mbind()
      migrate: remove VM_HUGETLB from vma flag check in vma_migratable()
      memory-hotplug: enable memory hotplug to handle hugepage
      migrate: check movability of hugepage in unmap_and_move_huge_page()
      prepare to remove /proc/sys/vm/hugepages_treat_as_movable

 Documentation/sysctl/vm.txt   |  13 +---
 arch/arm/mm/hugetlbpage.c     |   5 ++
 arch/arm64/mm/hugetlbpage.c   |   5 ++
 arch/ia64/mm/hugetlbpage.c    |   5 ++
 arch/metag/mm/hugetlbpage.c   |   5 ++
 arch/mips/mm/hugetlbpage.c    |   5 ++
 arch/powerpc/mm/hugetlbpage.c |  10 ++++
 arch/s390/mm/hugetlbpage.c    |   5 ++
 arch/sh/mm/hugetlbpage.c      |   5 ++
 arch/sparc/mm/hugetlbpage.c   |   5 ++
 arch/tile/mm/hugetlbpage.c    |   5 ++
 arch/x86/mm/hugetlbpage.c     |   8 +++
 include/linux/hugetlb.h       |  25 ++++++++
 include/linux/mempolicy.h     |   2 +-
 include/linux/migrate.h       |   5 --
 mm/hugetlb.c                  | 134 +++++++++++++++++++++++++++++++++++++-----
 mm/memory-failure.c           |  15 ++++-
 mm/memory.c                   |  17 +++++-
 mm/memory_hotplug.c           |  42 ++++++++++---
 mm/mempolicy.c                |  46 +++++++++++++--
 mm/migrate.c                  |  61 ++++++++++---------
 mm/page_alloc.c               |  12 ++++
 mm/page_isolation.c           |  14 +++++
 23 files changed, 371 insertions(+), 78 deletions(-)

[PATCH 1/9] migrate: make core migration code aware of hugepage

[Naoya Horiguchi]

Before enabling each user of page migration to support hugepage,
this patch enables the list of pages for migration to link not only
LRU pages, but also hugepages. As a result, putback_movable_pages()
and migrate_pages() can handle both of LRU pages and hugepages.

ChangeLog v5:
 - remove putback_active_hugepages

ChangeLog v4:
 - make some macros return 'do {} while(0)'
 - use more readable variable name

ChangeLog v3:
 - revert introducing migrate_movable_pages
 - add isolate_huge_page

ChangeLog v2:
 - move code removing VM_HUGETLB from vma_migratable check into a
   separate patch
 - hold hugetlb_lock in putback_active_hugepage
 - update comment near the definition of hugetlb_lock

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Acked-by: Hillf Danton <dhillf@gmail.com>
---
 include/linux/hugetlb.h |  4 ++++
 mm/hugetlb.c            | 23 ++++++++++++++++++++++-
 mm/migrate.c            | 10 +++++++++-
 3 files changed, 35 insertions(+), 2 deletions(-)

diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
index c2b1801..bc8d837 100644
--- v3.11-rc3.orig/include/linux/hugetlb.h
+++ v3.11-rc3/include/linux/hugetlb.h
@@ -66,6 +66,8 @@ int hugetlb_reserve_pages(struct inode *inode, long from, long to,
 						vm_flags_t vm_flags);
 void hugetlb_unreserve_pages(struct inode *inode, long offset, long freed);
 int dequeue_hwpoisoned_huge_page(struct page *page);
+bool isolate_huge_page(struct page *page, struct list_head *list);
+void putback_active_hugepage(struct page *page);
 void copy_huge_page(struct page *dst, struct page *src);
 
 #ifdef CONFIG_ARCH_WANT_HUGE_PMD_SHARE
@@ -134,6 +136,8 @@ static inline int dequeue_hwpoisoned_huge_page(struct page *page)
 	return 0;
 }
 
+#define isolate_huge_page(p, l) false
+#define putback_active_hugepage(p)	do {} while (0)
 static inline void copy_huge_page(struct page *dst, struct page *src)
 {
 }
diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
index 83aff0a..649771c 100644
--- v3.11-rc3.orig/mm/hugetlb.c
+++ v3.11-rc3/mm/hugetlb.c
@@ -48,7 +48,8 @@ static unsigned long __initdata default_hstate_max_huge_pages;
 static unsigned long __initdata default_hstate_size;
 
 /*
- * Protects updates to hugepage_freelists, nr_huge_pages, and free_huge_pages
+ * Protects updates to hugepage_freelists, hugepage_activelist, nr_huge_pages,
+ * free_huge_pages, and surplus_huge_pages.
  */
 DEFINE_SPINLOCK(hugetlb_lock);
 
@@ -3431,3 +3432,23 @@ int dequeue_hwpoisoned_huge_page(struct page *hpage)
 	return ret;
 }
 #endif
+
+bool isolate_huge_page(struct page *page, struct list_head *list)
+{
+	VM_BUG_ON(!PageHead(page));
+	if (!get_page_unless_zero(page))
+		return false;
+	spin_lock(&hugetlb_lock);
+	list_move_tail(&page->lru, list);
+	spin_unlock(&hugetlb_lock);
+	return true;
+}
+
+void putback_active_hugepage(struct page *page)
+{
+	VM_BUG_ON(!PageHead(page));
+	spin_lock(&hugetlb_lock);
+	list_move_tail(&page->lru, &(page_hstate(page))->hugepage_activelist);
+	spin_unlock(&hugetlb_lock);
+	put_page(page);
+}
diff --git v3.11-rc3.orig/mm/migrate.c v3.11-rc3/mm/migrate.c
index 6f0c244..b44a067 100644
--- v3.11-rc3.orig/mm/migrate.c
+++ v3.11-rc3/mm/migrate.c
@@ -100,6 +100,10 @@ void putback_movable_pages(struct list_head *l)
 	struct page *page2;
 
 	list_for_each_entry_safe(page, page2, l, lru) {
+		if (unlikely(PageHuge(page))) {
+			putback_active_hugepage(page);
+			continue;
+		}
 		list_del(&page->lru);
 		dec_zone_page_state(page, NR_ISOLATED_ANON +
 				page_is_file_cache(page));
@@ -1025,7 +1029,11 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,
 		list_for_each_entry_safe(page, page2, from, lru) {
 			cond_resched();
 
-			rc = unmap_and_move(get_new_page, private,
+			if (PageHuge(page))
+				rc = unmap_and_move_huge_page(get_new_page,
+						private, page, pass > 2, mode);
+			else
+				rc = unmap_and_move(get_new_page, private,
 						page, pass > 2, mode);
 
 			switch(rc) {
-- 
1.8.3.1

[PATCH 2/9] soft-offline: use migrate_pages() instead of migrate_huge_page()

[Naoya Horiguchi]

Currently migrate_huge_page() takes a pointer to a hugepage to be
migrated as an argument, instead of taking a pointer to the list of
hugepages to be migrated. This behavior was introduced in commit
189ebff28 ("hugetlb: simplify migrate_huge_page()"), and was OK
because until now hugepage migration is enabled only for soft-offlining
which migrates only one hugepage in a single call.

But the situation will change in the later patches in this series
which enable other users of page migration to support hugepage migration.
They can kick migration for both of normal pages and hugepages
in a single call, so we need to go back to original implementation
which uses linked lists to collect the hugepages to be migrated.

With this patch, soft_offline_huge_page() switches to use migrate_pages(),
and migrate_huge_page() is not used any more. So let's remove it.

ChangeLog v3:
 - Merged with another cleanup patch (4/10 in previous version)

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Acked-by: Hillf Danton <dhillf@gmail.com>
---
 include/linux/migrate.h |  5 -----
 mm/memory-failure.c     | 15 ++++++++++++---
 mm/migrate.c            | 28 ++--------------------------
 3 files changed, 14 insertions(+), 34 deletions(-)

diff --git v3.11-rc3.orig/include/linux/migrate.h v3.11-rc3/include/linux/migrate.h
index a405d3dc..6fe5214 100644
--- v3.11-rc3.orig/include/linux/migrate.h
+++ v3.11-rc3/include/linux/migrate.h
@@ -41,8 +41,6 @@ extern int migrate_page(struct address_space *,
 			struct page *, struct page *, enum migrate_mode);
 extern int migrate_pages(struct list_head *l, new_page_t x,
 		unsigned long private, enum migrate_mode mode, int reason);
-extern int migrate_huge_page(struct page *, new_page_t x,
-		unsigned long private, enum migrate_mode mode);
 
 extern int fail_migrate_page(struct address_space *,
 			struct page *, struct page *);
@@ -62,9 +60,6 @@ static inline void putback_movable_pages(struct list_head *l) {}
 static inline int migrate_pages(struct list_head *l, new_page_t x,
 		unsigned long private, enum migrate_mode mode, int reason)
 	{ return -ENOSYS; }
-static inline int migrate_huge_page(struct page *page, new_page_t x,
-		unsigned long private, enum migrate_mode mode)
-	{ return -ENOSYS; }
 
 static inline int migrate_prep(void) { return -ENOSYS; }
 static inline int migrate_prep_local(void) { return -ENOSYS; }
diff --git v3.11-rc3.orig/mm/memory-failure.c v3.11-rc3/mm/memory-failure.c
index 2c13aa7..af6f61c 100644
--- v3.11-rc3.orig/mm/memory-failure.c
+++ v3.11-rc3/mm/memory-failure.c
@@ -1467,6 +1467,7 @@ static int soft_offline_huge_page(struct page *page, int flags)
 	int ret;
 	unsigned long pfn = page_to_pfn(page);
 	struct page *hpage = compound_head(page);
+	LIST_HEAD(pagelist);
 
 	/*
 	 * This double-check of PageHWPoison is to avoid the race with
@@ -1482,12 +1483,20 @@ static int soft_offline_huge_page(struct page *page, int flags)
 	unlock_page(hpage);
 
 	/* Keep page count to indicate a given hugepage is isolated. */
-	ret = migrate_huge_page(hpage, new_page, MPOL_MF_MOVE_ALL,
-				MIGRATE_SYNC);
-	put_page(hpage);
+	list_move(&hpage->lru, &pagelist);
+	ret = migrate_pages(&pagelist, new_page, MPOL_MF_MOVE_ALL,
+				MIGRATE_SYNC, MR_MEMORY_FAILURE);
 	if (ret) {
 		pr_info("soft offline: %#lx: migration failed %d, type %lx\n",
 			pfn, ret, page->flags);
+		/*
+		 * We know that soft_offline_huge_page() tries to migrate
+		 * only one hugepage pointed to by hpage, so we need not
+		 * run through the pagelist here.
+		 */
+		putback_active_hugepage(hpage);
+		if (ret > 0)
+			ret = -EIO;
 	} else {
 		set_page_hwpoison_huge_page(hpage);
 		dequeue_hwpoisoned_huge_page(hpage);
diff --git v3.11-rc3.orig/mm/migrate.c v3.11-rc3/mm/migrate.c
index b44a067..3ec47d3 100644
--- v3.11-rc3.orig/mm/migrate.c
+++ v3.11-rc3/mm/migrate.c
@@ -979,6 +979,8 @@ static int unmap_and_move_huge_page(new_page_t get_new_page,
 
 	unlock_page(hpage);
 out:
+	if (rc != -EAGAIN)
+		putback_active_hugepage(hpage);
 	put_page(new_hpage);
 	if (result) {
 		if (rc)
@@ -1066,32 +1068,6 @@ int migrate_pages(struct list_head *from, new_page_t get_new_page,
 	return rc;
 }
 
-int migrate_huge_page(struct page *hpage, new_page_t get_new_page,
-		      unsigned long private, enum migrate_mode mode)
-{
-	int pass, rc;
-
-	for (pass = 0; pass < 10; pass++) {
-		rc = unmap_and_move_huge_page(get_new_page, private,
-						hpage, pass > 2, mode);
-		switch (rc) {
-		case -ENOMEM:
-			goto out;
-		case -EAGAIN:
-			/* try again */
-			cond_resched();
-			break;
-		case MIGRATEPAGE_SUCCESS:
-			goto out;
-		default:
-			rc = -EIO;
-			goto out;
-		}
-	}
-out:
-	return rc;
-}
-
 #ifdef CONFIG_NUMA
 /*
  * Move a list of individual pages
-- 
1.8.3.1

[PATCH 3/9] migrate: add hugepage migration code to migrate_pages()

[Naoya Horiguchi]

This patch extends check_range() to handle vma with VM_HUGETLB set.
We will be able to migrate hugepage with migrate_pages(2) after
applying the enablement patch which comes later in this series.

Note that for larger hugepages (covered by pud entries, 1GB for
x86_64 for example), we simply skip it now.

Note that using pmd_huge/pud_huge assumes that hugepages are pointed to
by pmd/pud. This is not true in some architectures implementing hugepage
with other mechanisms like ia64, but it's OK because pmd_huge/pud_huge
simply return 0 in such arch and page walker simply ignores such hugepages.

ChangeLog v4:
 - refactored check_hugetlb_pmd_range for better readability

ChangeLog v3:
 - revert introducing migrate_movable_pages
 - use isolate_huge_page

ChangeLog v2:
 - remove unnecessary extern
 - fix page table lock in check_hugetlb_pmd_range
 - updated description and renamed patch title

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Acked-by: Hillf Danton <dhillf@gmail.com>
---
 mm/mempolicy.c | 42 +++++++++++++++++++++++++++++++++++++-----
 1 file changed, 37 insertions(+), 5 deletions(-)

diff --git v3.11-rc3.orig/mm/mempolicy.c v3.11-rc3/mm/mempolicy.c
index 7431001..d96afc1 100644
--- v3.11-rc3.orig/mm/mempolicy.c
+++ v3.11-rc3/mm/mempolicy.c
@@ -512,6 +512,30 @@ static int check_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 	return addr != end;
 }
 
+static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
+		const nodemask_t *nodes, unsigned long flags,
+				    void *private)
+{
+#ifdef CONFIG_HUGETLB_PAGE
+	int nid;
+	struct page *page;
+
+	spin_lock(&vma->vm_mm->page_table_lock);
+	page = pte_page(huge_ptep_get((pte_t *)pmd));
+	nid = page_to_nid(page);
+	if (node_isset(nid, *nodes) == !!(flags & MPOL_MF_INVERT))
+		goto unlock;
+	/* With MPOL_MF_MOVE, we migrate only unshared hugepage. */
+	if (flags & (MPOL_MF_MOVE_ALL) ||
+	    (flags & MPOL_MF_MOVE && page_mapcount(page) == 1))
+		isolate_huge_page(page, private);
+unlock:
+	spin_unlock(&vma->vm_mm->page_table_lock);
+#else
+	BUG();
+#endif
+}
+
 static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
 		unsigned long addr, unsigned long end,
 		const nodemask_t *nodes, unsigned long flags,
@@ -523,6 +547,11 @@ static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
 	pmd = pmd_offset(pud, addr);
 	do {
 		next = pmd_addr_end(addr, end);
+		if (pmd_huge(*pmd) && is_vm_hugetlb_page(vma)) {
+			check_hugetlb_pmd_range(vma, pmd, nodes,
+						flags, private);
+			continue;
+		}
 		split_huge_page_pmd(vma, addr, pmd);
 		if (pmd_none_or_trans_huge_or_clear_bad(pmd))
 			continue;
@@ -544,6 +573,8 @@ static inline int check_pud_range(struct vm_area_struct *vma, pgd_t *pgd,
 	pud = pud_offset(pgd, addr);
 	do {
 		next = pud_addr_end(addr, end);
+		if (pud_huge(*pud) && is_vm_hugetlb_page(vma))
+			continue;
 		if (pud_none_or_clear_bad(pud))
 			continue;
 		if (check_pmd_range(vma, pud, addr, next, nodes,
@@ -635,9 +666,6 @@ check_range(struct mm_struct *mm, unsigned long start, unsigned long end,
 				return ERR_PTR(-EFAULT);
 		}
 
-		if (is_vm_hugetlb_page(vma))
-			goto next;
-
 		if (flags & MPOL_MF_LAZY) {
 			change_prot_numa(vma, start, endvma);
 			goto next;
@@ -986,7 +1014,11 @@ static void migrate_page_add(struct page *page, struct list_head *pagelist,
 
 static struct page *new_node_page(struct page *page, unsigned long node, int **x)
 {
-	return alloc_pages_exact_node(node, GFP_HIGHUSER_MOVABLE, 0);
+	if (PageHuge(page))
+		return alloc_huge_page_node(page_hstate(compound_head(page)),
+					node);
+	else
+		return alloc_pages_exact_node(node, GFP_HIGHUSER_MOVABLE, 0);
 }
 
 /*
@@ -1016,7 +1048,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
 		err = migrate_pages(&pagelist, new_node_page, dest,
 					MIGRATE_SYNC, MR_SYSCALL);
 		if (err)
-			putback_lru_pages(&pagelist);
+			putback_movable_pages(&pagelist);
 	}
 
 	return err;
-- 
1.8.3.1

[PATCH 4/9] migrate: add hugepage migration code to move_pages()

[Naoya Horiguchi]

This patch extends move_pages() to handle vma with VM_HUGETLB set.
We will be able to migrate hugepage with move_pages(2) after
applying the enablement patch which comes later in this series.

We avoid getting refcount on tail pages of hugepage, because unlike thp,
hugepage is not split and we need not care about races with splitting.

And migration of larger (1GB for x86_64) hugepage are not enabled.

ChangeLog v4:
 - use get_page instead of get_page_foll
 - add comment in follow_page_mask

ChangeLog v3:
 - revert introducing migrate_movable_pages
 - follow_page_mask(FOLL_GET) returns NULL for tail pages
 - use isolate_huge_page

ChangeLog v2:
 - updated description and renamed patch title

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
---
 mm/memory.c  | 17 +++++++++++++++--
 mm/migrate.c | 13 +++++++++++--
 2 files changed, 26 insertions(+), 4 deletions(-)

diff --git v3.11-rc3.orig/mm/memory.c v3.11-rc3/mm/memory.c
index 1ce2e2a..7ec1252 100644
--- v3.11-rc3.orig/mm/memory.c
+++ v3.11-rc3/mm/memory.c
@@ -1496,7 +1496,8 @@ struct page *follow_page_mask(struct vm_area_struct *vma,
 	if (pud_none(*pud))
 		goto no_page_table;
 	if (pud_huge(*pud) && vma->vm_flags & VM_HUGETLB) {
-		BUG_ON(flags & FOLL_GET);
+		if (flags & FOLL_GET)
+			goto out;
 		page = follow_huge_pud(mm, address, pud, flags & FOLL_WRITE);
 		goto out;
 	}
@@ -1507,8 +1508,20 @@ struct page *follow_page_mask(struct vm_area_struct *vma,
 	if (pmd_none(*pmd))
 		goto no_page_table;
 	if (pmd_huge(*pmd) && vma->vm_flags & VM_HUGETLB) {
-		BUG_ON(flags & FOLL_GET);
 		page = follow_huge_pmd(mm, address, pmd, flags & FOLL_WRITE);
+		if (flags & FOLL_GET) {
+			/*
+			 * Refcount on tail pages are not well-defined and
+			 * shouldn't be taken. The caller should handle a NULL
+			 * return when trying to follow tail pages.
+			 */
+			if (PageHead(page))
+				get_page(page);
+			else {
+				page = NULL;
+				goto out;
+			}
+		}
 		goto out;
 	}
 	if ((flags & FOLL_NUMA) && pmd_numa(*pmd))
diff --git v3.11-rc3.orig/mm/migrate.c v3.11-rc3/mm/migrate.c
index 3ec47d3..d313737 100644
--- v3.11-rc3.orig/mm/migrate.c
+++ v3.11-rc3/mm/migrate.c
@@ -1092,7 +1092,11 @@ static struct page *new_page_node(struct page *p, unsigned long private,
 
 	*result = &pm->status;
 
-	return alloc_pages_exact_node(pm->node,
+	if (PageHuge(p))
+		return alloc_huge_page_node(page_hstate(compound_head(p)),
+					pm->node);
+	else
+		return alloc_pages_exact_node(pm->node,
 				GFP_HIGHUSER_MOVABLE | GFP_THISNODE, 0);
 }
 
@@ -1152,6 +1156,11 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 				!migrate_all)
 			goto put_and_set;
 
+		if (PageHuge(page)) {
+			isolate_huge_page(page, &pagelist);
+			goto put_and_set;
+		}
+
 		err = isolate_lru_page(page);
 		if (!err) {
 			list_add_tail(&page->lru, &pagelist);
@@ -1174,7 +1183,7 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 		err = migrate_pages(&pagelist, new_page_node,
 				(unsigned long)pm, MIGRATE_SYNC, MR_SYSCALL);
 		if (err)
-			putback_lru_pages(&pagelist);
+			putback_movable_pages(&pagelist);
 	}
 
 	up_read(&mm->mmap_sem);
-- 
1.8.3.1

[PATCH 5/9] mbind: add hugepage migration code to mbind()

[Naoya Horiguchi]

This patch extends do_mbind() to handle vma with VM_HUGETLB set.
We will be able to migrate hugepage with mbind(2) after
applying the enablement patch which comes later in this series.

ChangeLog v3:
 - revert introducing migrate_movable_pages
 - added alloc_huge_page_noerr free from ERR_VALUE

ChangeLog v2:
 - updated description and renamed patch title

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
Acked-by: Hillf Danton <dhillf@gmail.com>
---
 include/linux/hugetlb.h |  3 +++
 mm/hugetlb.c            | 14 ++++++++++++++
 mm/mempolicy.c          |  4 +++-
 3 files changed, 20 insertions(+), 1 deletion(-)

diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
index bc8d837..d1db007 100644
--- v3.11-rc3.orig/include/linux/hugetlb.h
+++ v3.11-rc3/include/linux/hugetlb.h
@@ -265,6 +265,8 @@ struct huge_bootmem_page {
 };
 
 struct page *alloc_huge_page_node(struct hstate *h, int nid);
+struct page *alloc_huge_page_noerr(struct vm_area_struct *vma,
+				unsigned long addr, int avoid_reserve);
 
 /* arch callback */
 int __init alloc_bootmem_huge_page(struct hstate *h);
@@ -378,6 +380,7 @@ static inline pgoff_t basepage_index(struct page *page)
 #else	/* CONFIG_HUGETLB_PAGE */
 struct hstate {};
 #define alloc_huge_page_node(h, nid) NULL
+#define alloc_huge_page_noerr(v, a, r) NULL
 #define alloc_bootmem_huge_page(h) NULL
 #define hstate_file(f) NULL
 #define hstate_sizelog(s) NULL
diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
index 649771c..ee764b0 100644
--- v3.11-rc3.orig/mm/hugetlb.c
+++ v3.11-rc3/mm/hugetlb.c
@@ -1195,6 +1195,20 @@ static struct page *alloc_huge_page(struct vm_area_struct *vma,
 	return page;
 }
 
+/*
+ * alloc_huge_page()'s wrapper which simply returns the page if allocation
+ * succeeds, otherwise NULL. This function is called from new_vma_page(),
+ * where no ERR_VALUE is expected to be returned.
+ */
+struct page *alloc_huge_page_noerr(struct vm_area_struct *vma,
+				unsigned long addr, int avoid_reserve)
+{
+	struct page *page = alloc_huge_page(vma, addr, avoid_reserve);
+	if (IS_ERR(page))
+		page = NULL;
+	return page;
+}
+
 int __weak alloc_bootmem_huge_page(struct hstate *h)
 {
 	struct huge_bootmem_page *m;
diff --git v3.11-rc3.orig/mm/mempolicy.c v3.11-rc3/mm/mempolicy.c
index d96afc1..4a03c14 100644
--- v3.11-rc3.orig/mm/mempolicy.c
+++ v3.11-rc3/mm/mempolicy.c
@@ -1183,6 +1183,8 @@ static struct page *new_vma_page(struct page *page, unsigned long private, int *
 		vma = vma->vm_next;
 	}
 
+	if (PageHuge(page))
+		return alloc_huge_page_noerr(vma, address, 1);
 	/*
 	 * if !vma, alloc_page_vma() will use task or system default policy
 	 */
@@ -1293,7 +1295,7 @@ static long do_mbind(unsigned long start, unsigned long len,
 					(unsigned long)vma,
 					MIGRATE_SYNC, MR_MEMPOLICY_MBIND);
 			if (nr_failed)
-				putback_lru_pages(&pagelist);
+				putback_movable_pages(&pagelist);
 		}
 
 		if (nr_failed && (flags & MPOL_MF_STRICT))
-- 
1.8.3.1

[PATCH 6/9] migrate: remove VM_HUGETLB from vma flag check in vma_migratable()

[Naoya Horiguchi]

This patch enables hugepage migration from migrate_pages(2),
move_pages(2), and mbind(2).

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Hillf Danton <dhillf@gmail.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
---
 include/linux/mempolicy.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git v3.11-rc3.orig/include/linux/mempolicy.h v3.11-rc3/include/linux/mempolicy.h
index 0d7df39..2e475b5 100644
--- v3.11-rc3.orig/include/linux/mempolicy.h
+++ v3.11-rc3/include/linux/mempolicy.h
@@ -173,7 +173,7 @@ extern int mpol_to_str(char *buffer, int maxlen, struct mempolicy *pol);
 /* Check if a vma is migratable */
 static inline int vma_migratable(struct vm_area_struct *vma)
 {
-	if (vma->vm_flags & (VM_IO | VM_HUGETLB | VM_PFNMAP))
+	if (vma->vm_flags & (VM_IO | VM_PFNMAP))
 		return 0;
 	/*
 	 * Migration allocates pages in the highest zone. If we cannot
-- 
1.8.3.1

[PATCH 7/9] memory-hotplug: enable memory hotplug to handle hugepage

[Naoya Horiguchi]

Until now we can't offline memory blocks which contain hugepages because
a hugepage is considered as an unmovable page. But now with this patch
series, a hugepage has become movable, so by using hugepage migration we
can offline such memory blocks.

What's different from other users of hugepage migration is that we need
to decompose all the hugepages inside the target memory block into free
buddy pages after hugepage migration, because otherwise free hugepages
remaining in the memory block intervene the memory offlining.
For this reason we introduce new functions dissolve_free_huge_page() and
dissolve_free_huge_pages().

Other than that, what this patch does is straightforwardly to add hugepage
migration code, that is, adding hugepage code to the functions which scan
over pfn and collect hugepages to be migrated, and adding a hugepage
allocation function to alloc_migrate_target().

As for larger hugepages (1GB for x86_64), it's not easy to do hotremove
over them because it's larger than memory block. So we now simply leave
it to fail as it is.

ChangeLog v4:
 - add comment on dequeue_huge_page_node
 - alloc_migrate_target allocates destination hugepage from the next node
   of source node

ChangeLog v3:
 - revert introducing migrate_movable_pages (the function was opened)
 - add migratetype check in dequeue_huge_page_node to close the race
   between scan and allocation
 - make is_hugepage_movable use refcount to find active hugepages
   instead of running through hugepage_activelist
 - rename is_hugepage_movable to is_hugepage_active
 - add alignment check in dissolve_free_huge_pages
 - use round_up in calculating next scanning pfn
 - use isolate_huge_page

ChangeLog v2:
 - changed return value type of is_hugepage_movable() to bool
 - is_hugepage_movable() uses list_for_each_entry() instead of *_safe()
 - moved if(PageHuge) block before get_page_unless_zero() in do_migrate_range()
 - do_migrate_range() returns -EBUSY for hugepages larger than memory block
 - dissolve_free_huge_pages() calculates scan step and sets it to minimum
   hugepage size

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
---
 include/linux/hugetlb.h |  6 +++++
 mm/hugetlb.c            | 71 +++++++++++++++++++++++++++++++++++++++++++++++--
 mm/memory_hotplug.c     | 42 ++++++++++++++++++++++++-----
 mm/page_alloc.c         | 12 +++++++++
 mm/page_isolation.c     | 14 ++++++++++
 5 files changed, 136 insertions(+), 9 deletions(-)

diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
index d1db007..2e02c4e 100644
--- v3.11-rc3.orig/include/linux/hugetlb.h
+++ v3.11-rc3/include/linux/hugetlb.h
@@ -68,6 +68,7 @@ void hugetlb_unreserve_pages(struct inode *inode, long offset, long freed);
 int dequeue_hwpoisoned_huge_page(struct page *page);
 bool isolate_huge_page(struct page *page, struct list_head *list);
 void putback_active_hugepage(struct page *page);
+bool is_hugepage_active(struct page *page);
 void copy_huge_page(struct page *dst, struct page *src);
 
 #ifdef CONFIG_ARCH_WANT_HUGE_PMD_SHARE
@@ -138,6 +139,7 @@ static inline int dequeue_hwpoisoned_huge_page(struct page *page)
 
 #define isolate_huge_page(p, l) false
 #define putback_active_hugepage(p)	do {} while (0)
+#define is_hugepage_active(x)	false
 static inline void copy_huge_page(struct page *dst, struct page *src)
 {
 }
@@ -377,6 +379,9 @@ static inline pgoff_t basepage_index(struct page *page)
 	return __basepage_index(page);
 }
 
+extern void dissolve_free_huge_pages(unsigned long start_pfn,
+				     unsigned long end_pfn);
+
 #else	/* CONFIG_HUGETLB_PAGE */
 struct hstate {};
 #define alloc_huge_page_node(h, nid) NULL
@@ -403,6 +408,7 @@ static inline pgoff_t basepage_index(struct page *page)
 {
 	return page->index;
 }
+#define dissolve_free_huge_pages(s, e)	do {} while (0)
 #endif	/* CONFIG_HUGETLB_PAGE */
 
 #endif /* _LINUX_HUGETLB_H */
diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
index ee764b0..3121915 100644
--- v3.11-rc3.orig/mm/hugetlb.c
+++ v3.11-rc3/mm/hugetlb.c
@@ -21,6 +21,7 @@
 #include <linux/rmap.h>
 #include <linux/swap.h>
 #include <linux/swapops.h>
+#include <linux/page-isolation.h>
 
 #include <asm/page.h>
 #include <asm/pgtable.h>
@@ -518,9 +519,15 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid)
 {
 	struct page *page;
 
-	if (list_empty(&h->hugepage_freelists[nid]))
+	list_for_each_entry(page, &h->hugepage_freelists[nid], lru)
+		if (!is_migrate_isolate_page(page))
+			break;
+	/*
+	 * if 'non-isolated free hugepage' not found on the list,
+	 * the allocation fails.
+	 */
+	if (&h->hugepage_freelists[nid] == &page->lru)
 		return NULL;
-	page = list_entry(h->hugepage_freelists[nid].next, struct page, lru);
 	list_move(&page->lru, &h->hugepage_activelist);
 	set_page_refcounted(page);
 	h->free_huge_pages--;
@@ -861,6 +868,44 @@ static int free_pool_huge_page(struct hstate *h, nodemask_t *nodes_allowed,
 	return ret;
 }
 
+/*
+ * Dissolve a given free hugepage into free buddy pages. This function does
+ * nothing for in-use (including surplus) hugepages.
+ */
+static void dissolve_free_huge_page(struct page *page)
+{
+	spin_lock(&hugetlb_lock);
+	if (PageHuge(page) && !page_count(page)) {
+		struct hstate *h = page_hstate(page);
+		int nid = page_to_nid(page);
+		list_del(&page->lru);
+		h->free_huge_pages--;
+		h->free_huge_pages_node[nid]--;
+		update_and_free_page(h, page);
+	}
+	spin_unlock(&hugetlb_lock);
+}
+
+/*
+ * Dissolve free hugepages in a given pfn range. Used by memory hotplug to
+ * make specified memory blocks removable from the system.
+ * Note that start_pfn should aligned with (minimum) hugepage size.
+ */
+void dissolve_free_huge_pages(unsigned long start_pfn, unsigned long end_pfn)
+{
+	unsigned int order = 8 * sizeof(void *);
+	unsigned long pfn;
+	struct hstate *h;
+
+	/* Set scan step to minimum hugepage size */
+	for_each_hstate(h)
+		if (order > huge_page_order(h))
+			order = huge_page_order(h);
+	VM_BUG_ON(!IS_ALIGNED(start_pfn, 1 << order));
+	for (pfn = start_pfn; pfn < end_pfn; pfn += 1 << order)
+		dissolve_free_huge_page(pfn_to_page(pfn));
+}
+
 static struct page *alloc_buddy_huge_page(struct hstate *h, int nid)
 {
 	struct page *page;
@@ -3418,6 +3463,28 @@ static int is_hugepage_on_freelist(struct page *hpage)
 	return 0;
 }
 
+bool is_hugepage_active(struct page *page)
+{
+	VM_BUG_ON(!PageHuge(page));
+	/*
+	 * This function can be called for a tail page because the caller,
+	 * scan_movable_pages, scans through a given pfn-range which typically
+	 * covers one memory block. In systems using gigantic hugepage (1GB
+	 * for x86_64,) a hugepage is larger than a memory block, and we don't
+	 * support migrating such large hugepages for now, so return false
+	 * when called for tail pages.
+	 */
+	if (PageTail(page))
+		return false;
+	/*
+	 * Refcount of a hwpoisoned hugepages is 1, but they are not active,
+	 * so we should return false for them.
+	 */
+	if (unlikely(PageHWPoison(page)))
+		return false;
+	return page_count(page) > 0;
+}
+
 /*
  * This function is called from memory failure code.
  * Assume the caller holds page lock of the head page.
diff --git v3.11-rc3.orig/mm/memory_hotplug.c v3.11-rc3/mm/memory_hotplug.c
index ca1dd3a..31f08fa 100644
--- v3.11-rc3.orig/mm/memory_hotplug.c
+++ v3.11-rc3/mm/memory_hotplug.c
@@ -30,6 +30,7 @@
 #include <linux/mm_inline.h>
 #include <linux/firmware-map.h>
 #include <linux/stop_machine.h>
+#include <linux/hugetlb.h>
 
 #include <asm/tlbflush.h>
 
@@ -1208,10 +1209,12 @@ static int test_pages_in_a_zone(unsigned long start_pfn, unsigned long end_pfn)
 }
 
 /*
- * Scanning pfn is much easier than scanning lru list.
- * Scan pfn from start to end and Find LRU page.
+ * Scan pfn range [start,end) to find movable/migratable pages (LRU pages
+ * and hugepages). We scan pfn because it's much easier than scanning over
+ * linked list. This function returns the pfn of the first found movable
+ * page if it's found, otherwise 0.
  */
-static unsigned long scan_lru_pages(unsigned long start, unsigned long end)
+static unsigned long scan_movable_pages(unsigned long start, unsigned long end)
 {
 	unsigned long pfn;
 	struct page *page;
@@ -1220,6 +1223,13 @@ static unsigned long scan_lru_pages(unsigned long start, unsigned long end)
 			page = pfn_to_page(pfn);
 			if (PageLRU(page))
 				return pfn;
+			if (PageHuge(page)) {
+				if (is_hugepage_active(page))
+					return pfn;
+				else
+					pfn = round_up(pfn + 1,
+						1 << compound_order(page)) - 1;
+			}
 		}
 	}
 	return 0;
@@ -1240,6 +1250,19 @@ do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 		if (!pfn_valid(pfn))
 			continue;
 		page = pfn_to_page(pfn);
+
+		if (PageHuge(page)) {
+			struct page *head = compound_head(page);
+			pfn = page_to_pfn(head) + (1<<compound_order(head)) - 1;
+			if (compound_order(head) > PFN_SECTION_SHIFT) {
+				ret = -EBUSY;
+				break;
+			}
+			if (isolate_huge_page(page, &source))
+				move_pages -= 1 << compound_order(head);
+			continue;
+		}
+
 		if (!get_page_unless_zero(page))
 			continue;
 		/*
@@ -1272,7 +1295,7 @@ do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 	}
 	if (!list_empty(&source)) {
 		if (not_managed) {
-			putback_lru_pages(&source);
+			putback_movable_pages(&source);
 			goto out;
 		}
 
@@ -1283,7 +1306,7 @@ do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 		ret = migrate_pages(&source, alloc_migrate_target, 0,
 					MIGRATE_SYNC, MR_MEMORY_HOTPLUG);
 		if (ret)
-			putback_lru_pages(&source);
+			putback_movable_pages(&source);
 	}
 out:
 	return ret;
@@ -1527,8 +1550,8 @@ static int __ref __offline_pages(unsigned long start_pfn,
 		drain_all_pages();
 	}
 
-	pfn = scan_lru_pages(start_pfn, end_pfn);
-	if (pfn) { /* We have page on LRU */
+	pfn = scan_movable_pages(start_pfn, end_pfn);
+	if (pfn) { /* We have movable pages */
 		ret = do_migrate_range(pfn, end_pfn);
 		if (!ret) {
 			drain = 1;
@@ -1547,6 +1570,11 @@ static int __ref __offline_pages(unsigned long start_pfn,
 	yield();
 	/* drain pcp pages, this is synchronous. */
 	drain_all_pages();
+	/*
+	 * dissolve free hugepages in the memory block before doing offlining
+	 * actually in order to make hugetlbfs's object counting consistent.
+	 */
+	dissolve_free_huge_pages(start_pfn, end_pfn);
 	/* check again */
 	offlined_pages = check_pages_isolated(start_pfn, end_pfn);
 	if (offlined_pages < 0) {
diff --git v3.11-rc3.orig/mm/page_alloc.c v3.11-rc3/mm/page_alloc.c
index b100255..24fe228 100644
--- v3.11-rc3.orig/mm/page_alloc.c
+++ v3.11-rc3/mm/page_alloc.c
@@ -60,6 +60,7 @@
 #include <linux/page-debug-flags.h>
 #include <linux/hugetlb.h>
 #include <linux/sched/rt.h>
+#include <linux/hugetlb.h>
 
 #include <asm/sections.h>
 #include <asm/tlbflush.h>
@@ -5928,6 +5929,17 @@ bool has_unmovable_pages(struct zone *zone, struct page *page, int count,
 			continue;
 
 		page = pfn_to_page(check);
+
+		/*
+		 * Hugepages are not in LRU lists, but they're movable.
+		 * We need not scan over tail pages bacause we don't
+		 * handle each tail page individually in migration.
+		 */
+		if (PageHuge(page)) {
+			iter = round_up(iter + 1, 1<<compound_order(page)) - 1;
+			continue;
+		}
+
 		/*
 		 * We can't use page_count without pin a page
 		 * because another CPU can free compound page.
diff --git v3.11-rc3.orig/mm/page_isolation.c v3.11-rc3/mm/page_isolation.c
index 383bdbb..229d66f 100644
--- v3.11-rc3.orig/mm/page_isolation.c
+++ v3.11-rc3/mm/page_isolation.c
@@ -6,6 +6,7 @@
 #include <linux/page-isolation.h>
 #include <linux/pageblock-flags.h>
 #include <linux/memory.h>
+#include <linux/hugetlb.h>
 #include "internal.h"
 
 int set_migratetype_isolate(struct page *page, bool skip_hwpoisoned_pages)
@@ -252,6 +253,19 @@ struct page *alloc_migrate_target(struct page *page, unsigned long private,
 {
 	gfp_t gfp_mask = GFP_USER | __GFP_MOVABLE;
 
+	/*
+	 * TODO: allocate a destination hugepage from a nearest neighbor node,
+	 * accordance with memory policy of the user process if possible. For
+	 * now as a simple work-around, we use the next node for destination.
+	 */
+	if (PageHuge(page)) {
+		nodemask_t src = nodemask_of_node(page_to_nid(page));
+		nodemask_t dst;
+		nodes_complement(dst, src);
+		return alloc_huge_page_node(page_hstate(compound_head(page)),
+					    next_node(page_to_nid(page), dst));
+	}
+
 	if (PageHighMem(page))
 		gfp_mask |= __GFP_HIGHMEM;
 
-- 
1.8.3.1

[PATCH 8/9] migrate: check movability of hugepage in unmap_and_move_huge_page()

[Naoya Horiguchi]

Currently hugepage migration works well only for pmd-based hugepages
(mainly due to lack of testing,) so we had better not enable migration
of other levels of hugepages until we are ready for it.

Some users of hugepage migration (mbind, move_pages, and migrate_pages)
do page table walk and check pud/pmd_huge() there, so they are safe.
But the other users (softoffline and memory hotremove) don't do this,
so without this patch they can try to migrate unexpected types of hugepages.

To prevent this, we introduce hugepage_migration_support() as an architecture
dependent check of whether hugepage are implemented on a pmd basis or not.
And on some architecture multiple sizes of hugepages are available, so
hugepage_migration_support() also checks hugepage size.

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 arch/arm/mm/hugetlbpage.c     |  5 +++++
 arch/arm64/mm/hugetlbpage.c   |  5 +++++
 arch/ia64/mm/hugetlbpage.c    |  5 +++++
 arch/metag/mm/hugetlbpage.c   |  5 +++++
 arch/mips/mm/hugetlbpage.c    |  5 +++++
 arch/powerpc/mm/hugetlbpage.c | 10 ++++++++++
 arch/s390/mm/hugetlbpage.c    |  5 +++++
 arch/sh/mm/hugetlbpage.c      |  5 +++++
 arch/sparc/mm/hugetlbpage.c   |  5 +++++
 arch/tile/mm/hugetlbpage.c    |  5 +++++
 arch/x86/mm/hugetlbpage.c     |  8 ++++++++
 include/linux/hugetlb.h       | 12 ++++++++++++
 mm/migrate.c                  | 10 ++++++++++
 13 files changed, 85 insertions(+)

diff --git v3.11-rc3.orig/arch/arm/mm/hugetlbpage.c v3.11-rc3/arch/arm/mm/hugetlbpage.c
index 3d1e4a2..3f3b6a7 100644
--- v3.11-rc3.orig/arch/arm/mm/hugetlbpage.c
+++ v3.11-rc3/arch/arm/mm/hugetlbpage.c
@@ -99,3 +99,8 @@ int pmd_huge(pmd_t pmd)
 {
 	return pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT);
 }
+
+int pmd_huge_support(void)
+{
+	return 1;
+}
diff --git v3.11-rc3.orig/arch/arm64/mm/hugetlbpage.c v3.11-rc3/arch/arm64/mm/hugetlbpage.c
index 2fc8258..5e9aec3 100644
--- v3.11-rc3.orig/arch/arm64/mm/hugetlbpage.c
+++ v3.11-rc3/arch/arm64/mm/hugetlbpage.c
@@ -54,6 +54,11 @@ int pud_huge(pud_t pud)
 	return !(pud_val(pud) & PUD_TABLE_BIT);
 }
 
+int pmd_huge_support(void)
+{
+	return 1;
+}
+
 static __init int setup_hugepagesz(char *opt)
 {
 	unsigned long ps = memparse(opt, &opt);
diff --git v3.11-rc3.orig/arch/ia64/mm/hugetlbpage.c v3.11-rc3/arch/ia64/mm/hugetlbpage.c
index 76069c1..68232db 100644
--- v3.11-rc3.orig/arch/ia64/mm/hugetlbpage.c
+++ v3.11-rc3/arch/ia64/mm/hugetlbpage.c
@@ -114,6 +114,11 @@ int pud_huge(pud_t pud)
 	return 0;
 }
 
+int pmd_huge_support(void)
+{
+	return 0;
+}
+
 struct page *
 follow_huge_pmd(struct mm_struct *mm, unsigned long address, pmd_t *pmd, int write)
 {
diff --git v3.11-rc3.orig/arch/metag/mm/hugetlbpage.c v3.11-rc3/arch/metag/mm/hugetlbpage.c
index 3c52fa6..0424315 100644
--- v3.11-rc3.orig/arch/metag/mm/hugetlbpage.c
+++ v3.11-rc3/arch/metag/mm/hugetlbpage.c
@@ -110,6 +110,11 @@ int pud_huge(pud_t pud)
 	return 0;
 }
 
+int pmd_huge_support(void)
+{
+	return 1;
+}
+
 struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 			     pmd_t *pmd, int write)
 {
diff --git v3.11-rc3.orig/arch/mips/mm/hugetlbpage.c v3.11-rc3/arch/mips/mm/hugetlbpage.c
index a7fee0d..01fda44 100644
--- v3.11-rc3.orig/arch/mips/mm/hugetlbpage.c
+++ v3.11-rc3/arch/mips/mm/hugetlbpage.c
@@ -85,6 +85,11 @@ int pud_huge(pud_t pud)
 	return (pud_val(pud) & _PAGE_HUGE) != 0;
 }
 
+int pmd_huge_support(void)
+{
+	return 1;
+}
+
 struct page *
 follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 		pmd_t *pmd, int write)
diff --git v3.11-rc3.orig/arch/powerpc/mm/hugetlbpage.c v3.11-rc3/arch/powerpc/mm/hugetlbpage.c
index 834ca8e..d67db4b 100644
--- v3.11-rc3.orig/arch/powerpc/mm/hugetlbpage.c
+++ v3.11-rc3/arch/powerpc/mm/hugetlbpage.c
@@ -86,6 +86,11 @@ int pgd_huge(pgd_t pgd)
 	 */
 	return ((pgd_val(pgd) & 0x3) != 0x0);
 }
+
+int pmd_huge_support(void)
+{
+	return 1;
+}
 #else
 int pmd_huge(pmd_t pmd)
 {
@@ -101,6 +106,11 @@ int pgd_huge(pgd_t pgd)
 {
 	return 0;
 }
+
+int pmd_huge_support(void)
+{
+	return 0;
+}
 #endif
 
 pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
diff --git v3.11-rc3.orig/arch/s390/mm/hugetlbpage.c v3.11-rc3/arch/s390/mm/hugetlbpage.c
index 121089d..951ee25 100644
--- v3.11-rc3.orig/arch/s390/mm/hugetlbpage.c
+++ v3.11-rc3/arch/s390/mm/hugetlbpage.c
@@ -117,6 +117,11 @@ int pud_huge(pud_t pud)
 	return 0;
 }
 
+int pmd_huge_support(void)
+{
+	return 1;
+}
+
 struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 			     pmd_t *pmdp, int write)
 {
diff --git v3.11-rc3.orig/arch/sh/mm/hugetlbpage.c v3.11-rc3/arch/sh/mm/hugetlbpage.c
index d776234..0d676a4 100644
--- v3.11-rc3.orig/arch/sh/mm/hugetlbpage.c
+++ v3.11-rc3/arch/sh/mm/hugetlbpage.c
@@ -83,6 +83,11 @@ int pud_huge(pud_t pud)
 	return 0;
 }
 
+int pmd_huge_support(void)
+{
+	return 0;
+}
+
 struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 			     pmd_t *pmd, int write)
 {
diff --git v3.11-rc3.orig/arch/sparc/mm/hugetlbpage.c v3.11-rc3/arch/sparc/mm/hugetlbpage.c
index d2b5944..9639964 100644
--- v3.11-rc3.orig/arch/sparc/mm/hugetlbpage.c
+++ v3.11-rc3/arch/sparc/mm/hugetlbpage.c
@@ -234,6 +234,11 @@ int pud_huge(pud_t pud)
 	return 0;
 }
 
+int pmd_huge_support(void)
+{
+	return 0;
+}
+
 struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 			     pmd_t *pmd, int write)
 {
diff --git v3.11-rc3.orig/arch/tile/mm/hugetlbpage.c v3.11-rc3/arch/tile/mm/hugetlbpage.c
index 650ccff..0ac3599 100644
--- v3.11-rc3.orig/arch/tile/mm/hugetlbpage.c
+++ v3.11-rc3/arch/tile/mm/hugetlbpage.c
@@ -198,6 +198,11 @@ int pud_huge(pud_t pud)
 	return !!(pud_val(pud) & _PAGE_HUGE_PAGE);
 }
 
+int pmd_huge_support(void)
+{
+	return 1;
+}
+
 struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 			     pmd_t *pmd, int write)
 {
diff --git v3.11-rc3.orig/arch/x86/mm/hugetlbpage.c v3.11-rc3/arch/x86/mm/hugetlbpage.c
index 7e73e8c..9d980d8 100644
--- v3.11-rc3.orig/arch/x86/mm/hugetlbpage.c
+++ v3.11-rc3/arch/x86/mm/hugetlbpage.c
@@ -59,6 +59,10 @@ follow_huge_pmd(struct mm_struct *mm, unsigned long address,
 	return NULL;
 }
 
+int pmd_huge_support(void)
+{
+	return 0;
+}
 #else
 
 struct page *
@@ -77,6 +81,10 @@ int pud_huge(pud_t pud)
 	return !!(pud_val(pud) & _PAGE_PSE);
 }
 
+int pmd_huge_support(void)
+{
+	return 1;
+}
 #endif
 
 /* x86_64 also uses this file */
diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
index 2e02c4e..0393270 100644
--- v3.11-rc3.orig/include/linux/hugetlb.h
+++ v3.11-rc3/include/linux/hugetlb.h
@@ -381,6 +381,16 @@ static inline pgoff_t basepage_index(struct page *page)
 
 extern void dissolve_free_huge_pages(unsigned long start_pfn,
 				     unsigned long end_pfn);
+int pmd_huge_support(void);
+/*
+ * Currently hugepage migration is enabled only for pmd-based hugepage.
+ * This function will be updated when hugepage migration is more widely
+ * supported.
+ */
+static inline int hugepage_migration_support(struct hstate *h)
+{
+	return pmd_huge_support() && (huge_page_shift(h) == PMD_SHIFT);
+}
 
 #else	/* CONFIG_HUGETLB_PAGE */
 struct hstate {};
@@ -409,6 +419,8 @@ static inline pgoff_t basepage_index(struct page *page)
 	return page->index;
 }
 #define dissolve_free_huge_pages(s, e)	do {} while (0)
+#define pmd_huge_support()	0
+#define hugepage_migration_support(h)	0
 #endif	/* CONFIG_HUGETLB_PAGE */
 
 #endif /* _LINUX_HUGETLB_H */
diff --git v3.11-rc3.orig/mm/migrate.c v3.11-rc3/mm/migrate.c
index d313737..61f14a1 100644
--- v3.11-rc3.orig/mm/migrate.c
+++ v3.11-rc3/mm/migrate.c
@@ -949,6 +949,16 @@ static int unmap_and_move_huge_page(new_page_t get_new_page,
 	struct page *new_hpage = get_new_page(hpage, private, &result);
 	struct anon_vma *anon_vma = NULL;
 
+	/*
+	 * Movability of hugepages depends on architectures and hugepage size.
+	 * This check is necessary because some callers of hugepage migration
+	 * like soft offline and memory hotremove don't walk through page
+	 * tables or check whether the hugepage is pmd-based or not before
+	 * kicking migration.
+	 */
+	if (!hugepage_migration_support(page_hstate(hpage)))
+		return -ENOSYS;
+
 	if (!new_hpage)
 		return -ENOMEM;
 
-- 
1.8.3.1

[PATCH 9/9] prepare to remove /proc/sys/vm/hugepages_treat_as_movable

[Naoya Horiguchi]

Now we have extended hugepage migration and it's opened to many users
of page migration, which is a good reason to consider hugepage as movable.
So we can go to the direction to remove this parameter. In order to
allow userspace to prepare for the removal, let's leave this sysctl handler
as noop for a while.

Note that hugepage migration is available only for the architectures
which implement hugepage on a pmd basis. On the other architectures,
allocating hugepages from MOVABLE is not a good idea because it can
break memory hotremove (which expects that all pages of ZONE_MOVABLE are
movable.) So we choose GFP flags in accordance with mobility of hugepage.

ChangeLog v5:
 - choose GFP flags in accordance with mobility of hugepage

ChangeLog v3:
 - use WARN_ON_ONCE

ChangeLog v2:
 - shift to noop function instead of completely removing the parameter
 - rename patch title

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Acked-by: Andi Kleen <ak@linux.intel.com>
Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
---
 Documentation/sysctl/vm.txt | 13 ++-----------
 mm/hugetlb.c                | 26 +++++++++++++++-----------
 2 files changed, 17 insertions(+), 22 deletions(-)

diff --git v3.11-rc3.orig/Documentation/sysctl/vm.txt v3.11-rc3/Documentation/sysctl/vm.txt
index 36ecc26..6e211a1 100644
--- v3.11-rc3.orig/Documentation/sysctl/vm.txt
+++ v3.11-rc3/Documentation/sysctl/vm.txt
@@ -200,17 +200,8 @@ fragmentation index is <= extfrag_threshold. The default value is 500.
 
 hugepages_treat_as_movable
 
-This parameter is only useful when kernelcore= is specified at boot time to
-create ZONE_MOVABLE for pages that may be reclaimed or migrated. Huge pages
-are not movable so are not normally allocated from ZONE_MOVABLE. A non-zero
-value written to hugepages_treat_as_movable allows huge pages to be allocated
-from ZONE_MOVABLE.
-
-Once enabled, the ZONE_MOVABLE is treated as an area of memory the huge
-pages pool can easily grow or shrink within. Assuming that applications are
-not running that mlock() a lot of memory, it is likely the huge pages pool
-can grow to the size of ZONE_MOVABLE by repeatedly entering the desired value
-into nr_hugepages and triggering page reclaim.
+This parameter is obsolete and planned to be removed. The value has no effect
+on kernel's behavior.
 
 ==============================================================
 
diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
index 3121915..b888873 100644
--- v3.11-rc3.orig/mm/hugetlb.c
+++ v3.11-rc3/mm/hugetlb.c
@@ -34,7 +34,6 @@
 #include "internal.h"
 
 const unsigned long hugetlb_zero = 0, hugetlb_infinity = ~0UL;
-static gfp_t htlb_alloc_mask = GFP_HIGHUSER;
 unsigned long hugepages_treat_as_movable;
 
 int hugetlb_max_hstate __read_mostly;
@@ -535,6 +534,15 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid)
 	return page;
 }
 
+/* Movability of hugepages depends on migration support. */
+static inline int htlb_alloc_mask(struct hstate *h)
+{
+	if (hugepage_migration_support(h))
+		return GFP_HIGHUSER_MOVABLE;
+	else
+		return GFP_HIGHUSER;
+}
+
 static struct page *dequeue_huge_page_vma(struct hstate *h,
 				struct vm_area_struct *vma,
 				unsigned long address, int avoid_reserve)
@@ -550,7 +558,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 retry_cpuset:
 	cpuset_mems_cookie = get_mems_allowed();
 	zonelist = huge_zonelist(vma, address,
-					htlb_alloc_mask, &mpol, &nodemask);
+					htlb_alloc_mask(h), &mpol, &nodemask);
 	/*
 	 * A child process with MAP_PRIVATE mappings created by their parent
 	 * have no page reserves. This check ensures that reservations are
@@ -566,7 +574,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 						MAX_NR_ZONES - 1, nodemask) {
-		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask)) {
+		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask(h))) {
 			page = dequeue_huge_page_node(h, zone_to_nid(zone));
 			if (page) {
 				if (!avoid_reserve)
@@ -723,7 +731,7 @@ static struct page *alloc_fresh_huge_page_node(struct hstate *h, int nid)
 		return NULL;
 
 	page = alloc_pages_exact_node(nid,
-		htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
+		htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
 						__GFP_REPEAT|__GFP_NOWARN,
 		huge_page_order(h));
 	if (page) {
@@ -948,12 +956,12 @@ static struct page *alloc_buddy_huge_page(struct hstate *h, int nid)
 	spin_unlock(&hugetlb_lock);
 
 	if (nid == NUMA_NO_NODE)
-		page = alloc_pages(htlb_alloc_mask|__GFP_COMP|
+		page = alloc_pages(htlb_alloc_mask(h)|__GFP_COMP|
 				   __GFP_REPEAT|__GFP_NOWARN,
 				   huge_page_order(h));
 	else
 		page = alloc_pages_exact_node(nid,
-			htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
+			htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
 			__GFP_REPEAT|__GFP_NOWARN, huge_page_order(h));
 
 	if (page && arch_prepare_hugepage(page)) {
@@ -2132,11 +2140,7 @@ int hugetlb_treat_movable_handler(struct ctl_table *table, int write,
 			void __user *buffer,
 			size_t *length, loff_t *ppos)
 {
-	proc_dointvec(table, write, buffer, length, ppos);
-	if (hugepages_treat_as_movable)
-		htlb_alloc_mask = GFP_HIGHUSER_MOVABLE;
-	else
-		htlb_alloc_mask = GFP_HIGHUSER;
+	WARN_ON_ONCE("This knob is obsolete and has no effect. It is scheduled for removal.\n");
 	return 0;
 }
 
-- 
1.8.3.1

Re: [PATCH 9/9] prepare to remove /proc/sys/vm/hugepages_treat_as_movable

[Aneesh Kumar K.V]

Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> writes:

> Now we have extended hugepage migration and it's opened to many users
> of page migration, which is a good reason to consider hugepage as movable.
> So we can go to the direction to remove this parameter. In order to
> allow userspace to prepare for the removal, let's leave this sysctl handler
> as noop for a while.
>
> Note that hugepage migration is available only for the architectures
> which implement hugepage on a pmd basis. On the other architectures,
> allocating hugepages from MOVABLE is not a good idea because it can
> break memory hotremove (which expects that all pages of ZONE_MOVABLE are
> movable.) So we choose GFP flags in accordance with mobility of hugepage.
>
> ChangeLog v5:
>  - choose GFP flags in accordance with mobility of hugepage
>
> ChangeLog v3:
>  - use WARN_ON_ONCE
>
> ChangeLog v2:
>  - shift to noop function instead of completely removing the parameter
>  - rename patch title
>
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Acked-by: Andi Kleen <ak@linux.intel.com>
> Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>

Ok that mostly address the issue I raised last time, but how do archs
that don't support hugepage migration force the hugepage allocation from
movable zone. They were able to do that before with the sysctl you are
removing in this patch isn't it ? May be keep that interface and print a
warn_on of archs that support hugepage migration ?


> ---
>  Documentation/sysctl/vm.txt | 13 ++-----------
>  mm/hugetlb.c                | 26 +++++++++++++++-----------
>  2 files changed, 17 insertions(+), 22 deletions(-)
>
> diff --git v3.11-rc3.orig/Documentation/sysctl/vm.txt v3.11-rc3/Documentation/sysctl/vm.txt
> index 36ecc26..6e211a1 100644
> --- v3.11-rc3.orig/Documentation/sysctl/vm.txt
> +++ v3.11-rc3/Documentation/sysctl/vm.txt
> @@ -200,17 +200,8 @@ fragmentation index is <= extfrag_threshold. The default value is 500.
>
>  hugepages_treat_as_movable
>
> -This parameter is only useful when kernelcore= is specified at boot time to
> -create ZONE_MOVABLE for pages that may be reclaimed or migrated. Huge pages
> -are not movable so are not normally allocated from ZONE_MOVABLE. A non-zero
> -value written to hugepages_treat_as_movable allows huge pages to be allocated
> -from ZONE_MOVABLE.
> -
> -Once enabled, the ZONE_MOVABLE is treated as an area of memory the huge
> -pages pool can easily grow or shrink within. Assuming that applications are
> -not running that mlock() a lot of memory, it is likely the huge pages pool
> -can grow to the size of ZONE_MOVABLE by repeatedly entering the desired value
> -into nr_hugepages and triggering page reclaim.
> +This parameter is obsolete and planned to be removed. The value has no effect
> +on kernel's behavior.
>
>  ==============================================================
>
> diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
> index 3121915..b888873 100644
> --- v3.11-rc3.orig/mm/hugetlb.c
> +++ v3.11-rc3/mm/hugetlb.c
> @@ -34,7 +34,6 @@
>  #include "internal.h"
>
>  const unsigned long hugetlb_zero = 0, hugetlb_infinity = ~0UL;
> -static gfp_t htlb_alloc_mask = GFP_HIGHUSER;
>  unsigned long hugepages_treat_as_movable;
>
>  int hugetlb_max_hstate __read_mostly;
> @@ -535,6 +534,15 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid)
>  	return page;
>  }
>
> +/* Movability of hugepages depends on migration support. */
> +static inline int htlb_alloc_mask(struct hstate *h)
> +{
> +	if (hugepage_migration_support(h))
> +		return GFP_HIGHUSER_MOVABLE;
> +	else
> +		return GFP_HIGHUSER;
> +}
> +
>  static struct page *dequeue_huge_page_vma(struct hstate *h,
>  				struct vm_area_struct *vma,
>  				unsigned long address, int avoid_reserve)
> @@ -550,7 +558,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
>  retry_cpuset:
>  	cpuset_mems_cookie = get_mems_allowed();
>  	zonelist = huge_zonelist(vma, address,
> -					htlb_alloc_mask, &mpol, &nodemask);
> +					htlb_alloc_mask(h), &mpol, &nodemask);
>  	/*
>  	 * A child process with MAP_PRIVATE mappings created by their parent
>  	 * have no page reserves. This check ensures that reservations are
> @@ -566,7 +574,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
>
>  	for_each_zone_zonelist_nodemask(zone, z, zonelist,
>  						MAX_NR_ZONES - 1, nodemask) {
> -		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask)) {
> +		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask(h))) {
>  			page = dequeue_huge_page_node(h, zone_to_nid(zone));
>  			if (page) {
>  				if (!avoid_reserve)
> @@ -723,7 +731,7 @@ static struct page *alloc_fresh_huge_page_node(struct hstate *h, int nid)
>  		return NULL;
>
>  	page = alloc_pages_exact_node(nid,
> -		htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
> +		htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
>  						__GFP_REPEAT|__GFP_NOWARN,
>  		huge_page_order(h));
>  	if (page) {
> @@ -948,12 +956,12 @@ static struct page *alloc_buddy_huge_page(struct hstate *h, int nid)
>  	spin_unlock(&hugetlb_lock);
>
>  	if (nid == NUMA_NO_NODE)
> -		page = alloc_pages(htlb_alloc_mask|__GFP_COMP|
> +		page = alloc_pages(htlb_alloc_mask(h)|__GFP_COMP|
>  				   __GFP_REPEAT|__GFP_NOWARN,
>  				   huge_page_order(h));
>  	else
>  		page = alloc_pages_exact_node(nid,
> -			htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
> +			htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
>  			__GFP_REPEAT|__GFP_NOWARN, huge_page_order(h));
>
>  	if (page && arch_prepare_hugepage(page)) {
> @@ -2132,11 +2140,7 @@ int hugetlb_treat_movable_handler(struct ctl_table *table, int write,
>  			void __user *buffer,
>  			size_t *length, loff_t *ppos)
>  {
> -	proc_dointvec(table, write, buffer, length, ppos);
> -	if (hugepages_treat_as_movable)
> -		htlb_alloc_mask = GFP_HIGHUSER_MOVABLE;
> -	else
> -		htlb_alloc_mask = GFP_HIGHUSER;
> +	WARN_ON_ONCE("This knob is obsolete and has no effect. It is scheduled for removal.\n");
>  	return 0;
>  }
>
> -- 
> 1.8.3.1

Re: [PATCH 8/9] migrate: check movability of hugepage in unmap_and_move_huge_page()

[Aneesh Kumar K.V]

Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> writes:

> Currently hugepage migration works well only for pmd-based hugepages
> (mainly due to lack of testing,) so we had better not enable migration
> of other levels of hugepages until we are ready for it.
>
> Some users of hugepage migration (mbind, move_pages, and migrate_pages)
> do page table walk and check pud/pmd_huge() there, so they are safe.
> But the other users (softoffline and memory hotremove) don't do this,
> so without this patch they can try to migrate unexpected types of hugepages.
>
> To prevent this, we introduce hugepage_migration_support() as an architecture
> dependent check of whether hugepage are implemented on a pmd basis or not.
> And on some architecture multiple sizes of hugepages are available, so
> hugepage_migration_support() also checks hugepage size.
>
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> ---
>  arch/arm/mm/hugetlbpage.c     |  5 +++++
>  arch/arm64/mm/hugetlbpage.c   |  5 +++++
>  arch/ia64/mm/hugetlbpage.c    |  5 +++++
>  arch/metag/mm/hugetlbpage.c   |  5 +++++
>  arch/mips/mm/hugetlbpage.c    |  5 +++++
>  arch/powerpc/mm/hugetlbpage.c | 10 ++++++++++
>  arch/s390/mm/hugetlbpage.c    |  5 +++++
>  arch/sh/mm/hugetlbpage.c      |  5 +++++
>  arch/sparc/mm/hugetlbpage.c   |  5 +++++
>  arch/tile/mm/hugetlbpage.c    |  5 +++++
>  arch/x86/mm/hugetlbpage.c     |  8 ++++++++
>  include/linux/hugetlb.h       | 12 ++++++++++++
>  mm/migrate.c                  | 10 ++++++++++
>  13 files changed, 85 insertions(+)
>
> diff --git v3.11-rc3.orig/arch/arm/mm/hugetlbpage.c v3.11-rc3/arch/arm/mm/hugetlbpage.c
> index 3d1e4a2..3f3b6a7 100644
> --- v3.11-rc3.orig/arch/arm/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/arm/mm/hugetlbpage.c
> @@ -99,3 +99,8 @@ int pmd_huge(pmd_t pmd)
>  {
>  	return pmd_val(pmd) && !(pmd_val(pmd) & PMD_TABLE_BIT);
>  }
> +
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
> diff --git v3.11-rc3.orig/arch/arm64/mm/hugetlbpage.c v3.11-rc3/arch/arm64/mm/hugetlbpage.c
> index 2fc8258..5e9aec3 100644
> --- v3.11-rc3.orig/arch/arm64/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/arm64/mm/hugetlbpage.c
> @@ -54,6 +54,11 @@ int pud_huge(pud_t pud)
>  	return !(pud_val(pud) & PUD_TABLE_BIT);
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
> +
>  static __init int setup_hugepagesz(char *opt)
>  {
>  	unsigned long ps = memparse(opt, &opt);
> diff --git v3.11-rc3.orig/arch/ia64/mm/hugetlbpage.c v3.11-rc3/arch/ia64/mm/hugetlbpage.c
> index 76069c1..68232db 100644
> --- v3.11-rc3.orig/arch/ia64/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/ia64/mm/hugetlbpage.c
> @@ -114,6 +114,11 @@ int pud_huge(pud_t pud)
>  	return 0;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 0;
> +}
> +
>  struct page *
>  follow_huge_pmd(struct mm_struct *mm, unsigned long address, pmd_t *pmd, int write)
>  {
> diff --git v3.11-rc3.orig/arch/metag/mm/hugetlbpage.c v3.11-rc3/arch/metag/mm/hugetlbpage.c
> index 3c52fa6..0424315 100644
> --- v3.11-rc3.orig/arch/metag/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/metag/mm/hugetlbpage.c
> @@ -110,6 +110,11 @@ int pud_huge(pud_t pud)
>  	return 0;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
> +
>  struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  			     pmd_t *pmd, int write)
>  {
> diff --git v3.11-rc3.orig/arch/mips/mm/hugetlbpage.c v3.11-rc3/arch/mips/mm/hugetlbpage.c
> index a7fee0d..01fda44 100644
> --- v3.11-rc3.orig/arch/mips/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/mips/mm/hugetlbpage.c
> @@ -85,6 +85,11 @@ int pud_huge(pud_t pud)
>  	return (pud_val(pud) & _PAGE_HUGE) != 0;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
> +
>  struct page *
>  follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  		pmd_t *pmd, int write)
> diff --git v3.11-rc3.orig/arch/powerpc/mm/hugetlbpage.c v3.11-rc3/arch/powerpc/mm/hugetlbpage.c
> index 834ca8e..d67db4b 100644
> --- v3.11-rc3.orig/arch/powerpc/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/powerpc/mm/hugetlbpage.c
> @@ -86,6 +86,11 @@ int pgd_huge(pgd_t pgd)
>  	 */
>  	return ((pgd_val(pgd) & 0x3) != 0x0);
>  }
> +
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
>  #else
>  int pmd_huge(pmd_t pmd)
>  {
> @@ -101,6 +106,11 @@ int pgd_huge(pgd_t pgd)
>  {
>  	return 0;
>  }
> +
> +int pmd_huge_support(void)
> +{
> +	return 0;
> +}
>  #endif
>
>  pte_t *huge_pte_offset(struct mm_struct *mm, unsigned long addr)
> diff --git v3.11-rc3.orig/arch/s390/mm/hugetlbpage.c v3.11-rc3/arch/s390/mm/hugetlbpage.c
> index 121089d..951ee25 100644
> --- v3.11-rc3.orig/arch/s390/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/s390/mm/hugetlbpage.c
> @@ -117,6 +117,11 @@ int pud_huge(pud_t pud)
>  	return 0;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
> +
>  struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  			     pmd_t *pmdp, int write)
>  {
> diff --git v3.11-rc3.orig/arch/sh/mm/hugetlbpage.c v3.11-rc3/arch/sh/mm/hugetlbpage.c
> index d776234..0d676a4 100644
> --- v3.11-rc3.orig/arch/sh/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/sh/mm/hugetlbpage.c
> @@ -83,6 +83,11 @@ int pud_huge(pud_t pud)
>  	return 0;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 0;
> +}
> +
>  struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  			     pmd_t *pmd, int write)
>  {
> diff --git v3.11-rc3.orig/arch/sparc/mm/hugetlbpage.c v3.11-rc3/arch/sparc/mm/hugetlbpage.c
> index d2b5944..9639964 100644
> --- v3.11-rc3.orig/arch/sparc/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/sparc/mm/hugetlbpage.c
> @@ -234,6 +234,11 @@ int pud_huge(pud_t pud)
>  	return 0;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 0;
> +}
> +
>  struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  			     pmd_t *pmd, int write)
>  {
> diff --git v3.11-rc3.orig/arch/tile/mm/hugetlbpage.c v3.11-rc3/arch/tile/mm/hugetlbpage.c
> index 650ccff..0ac3599 100644
> --- v3.11-rc3.orig/arch/tile/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/tile/mm/hugetlbpage.c
> @@ -198,6 +198,11 @@ int pud_huge(pud_t pud)
>  	return !!(pud_val(pud) & _PAGE_HUGE_PAGE);
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
> +
>  struct page *follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  			     pmd_t *pmd, int write)
>  {
> diff --git v3.11-rc3.orig/arch/x86/mm/hugetlbpage.c v3.11-rc3/arch/x86/mm/hugetlbpage.c
> index 7e73e8c..9d980d8 100644
> --- v3.11-rc3.orig/arch/x86/mm/hugetlbpage.c
> +++ v3.11-rc3/arch/x86/mm/hugetlbpage.c
> @@ -59,6 +59,10 @@ follow_huge_pmd(struct mm_struct *mm, unsigned long address,
>  	return NULL;
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 0;
> +}
>  #else
>
>  struct page *
> @@ -77,6 +81,10 @@ int pud_huge(pud_t pud)
>  	return !!(pud_val(pud) & _PAGE_PSE);
>  }
>
> +int pmd_huge_support(void)
> +{
> +	return 1;
> +}
>  #endif
>
>  /* x86_64 also uses this file */
> diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
> index 2e02c4e..0393270 100644
> --- v3.11-rc3.orig/include/linux/hugetlb.h
> +++ v3.11-rc3/include/linux/hugetlb.h
> @@ -381,6 +381,16 @@ static inline pgoff_t basepage_index(struct page *page)
>
>  extern void dissolve_free_huge_pages(unsigned long start_pfn,
>  				     unsigned long end_pfn);
> +int pmd_huge_support(void);
> +/*
> + * Currently hugepage migration is enabled only for pmd-based hugepage.
> + * This function will be updated when hugepage migration is more widely
> + * supported.
> + */
> +static inline int hugepage_migration_support(struct hstate *h)
> +{
> +	return pmd_huge_support() && (huge_page_shift(h) == PMD_SHIFT);
> +}
>

For pmd_huge_support to be true, it needs huge_page_shift == PMD-SHIFT
right ? So why check again ?  Also are we only supporting hugepage migration
for hugepages at PMD level ? Why not at other levels ? Also isn't it
better to use zone movable allocation for larger size hugepags ?


-aneesh

Re: [PATCH 9/9] prepare to remove /proc/sys/vm/hugepages_treat_as_movable

[Naoya Horiguchi]

Hi Aneesh,

On Sun, Aug 11, 2013 at 11:13:58PM +0530, Aneesh Kumar K.V wrote:
> Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> writes:
> 
> > Now we have extended hugepage migration and it's opened to many users
> > of page migration, which is a good reason to consider hugepage as movable.
> > So we can go to the direction to remove this parameter. In order to
> > allow userspace to prepare for the removal, let's leave this sysctl handler
> > as noop for a while.
> >
> > Note that hugepage migration is available only for the architectures
> > which implement hugepage on a pmd basis. On the other architectures,
> > allocating hugepages from MOVABLE is not a good idea because it can
> > break memory hotremove (which expects that all pages of ZONE_MOVABLE are
> > movable.) So we choose GFP flags in accordance with mobility of hugepage.
> >
> > ChangeLog v5:
> >  - choose GFP flags in accordance with mobility of hugepage
> >
> > ChangeLog v3:
> >  - use WARN_ON_ONCE
> >
> > ChangeLog v2:
> >  - shift to noop function instead of completely removing the parameter
> >  - rename patch title
> >
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > Acked-by: Andi Kleen <ak@linux.intel.com>
> > Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> 
> Ok that mostly address the issue I raised last time, but how do archs
> that don't support hugepage migration force the hugepage allocation from
> movable zone. They were able to do that before with the sysctl you are
> removing in this patch isn't it ? May be keep that interface and print a
> warn_on of archs that support hugepage migration ?

OK, I might have persisted on the idea of removing the parameter, sorry.
If we keep it, I don't think we need this WARN_ON any more.
As the result the sysctl handler is a noop, and can be replaced with
a common function.

Attached the revised one, does it work for you?

Thanks,
Naoya Horiguchi
---
From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Date: Mon, 12 Aug 2013 11:49:37 -0400
Subject: [PATCH] hugetlb: make htlb_alloc_mask dependent on migration support

Now hugepage migration is enabled, although restricted on pmd-based hugepages
for now (due to lack of testing.) So we should allocate migratable hugepages
from ZONE_MOVABLE if possible.

This patch makes GFP flags in hugepage allocation dependent on migration
support, not only the value of hugepages_treat_as_movable.
It provides no change on the behavior for architectures which do not support
hugepage migration,

ChangeLog v5:
 - choose GFP flags in accordance with mobility of hugepage
 - keep hugepages_treat_as_movable

ChangeLog v3:
 - use WARN_ON_ONCE

ChangeLog v2:
 - shift to noop function instead of completely removing the parameter
 - rename patch title

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 Documentation/sysctl/vm.txt | 30 +++++++++++++++++++-----------
 kernel/sysctl.c             |  2 +-
 mm/hugetlb.c                | 32 ++++++++++++++------------------
 3 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index 36ecc26..79a797e 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -200,17 +200,25 @@ fragmentation index is <= extfrag_threshold. The default value is 500.
 
 hugepages_treat_as_movable
 
-This parameter is only useful when kernelcore= is specified at boot time to
-create ZONE_MOVABLE for pages that may be reclaimed or migrated. Huge pages
-are not movable so are not normally allocated from ZONE_MOVABLE. A non-zero
-value written to hugepages_treat_as_movable allows huge pages to be allocated
-from ZONE_MOVABLE.
-
-Once enabled, the ZONE_MOVABLE is treated as an area of memory the huge
-pages pool can easily grow or shrink within. Assuming that applications are
-not running that mlock() a lot of memory, it is likely the huge pages pool
-can grow to the size of ZONE_MOVABLE by repeatedly entering the desired value
-into nr_hugepages and triggering page reclaim.
+This parameter controls whether we can allocate hugepages from ZONE_MOVABLE
+or not. If set to non-zero, hugepages can be allocated from ZONE_MOVABLE.
+ZONE_MOVABLE is created when kernel boot parameter kernelcore= is specified,
+so this parameter has no effect if used without kernelcore=.
+
+Hugepage migration is now available in some situations which depend on the
+architecture and/or the hugepage size. If a hugepage supports migration,
+allocation from ZONE_MOVABLE is always enabled for the hugepage regardless
+of the value of this parameter.
+IOW, this parameter affects only non-migratable hugepages.
+
+Assuming that hugepages are not migratable in your system, one usecase of
+this parameter is that users can make hugepage pool more extensible by
+enabling the allocation from ZONE_MOVABLE. This is because on ZONE_MOVABLE
+page reclaim/migration/compaction work more and you can get contiguous
+memory more likely. Note that using ZONE_MOVABLE for non-migratable
+hugepages can do harm to other features like memory hotremove (because
+memory hotremove expects that memory blocks on ZONE_MOVABLE are always
+removable,) so it's a trade-off responsible for the users.
 
 ==============================================================
 
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index ac09d98..b4b32f5 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -1225,7 +1225,7 @@ static struct ctl_table vm_table[] = {
 		.data		= &hugepages_treat_as_movable,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
-		.proc_handler	= hugetlb_treat_movable_handler,
+		.proc_handler	= proc_dointvec,
 	},
 	{
 		.procname	= "nr_overcommit_hugepages",
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 3121915..364f745 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -34,7 +34,6 @@
 #include "internal.h"
 
 const unsigned long hugetlb_zero = 0, hugetlb_infinity = ~0UL;
-static gfp_t htlb_alloc_mask = GFP_HIGHUSER;
 unsigned long hugepages_treat_as_movable;
 
 int hugetlb_max_hstate __read_mostly;
@@ -535,6 +534,15 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid)
 	return page;
 }
 
+/* Movability of hugepages depends on migration support. */
+static inline int htlb_alloc_mask(struct hstate *h)
+{
+	if (hugepages_treat_as_movable || hugepage_migration_support(h))
+		return GFP_HIGHUSER_MOVABLE;
+	else
+		return GFP_HIGHUSER;
+}
+
 static struct page *dequeue_huge_page_vma(struct hstate *h,
 				struct vm_area_struct *vma,
 				unsigned long address, int avoid_reserve)
@@ -550,7 +558,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 retry_cpuset:
 	cpuset_mems_cookie = get_mems_allowed();
 	zonelist = huge_zonelist(vma, address,
-					htlb_alloc_mask, &mpol, &nodemask);
+					htlb_alloc_mask(h), &mpol, &nodemask);
 	/*
 	 * A child process with MAP_PRIVATE mappings created by their parent
 	 * have no page reserves. This check ensures that reservations are
@@ -566,7 +574,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 						MAX_NR_ZONES - 1, nodemask) {
-		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask)) {
+		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask(h))) {
 			page = dequeue_huge_page_node(h, zone_to_nid(zone));
 			if (page) {
 				if (!avoid_reserve)
@@ -723,7 +731,7 @@ static struct page *alloc_fresh_huge_page_node(struct hstate *h, int nid)
 		return NULL;
 
 	page = alloc_pages_exact_node(nid,
-		htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
+		htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
 						__GFP_REPEAT|__GFP_NOWARN,
 		huge_page_order(h));
 	if (page) {
@@ -948,12 +956,12 @@ static struct page *alloc_buddy_huge_page(struct hstate *h, int nid)
 	spin_unlock(&hugetlb_lock);
 
 	if (nid == NUMA_NO_NODE)
-		page = alloc_pages(htlb_alloc_mask|__GFP_COMP|
+		page = alloc_pages(htlb_alloc_mask(h)|__GFP_COMP|
 				   __GFP_REPEAT|__GFP_NOWARN,
 				   huge_page_order(h));
 	else
 		page = alloc_pages_exact_node(nid,
-			htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
+			htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
 			__GFP_REPEAT|__GFP_NOWARN, huge_page_order(h));
 
 	if (page && arch_prepare_hugepage(page)) {
@@ -2128,18 +2136,6 @@ int hugetlb_mempolicy_sysctl_handler(struct ctl_table *table, int write,
 }
 #endif /* CONFIG_NUMA */
 
-int hugetlb_treat_movable_handler(struct ctl_table *table, int write,
-			void __user *buffer,
-			size_t *length, loff_t *ppos)
-{
-	proc_dointvec(table, write, buffer, length, ppos);
-	if (hugepages_treat_as_movable)
-		htlb_alloc_mask = GFP_HIGHUSER_MOVABLE;
-	else
-		htlb_alloc_mask = GFP_HIGHUSER;
-	return 0;
-}
-
 int hugetlb_overcommit_handler(struct ctl_table *table, int write,
 			void __user *buffer,
 			size_t *length, loff_t *ppos)
-- 
1.8.3.1

Re: [PATCH v5 0/9] extend hugepage migration

[Andrew Morton]

On Fri,  9 Aug 2013 01:21:33 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:

> Here is the 5th version of hugepage migration patchset.
> Changes in this version are as follows:
>  - removed putback_active_hugepages() as a cleanup (1/9)
>  - added code to check movability of a given hugepage (8/9)
>  - set GFP MOVABLE flag depending on the movability of hugepage (9/9).
> 
> I feel that 8/9 and 9/9 contain some new things, so need reviews on them.
> 
> TODOs: (likely to be done after this work)
>  - split page table lock for pmd/pud based hugepage (maybe applicable to thp)
>  - improve alloc_migrate_target (especially in node choice)
>  - using page walker in check_range

This is a pretty large and complex patchset.  I skimmed the patches
(and have one trivial comment) then queued them up for a bit of
testing.  I've asked Mel if he can find time to review the changes
(please).

btw, it would be helpful if this [patch 0/n] had a decent overview of
the patch series - what are the objectives, how were they achieved,
what value they have to our users, testing results, etc.

mm-prepare-to-remove-proc-sys-vm-hugepages_treat_as_movable.patch had a
conflict with
http://ozlabs.org/~akpm/mmots/broken-out/mm-hugetlb-move-up-the-code-which-check-availability-of-free-huge-page.patch
which I resolved in the obvious manner.  Please check that from a
runtime perspective.

Re: [PATCH 3/9] migrate: add hugepage migration code to migrate_pages()

[Andrew Morton]

On Fri,  9 Aug 2013 01:21:36 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:

> +static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
> +		const nodemask_t *nodes, unsigned long flags,
> +				    void *private)
> +{
> +#ifdef CONFIG_HUGETLB_PAGE
> +	int nid;
> +	struct page *page;
> +
> +	spin_lock(&vma->vm_mm->page_table_lock);
> +	page = pte_page(huge_ptep_get((pte_t *)pmd));
> +	nid = page_to_nid(page);
> +	if (node_isset(nid, *nodes) == !!(flags & MPOL_MF_INVERT))
> +		goto unlock;
> +	/* With MPOL_MF_MOVE, we migrate only unshared hugepage. */
> +	if (flags & (MPOL_MF_MOVE_ALL) ||
> +	    (flags & MPOL_MF_MOVE && page_mapcount(page) == 1))
> +		isolate_huge_page(page, private);
> +unlock:
> +	spin_unlock(&vma->vm_mm->page_table_lock);
> +#else
> +	BUG();
> +#endif
> +}

The function is poorly named.  What does it "check"?  And it does more
than checking things - at actually makes alterations!

Can we have a better name here please, and some docmentation explaining
what it does and why it does it?

Re: [PATCH 3/9] migrate: add hugepage migration code to migrate_pages()

[Naoya Horiguchi]

On Wed, Aug 14, 2013 at 04:41:00PM -0700, Andrew Morton wrote:
> On Fri,  9 Aug 2013 01:21:36 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:
> 
> > +static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
> > +		const nodemask_t *nodes, unsigned long flags,
> > +				    void *private)
> > +{
> > +#ifdef CONFIG_HUGETLB_PAGE
> > +	int nid;
> > +	struct page *page;
> > +
> > +	spin_lock(&vma->vm_mm->page_table_lock);
> > +	page = pte_page(huge_ptep_get((pte_t *)pmd));
> > +	nid = page_to_nid(page);
> > +	if (node_isset(nid, *nodes) == !!(flags & MPOL_MF_INVERT))
> > +		goto unlock;
> > +	/* With MPOL_MF_MOVE, we migrate only unshared hugepage. */
> > +	if (flags & (MPOL_MF_MOVE_ALL) ||
> > +	    (flags & MPOL_MF_MOVE && page_mapcount(page) == 1))
> > +		isolate_huge_page(page, private);
> > +unlock:
> > +	spin_unlock(&vma->vm_mm->page_table_lock);
> > +#else
> > +	BUG();
> > +#endif
> > +}
> 
> The function is poorly named.  What does it "check"?  And it does more
> than checking things - at actually makes alterations!

I named like this because it became a new member of check_range() family.
So it's better to rename all of the family. An attached patch does this.

> Can we have a better name here please, and some docmentation explaining
> what it does and why it does it?

I also added some comment on check_range().

Thanks,
Naoya
---
From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Date: Wed, 14 Aug 2013 20:28:35 -0400
Subject: [PATCH] mempolicy: rename check_*range to queue_pages_*range

The function check_range() (and its family) is not well-named, because
it does not only checking something, but moving pages from list to list
to do page migration for them.
So queue_pages_*range is more desirable name.

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 mm/mempolicy.c | 41 +++++++++++++++++++++++------------------
 1 file changed, 23 insertions(+), 18 deletions(-)

diff --git a/mm/mempolicy.c b/mm/mempolicy.c
index 4a03c14..dca5225 100644
--- a/mm/mempolicy.c
+++ b/mm/mempolicy.c
@@ -473,8 +473,11 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = {
 static void migrate_page_add(struct page *page, struct list_head *pagelist,
 				unsigned long flags);
 
-/* Scan through pages checking if pages follow certain conditions. */
-static int check_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
+/*
+ * Scan through pages checking if pages follow certain conditions,
+ * and move them to the pagelist if they do.
+ */
+static int queue_pages_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 		unsigned long addr, unsigned long end,
 		const nodemask_t *nodes, unsigned long flags,
 		void *private)
@@ -512,8 +515,8 @@ static int check_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
 	return addr != end;
 }
 
-static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
-		const nodemask_t *nodes, unsigned long flags,
+static void queue_pages_hugetlb_pmd_range(struct vm_area_struct *vma,
+		pmd_t *pmd, const nodemask_t *nodes, unsigned long flags,
 				    void *private)
 {
 #ifdef CONFIG_HUGETLB_PAGE
@@ -536,7 +539,7 @@ static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
 #endif
 }
 
-static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
+static inline int queue_pages_pmd_range(struct vm_area_struct *vma, pud_t *pud,
 		unsigned long addr, unsigned long end,
 		const nodemask_t *nodes, unsigned long flags,
 		void *private)
@@ -548,21 +551,21 @@ static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
 	do {
 		next = pmd_addr_end(addr, end);
 		if (pmd_huge(*pmd) && is_vm_hugetlb_page(vma)) {
-			check_hugetlb_pmd_range(vma, pmd, nodes,
+			queue_pages_hugetlb_pmd_range(vma, pmd, nodes,
 						flags, private);
 			continue;
 		}
 		split_huge_page_pmd(vma, addr, pmd);
 		if (pmd_none_or_trans_huge_or_clear_bad(pmd))
 			continue;
-		if (check_pte_range(vma, pmd, addr, next, nodes,
+		if (queue_pages_pte_range(vma, pmd, addr, next, nodes,
 				    flags, private))
 			return -EIO;
 	} while (pmd++, addr = next, addr != end);
 	return 0;
 }
 
-static inline int check_pud_range(struct vm_area_struct *vma, pgd_t *pgd,
+static inline int queue_pages_pud_range(struct vm_area_struct *vma, pgd_t *pgd,
 		unsigned long addr, unsigned long end,
 		const nodemask_t *nodes, unsigned long flags,
 		void *private)
@@ -577,14 +580,14 @@ static inline int check_pud_range(struct vm_area_struct *vma, pgd_t *pgd,
 			continue;
 		if (pud_none_or_clear_bad(pud))
 			continue;
-		if (check_pmd_range(vma, pud, addr, next, nodes,
+		if (queue_pages_pmd_range(vma, pud, addr, next, nodes,
 				    flags, private))
 			return -EIO;
 	} while (pud++, addr = next, addr != end);
 	return 0;
 }
 
-static inline int check_pgd_range(struct vm_area_struct *vma,
+static inline int queue_pages_pgd_range(struct vm_area_struct *vma,
 		unsigned long addr, unsigned long end,
 		const nodemask_t *nodes, unsigned long flags,
 		void *private)
@@ -597,7 +600,7 @@ static inline int check_pgd_range(struct vm_area_struct *vma,
 		next = pgd_addr_end(addr, end);
 		if (pgd_none_or_clear_bad(pgd))
 			continue;
-		if (check_pud_range(vma, pgd, addr, next, nodes,
+		if (queue_pages_pud_range(vma, pgd, addr, next, nodes,
 				    flags, private))
 			return -EIO;
 	} while (pgd++, addr = next, addr != end);
@@ -635,12 +638,14 @@ static unsigned long change_prot_numa(struct vm_area_struct *vma,
 #endif /* CONFIG_ARCH_USES_NUMA_PROT_NONE */
 
 /*
- * Check if all pages in a range are on a set of nodes.
- * If pagelist != NULL then isolate pages from the LRU and
- * put them on the pagelist.
+ * Walk through page tables and collect pages to be migrated.
+ *
+ * If pages found in a given range are on a set of nodes (determined by
+ * @nodes and @flags,) it's isolated and queued to the pagelist which is
+ * passed via @private.)
  */
 static struct vm_area_struct *
-check_range(struct mm_struct *mm, unsigned long start, unsigned long end,
+queue_pages_range(struct mm_struct *mm, unsigned long start, unsigned long end,
 		const nodemask_t *nodes, unsigned long flags, void *private)
 {
 	int err;
@@ -675,7 +680,7 @@ check_range(struct mm_struct *mm, unsigned long start, unsigned long end,
 		     ((flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)) &&
 		      vma_migratable(vma))) {
 
-			err = check_pgd_range(vma, start, endvma, nodes,
+			err = queue_pages_pgd_range(vma, start, endvma, nodes,
 						flags, private);
 			if (err) {
 				first = ERR_PTR(err);
@@ -1041,7 +1046,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
 	 * space range and MPOL_MF_DISCONTIG_OK, this call can not fail.
 	 */
 	VM_BUG_ON(!(flags & (MPOL_MF_MOVE | MPOL_MF_MOVE_ALL)));
-	check_range(mm, mm->mmap->vm_start, mm->task_size, &nmask,
+	queue_pages_range(mm, mm->mmap->vm_start, mm->task_size, &nmask,
 			flags | MPOL_MF_DISCONTIG_OK, &pagelist);
 
 	if (!list_empty(&pagelist)) {
@@ -1279,7 +1284,7 @@ static long do_mbind(unsigned long start, unsigned long len,
 	if (err)
 		goto mpol_out;
 
-	vma = check_range(mm, start, end, nmask,
+	vma = queue_pages_range(mm, start, end, nmask,
 			  flags | MPOL_MF_INVERT, &pagelist);
 
 	err = PTR_ERR(vma);	/* maybe ... */
-- 
1.8.3.1

Re: [PATCH v5 0/9] extend hugepage migration

[Naoya Horiguchi]

On Wed, Aug 14, 2013 at 04:40:52PM -0700, Andrew Morton wrote:
> On Fri,  9 Aug 2013 01:21:33 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:
> 
> > Here is the 5th version of hugepage migration patchset.
> > Changes in this version are as follows:
> >  - removed putback_active_hugepages() as a cleanup (1/9)
> >  - added code to check movability of a given hugepage (8/9)
> >  - set GFP MOVABLE flag depending on the movability of hugepage (9/9).
> > 
> > I feel that 8/9 and 9/9 contain some new things, so need reviews on them.
> > 
> > TODOs: (likely to be done after this work)
> >  - split page table lock for pmd/pud based hugepage (maybe applicable to thp)
> >  - improve alloc_migrate_target (especially in node choice)
> >  - using page walker in check_range
> 
> This is a pretty large and complex patchset.  I skimmed the patches
> (and have one trivial comment) then queued them up for a bit of
> testing.  I've asked Mel if he can find time to review the changes
> (please).
> 
> btw, it would be helpful if this [patch 0/n] had a decent overview of
> the patch series - what are the objectives, how were they achieved,
> what value they have to our users, testing results, etc.

Here is the general description:
---
Currently hugepage migration is available only for soft offlining, but
it's also useful for some other users of page migration (clearly because
users of hugepage can enjoy the benefit of mempolicy and memory hotplug.)
So this patchset tries to extend such users to support hugepage migration.

The target of this patchset is to enable hugepage migration for NUMA
related system calls (migrate_pages(2), move_pages(2), and mbind(2)),
and memory hotplug.
This patchset does not add hugepage migration for memory compaction,
because users of memory compaction mainly expect to construct thp by
arranging raw pages, and there's little or no need to compact hugepages.
CMA, another user of page migration, can have benefit from hugepage
migration, but is not enabled to support it for now (just because of
lack of testing and expertise in CMA.)

Hugepage migration of non pmd-based hugepage (for example 1GB hugepage in
x86_64, or hugepages in architectures like ia64) is not enabled for now
(again, because of lack of testing.)
---

As for how these are achived, I extended the API (migrate_pages()) to
handle hugepage (with patch 1 and 2) and adjusted code of each caller
to check and collect movable hugepages (with patch 3-7). Remaining 2
patches are kind of miscellaneous ones to avoid unexpected behavior.
Patch 8 is about making sure that we only migrate pmd-based hugepages.
And patch 9 is about choosing appropriate zone for hugepage allocation.

My test is mainly functional one, simply kicking hugepage migration via
each entry point and confirm that migration is done correctly. Test code
is available here:
  git://github.com/Naoya-Horiguchi/test_hugepage_migration_extension.git

And I always run libhugetlbfs test when changing hugetlbfs's code.
With this patchset, no regression was found in the test.

> mm-prepare-to-remove-proc-sys-vm-hugepages_treat_as_movable.patch had a
> conflict with
> http://ozlabs.org/~akpm/mmots/broken-out/mm-hugetlb-move-up-the-code-which-check-availability-of-free-huge-page.patch
> which I resolved in the obvious manner.  Please check that from a
> runtime perspective.

As replied to the mm-commits notification ("Subject: + mm-prepare-to-remove-
proc-sys-vm-hugepages_treat_as_movable.patch added to -mm tree",)
I want to replace that patch with another one ("Subject: [PATCH] hugetlb:
make htlb_alloc_mask dependent on migration support").
With the new patch, the conflict with the Joonsoo's patch changes a little
bit (only difference of htlb_alloc_mask and htlb_alloc_mask(h).)
And I confirmed that the conflict had no harm on runtime behavior (passed
my testing.)

---
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 364f745..510b232 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -555,10 +555,6 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 	struct zoneref *z;
 	unsigned int cpuset_mems_cookie;
 
-retry_cpuset:
-	cpuset_mems_cookie = get_mems_allowed();
-	zonelist = huge_zonelist(vma, address,
-					htlb_alloc_mask(h), &mpol, &nodemask);
 	/*
 	 * A child process with MAP_PRIVATE mappings created by their parent
 	 * have no page reserves. This check ensures that reservations are
@@ -572,6 +568,11 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 	if (avoid_reserve && h->free_huge_pages - h->resv_huge_pages == 0)
 		goto err;
 
+retry_cpuset:
+	cpuset_mems_cookie = get_mems_allowed();
+	zonelist = huge_zonelist(vma, address,
+					htlb_alloc_mask(h), &mpol, &nodemask);
+
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 						MAX_NR_ZONES - 1, nodemask) {
 		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask(h))) {
@@ -590,7 +591,6 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 	return page;
 
 err:
-	mpol_cond_put(mpol);
 	return NULL;
 }

Re: [PATCH v5 0/9] extend hugepage migration

[Andrew Morton]

On Thu, 15 Aug 2013 02:23:40 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:

>
> ...
>
> > mm-prepare-to-remove-proc-sys-vm-hugepages_treat_as_movable.patch had a
> > conflict with
> > http://ozlabs.org/~akpm/mmots/broken-out/mm-hugetlb-move-up-the-code-which-check-availability-of-free-huge-page.patch
> > which I resolved in the obvious manner.  Please check that from a
> > runtime perspective.
> 
> As replied to the mm-commits notification ("Subject: + mm-prepare-to-remove-
> proc-sys-vm-hugepages_treat_as_movable.patch added to -mm tree",)

I can't find that email anywhere :(

> I want to replace that patch with another one ("Subject: [PATCH] hugetlb:
> make htlb_alloc_mask dependent on migration support").

Can you please formally resend this?

Re: [PATCH v5 0/9] extend hugepage migration

[Naoya Horiguchi]

On Thu, Aug 15, 2013 at 12:39:03PM -0700, Andrew Morton wrote:
> On Thu, 15 Aug 2013 02:23:40 -0400 Naoya Horiguchi <n-horiguchi@ah.jp.nec.com> wrote:
> 
> >
> > ...
> >
> > > mm-prepare-to-remove-proc-sys-vm-hugepages_treat_as_movable.patch had a
> > > conflict with
> > > http://ozlabs.org/~akpm/mmots/broken-out/mm-hugetlb-move-up-the-code-which-check-availability-of-free-huge-page.patch
> > > which I resolved in the obvious manner.  Please check that from a
> > > runtime perspective.
> > 
> > As replied to the mm-commits notification ("Subject: + mm-prepare-to-remove-
> > proc-sys-vm-hugepages_treat_as_movable.patch added to -mm tree",)
> 
> I can't find that email anywhere :(

I meant this one:
http://marc.info/?l=linux-mm&m=137653062031460&w=2

> > I want to replace that patch with another one ("Subject: [PATCH] hugetlb:
> > make htlb_alloc_mask dependent on migration support").
> 
> Can you please formally resend this?

Attached it.

Thanks,
Naoya
---
From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Date: Mon, 12 Aug 2013 11:49:37 -0400
Subject: [PATCH] hugetlb: make htlb_alloc_mask dependent on migration support

Now hugepage migration is enabled, although restricted on pmd-based hugepages
for now (due to lack of testing.) So we should allocate migratable hugepages
from ZONE_MOVABLE if possible.

This patch makes GFP flags in hugepage allocation dependent on migration
support, not only the value of hugepages_treat_as_movable.
It provides no change on the behavior for architectures which do not support
hugepage migration,

ChangeLog v5:
 - choose GFP flags in accordance with mobility of hugepage
 - keep hugepages_treat_as_movable

ChangeLog v3:
 - use WARN_ON_ONCE

ChangeLog v2:
 - shift to noop function instead of completely removing the parameter
 - rename patch title

Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 Documentation/sysctl/vm.txt | 30 +++++++++++++++++++-----------
 kernel/sysctl.c             |  2 +-
 mm/hugetlb.c                | 32 ++++++++++++++------------------
 3 files changed, 34 insertions(+), 30 deletions(-)

diff --git a/Documentation/sysctl/vm.txt b/Documentation/sysctl/vm.txt
index 36ecc26..79a797e 100644
--- a/Documentation/sysctl/vm.txt
+++ b/Documentation/sysctl/vm.txt
@@ -200,17 +200,25 @@ fragmentation index is <= extfrag_threshold. The default value is 500.
 
 hugepages_treat_as_movable
 
-This parameter is only useful when kernelcore= is specified at boot time to
-create ZONE_MOVABLE for pages that may be reclaimed or migrated. Huge pages
-are not movable so are not normally allocated from ZONE_MOVABLE. A non-zero
-value written to hugepages_treat_as_movable allows huge pages to be allocated
-from ZONE_MOVABLE.
-
-Once enabled, the ZONE_MOVABLE is treated as an area of memory the huge
-pages pool can easily grow or shrink within. Assuming that applications are
-not running that mlock() a lot of memory, it is likely the huge pages pool
-can grow to the size of ZONE_MOVABLE by repeatedly entering the desired value
-into nr_hugepages and triggering page reclaim.
+This parameter controls whether we can allocate hugepages from ZONE_MOVABLE
+or not. If set to non-zero, hugepages can be allocated from ZONE_MOVABLE.
+ZONE_MOVABLE is created when kernel boot parameter kernelcore= is specified,
+so this parameter has no effect if used without kernelcore=.
+
+Hugepage migration is now available in some situations which depend on the
+architecture and/or the hugepage size. If a hugepage supports migration,
+allocation from ZONE_MOVABLE is always enabled for the hugepage regardless
+of the value of this parameter.
+IOW, this parameter affects only non-migratable hugepages.
+
+Assuming that hugepages are not migratable in your system, one usecase of
+this parameter is that users can make hugepage pool more extensible by
+enabling the allocation from ZONE_MOVABLE. This is because on ZONE_MOVABLE
+page reclaim/migration/compaction work more and you can get contiguous
+memory more likely. Note that using ZONE_MOVABLE for non-migratable
+hugepages can do harm to other features like memory hotremove (because
+memory hotremove expects that memory blocks on ZONE_MOVABLE are always
+removable,) so it's a trade-off responsible for the users.
 
 ==============================================================
 
diff --git a/kernel/sysctl.c b/kernel/sysctl.c
index ac09d98..b4b32f5 100644
--- a/kernel/sysctl.c
+++ b/kernel/sysctl.c
@@ -1225,7 +1225,7 @@ static struct ctl_table vm_table[] = {
 		.data		= &hugepages_treat_as_movable,
 		.maxlen		= sizeof(int),
 		.mode		= 0644,
-		.proc_handler	= hugetlb_treat_movable_handler,
+		.proc_handler	= proc_dointvec,
 	},
 	{
 		.procname	= "nr_overcommit_hugepages",
diff --git a/mm/hugetlb.c b/mm/hugetlb.c
index 3121915..364f745 100644
--- a/mm/hugetlb.c
+++ b/mm/hugetlb.c
@@ -34,7 +34,6 @@
 #include "internal.h"
 
 const unsigned long hugetlb_zero = 0, hugetlb_infinity = ~0UL;
-static gfp_t htlb_alloc_mask = GFP_HIGHUSER;
 unsigned long hugepages_treat_as_movable;
 
 int hugetlb_max_hstate __read_mostly;
@@ -535,6 +534,15 @@ static struct page *dequeue_huge_page_node(struct hstate *h, int nid)
 	return page;
 }
 
+/* Movability of hugepages depends on migration support. */
+static inline int htlb_alloc_mask(struct hstate *h)
+{
+	if (hugepages_treat_as_movable || hugepage_migration_support(h))
+		return GFP_HIGHUSER_MOVABLE;
+	else
+		return GFP_HIGHUSER;
+}
+
 static struct page *dequeue_huge_page_vma(struct hstate *h,
 				struct vm_area_struct *vma,
 				unsigned long address, int avoid_reserve)
@@ -550,7 +558,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 retry_cpuset:
 	cpuset_mems_cookie = get_mems_allowed();
 	zonelist = huge_zonelist(vma, address,
-					htlb_alloc_mask, &mpol, &nodemask);
+					htlb_alloc_mask(h), &mpol, &nodemask);
 	/*
 	 * A child process with MAP_PRIVATE mappings created by their parent
 	 * have no page reserves. This check ensures that reservations are
@@ -566,7 +574,7 @@ static struct page *dequeue_huge_page_vma(struct hstate *h,
 
 	for_each_zone_zonelist_nodemask(zone, z, zonelist,
 						MAX_NR_ZONES - 1, nodemask) {
-		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask)) {
+		if (cpuset_zone_allowed_softwall(zone, htlb_alloc_mask(h))) {
 			page = dequeue_huge_page_node(h, zone_to_nid(zone));
 			if (page) {
 				if (!avoid_reserve)
@@ -723,7 +731,7 @@ static struct page *alloc_fresh_huge_page_node(struct hstate *h, int nid)
 		return NULL;
 
 	page = alloc_pages_exact_node(nid,
-		htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
+		htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
 						__GFP_REPEAT|__GFP_NOWARN,
 		huge_page_order(h));
 	if (page) {
@@ -948,12 +956,12 @@ static struct page *alloc_buddy_huge_page(struct hstate *h, int nid)
 	spin_unlock(&hugetlb_lock);
 
 	if (nid == NUMA_NO_NODE)
-		page = alloc_pages(htlb_alloc_mask|__GFP_COMP|
+		page = alloc_pages(htlb_alloc_mask(h)|__GFP_COMP|
 				   __GFP_REPEAT|__GFP_NOWARN,
 				   huge_page_order(h));
 	else
 		page = alloc_pages_exact_node(nid,
-			htlb_alloc_mask|__GFP_COMP|__GFP_THISNODE|
+			htlb_alloc_mask(h)|__GFP_COMP|__GFP_THISNODE|
 			__GFP_REPEAT|__GFP_NOWARN, huge_page_order(h));
 
 	if (page && arch_prepare_hugepage(page)) {
@@ -2128,18 +2136,6 @@ int hugetlb_mempolicy_sysctl_handler(struct ctl_table *table, int write,
 }
 #endif /* CONFIG_NUMA */
 
-int hugetlb_treat_movable_handler(struct ctl_table *table, int write,
-			void __user *buffer,
-			size_t *length, loff_t *ppos)
-{
-	proc_dointvec(table, write, buffer, length, ppos);
-	if (hugepages_treat_as_movable)
-		htlb_alloc_mask = GFP_HIGHUSER_MOVABLE;
-	else
-		htlb_alloc_mask = GFP_HIGHUSER;
-	return 0;
-}
-
 int hugetlb_overcommit_handler(struct ctl_table *table, int write,
 			void __user *buffer,
 			size_t *length, loff_t *ppos)
-- 
1.8.3.1

Re: [PATCH 1/9] migrate: make core migration code aware of hugepage

[Mel Gorman]

On Fri, Aug 09, 2013 at 01:21:34AM -0400, Naoya Horiguchi wrote:
> Before enabling each user of page migration to support hugepage,
> this patch enables the list of pages for migration to link not only
> LRU pages, but also hugepages. As a result, putback_movable_pages()
> and migrate_pages() can handle both of LRU pages and hugepages.
> 

LRU pages and *allocated* hugepages.

On its own the patch looks ok but it's not obvious at this point what
happens for pages that are on the hugetlbfs pool lists but not allocated
by any process. They will fail to isolate because of the
get_page_unless_zero() check. Maybe it's handled by a later patch.

-- 
Mel Gorman
SUSE Labs

Re: [PATCH 3/9] migrate: add hugepage migration code to migrate_pages()

[Mel Gorman]

On Fri, Aug 09, 2013 at 01:21:36AM -0400, Naoya Horiguchi wrote:
> This patch extends check_range() to handle vma with VM_HUGETLB set.
> We will be able to migrate hugepage with migrate_pages(2) after
> applying the enablement patch which comes later in this series.
> 
> Note that for larger hugepages (covered by pud entries, 1GB for
> x86_64 for example), we simply skip it now.
> 
> Note that using pmd_huge/pud_huge assumes that hugepages are pointed to
> by pmd/pud. This is not true in some architectures implementing hugepage
> with other mechanisms like ia64, but it's OK because pmd_huge/pud_huge
> simply return 0 in such arch and page walker simply ignores such hugepages.
> 
> ChangeLog v4:
>  - refactored check_hugetlb_pmd_range for better readability
> 
> ChangeLog v3:
>  - revert introducing migrate_movable_pages
>  - use isolate_huge_page
> 
> ChangeLog v2:
>  - remove unnecessary extern
>  - fix page table lock in check_hugetlb_pmd_range
>  - updated description and renamed patch title
> 
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Acked-by: Andi Kleen <ak@linux.intel.com>
> Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> Acked-by: Hillf Danton <dhillf@gmail.com>
> ---
>  mm/mempolicy.c | 42 +++++++++++++++++++++++++++++++++++++-----
>  1 file changed, 37 insertions(+), 5 deletions(-)
> 
> diff --git v3.11-rc3.orig/mm/mempolicy.c v3.11-rc3/mm/mempolicy.c
> index 7431001..d96afc1 100644
> --- v3.11-rc3.orig/mm/mempolicy.c
> +++ v3.11-rc3/mm/mempolicy.c
> @@ -512,6 +512,30 @@ static int check_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
>  	return addr != end;
>  }
>  
> +static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
> +		const nodemask_t *nodes, unsigned long flags,
> +				    void *private)
> +{
> +#ifdef CONFIG_HUGETLB_PAGE
> +	int nid;
> +	struct page *page;
> +
> +	spin_lock(&vma->vm_mm->page_table_lock);
> +	page = pte_page(huge_ptep_get((pte_t *)pmd));
> +	nid = page_to_nid(page);
> +	if (node_isset(nid, *nodes) == !!(flags & MPOL_MF_INVERT))
> +		goto unlock;
> +	/* With MPOL_MF_MOVE, we migrate only unshared hugepage. */
> +	if (flags & (MPOL_MF_MOVE_ALL) ||
> +	    (flags & MPOL_MF_MOVE && page_mapcount(page) == 1))
> +		isolate_huge_page(page, private);
> +unlock:
> +	spin_unlock(&vma->vm_mm->page_table_lock);
> +#else
> +	BUG();
> +#endif
> +}
> +
>  static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
>  		unsigned long addr, unsigned long end,
>  		const nodemask_t *nodes, unsigned long flags,
> @@ -523,6 +547,11 @@ static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
>  	pmd = pmd_offset(pud, addr);
>  	do {
>  		next = pmd_addr_end(addr, end);
> +		if (pmd_huge(*pmd) && is_vm_hugetlb_page(vma)) {
> +			check_hugetlb_pmd_range(vma, pmd, nodes,
> +						flags, private);
> +			continue;
> +		}
>  		split_huge_page_pmd(vma, addr, pmd);
>  		if (pmd_none_or_trans_huge_or_clear_bad(pmd))
>  			continue;

If a hugepage is currently being migrated then a migration entry should
be in its place which is a type of swap entry. Will the pmd_huge check
still do the right thing if migration is already in progress?

>  		if (check_pmd_range(vma, pud, addr, next, nodes,
> @@ -635,9 +666,6 @@ check_range(struct mm_struct *mm, unsigned long start, unsigned long end,
>  				return ERR_PTR(-EFAULT);
>  		}
>  
> -		if (is_vm_hugetlb_page(vma))
> -			goto next;
> -
>  		if (flags & MPOL_MF_LAZY) {
>  			change_prot_numa(vma, start, endvma);
>  			goto next;
> @@ -986,7 +1014,11 @@ static void migrate_page_add(struct page *page, struct list_head *pagelist,
>  
>  static struct page *new_node_page(struct page *page, unsigned long node, int **x)
>  {
> -	return alloc_pages_exact_node(node, GFP_HIGHUSER_MOVABLE, 0);
> +	if (PageHuge(page))
> +		return alloc_huge_page_node(page_hstate(compound_head(page)),
> +					node);
> +	else
> +		return alloc_pages_exact_node(node, GFP_HIGHUSER_MOVABLE, 0);
>  }
>  
>  /*
> @@ -1016,7 +1048,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
>  		err = migrate_pages(&pagelist, new_node_page, dest,
>  					MIGRATE_SYNC, MR_SYSCALL);
>  		if (err)
> -			putback_lru_pages(&pagelist);
> +			putback_movable_pages(&pagelist);
>  	}
>  
>  	return err;
> -- 
> 1.8.3.1
> 

-- 
Mel Gorman
SUSE Labs

Re: [PATCH 5/9] mbind: add hugepage migration code to mbind()

[Mel Gorman]

On Fri, Aug 09, 2013 at 01:21:38AM -0400, Naoya Horiguchi wrote:
> This patch extends do_mbind() to handle vma with VM_HUGETLB set.
> We will be able to migrate hugepage with mbind(2) after
> applying the enablement patch which comes later in this series.
> 
> ChangeLog v3:
>  - revert introducing migrate_movable_pages
>  - added alloc_huge_page_noerr free from ERR_VALUE
> 
> ChangeLog v2:
>  - updated description and renamed patch title
> 
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Acked-by: Andi Kleen <ak@linux.intel.com>
> Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> Acked-by: Hillf Danton <dhillf@gmail.com>
> ---
>  include/linux/hugetlb.h |  3 +++
>  mm/hugetlb.c            | 14 ++++++++++++++
>  mm/mempolicy.c          |  4 +++-
>  3 files changed, 20 insertions(+), 1 deletion(-)
> 
> diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
> index bc8d837..d1db007 100644
> --- v3.11-rc3.orig/include/linux/hugetlb.h
> +++ v3.11-rc3/include/linux/hugetlb.h
> @@ -265,6 +265,8 @@ struct huge_bootmem_page {
>  };
>  
>  struct page *alloc_huge_page_node(struct hstate *h, int nid);
> +struct page *alloc_huge_page_noerr(struct vm_area_struct *vma,
> +				unsigned long addr, int avoid_reserve);
>  
>  /* arch callback */
>  int __init alloc_bootmem_huge_page(struct hstate *h);
> @@ -378,6 +380,7 @@ static inline pgoff_t basepage_index(struct page *page)
>  #else	/* CONFIG_HUGETLB_PAGE */
>  struct hstate {};
>  #define alloc_huge_page_node(h, nid) NULL
> +#define alloc_huge_page_noerr(v, a, r) NULL
>  #define alloc_bootmem_huge_page(h) NULL
>  #define hstate_file(f) NULL
>  #define hstate_sizelog(s) NULL
> diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
> index 649771c..ee764b0 100644
> --- v3.11-rc3.orig/mm/hugetlb.c
> +++ v3.11-rc3/mm/hugetlb.c
> @@ -1195,6 +1195,20 @@ static struct page *alloc_huge_page(struct vm_area_struct *vma,
>  	return page;
>  }
>  
> +/*
> + * alloc_huge_page()'s wrapper which simply returns the page if allocation
> + * succeeds, otherwise NULL. This function is called from new_vma_page(),
> + * where no ERR_VALUE is expected to be returned.
> + */
> +struct page *alloc_huge_page_noerr(struct vm_area_struct *vma,
> +				unsigned long addr, int avoid_reserve)
> +{
> +	struct page *page = alloc_huge_page(vma, addr, avoid_reserve);
> +	if (IS_ERR(page))
> +		page = NULL;
> +	return page;
> +}
> +
>  int __weak alloc_bootmem_huge_page(struct hstate *h)
>  {
>  	struct huge_bootmem_page *m;
> diff --git v3.11-rc3.orig/mm/mempolicy.c v3.11-rc3/mm/mempolicy.c
> index d96afc1..4a03c14 100644
> --- v3.11-rc3.orig/mm/mempolicy.c
> +++ v3.11-rc3/mm/mempolicy.c
> @@ -1183,6 +1183,8 @@ static struct page *new_vma_page(struct page *page, unsigned long private, int *
>  		vma = vma->vm_next;
>  	}
>  
> +	if (PageHuge(page))
> +		return alloc_huge_page_noerr(vma, address, 1);
>  	/*
>  	 * if !vma, alloc_page_vma() will use task or system default policy
>  	 */

It's interesting to note that it will be tricky to configure a system to
allow this sort of migration to succeed.

This call correctly uses avoid_reserve but that does mean that for it
to work that there there must be free pages statically allocated in the
hugepage pool of the destination node or hugepage dynamic pool resizing
must be enabled. The former option is going to waste memory because pages
allocated to the static pool cannot be used for any other purpose and
using dynamic hugepage pool resizing may fail.

It makes me wonder how actually useful generic hugetlbfs page migration
will be in practice. Are there really usecases where the system
administrator is willing to create unused hugepage pools on each node
just to enable migration?

> @@ -1293,7 +1295,7 @@ static long do_mbind(unsigned long start, unsigned long len,
>  					(unsigned long)vma,
>  					MIGRATE_SYNC, MR_MEMPOLICY_MBIND);
>  			if (nr_failed)
> -				putback_lru_pages(&pagelist);
> +				putback_movable_pages(&pagelist);
>  		}
>  
>  		if (nr_failed && (flags & MPOL_MF_STRICT))
> -- 
> 1.8.3.1
> 

-- 
Mel Gorman
SUSE Labs

Re: [PATCH 9/9] prepare to remove /proc/sys/vm/hugepages_treat_as_movable

[Mel Gorman]

On Fri, Aug 09, 2013 at 01:21:42AM -0400, Naoya Horiguchi wrote:
> Now we have extended hugepage migration and it's opened to many users
> of page migration, which is a good reason to consider hugepage as movable.
> So we can go to the direction to remove this parameter. In order to
> allow userspace to prepare for the removal, let's leave this sysctl handler
> as noop for a while.
> 

Note that this assumes that users interested in memory hot-remove and
hugepages are also willing to resize the hugepage pool on the target nodes
before attempting the hot-remove operation. I guess that technically the
necessary setup steps could be done from userspace or manually by the
system administrator but it may not be obvious to the system
administrator that the step is required.

-- 
Mel Gorman
SUSE Labs

Re: [PATCH 1/9] migrate: make core migration code aware of hugepage

[Naoya Horiguchi]

On Tue, Sep 10, 2013 at 02:51:30PM +0100, Mel Gorman wrote:
> On Fri, Aug 09, 2013 at 01:21:34AM -0400, Naoya Horiguchi wrote:
> > Before enabling each user of page migration to support hugepage,
> > this patch enables the list of pages for migration to link not only
> > LRU pages, but also hugepages. As a result, putback_movable_pages()
> > and migrate_pages() can handle both of LRU pages and hugepages.
> > 
> 
> LRU pages and *allocated* hugepages.

Right.

> On its own the patch looks ok but it's not obvious at this point what
> happens for pages that are on the hugetlbfs pool lists but not allocated
> by any process.

OK. I'll add comments about clarifying that. Now I'm preparing the next
patchset for migration of 1GB hugepages, so it's done in that series.

> They will fail to isolate because of the
> get_page_unless_zero() check. Maybe it's handled by a later patch.

The callers which determine the target pages with virtual address (like
mbind, migrate_pages) don't try to migrate hugepages in the hugetlbfs
pool. And the other callers which determine targets with physical address
(like memory hotplug and soft offline) have their own check not to migrate
free hugepages.

Thanks,
Naoya Horiguchi

Re: [PATCH 3/9] migrate: add hugepage migration code to migrate_pages()

[Naoya Horiguchi]

On Tue, Sep 10, 2013 at 03:33:26PM +0100, Mel Gorman wrote:
> On Fri, Aug 09, 2013 at 01:21:36AM -0400, Naoya Horiguchi wrote:
> > This patch extends check_range() to handle vma with VM_HUGETLB set.
> > We will be able to migrate hugepage with migrate_pages(2) after
> > applying the enablement patch which comes later in this series.
> > 
> > Note that for larger hugepages (covered by pud entries, 1GB for
> > x86_64 for example), we simply skip it now.
> > 
> > Note that using pmd_huge/pud_huge assumes that hugepages are pointed to
> > by pmd/pud. This is not true in some architectures implementing hugepage
> > with other mechanisms like ia64, but it's OK because pmd_huge/pud_huge
> > simply return 0 in such arch and page walker simply ignores such hugepages.
> > 
> > ChangeLog v4:
> >  - refactored check_hugetlb_pmd_range for better readability
> > 
> > ChangeLog v3:
> >  - revert introducing migrate_movable_pages
> >  - use isolate_huge_page
> > 
> > ChangeLog v2:
> >  - remove unnecessary extern
> >  - fix page table lock in check_hugetlb_pmd_range
> >  - updated description and renamed patch title
> > 
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > Acked-by: Andi Kleen <ak@linux.intel.com>
> > Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> > Acked-by: Hillf Danton <dhillf@gmail.com>
> > ---
> >  mm/mempolicy.c | 42 +++++++++++++++++++++++++++++++++++++-----
> >  1 file changed, 37 insertions(+), 5 deletions(-)
> > 
> > diff --git v3.11-rc3.orig/mm/mempolicy.c v3.11-rc3/mm/mempolicy.c
> > index 7431001..d96afc1 100644
> > --- v3.11-rc3.orig/mm/mempolicy.c
> > +++ v3.11-rc3/mm/mempolicy.c
> > @@ -512,6 +512,30 @@ static int check_pte_range(struct vm_area_struct *vma, pmd_t *pmd,
> >  	return addr != end;
> >  }
> >  
> > +static void check_hugetlb_pmd_range(struct vm_area_struct *vma, pmd_t *pmd,
> > +		const nodemask_t *nodes, unsigned long flags,
> > +				    void *private)
> > +{
> > +#ifdef CONFIG_HUGETLB_PAGE
> > +	int nid;
> > +	struct page *page;
> > +
> > +	spin_lock(&vma->vm_mm->page_table_lock);
> > +	page = pte_page(huge_ptep_get((pte_t *)pmd));
> > +	nid = page_to_nid(page);
> > +	if (node_isset(nid, *nodes) == !!(flags & MPOL_MF_INVERT))
> > +		goto unlock;
> > +	/* With MPOL_MF_MOVE, we migrate only unshared hugepage. */
> > +	if (flags & (MPOL_MF_MOVE_ALL) ||
> > +	    (flags & MPOL_MF_MOVE && page_mapcount(page) == 1))
> > +		isolate_huge_page(page, private);
> > +unlock:
> > +	spin_unlock(&vma->vm_mm->page_table_lock);
> > +#else
> > +	BUG();
> > +#endif
> > +}
> > +
> >  static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
> >  		unsigned long addr, unsigned long end,
> >  		const nodemask_t *nodes, unsigned long flags,
> > @@ -523,6 +547,11 @@ static inline int check_pmd_range(struct vm_area_struct *vma, pud_t *pud,
> >  	pmd = pmd_offset(pud, addr);
> >  	do {
> >  		next = pmd_addr_end(addr, end);
> > +		if (pmd_huge(*pmd) && is_vm_hugetlb_page(vma)) {
> > +			check_hugetlb_pmd_range(vma, pmd, nodes,
> > +						flags, private);
> > +			continue;
> > +		}
> >  		split_huge_page_pmd(vma, addr, pmd);
> >  		if (pmd_none_or_trans_huge_or_clear_bad(pmd))
> >  			continue;
> 
> If a hugepage is currently being migrated then a migration entry should
> be in its place which is a type of swap entry. Will the pmd_huge check
> still do the right thing if migration is already in progress?

Ah, we need migration entry check before pmd_huge check. Thank you.

Thanks,
Naoya Horiguchi

> 
> >  		if (check_pmd_range(vma, pud, addr, next, nodes,
> > @@ -635,9 +666,6 @@ check_range(struct mm_struct *mm, unsigned long start, unsigned long end,
> >  				return ERR_PTR(-EFAULT);
> >  		}
> >  
> > -		if (is_vm_hugetlb_page(vma))
> > -			goto next;
> > -
> >  		if (flags & MPOL_MF_LAZY) {
> >  			change_prot_numa(vma, start, endvma);
> >  			goto next;
> > @@ -986,7 +1014,11 @@ static void migrate_page_add(struct page *page, struct list_head *pagelist,
> >  
> >  static struct page *new_node_page(struct page *page, unsigned long node, int **x)
> >  {
> > -	return alloc_pages_exact_node(node, GFP_HIGHUSER_MOVABLE, 0);
> > +	if (PageHuge(page))
> > +		return alloc_huge_page_node(page_hstate(compound_head(page)),
> > +					node);
> > +	else
> > +		return alloc_pages_exact_node(node, GFP_HIGHUSER_MOVABLE, 0);
> >  }
> >  
> >  /*
> > @@ -1016,7 +1048,7 @@ static int migrate_to_node(struct mm_struct *mm, int source, int dest,
> >  		err = migrate_pages(&pagelist, new_node_page, dest,
> >  					MIGRATE_SYNC, MR_SYSCALL);
> >  		if (err)
> > -			putback_lru_pages(&pagelist);
> > +			putback_movable_pages(&pagelist);
> >  	}
> >  
> >  	return err;
> > -- 
> > 1.8.3.1
> > 
> 
> -- 
> Mel Gorman
> SUSE Labs
>

Re: [PATCH 5/9] mbind: add hugepage migration code to mbind()

[Naoya Horiguchi]

On Tue, Sep 10, 2013 at 03:41:09PM +0100, Mel Gorman wrote:
> On Fri, Aug 09, 2013 at 01:21:38AM -0400, Naoya Horiguchi wrote:
> > This patch extends do_mbind() to handle vma with VM_HUGETLB set.
> > We will be able to migrate hugepage with mbind(2) after
> > applying the enablement patch which comes later in this series.
> > 
> > ChangeLog v3:
> >  - revert introducing migrate_movable_pages
> >  - added alloc_huge_page_noerr free from ERR_VALUE
> > 
> > ChangeLog v2:
> >  - updated description and renamed patch title
> > 
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > Acked-by: Andi Kleen <ak@linux.intel.com>
> > Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> > Acked-by: Hillf Danton <dhillf@gmail.com>
> > ---
> >  include/linux/hugetlb.h |  3 +++
> >  mm/hugetlb.c            | 14 ++++++++++++++
> >  mm/mempolicy.c          |  4 +++-
> >  3 files changed, 20 insertions(+), 1 deletion(-)
> > 
> > diff --git v3.11-rc3.orig/include/linux/hugetlb.h v3.11-rc3/include/linux/hugetlb.h
> > index bc8d837..d1db007 100644
> > --- v3.11-rc3.orig/include/linux/hugetlb.h
> > +++ v3.11-rc3/include/linux/hugetlb.h
> > @@ -265,6 +265,8 @@ struct huge_bootmem_page {
> >  };
> >  
> >  struct page *alloc_huge_page_node(struct hstate *h, int nid);
> > +struct page *alloc_huge_page_noerr(struct vm_area_struct *vma,
> > +				unsigned long addr, int avoid_reserve);
> >  
> >  /* arch callback */
> >  int __init alloc_bootmem_huge_page(struct hstate *h);
> > @@ -378,6 +380,7 @@ static inline pgoff_t basepage_index(struct page *page)
> >  #else	/* CONFIG_HUGETLB_PAGE */
> >  struct hstate {};
> >  #define alloc_huge_page_node(h, nid) NULL
> > +#define alloc_huge_page_noerr(v, a, r) NULL
> >  #define alloc_bootmem_huge_page(h) NULL
> >  #define hstate_file(f) NULL
> >  #define hstate_sizelog(s) NULL
> > diff --git v3.11-rc3.orig/mm/hugetlb.c v3.11-rc3/mm/hugetlb.c
> > index 649771c..ee764b0 100644
> > --- v3.11-rc3.orig/mm/hugetlb.c
> > +++ v3.11-rc3/mm/hugetlb.c
> > @@ -1195,6 +1195,20 @@ static struct page *alloc_huge_page(struct vm_area_struct *vma,
> >  	return page;
> >  }
> >  
> > +/*
> > + * alloc_huge_page()'s wrapper which simply returns the page if allocation
> > + * succeeds, otherwise NULL. This function is called from new_vma_page(),
> > + * where no ERR_VALUE is expected to be returned.
> > + */
> > +struct page *alloc_huge_page_noerr(struct vm_area_struct *vma,
> > +				unsigned long addr, int avoid_reserve)
> > +{
> > +	struct page *page = alloc_huge_page(vma, addr, avoid_reserve);
> > +	if (IS_ERR(page))
> > +		page = NULL;
> > +	return page;
> > +}
> > +
> >  int __weak alloc_bootmem_huge_page(struct hstate *h)
> >  {
> >  	struct huge_bootmem_page *m;
> > diff --git v3.11-rc3.orig/mm/mempolicy.c v3.11-rc3/mm/mempolicy.c
> > index d96afc1..4a03c14 100644
> > --- v3.11-rc3.orig/mm/mempolicy.c
> > +++ v3.11-rc3/mm/mempolicy.c
> > @@ -1183,6 +1183,8 @@ static struct page *new_vma_page(struct page *page, unsigned long private, int *
> >  		vma = vma->vm_next;
> >  	}
> >  
> > +	if (PageHuge(page))
> > +		return alloc_huge_page_noerr(vma, address, 1);
> >  	/*
> >  	 * if !vma, alloc_page_vma() will use task or system default policy
> >  	 */
> 
> It's interesting to note that it will be tricky to configure a system to
> allow this sort of migration to succeed.
> 
> This call correctly uses avoid_reserve but that does mean that for it
> to work that there there must be free pages statically allocated in the
> hugepage pool of the destination node or hugepage dynamic pool resizing
> must be enabled. The former option is going to waste memory because pages
> allocated to the static pool cannot be used for any other purpose and
> using dynamic hugepage pool resizing may fail.

Yes, that's interesting because it's important to make page migration
more likely to succeed. I guess that dynamic pool resizing can affect
the pool configuration without administrators' knowing, so allocating
surplus hugepages directly from buddy seems more preferable.

> It makes me wonder how actually useful generic hugetlbfs page migration
> will be in practice. Are there really usecases where the system
> administrator is willing to create unused hugepage pools on each node
> just to enable migration?

Maybe most users don't want it.

Thanks,
Naoya Horiguchi

> > @@ -1293,7 +1295,7 @@ static long do_mbind(unsigned long start, unsigned long len,
> >  					(unsigned long)vma,
> >  					MIGRATE_SYNC, MR_MEMPOLICY_MBIND);
> >  			if (nr_failed)
> > -				putback_lru_pages(&pagelist);
> > +				putback_movable_pages(&pagelist);
> >  		}
> >  
> >  		if (nr_failed && (flags & MPOL_MF_STRICT))
> > -- 
> > 1.8.3.1
> > 
> 
> -- 
> Mel Gorman
> SUSE Labs
>

Re: [PATCH 5/9] mbind: add hugepage migration code to mbind()

[Andi Kleen]

> > It makes me wonder how actually useful generic hugetlbfs page migration
> > will be in practice. Are there really usecases where the system
> > administrator is willing to create unused hugepage pools on each node
> > just to enable migration?
> 
> Maybe most users don't want it.

I'm sure some power users will be willing to do that.
Of course for a lot of others THP is enough.

-Andi

Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages()

[Borislav Petkov]

On Fri, Aug 09, 2013 at 01:21:37AM -0400, Naoya Horiguchi wrote:
> This patch extends move_pages() to handle vma with VM_HUGETLB set.
> We will be able to migrate hugepage with move_pages(2) after
> applying the enablement patch which comes later in this series.
> 
> We avoid getting refcount on tail pages of hugepage, because unlike thp,
> hugepage is not split and we need not care about races with splitting.
> 
> And migration of larger (1GB for x86_64) hugepage are not enabled.
> 
> ChangeLog v4:
>  - use get_page instead of get_page_foll
>  - add comment in follow_page_mask
> 
> ChangeLog v3:
>  - revert introducing migrate_movable_pages
>  - follow_page_mask(FOLL_GET) returns NULL for tail pages
>  - use isolate_huge_page
> 
> ChangeLog v2:
>  - updated description and renamed patch title
> 
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Acked-by: Andi Kleen <ak@linux.intel.com>
> Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> ---
>  mm/memory.c  | 17 +++++++++++++++--
>  mm/migrate.c | 13 +++++++++++--
>  2 files changed, 26 insertions(+), 4 deletions(-)

...

> diff --git v3.11-rc3.orig/mm/migrate.c v3.11-rc3/mm/migrate.c
> index 3ec47d3..d313737 100644
> --- v3.11-rc3.orig/mm/migrate.c
> +++ v3.11-rc3/mm/migrate.c
> @@ -1092,7 +1092,11 @@ static struct page *new_page_node(struct page *p, unsigned long private,
>  
>  	*result = &pm->status;
>  
> -	return alloc_pages_exact_node(pm->node,
> +	if (PageHuge(p))
> +		return alloc_huge_page_node(page_hstate(compound_head(p)),
> +					pm->node);
> +	else
> +		return alloc_pages_exact_node(pm->node,
>  				GFP_HIGHUSER_MOVABLE | GFP_THISNODE, 0);
>  }
>  
> @@ -1152,6 +1156,11 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
>  				!migrate_all)
>  			goto put_and_set;
>  
> +		if (PageHuge(page)) {
> +			isolate_huge_page(page, &pagelist);
> +			goto put_and_set;
> +		}

This gives

In file included from mm/migrate.c:35:0:
mm/migrate.c: In function ‘do_move_page_to_node_array’:
include/linux/hugetlb.h:140:33: warning: statement with no effect [-Wunused-value]
 #define isolate_huge_page(p, l) false
                                 ^
mm/migrate.c:1170:4: note: in expansion of macro ‘isolate_huge_page’
    isolate_huge_page(page, &pagelist);

on a

# CONFIG_HUGETLBFS is not set

.config.

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages()

[Naoya Horiguchi]

On Sat, Sep 28, 2013 at 07:26:02PM +0200, Borislav Petkov wrote:
> On Fri, Aug 09, 2013 at 01:21:37AM -0400, Naoya Horiguchi wrote:
> > This patch extends move_pages() to handle vma with VM_HUGETLB set.
> > We will be able to migrate hugepage with move_pages(2) after
> > applying the enablement patch which comes later in this series.
> > 
> > We avoid getting refcount on tail pages of hugepage, because unlike thp,
> > hugepage is not split and we need not care about races with splitting.
> > 
> > And migration of larger (1GB for x86_64) hugepage are not enabled.
> > 
> > ChangeLog v4:
> >  - use get_page instead of get_page_foll
> >  - add comment in follow_page_mask
> > 
> > ChangeLog v3:
> >  - revert introducing migrate_movable_pages
> >  - follow_page_mask(FOLL_GET) returns NULL for tail pages
> >  - use isolate_huge_page
> > 
> > ChangeLog v2:
> >  - updated description and renamed patch title
> > 
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > Acked-by: Andi Kleen <ak@linux.intel.com>
> > Reviewed-by: Wanpeng Li <liwanp@linux.vnet.ibm.com>
> > ---
> >  mm/memory.c  | 17 +++++++++++++++--
> >  mm/migrate.c | 13 +++++++++++--
> >  2 files changed, 26 insertions(+), 4 deletions(-)
> 
> ...
> 
> > diff --git v3.11-rc3.orig/mm/migrate.c v3.11-rc3/mm/migrate.c
> > index 3ec47d3..d313737 100644
> > --- v3.11-rc3.orig/mm/migrate.c
> > +++ v3.11-rc3/mm/migrate.c
> > @@ -1092,7 +1092,11 @@ static struct page *new_page_node(struct page *p, unsigned long private,
> >  
> >  	*result = &pm->status;
> >  
> > -	return alloc_pages_exact_node(pm->node,
> > +	if (PageHuge(p))
> > +		return alloc_huge_page_node(page_hstate(compound_head(p)),
> > +					pm->node);
> > +	else
> > +		return alloc_pages_exact_node(pm->node,
> >  				GFP_HIGHUSER_MOVABLE | GFP_THISNODE, 0);
> >  }
> >  
> > @@ -1152,6 +1156,11 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
> >  				!migrate_all)
> >  			goto put_and_set;
> >  
> > +		if (PageHuge(page)) {
> > +			isolate_huge_page(page, &pagelist);
> > +			goto put_and_set;
> > +		}
> 
> This gives
> 
> In file included from mm/migrate.c:35:0:
> mm/migrate.c: In function ‘do_move_page_to_node_array’:
> include/linux/hugetlb.h:140:33: warning: statement with no effect [-Wunused-value]
>  #define isolate_huge_page(p, l) false
>                                  ^
> mm/migrate.c:1170:4: note: in expansion of macro ‘isolate_huge_page’
>     isolate_huge_page(page, &pagelist);
> 
> on a
> 
> # CONFIG_HUGETLBFS is not set
> 
> .config.

Thanks for reporting. The patch should fix this.

Naoya Horiguchi
---
From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Date: Mon, 30 Sep 2013 10:22:26 -0400
Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()

Introduces a cosmetic substitution of the returned value of isolate_huge_page()
to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.

Reported-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 mm/migrate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 4cd63c2..4a26042 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -1168,7 +1168,7 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 			goto put_and_set;
 
 		if (PageHuge(page)) {
-			isolate_huge_page(page, &pagelist);
+			err = isolate_huge_page(page, &pagelist);
 			goto put_and_set;
 		}
 
-- 
1.8.3.1

Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages()

[Borislav Petkov]

On Mon, Sep 30, 2013 at 11:01:03AM -0400, Naoya Horiguchi wrote:
> Thanks for reporting. The patch should fix this.
> 
> Naoya Horiguchi
> ---
> From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Date: Mon, 30 Sep 2013 10:22:26 -0400
> Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()
> 
> Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> 
> Reported-by: Borislav Petkov <bp@alien8.de>
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>

Thanks for this. Unfortunately, I cannot trigger it anymore. :\ Maybe it
is because I pulled latest git and this was triggering only on a older
repo state, hmmm.

The patch looks obviously correct though so you could send it up or hold
on to it until someone else reports it.

Anyway, sorry for the trouble.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages()

[Naoya Horiguchi]

On Mon, Sep 30, 2013 at 06:04:50PM +0200, Borislav Petkov wrote:
> On Mon, Sep 30, 2013 at 11:01:03AM -0400, Naoya Horiguchi wrote:
> > Thanks for reporting. The patch should fix this.
> > 
> > Naoya Horiguchi
> > ---
> > From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > Date: Mon, 30 Sep 2013 10:22:26 -0400
> > Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()
> > 
> > Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> > to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> > 
> > Reported-by: Borislav Petkov <bp@alien8.de>
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> 
> Thanks for this. Unfortunately, I cannot trigger it anymore. :\ Maybe it
> is because I pulled latest git and this was triggering only on a older
> repo state, hmmm.
> 
> The patch looks obviously correct though so you could send it up or hold
> on to it until someone else reports it.
> 
> Anyway, sorry for the trouble.

OK, no problem :)

Thanks,
Naoya

Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages()

[Borislav Petkov]

[-- Attachment #1: Type: text/plain, Size: 1893 bytes --]

On Mon, Sep 30, 2013 at 12:08:44PM -0400, Naoya Horiguchi wrote:
> On Mon, Sep 30, 2013 at 06:04:50PM +0200, Borislav Petkov wrote:
> > On Mon, Sep 30, 2013 at 11:01:03AM -0400, Naoya Horiguchi wrote:
> > > Thanks for reporting. The patch should fix this.
> > > 
> > > Naoya Horiguchi
> > > ---
> > > From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > > Date: Mon, 30 Sep 2013 10:22:26 -0400
> > > Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()
> > > 
> > > Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> > > to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> > > 
> > > Reported-by: Borislav Petkov <bp@alien8.de>
> > > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > 
> > Thanks for this. Unfortunately, I cannot trigger it anymore. :\ Maybe it
> > is because I pulled latest git and this was triggering only on a older
> > repo state, hmmm.
> > 
> > The patch looks obviously correct though so you could send it up or hold
> > on to it until someone else reports it.
> > 
> > Anyway, sorry for the trouble.
> 
> OK, no problem :)

Hey Naoya,

I can trigger this issue again.

Kernel is latest Linus: v3.12-4849-g10d0c9705e80

Compiler is: gcc (Debian 4.8.1-10) 4.8.1, config is attached.

And yes, the patch you sent me previously is still good and fixes the
warning so feel free to add my Tested-by: tag.

In file included from mm/migrate.c:35:0:
mm/migrate.c: In function ‘do_move_page_to_node_array’:
include/linux/hugetlb.h:140:33: warning: statement with no effect [-Wunused-value]
 #define isolate_huge_page(p, l) false
                                 ^
mm/migrate.c:1181:4: note: in expansion of macro ‘isolate_huge_page’
    isolate_huge_page(page, &pagelist);
    ^

Thanks.

-- 
Regards/Gruss,
    Boris.

Sent from a fat crate under my desk. Formatting is fine.
--

[-- Attachment #2: config.gz --]
[-- Type: application/octet-stream, Size: 20043 bytes --]

[PATCH] mm/migrate.c: take returned value of isolate_huge_page()(Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages())

[Naoya Horiguchi]

On Tue, Nov 12, 2013 at 12:56:33PM +0100, Borislav Petkov wrote:
> On Mon, Sep 30, 2013 at 12:08:44PM -0400, Naoya Horiguchi wrote:
> > On Mon, Sep 30, 2013 at 06:04:50PM +0200, Borislav Petkov wrote:
> > > On Mon, Sep 30, 2013 at 11:01:03AM -0400, Naoya Horiguchi wrote:
> > > > Thanks for reporting. The patch should fix this.
> > > > 
> > > > Naoya Horiguchi
> > > > ---
> > > > From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > > > Date: Mon, 30 Sep 2013 10:22:26 -0400
> > > > Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()
> > > > 
> > > > Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> > > > to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> > > > 
> > > > Reported-by: Borislav Petkov <bp@alien8.de>
> > > > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > > 
> > > Thanks for this. Unfortunately, I cannot trigger it anymore. :\ Maybe it
> > > is because I pulled latest git and this was triggering only on a older
> > > repo state, hmmm.
> > > 
> > > The patch looks obviously correct though so you could send it up or hold
> > > on to it until someone else reports it.
> > > 
> > > Anyway, sorry for the trouble.
> > 
> > OK, no problem :)
> 
> Hey Naoya,
> 
> I can trigger this issue again.
> 
> Kernel is latest Linus: v3.12-4849-g10d0c9705e80
> 
> Compiler is: gcc (Debian 4.8.1-10) 4.8.1, config is attached.
> 
> And yes, the patch you sent me previously is still good and fixes the
> warning so feel free to add my Tested-by: tag.

Sorry for late response, and thanks for testing!
Andrew, can you apply this fix?

Thanks,
Naoya Horiguchi
---
From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Date: Mon, 30 Sep 2013 10:22:26 -0400
Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()

Introduces a cosmetic substitution of the returned value of isolate_huge_page()
to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.

Reported-by: Borislav Petkov <bp@alien8.de>
Tested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 mm/migrate.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/mm/migrate.c b/mm/migrate.c
index 4cd63c2..4a26042 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -1168,7 +1168,7 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
 			goto put_and_set;
 
 		if (PageHuge(page)) {
-			isolate_huge_page(page, &pagelist);
+			err = isolate_huge_page(page, &pagelist);
 			goto put_and_set;
 		}
 
-- 
1.8.3.1

Re: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()(Re: [PATCH 4/9] migrate: add hugepage migration code to move_pages())

[David Rientjes]

On Thu, 14 Nov 2013, Naoya Horiguchi wrote:

> Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> 
> Reported-by: Borislav Petkov <bp@alien8.de>
> Tested-by: Borislav Petkov <bp@alien8.de>
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> ---
>  mm/migrate.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/mm/migrate.c b/mm/migrate.c
> index 4cd63c2..4a26042 100644
> --- a/mm/migrate.c
> +++ b/mm/migrate.c
> @@ -1168,7 +1168,7 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
>  			goto put_and_set;
>  
>  		if (PageHuge(page)) {
> -			isolate_huge_page(page, &pagelist);
> +			err = isolate_huge_page(page, &pagelist);
>  			goto put_and_set;
>  		}
>  

I think it would be better to just fix hugetlb.h to do

	static inline bool isolate_huge_page(struct page *page, struct list_head *list)
	{
		return false;
	}

for the !CONFIG_HUGETLB_PAGE variant.

Re: [PATCH] mm/migrate.c: take returned value ofisolate_huge_page()(Re: [PATCH 4/9] migrate: add hugepage migration code tomove_pages())

[Naoya Horiguchi]

On Thu, Nov 14, 2013 at 03:11:21PM -0800, David Rientjes wrote:
> On Thu, 14 Nov 2013, Naoya Horiguchi wrote:
> 
> > Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> > to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> > 
> > Reported-by: Borislav Petkov <bp@alien8.de>
> > Tested-by: Borislav Petkov <bp@alien8.de>
> > Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> > ---
> >  mm/migrate.c | 2 +-
> >  1 file changed, 1 insertion(+), 1 deletion(-)
> > 
> > diff --git a/mm/migrate.c b/mm/migrate.c
> > index 4cd63c2..4a26042 100644
> > --- a/mm/migrate.c
> > +++ b/mm/migrate.c
> > @@ -1168,7 +1168,7 @@ static int do_move_page_to_node_array(struct mm_struct *mm,
> >  			goto put_and_set;
> >  
> >  		if (PageHuge(page)) {
> > -			isolate_huge_page(page, &pagelist);
> > +			err = isolate_huge_page(page, &pagelist);
> >  			goto put_and_set;
> >  		}
> >  
> 
> I think it would be better to just fix hugetlb.h to do
> 
> 	static inline bool isolate_huge_page(struct page *page, struct list_head *list)
> 	{
> 		return false;
> 	}
> 
> for the !CONFIG_HUGETLB_PAGE variant.

Right. I confirmed that it fixes the warning with Borislav's .config.
Here is new one. Could you add some credit tag?

Thanks,
Naoya
---
From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
Date: Fri, 15 Nov 2013 09:00:15 -0500
Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()

Introduces a cosmetic substitution of the returned value of isolate_huge_page()
to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.

Reported-by: Borislav Petkov <bp@alien8.de>
Tested-by: Borislav Petkov <bp@alien8.de>
Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
---
 include/linux/hugetlb.h | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/include/linux/hugetlb.h b/include/linux/hugetlb.h
index acd2010328f3..25cdb9b285a9 100644
--- a/include/linux/hugetlb.h
+++ b/include/linux/hugetlb.h
@@ -137,7 +137,10 @@ static inline int dequeue_hwpoisoned_huge_page(struct page *page)
 	return 0;
 }
 
-#define isolate_huge_page(p, l) false
+static inline bool isolate_huge_page(struct page *page, struct list_head *list)
+{
+	return false;
+}
 #define putback_active_hugepage(p)	do {} while (0)
 #define is_hugepage_active(x)	false
 static inline void copy_huge_page(struct page *dst, struct page *src)
-- 
1.8.3.1

Re: [PATCH] mm/migrate.c: take returned value ofisolate_huge_page()(Re: [PATCH 4/9] migrate: add hugepage migration code tomove_pages())

[David Rientjes]

On Fri, 15 Nov 2013, Naoya Horiguchi wrote:

> From: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>
> Date: Fri, 15 Nov 2013 09:00:15 -0500
> Subject: [PATCH] mm/migrate.c: take returned value of isolate_huge_page()
> 
> Introduces a cosmetic substitution of the returned value of isolate_huge_page()
> to suppress a build warning when !CONFIG_HUGETLBFS. No behavioral change.
> 
> Reported-by: Borislav Petkov <bp@alien8.de>
> Tested-by: Borislav Petkov <bp@alien8.de>
> Signed-off-by: Naoya Horiguchi <n-horiguchi@ah.jp.nec.com>

Acked-by: David Rientjes <rientjes@google.com>