[PATCH v4 0/7] Split a folio to any lower order folios

[PATCH v4 0/7] Split a folio to any lower order folios

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

Hi all,

File folio supports any order and multi-size THP is upstreamed[1], so both
file and anonymous folios can be >0 order. Currently, split_huge_page()
only splits a huge page to order-0 pages, but splitting to orders higher than
0 is going to better utilize large folios. In addition, Large Block
Sizes in XFS support would benefit from it[2]. This patchset adds support for
splitting a large folio to any lower order folios and uses it during file
folio truncate operations.

For Patch 6, Hugh did not like my approach to minimize the number of
folios for truncate[3]. I would like to get more feedback, especially
from FS people, on it to decide whether to keep it or not.

The patchset is on top of mm-everything-2024-02-13-01-26.

Changelog 
===

Since v3
---
1. Excluded shmem folios and pagecache folios without FS support from
splitting to any order (per Hugh Dickins).
2. Allowed splitting anonymous large folio to any lower order since
multi-size THP is upstreamed.
3. Adapted selftests code to new framework.

Since v2
---
1. Fixed an issue in __split_page_owner() introduced during my rebase

Since v1
---
1. Changed split_page_memcg() and split_page_owner() parameter to use order
2. Used folio_test_pmd_mappable() in place of the equivalent code

Details
===

* Patch 1 changes split_page_memcg() to use order instead of nr_pages
* Patch 2 changes split_page_owner() to use order instead of nr_pages
* Patch 3 and 4 add new_order parameter split_page_memcg() and
  split_page_owner() and prepare for upcoming changes.
* Patch 5 adds split_huge_page_to_list_to_order() to split a huge page
  to any lower order. The original split_huge_page_to_list() calls
  split_huge_page_to_list_to_order() with new_order = 0.
* Patch 6 uses split_huge_page_to_list_to_order() in large pagecache folio
  truncation instead of split the large folio all the way down to order-0.
* Patch 7 adds a test API to debugfs and test cases in
  split_huge_page_test selftests.

Comments and/or suggestions are welcome.

[1] https://lore.kernel.org/all/20231207161211.2374093-1-ryan.roberts@arm.com/
[2] https://lore.kernel.org/linux-mm/qzbcjn4gcyxla4gwuj6smlnwknz2wvo5wrjctin6eengjfqjei@lzkxv3iy6bol/
[3] https://lore.kernel.org/linux-mm/9dd96da-efa2-5123-20d4-4992136ef3ad@google.com/

Zi Yan (7):
  mm/memcg: use order instead of nr in split_page_memcg()
  mm/page_owner: use order instead of nr in split_page_owner()
  mm: memcg: make memcg huge page split support any order split.
  mm: page_owner: add support for splitting to any order in split
    page_owner.
  mm: thp: split huge page to any lower order pages (except order-1).
  mm: truncate: split huge page cache page to a non-zero order if
    possible.
  mm: huge_memory: enable debugfs to split huge pages to any order.

 include/linux/huge_mm.h                       |  21 +-
 include/linux/memcontrol.h                    |   4 +-
 include/linux/page_owner.h                    |  10 +-
 mm/huge_memory.c                              | 149 +++++++++---
 mm/memcontrol.c                               |  10 +-
 mm/page_alloc.c                               |   8 +-
 mm/page_owner.c                               |   8 +-
 mm/truncate.c                                 |  21 +-
 .../selftests/mm/split_huge_page_test.c       | 223 +++++++++++++++++-
 9 files changed, 382 insertions(+), 72 deletions(-)

-- 
2.43.0

[PATCH v4 1/7] mm/memcg: use order instead of nr in split_page_memcg()

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

We do not have non power of two pages, using nr is error prone if nr
is not power-of-two. Use page order instead.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 include/linux/memcontrol.h | 4 ++--
 mm/huge_memory.c           | 3 ++-
 mm/memcontrol.c            | 3 ++-
 mm/page_alloc.c            | 4 ++--
 4 files changed, 8 insertions(+), 6 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 4e4caeaea404..173bbb53c1ec 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1163,7 +1163,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm,
 	rcu_read_unlock();
 }
 
-void split_page_memcg(struct page *head, unsigned int nr);
+void split_page_memcg(struct page *head, int order);
 
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 						gfp_t gfp_mask,
@@ -1621,7 +1621,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx)
 {
 }
 
-static inline void split_page_memcg(struct page *head, unsigned int nr)
+static inline void split_page_memcg(struct page *head, int order)
 {
 }
 
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 016e20bd813e..0cd5fba0923c 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2877,9 +2877,10 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	unsigned long offset = 0;
 	unsigned int nr = thp_nr_pages(head);
 	int i, nr_dropped = 0;
+	int order = folio_order(folio);
 
 	/* complete memcg works before add pages to LRU */
-	split_page_memcg(head, nr);
+	split_page_memcg(head, order);
 
 	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
 		offset = swp_offset(folio->swap);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 93ad8640b741..404e529644c0 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -3608,11 +3608,12 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size)
 /*
  * Because page_memcg(head) is not set on tails, set it now.
  */
-void split_page_memcg(struct page *head, unsigned int nr)
+void split_page_memcg(struct page *head, int order)
 {
 	struct folio *folio = page_folio(head);
 	struct mem_cgroup *memcg = folio_memcg(folio);
 	int i;
+	unsigned int nr = 1 << order;
 
 	if (mem_cgroup_disabled() || !memcg)
 		return;
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7ae4b74c9e5c..7c927b84e16c 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2653,7 +2653,7 @@ void split_page(struct page *page, unsigned int order)
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
 	split_page_owner(page, 1 << order);
-	split_page_memcg(page, 1 << order);
+	split_page_memcg(page, order);
 }
 EXPORT_SYMBOL_GPL(split_page);
 
@@ -4838,7 +4838,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order,
 		struct page *last = page + nr;
 
 		split_page_owner(page, 1 << order);
-		split_page_memcg(page, 1 << order);
+		split_page_memcg(page, order);
 		while (page < --last)
 			set_page_refcounted(last);
 
-- 
2.43.0

[PATCH v4 2/7] mm/page_owner: use order instead of nr in split_page_owner()

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

We do not have non power of two pages, using nr is error prone if nr
is not power-of-two. Use page order instead.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 include/linux/page_owner.h | 8 ++++----
 mm/huge_memory.c           | 2 +-
 mm/page_alloc.c            | 4 ++--
 mm/page_owner.c            | 3 ++-
 4 files changed, 9 insertions(+), 8 deletions(-)

diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
index 119a0c9d2a8b..d7878523adfc 100644
--- a/include/linux/page_owner.h
+++ b/include/linux/page_owner.h
@@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops;
 extern void __reset_page_owner(struct page *page, unsigned short order);
 extern void __set_page_owner(struct page *page,
 			unsigned short order, gfp_t gfp_mask);
-extern void __split_page_owner(struct page *page, unsigned int nr);
+extern void __split_page_owner(struct page *page, int order);
 extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
 extern void __set_page_owner_migrate_reason(struct page *page, int reason);
 extern void __dump_page_owner(const struct page *page);
@@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page,
 		__set_page_owner(page, order, gfp_mask);
 }
 
-static inline void split_page_owner(struct page *page, unsigned int nr)
+static inline void split_page_owner(struct page *page, int order)
 {
 	if (static_branch_unlikely(&page_owner_inited))
-		__split_page_owner(page, nr);
+		__split_page_owner(page, order);
 }
 static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
 {
@@ -60,7 +60,7 @@ static inline void set_page_owner(struct page *page,
 {
 }
 static inline void split_page_owner(struct page *page,
-			unsigned short order)
+			int order)
 {
 }
 static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 0cd5fba0923c..f079b02f1f59 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2919,7 +2919,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	unlock_page_lruvec(lruvec);
 	/* Caller disabled irqs, so they are still disabled here */
 
-	split_page_owner(head, nr);
+	split_page_owner(head, order);
 
 	/* See comment in __split_huge_page_tail() */
 	if (PageAnon(head)) {
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 7c927b84e16c..b6e8fe6fed67 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2652,7 +2652,7 @@ void split_page(struct page *page, unsigned int order)
 
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
-	split_page_owner(page, 1 << order);
+	split_page_owner(page, order);
 	split_page_memcg(page, order);
 }
 EXPORT_SYMBOL_GPL(split_page);
@@ -4837,7 +4837,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order,
 		struct page *page = virt_to_page((void *)addr);
 		struct page *last = page + nr;
 
-		split_page_owner(page, 1 << order);
+		split_page_owner(page, order);
 		split_page_memcg(page, order);
 		while (page < --last)
 			set_page_refcounted(last);
diff --git a/mm/page_owner.c b/mm/page_owner.c
index c4f9e5506e93..1319e402c2cf 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -292,11 +292,12 @@ void __set_page_owner_migrate_reason(struct page *page, int reason)
 	page_ext_put(page_ext);
 }
 
-void __split_page_owner(struct page *page, unsigned int nr)
+void __split_page_owner(struct page *page, int order)
 {
 	int i;
 	struct page_ext *page_ext = page_ext_get(page);
 	struct page_owner *page_owner;
+	unsigned int nr = 1 << order;
 
 	if (unlikely(!page_ext))
 		return;
-- 
2.43.0

[PATCH v4 3/7] mm: memcg: make memcg huge page split support any order split.

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

It sets memcg information for the pages after the split. A new parameter
new_order is added to tell the order of subpages in the new page, always 0
for now. It prepares for upcoming changes to support split huge page to
any lower order.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 include/linux/memcontrol.h |  4 ++--
 mm/huge_memory.c           |  2 +-
 mm/memcontrol.c            | 11 ++++++-----
 mm/page_alloc.c            |  4 ++--
 4 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 173bbb53c1ec..9a2dea92be0e 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1163,7 +1163,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm,
 	rcu_read_unlock();
 }
 
-void split_page_memcg(struct page *head, int order);
+void split_page_memcg(struct page *head, int old_order, int new_order);
 
 unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 						gfp_t gfp_mask,
@@ -1621,7 +1621,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx)
 {
 }
 
-static inline void split_page_memcg(struct page *head, int order)
+static inline void split_page_memcg(struct page *head, int old_order, int new_order)
 {
 }
 
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index f079b02f1f59..3d30eccd3a7f 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2880,7 +2880,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	int order = folio_order(folio);
 
 	/* complete memcg works before add pages to LRU */
-	split_page_memcg(head, order);
+	split_page_memcg(head, order, 0);
 
 	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
 		offset = swp_offset(folio->swap);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 404e529644c0..27d53715d8dc 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -3608,23 +3608,24 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size)
 /*
  * Because page_memcg(head) is not set on tails, set it now.
  */
-void split_page_memcg(struct page *head, int order)
+void split_page_memcg(struct page *head, int old_order, int new_order)
 {
 	struct folio *folio = page_folio(head);
 	struct mem_cgroup *memcg = folio_memcg(folio);
 	int i;
-	unsigned int nr = 1 << order;
+	unsigned int old_nr = 1 << old_order;
+	unsigned int new_nr = 1 << new_order;
 
 	if (mem_cgroup_disabled() || !memcg)
 		return;
 
-	for (i = 1; i < nr; i++)
+	for (i = new_nr; i < old_nr; i += new_nr)
 		folio_page(folio, i)->memcg_data = folio->memcg_data;
 
 	if (folio_memcg_kmem(folio))
-		obj_cgroup_get_many(__folio_objcg(folio), nr - 1);
+		obj_cgroup_get_many(__folio_objcg(folio), old_nr / new_nr - 1);
 	else
-		css_get_many(&memcg->css, nr - 1);
+		css_get_many(&memcg->css, old_nr / new_nr - 1);
 }
 
 #ifdef CONFIG_SWAP
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index b6e8fe6fed67..9d4dd41d0647 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2653,7 +2653,7 @@ void split_page(struct page *page, unsigned int order)
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
 	split_page_owner(page, order);
-	split_page_memcg(page, order);
+	split_page_memcg(page, order, 0);
 }
 EXPORT_SYMBOL_GPL(split_page);
 
@@ -4838,7 +4838,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order,
 		struct page *last = page + nr;
 
 		split_page_owner(page, order);
-		split_page_memcg(page, order);
+		split_page_memcg(page, order, 0);
 		while (page < --last)
 			set_page_refcounted(last);
 
-- 
2.43.0

[PATCH v4 4/7] mm: page_owner: add support for splitting to any order in split page_owner.

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

It adds a new_order parameter to set new page order in page owner.
It prepares for upcoming changes to support split huge page to any
lower order.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 include/linux/page_owner.h | 10 +++++-----
 mm/huge_memory.c           |  2 +-
 mm/page_alloc.c            |  4 ++--
 mm/page_owner.c            |  9 +++++----
 4 files changed, 13 insertions(+), 12 deletions(-)

diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
index d7878523adfc..a784ba69f67f 100644
--- a/include/linux/page_owner.h
+++ b/include/linux/page_owner.h
@@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops;
 extern void __reset_page_owner(struct page *page, unsigned short order);
 extern void __set_page_owner(struct page *page,
 			unsigned short order, gfp_t gfp_mask);
-extern void __split_page_owner(struct page *page, int order);
+extern void __split_page_owner(struct page *page, int old_order, int new_order);
 extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
 extern void __set_page_owner_migrate_reason(struct page *page, int reason);
 extern void __dump_page_owner(const struct page *page);
@@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page,
 		__set_page_owner(page, order, gfp_mask);
 }
 
-static inline void split_page_owner(struct page *page, int order)
+static inline void split_page_owner(struct page *page, int old_order, int new_order)
 {
 	if (static_branch_unlikely(&page_owner_inited))
-		__split_page_owner(page, order);
+		__split_page_owner(page, old_order, new_order);
 }
 static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
 {
@@ -56,11 +56,11 @@ static inline void reset_page_owner(struct page *page, unsigned short order)
 {
 }
 static inline void set_page_owner(struct page *page,
-			unsigned int order, gfp_t gfp_mask)
+			unsigned short order, gfp_t gfp_mask)
 {
 }
 static inline void split_page_owner(struct page *page,
-			int order)
+			int old_order, int new_order)
 {
 }
 static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 3d30eccd3a7f..ad7133c97428 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2919,7 +2919,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	unlock_page_lruvec(lruvec);
 	/* Caller disabled irqs, so they are still disabled here */
 
-	split_page_owner(head, order);
+	split_page_owner(head, order, 0);
 
 	/* See comment in __split_huge_page_tail() */
 	if (PageAnon(head)) {
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 9d4dd41d0647..e0f107b21c98 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -2652,7 +2652,7 @@ void split_page(struct page *page, unsigned int order)
 
 	for (i = 1; i < (1 << order); i++)
 		set_page_refcounted(page + i);
-	split_page_owner(page, order);
+	split_page_owner(page, order, 0);
 	split_page_memcg(page, order, 0);
 }
 EXPORT_SYMBOL_GPL(split_page);
@@ -4837,7 +4837,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order,
 		struct page *page = virt_to_page((void *)addr);
 		struct page *last = page + nr;
 
-		split_page_owner(page, order);
+		split_page_owner(page, order, 0);
 		split_page_memcg(page, order, 0);
 		while (page < --last)
 			set_page_refcounted(last);
diff --git a/mm/page_owner.c b/mm/page_owner.c
index 1319e402c2cf..ebbffa0501db 100644
--- a/mm/page_owner.c
+++ b/mm/page_owner.c
@@ -292,19 +292,20 @@ void __set_page_owner_migrate_reason(struct page *page, int reason)
 	page_ext_put(page_ext);
 }
 
-void __split_page_owner(struct page *page, int order)
+void __split_page_owner(struct page *page, int old_order, int new_order)
 {
 	int i;
 	struct page_ext *page_ext = page_ext_get(page);
 	struct page_owner *page_owner;
-	unsigned int nr = 1 << order;
+	unsigned int old_nr = 1 << old_order;
+	unsigned int new_nr = 1 << new_order;
 
 	if (unlikely(!page_ext))
 		return;
 
-	for (i = 0; i < nr; i++) {
+	for (i = 0; i < old_nr; i += new_nr) {
 		page_owner = get_page_owner(page_ext);
-		page_owner->order = 0;
+		page_owner->order = new_order;
 		page_ext = page_ext_next(page_ext);
 	}
 	page_ext_put(page_ext);
-- 
2.43.0

[PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

To split a THP to any lower order (except order-1) pages, we need to
reform THPs on subpages at given order and add page refcount based on the
new page order. Also we need to reinitialize page_deferred_list after
removing the page from the split_queue, otherwise a subsequent split will
see list corruption when checking the page_deferred_list again.

It has many uses, like minimizing the number of pages after
truncating a huge pagecache page. For anonymous THPs, we can only split
them to order-0 like before until we add support for any size anonymous
THPs.

Order-1 folio is not supported because _deferred_list, which is used by
partially mapped folios, is stored in subpage 2 and an order-1 folio only
has subpage 0 and 1.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 include/linux/huge_mm.h |  21 +++++---
 mm/huge_memory.c        | 114 +++++++++++++++++++++++++++++++---------
 2 files changed, 101 insertions(+), 34 deletions(-)

diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
index 5adb86af35fc..de0c89105076 100644
--- a/include/linux/huge_mm.h
+++ b/include/linux/huge_mm.h
@@ -265,10 +265,11 @@ unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
 
 void folio_prep_large_rmappable(struct folio *folio);
 bool can_split_folio(struct folio *folio, int *pextra_pins);
-int split_huge_page_to_list(struct page *page, struct list_head *list);
+int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
+		unsigned int new_order);
 static inline int split_huge_page(struct page *page)
 {
-	return split_huge_page_to_list(page, NULL);
+	return split_huge_page_to_list_to_order(page, NULL, 0);
 }
 void deferred_split_folio(struct folio *folio);
 
@@ -422,7 +423,8 @@ can_split_folio(struct folio *folio, int *pextra_pins)
 	return false;
 }
 static inline int
-split_huge_page_to_list(struct page *page, struct list_head *list)
+split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
+		unsigned int new_order)
 {
 	return 0;
 }
@@ -519,17 +521,20 @@ static inline bool thp_migration_supported(void)
 }
 #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
 
-static inline int split_folio_to_list(struct folio *folio,
-		struct list_head *list)
+static inline int split_folio_to_list_to_order(struct folio *folio,
+		struct list_head *list, int new_order)
 {
-	return split_huge_page_to_list(&folio->page, list);
+	return split_huge_page_to_list_to_order(&folio->page, list, new_order);
 }
 
-static inline int split_folio(struct folio *folio)
+static inline int split_folio_to_order(struct folio *folio, int new_order)
 {
-	return split_folio_to_list(folio, NULL);
+	return split_folio_to_list_to_order(folio, NULL, new_order);
 }
 
+#define split_folio_to_list(f, l) split_folio_to_list_to_order(f, l, 0)
+#define split_folio(f) split_folio_to_order(f, 0)
+
 /*
  * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
  * limitations in the implementation like arm64 MTE can override this to
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index ad7133c97428..d0e555a8ea98 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2718,11 +2718,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
 
 static void unmap_folio(struct folio *folio)
 {
-	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD |
-		TTU_SYNC | TTU_BATCH_FLUSH;
+	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC |
+		TTU_BATCH_FLUSH;
 
 	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
 
+	if (folio_test_pmd_mappable(folio))
+		ttu_flags |= TTU_SPLIT_HUGE_PMD;
+
 	/*
 	 * Anon pages need migration entries to preserve them, but file
 	 * pages can simply be left unmapped, then faulted back on demand.
@@ -2756,7 +2759,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
 		struct lruvec *lruvec, struct list_head *list)
 {
 	VM_BUG_ON_PAGE(!PageHead(head), head);
-	VM_BUG_ON_PAGE(PageCompound(tail), head);
 	VM_BUG_ON_PAGE(PageLRU(tail), head);
 	lockdep_assert_held(&lruvec->lru_lock);
 
@@ -2777,7 +2779,8 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
 }
 
 static void __split_huge_page_tail(struct folio *folio, int tail,
-		struct lruvec *lruvec, struct list_head *list)
+		struct lruvec *lruvec, struct list_head *list,
+		unsigned int new_order)
 {
 	struct page *head = &folio->page;
 	struct page *page_tail = head + tail;
@@ -2847,10 +2850,15 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
 	 * which needs correct compound_head().
 	 */
 	clear_compound_head(page_tail);
+	if (new_order) {
+		prep_compound_page(page_tail, new_order);
+		folio_prep_large_rmappable(page_folio(page_tail));
+	}
 
 	/* Finally unfreeze refcount. Additional reference from page cache. */
-	page_ref_unfreeze(page_tail, 1 + (!folio_test_anon(folio) ||
-					  folio_test_swapcache(folio)));
+	page_ref_unfreeze(page_tail,
+		1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ?
+			     folio_nr_pages(page_folio(page_tail)) : 0));
 
 	if (folio_test_young(folio))
 		folio_set_young(new_folio);
@@ -2868,7 +2876,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
 }
 
 static void __split_huge_page(struct page *page, struct list_head *list,
-		pgoff_t end)
+		pgoff_t end, unsigned int new_order)
 {
 	struct folio *folio = page_folio(page);
 	struct page *head = &folio->page;
@@ -2877,10 +2885,11 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	unsigned long offset = 0;
 	unsigned int nr = thp_nr_pages(head);
 	int i, nr_dropped = 0;
+	unsigned int new_nr = 1 << new_order;
 	int order = folio_order(folio);
 
 	/* complete memcg works before add pages to LRU */
-	split_page_memcg(head, order, 0);
+	split_page_memcg(head, order, new_order);
 
 	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
 		offset = swp_offset(folio->swap);
@@ -2893,8 +2902,8 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 
 	ClearPageHasHWPoisoned(head);
 
-	for (i = nr - 1; i >= 1; i--) {
-		__split_huge_page_tail(folio, i, lruvec, list);
+	for (i = nr - new_nr; i >= new_nr; i -= new_nr) {
+		__split_huge_page_tail(folio, i, lruvec, list, new_order);
 		/* Some pages can be beyond EOF: drop them from page cache */
 		if (head[i].index >= end) {
 			struct folio *tail = page_folio(head + i);
@@ -2910,29 +2919,41 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 			__xa_store(&head->mapping->i_pages, head[i].index,
 					head + i, 0);
 		} else if (swap_cache) {
+			/*
+			 * split anonymous THPs (including swapped out ones) to
+			 * non-zero order not supported
+			 */
+			VM_WARN_ONCE(new_order,
+				"Split swap-cached anon folio to non-0 order not supported");
 			__xa_store(&swap_cache->i_pages, offset + i,
 					head + i, 0);
 		}
 	}
 
-	ClearPageCompound(head);
+	if (!new_order)
+		ClearPageCompound(head);
+	else {
+		struct folio *new_folio = (struct folio *)head;
+
+		folio_set_order(new_folio, new_order);
+	}
 	unlock_page_lruvec(lruvec);
 	/* Caller disabled irqs, so they are still disabled here */
 
-	split_page_owner(head, order, 0);
+	split_page_owner(head, order, new_order);
 
 	/* See comment in __split_huge_page_tail() */
 	if (PageAnon(head)) {
 		/* Additional pin to swap cache */
 		if (PageSwapCache(head)) {
-			page_ref_add(head, 2);
+			page_ref_add(head, 1 + new_nr);
 			xa_unlock(&swap_cache->i_pages);
 		} else {
 			page_ref_inc(head);
 		}
 	} else {
 		/* Additional pin to page cache */
-		page_ref_add(head, 2);
+		page_ref_add(head, 1 + new_nr);
 		xa_unlock(&head->mapping->i_pages);
 	}
 	local_irq_enable();
@@ -2944,7 +2965,15 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	if (folio_test_swapcache(folio))
 		split_swap_cluster(folio->swap);
 
-	for (i = 0; i < nr; i++) {
+	/*
+	 * set page to its compound_head when split to non order-0 pages, so
+	 * we can skip unlocking it below, since PG_locked is transferred to
+	 * the compound_head of the page and the caller will unlock it.
+	 */
+	if (new_order)
+		page = compound_head(page);
+
+	for (i = 0; i < nr; i += new_nr) {
 		struct page *subpage = head + i;
 		if (subpage == page)
 			continue;
@@ -2978,29 +3007,35 @@ bool can_split_folio(struct folio *folio, int *pextra_pins)
 }
 
 /*
- * This function splits huge page into normal pages. @page can point to any
- * subpage of huge page to split. Split doesn't change the position of @page.
+ * This function splits huge page into pages in @new_order. @page can point to
+ * any subpage of huge page to split. Split doesn't change the position of
+ * @page.
+ *
+ * NOTE: order-1 folio is not supported because _deferred_list, which is used
+ * by partially mapped folios, is stored in subpage 2 and an order-1 folio
+ * only has subpage 0 and 1.
  *
  * Only caller must hold pin on the @page, otherwise split fails with -EBUSY.
  * The huge page must be locked.
  *
  * If @list is null, tail pages will be added to LRU list, otherwise, to @list.
  *
- * Both head page and tail pages will inherit mapping, flags, and so on from
- * the hugepage.
+ * Pages in new_order will inherit mapping, flags, and so on from the hugepage.
  *
- * GUP pin and PG_locked transferred to @page. Rest subpages can be freed if
- * they are not mapped.
+ * GUP pin and PG_locked transferred to @page or the compound page @page belongs
+ * to. Rest subpages can be freed if they are not mapped.
  *
  * Returns 0 if the hugepage is split successfully.
  * Returns -EBUSY if the page is pinned or if anon_vma disappeared from under
  * us.
  */
-int split_huge_page_to_list(struct page *page, struct list_head *list)
+int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
+				     unsigned int new_order)
 {
 	struct folio *folio = page_folio(page);
 	struct deferred_split *ds_queue = get_deferred_split_queue(folio);
-	XA_STATE(xas, &folio->mapping->i_pages, folio->index);
+	/* reset xarray order to new order after split */
+	XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order);
 	struct anon_vma *anon_vma = NULL;
 	struct address_space *mapping = NULL;
 	int extra_pins, ret;
@@ -3010,6 +3045,26 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
 	VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
 	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
 
+	/* Cannot split THP to order-1 (no order-1 THPs) */
+	if (new_order == 1) {
+		VM_WARN_ONCE(1, "Cannot split to order-1 folio");
+		return -EINVAL;
+	}
+
+	if (new_order) {
+		/* Split shmem folio to non-zero order not supported */
+		if (shmem_mapping(folio->mapping)) {
+			VM_WARN_ONCE(1, "Split shmem folio to non-0 order not support");
+			return -EINVAL;
+		}
+		/* No split if the file system does not support large folio */
+		if (!mapping_large_folio_support(folio->mapping)) {
+			VM_WARN_ONCE(1, "Split file folio to non-0 order not support");
+			return -EINVAL;
+		}
+	}
+
+
 	is_hzp = is_huge_zero_page(&folio->page);
 	if (is_hzp) {
 		pr_warn_ratelimited("Called split_huge_page for huge zero page\n");
@@ -3105,14 +3160,21 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
 	if (folio_ref_freeze(folio, 1 + extra_pins)) {
 		if (!list_empty(&folio->_deferred_list)) {
 			ds_queue->split_queue_len--;
-			list_del(&folio->_deferred_list);
+			/*
+			 * Reinitialize page_deferred_list after removing the
+			 * page from the split_queue, otherwise a subsequent
+			 * split will see list corruption when checking the
+			 * page_deferred_list.
+			 */
+			list_del_init(&folio->_deferred_list);
 		}
 		spin_unlock(&ds_queue->split_queue_lock);
 		if (mapping) {
 			int nr = folio_nr_pages(folio);
 
 			xas_split(&xas, folio, folio_order(folio));
-			if (folio_test_pmd_mappable(folio)) {
+			if (folio_test_pmd_mappable(folio) &&
+			    new_order < HPAGE_PMD_ORDER) {
 				if (folio_test_swapbacked(folio)) {
 					__lruvec_stat_mod_folio(folio,
 							NR_SHMEM_THPS, -nr);
@@ -3124,7 +3186,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
 			}
 		}
 
-		__split_huge_page(page, list, end);
+		__split_huge_page(page, list, end, new_order);
 		ret = 0;
 	} else {
 		spin_unlock(&ds_queue->split_queue_lock);
-- 
2.43.0

[PATCH v4 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

To minimize the number of pages after a huge page truncation, we do not
need to split it all the way down to order-0. The huge page has at most
three parts, the part before offset, the part to be truncated, the part
remaining at the end. Find the greatest common divisor of them to
calculate the new page order from it, so we can split the huge
page to this order and keep the remaining pages as large and as few as
possible.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/truncate.c | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)

diff --git a/mm/truncate.c b/mm/truncate.c
index 725b150e47ac..49ddbbf7a617 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -21,6 +21,7 @@
 #include <linux/task_io_accounting_ops.h>
 #include <linux/shmem_fs.h>
 #include <linux/rmap.h>
+#include <linux/gcd.h>
 #include "internal.h"
 
 /*
@@ -210,7 +211,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio)
 bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
 {
 	loff_t pos = folio_pos(folio);
-	unsigned int offset, length;
+	unsigned int offset, length, remaining;
+	unsigned int new_order = folio_order(folio);
 
 	if (pos < start)
 		offset = start - pos;
@@ -221,6 +223,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
 		length = length - offset;
 	else
 		length = end + 1 - pos - offset;
+	remaining = folio_size(folio) - offset - length;
 
 	folio_wait_writeback(folio);
 	if (length == folio_size(folio)) {
@@ -235,11 +238,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
 	 */
 	folio_zero_range(folio, offset, length);
 
+	/*
+	 * Use the greatest common divisor of offset, length, and remaining
+	 * as the smallest page size and compute the new order from it. So we
+	 * can truncate a subpage as large as possible. Round up gcd to
+	 * PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0.
+	 */
+	new_order = ilog2(round_up(gcd(gcd(offset, length), remaining),
+				   PAGE_SIZE) / PAGE_SIZE);
+
+	/* order-1 THP not supported, downgrade to order-0 */
+	if (new_order == 1)
+		new_order = 0;
+
+
 	if (folio_has_private(folio))
 		folio_invalidate(folio, offset, length);
 	if (!folio_test_large(folio))
 		return true;
-	if (split_folio(folio) == 0)
+	if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0)
 		return true;
 	if (folio_test_dirty(folio))
 		return false;
-- 
2.43.0

[PATCH v4 7/7] mm: huge_memory: enable debugfs to split huge pages to any order.

[Zi Yan]

From: Zi Yan <ziy@nvidia.com>

It is used to test split_huge_page_to_list_to_order for pagecache THPs.
Also add test cases for split_huge_page_to_list_to_order via both
debugfs, truncating a file, and punching holes in a file.

Signed-off-by: Zi Yan <ziy@nvidia.com>
---
 mm/huge_memory.c                              |  34 ++-
 .../selftests/mm/split_huge_page_test.c       | 223 +++++++++++++++++-
 2 files changed, 239 insertions(+), 18 deletions(-)

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index d0e555a8ea98..0564b007cbd1 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3399,7 +3399,7 @@ static inline bool vma_not_suitable_for_thp_split(struct vm_area_struct *vma)
 }
 
 static int split_huge_pages_pid(int pid, unsigned long vaddr_start,
-				unsigned long vaddr_end)
+				unsigned long vaddr_end, unsigned int new_order)
 {
 	int ret = 0;
 	struct task_struct *task;
@@ -3463,13 +3463,19 @@ static int split_huge_pages_pid(int pid, unsigned long vaddr_start,
 			goto next;
 
 		total++;
-		if (!can_split_folio(folio, NULL))
+		/*
+		 * For folios with private, split_huge_page_to_list_to_order()
+		 * will try to drop it before split and then check if the folio
+		 * can be split or not. So skip the check here.
+		 */
+		if (!folio_test_private(folio) &&
+		    !can_split_folio(folio, NULL))
 			goto next;
 
 		if (!folio_trylock(folio))
 			goto next;
 
-		if (!split_folio(folio))
+		if (!split_folio_to_order(folio, new_order))
 			split++;
 
 		folio_unlock(folio);
@@ -3487,7 +3493,7 @@ static int split_huge_pages_pid(int pid, unsigned long vaddr_start,
 }
 
 static int split_huge_pages_in_file(const char *file_path, pgoff_t off_start,
-				pgoff_t off_end)
+				pgoff_t off_end, unsigned int new_order)
 {
 	struct filename *file;
 	struct file *candidate;
@@ -3526,7 +3532,7 @@ static int split_huge_pages_in_file(const char *file_path, pgoff_t off_start,
 		if (!folio_trylock(folio))
 			goto next;
 
-		if (!split_folio(folio))
+		if (!split_folio_to_order(folio, new_order))
 			split++;
 
 		folio_unlock(folio);
@@ -3551,10 +3557,14 @@ static ssize_t split_huge_pages_write(struct file *file, const char __user *buf,
 {
 	static DEFINE_MUTEX(split_debug_mutex);
 	ssize_t ret;
-	/* hold pid, start_vaddr, end_vaddr or file_path, off_start, off_end */
+	/*
+	 * hold pid, start_vaddr, end_vaddr, new_order or
+	 * file_path, off_start, off_end, new_order
+	 */
 	char input_buf[MAX_INPUT_BUF_SZ];
 	int pid;
 	unsigned long vaddr_start, vaddr_end;
+	unsigned int new_order = 0;
 
 	ret = mutex_lock_interruptible(&split_debug_mutex);
 	if (ret)
@@ -3583,29 +3593,29 @@ static ssize_t split_huge_pages_write(struct file *file, const char __user *buf,
 			goto out;
 		}
 
-		ret = sscanf(buf, "0x%lx,0x%lx", &off_start, &off_end);
-		if (ret != 2) {
+		ret = sscanf(buf, "0x%lx,0x%lx,%d", &off_start, &off_end, &new_order);
+		if (ret != 2 && ret != 3) {
 			ret = -EINVAL;
 			goto out;
 		}
-		ret = split_huge_pages_in_file(file_path, off_start, off_end);
+		ret = split_huge_pages_in_file(file_path, off_start, off_end, new_order);
 		if (!ret)
 			ret = input_len;
 
 		goto out;
 	}
 
-	ret = sscanf(input_buf, "%d,0x%lx,0x%lx", &pid, &vaddr_start, &vaddr_end);
+	ret = sscanf(input_buf, "%d,0x%lx,0x%lx,%d", &pid, &vaddr_start, &vaddr_end, &new_order);
 	if (ret == 1 && pid == 1) {
 		split_huge_pages_all();
 		ret = strlen(input_buf);
 		goto out;
-	} else if (ret != 3) {
+	} else if (ret != 3 && ret != 4) {
 		ret = -EINVAL;
 		goto out;
 	}
 
-	ret = split_huge_pages_pid(pid, vaddr_start, vaddr_end);
+	ret = split_huge_pages_pid(pid, vaddr_start, vaddr_end, new_order);
 	if (!ret)
 		ret = strlen(input_buf);
 out:
diff --git a/tools/testing/selftests/mm/split_huge_page_test.c b/tools/testing/selftests/mm/split_huge_page_test.c
index 7b698a848bab..ffed5ae24566 100644
--- a/tools/testing/selftests/mm/split_huge_page_test.c
+++ b/tools/testing/selftests/mm/split_huge_page_test.c
@@ -16,6 +16,7 @@
 #include <sys/mount.h>
 #include <malloc.h>
 #include <stdbool.h>
+#include <time.h>
 #include "vm_util.h"
 #include "../kselftest.h"
 
@@ -24,10 +25,12 @@ unsigned int pageshift;
 uint64_t pmd_pagesize;
 
 #define SPLIT_DEBUGFS "/sys/kernel/debug/split_huge_pages"
+#define SMAP_PATH "/proc/self/smaps"
+#define THP_FS_PATH "/mnt/thp_fs"
 #define INPUT_MAX 80
 
-#define PID_FMT "%d,0x%lx,0x%lx"
-#define PATH_FMT "%s,0x%lx,0x%lx"
+#define PID_FMT "%d,0x%lx,0x%lx,%d"
+#define PATH_FMT "%s,0x%lx,0x%lx,%d"
 
 #define PFN_MASK     ((1UL<<55)-1)
 #define KPF_THP      (1UL<<22)
@@ -102,7 +105,7 @@ void split_pmd_thp(void)
 
 	/* split all THPs */
 	write_debugfs(PID_FMT, getpid(), (uint64_t)one_page,
-		(uint64_t)one_page + len);
+		(uint64_t)one_page + len, 0);
 
 	for (i = 0; i < len; i++)
 		if (one_page[i] != (char)i)
@@ -177,7 +180,7 @@ void split_pte_mapped_thp(void)
 
 	/* split all remapped THPs */
 	write_debugfs(PID_FMT, getpid(), (uint64_t)pte_mapped,
-		      (uint64_t)pte_mapped + pagesize * 4);
+		      (uint64_t)pte_mapped + pagesize * 4, 0);
 
 	/* smap does not show THPs after mremap, use kpageflags instead */
 	thp_size = 0;
@@ -237,7 +240,7 @@ void split_file_backed_thp(void)
 	}
 
 	/* split the file-backed THP */
-	write_debugfs(PATH_FMT, testfile, pgoff_start, pgoff_end);
+	write_debugfs(PATH_FMT, testfile, pgoff_start, pgoff_end, 0);
 
 	status = unlink(testfile);
 	if (status) {
@@ -265,8 +268,188 @@ void split_file_backed_thp(void)
 	ksft_exit_fail_msg("Error occurred\n");
 }
 
+void create_pagecache_thp_and_fd(const char *testfile, size_t fd_size, int *fd, char **addr)
+{
+	size_t i;
+	int dummy;
+
+	srand(time(NULL));
+
+	*fd = open(testfile, O_CREAT | O_RDWR, 0664);
+	if (*fd == -1)
+		ksft_exit_fail_msg("Failed to create a file at "THP_FS_PATH);
+
+	for (i = 0; i < fd_size; i++) {
+		unsigned char byte = (unsigned char)i;
+
+		write(*fd, &byte, sizeof(byte));
+	}
+	close(*fd);
+	sync();
+	*fd = open("/proc/sys/vm/drop_caches", O_WRONLY);
+	if (*fd == -1) {
+		ksft_perror("open drop_caches");
+		goto err_out_unlink;
+	}
+	if (write(*fd, "3", 1) != 1) {
+		ksft_perror("write to drop_caches");
+		goto err_out_unlink;
+	}
+	close(*fd);
+
+	*fd = open(testfile, O_RDWR);
+	if (*fd == -1) {
+		ksft_perror("Failed to open a file at "THP_FS_PATH);
+		goto err_out_unlink;
+	}
+
+	*addr = mmap(NULL, fd_size, PROT_READ|PROT_WRITE, MAP_SHARED, *fd, 0);
+	if (*addr == (char *)-1) {
+		ksft_perror("cannot mmap");
+		goto err_out_close;
+	}
+	madvise(*addr, fd_size, MADV_HUGEPAGE);
+
+	for (size_t i = 0; i < fd_size; i++)
+		dummy += *(*addr + i);
+
+	if (!check_huge_file(*addr, fd_size / pmd_pagesize, pmd_pagesize)) {
+		ksft_print_msg("No large pagecache folio generated, please mount a filesystem supporting large folio at "THP_FS_PATH"\n");
+		goto err_out_close;
+	}
+	return;
+err_out_close:
+	close(*fd);
+err_out_unlink:
+	unlink(testfile);
+	ksft_exit_fail_msg("Failed to create large pagecache folios\n");
+}
+
+void split_thp_in_pagecache_to_order(size_t fd_size, int order)
+{
+	int fd;
+	char *addr;
+	size_t i;
+	const char testfile[] = THP_FS_PATH "/test";
+	int err = 0;
+
+	create_pagecache_thp_and_fd(testfile, fd_size, &fd, &addr);
+
+	write_debugfs(PID_FMT, getpid(), (uint64_t)addr, (uint64_t)addr + fd_size, order);
+
+	for (i = 0; i < fd_size; i++)
+		if (*(addr + i) != (char)i) {
+			ksft_print_msg("%lu byte corrupted in the file\n", i);
+			err = EXIT_FAILURE;
+			goto out;
+		}
+
+	if (!check_huge_file(addr, 0, pmd_pagesize)) {
+		ksft_print_msg("Still FilePmdMapped not split\n");
+		err = EXIT_FAILURE;
+		goto out;
+	}
+
+out:
+	close(fd);
+	unlink(testfile);
+	if (err)
+		ksft_exit_fail_msg("Split PMD-mapped pagecache folio to order %d failed\n", order);
+	ksft_test_result_pass("Split PMD-mapped pagecache folio to order %d passed\n", order);
+}
+
+void truncate_thp_in_pagecache_to_order(size_t fd_size, int order)
+{
+	int fd;
+	char *addr;
+	size_t i;
+	const char testfile[] = THP_FS_PATH "/test";
+	int err = 0;
+
+	create_pagecache_thp_and_fd(testfile, fd_size, &fd, &addr);
+
+	ftruncate(fd, pagesize << order);
+
+	for (i = 0; i < (pagesize << order); i++)
+		if (*(addr + i) != (char)i) {
+			ksft_print_msg("%lu byte corrupted in the file\n", i);
+			err = EXIT_FAILURE;
+			goto out;
+		}
+
+	if (!check_huge_file(addr, 0, pmd_pagesize)) {
+		ksft_print_msg("Still FilePmdMapped not split after truncate\n");
+		err = EXIT_FAILURE;
+		goto out;
+	}
+
+out:
+	close(fd);
+	unlink(testfile);
+	if (err)
+		ksft_exit_fail_msg("Truncate PMD-mapped pagecache folio to order %d failed\n", order);
+	ksft_test_result_pass("Truncate PMD-mapped pagecache folio to order %d passed\n", order);
+}
+
+void punch_hole_in_pagecache_thp(size_t fd_size, off_t offset[], off_t len[],
+			int n, int num_left_thps)
+{
+	int fd, j;
+	char *addr;
+	size_t i;
+	const char testfile[] = THP_FS_PATH "/test";
+	int err = 0;
+
+	create_pagecache_thp_and_fd(testfile, fd_size, &fd, &addr);
+
+	for (j = 0; j < n; j++) {
+		ksft_print_msg("punch a hole to %ld kB PMD-mapped pagecache page at addr: %lx, offset %ld, and len %ld ...\n",
+			fd_size >> 10, (unsigned long)addr, offset[j], len[j]);
+		fallocate(fd, FALLOC_FL_PUNCH_HOLE|FALLOC_FL_KEEP_SIZE, offset[j], len[j]);
+	}
+
+	for (i = 0; i < fd_size; i++) {
+		int in_hole = 0;
+
+		for (j = 0; j < n; j++)
+			if (i >= offset[j] && i < (offset[j] + len[j])) {
+				in_hole = 1;
+				break;
+			}
+
+		if (in_hole) {
+			if (*(addr + i)) {
+				ksft_print_msg("%lu byte non-zero after punch\n", i);
+				err = EXIT_FAILURE;
+				goto out;
+			}
+			continue;
+		}
+		if (*(addr + i) != (char)i) {
+			ksft_print_msg("%lu byte corrupted in the file\n", i);
+			err = EXIT_FAILURE;
+			goto out;
+		}
+	}
+
+	if (!check_huge_file(addr, num_left_thps, pmd_pagesize)) {
+		ksft_print_msg("Still FilePmdMapped not split after punch\n");
+		goto out;
+	}
+out:
+	close(fd);
+	unlink(testfile);
+	if (err)
+		ksft_exit_fail_msg("Punch holes in PMD-mapped pagecache folio failed\n");
+	ksft_test_result_pass("Punch holes PMD-mapped pagecache folio passed\n");
+}
+
 int main(int argc, char **argv)
 {
+	int i;
+	size_t fd_size;
+	off_t offset[2], len[2];
+
 	ksft_print_header();
 
 	if (geteuid() != 0) {
@@ -274,7 +457,7 @@ int main(int argc, char **argv)
 		ksft_finished();
 	}
 
-	ksft_set_plan(3);
+	ksft_set_plan(3+8+9+2);
 
 	pagesize = getpagesize();
 	pageshift = ffs(pagesize) - 1;
@@ -282,9 +465,37 @@ int main(int argc, char **argv)
 	if (!pmd_pagesize)
 		ksft_exit_fail_msg("Reading PMD pagesize failed\n");
 
+	fd_size = 2 * pmd_pagesize;
+
 	split_pmd_thp();
 	split_pte_mapped_thp();
 	split_file_backed_thp();
 
+	for (i = 8; i >= 0; i--)
+		if (i != 1)
+			split_thp_in_pagecache_to_order(fd_size, i);
+
+	/*
+	 * for i is 1, truncate code in the kernel should create order-0 pages
+	 * instead of order-1 THPs, since order-1 THP is not supported. No error
+	 * is expected.
+	 */
+	for (i = 8; i >= 0; i--)
+		truncate_thp_in_pagecache_to_order(fd_size, i);
+
+	offset[0] = 123;
+	offset[1] = 4 * pagesize;
+	len[0] = 200 * pagesize;
+	len[1] = 16 * pagesize;
+	punch_hole_in_pagecache_thp(fd_size, offset, len, 2, 1);
+
+	offset[0] = 259 * pagesize + pagesize / 2;
+	offset[1] = 33 * pagesize;
+	len[0] = 129 * pagesize;
+	len[1] = 16 * pagesize;
+	punch_hole_in_pagecache_thp(fd_size, offset, len, 2, 1);
+
 	ksft_finished();
+
+	return 0;
 }
-- 
2.43.0

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Luis Chamberlain]

On Tue, Feb 13, 2024 at 04:55:18PM -0500, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> Order-1 folio is not supported because _deferred_list, which is used by
> partially mapped folios, is stored in subpage 2 and an order-1 folio only
> has subpage 0 and 1.

The LBS patches has the patch from Matthew which enables and allowed us
to successfully test order 1. So this restriction could be dropped if
that gets merged.

  Luis

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[David Hildenbrand]

On 13.02.24 23:05, Luis Chamberlain wrote:
> On Tue, Feb 13, 2024 at 04:55:18PM -0500, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>> Order-1 folio is not supported because _deferred_list, which is used by
>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>> has subpage 0 and 1.
> 
> The LBS patches has the patch from Matthew which enables and allowed us
> to successfully test order 1. So this restriction could be dropped if
> that gets merged.

For anon folios it will still be in place, so the restriction will only 
be dropped for !anon.

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 616 bytes --]

On 13 Feb 2024, at 17:05, Luis Chamberlain wrote:

> On Tue, Feb 13, 2024 at 04:55:18PM -0500, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>> Order-1 folio is not supported because _deferred_list, which is used by
>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>> has subpage 0 and 1.
>
> The LBS patches has the patch from Matthew which enables and allowed us
> to successfully test order 1. So this restriction could be dropped if
> that gets merged.

OK. But it only applies to file-backed folios IIUC. Anonymous folios still
cannot be split to order-1.

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 1528 bytes --]

On 13 Feb 2024, at 17:15, Zi Yan wrote:

> On 13 Feb 2024, at 17:05, Luis Chamberlain wrote:
>
>> On Tue, Feb 13, 2024 at 04:55:18PM -0500, Zi Yan wrote:
>>> From: Zi Yan <ziy@nvidia.com>
>>> Order-1 folio is not supported because _deferred_list, which is used by
>>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>>> has subpage 0 and 1.
>>
>> The LBS patches has the patch from Matthew which enables and allowed us
>> to successfully test order 1. So this restriction could be dropped if
>> that gets merged.
>
> OK. But it only applies to file-backed folios IIUC. Anonymous folios still
> cannot be split to order-1.

Something like this would lift the restriction:

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 0564b007cbd1..7ee7f1906617 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -3045,9 +3045,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
        VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
        VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);

-       /* Cannot split THP to order-1 (no order-1 THPs) */
-       if (new_order == 1) {
-               VM_WARN_ONCE(1, "Cannot split to order-1 folio");
+       /* Cannot split anonymous folios to order-1 (no order-1 anon folios) */
+       if (new_order == 1 && folio_test_anon(folio)) {
+               VM_WARN_ONCE(1, "Cannot split to order-1 anonymous folio");
                return -EINVAL;
        }

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[David Hildenbrand]

On 13.02.24 22:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> Hi all,
> 
> File folio supports any order and multi-size THP is upstreamed[1], so both
> file and anonymous folios can be >0 order. Currently, split_huge_page()
> only splits a huge page to order-0 pages, but splitting to orders higher than
> 0 is going to better utilize large folios. In addition, Large Block
> Sizes in XFS support would benefit from it[2]. This patchset adds support for
> splitting a large folio to any lower order folios and uses it during file
> folio truncate operations.
> 
> For Patch 6, Hugh did not like my approach to minimize the number of
> folios for truncate[3]. I would like to get more feedback, especially
> from FS people, on it to decide whether to keep it or not.

I'm curious, would it make sense to exclude the "more" controversial 
parts (i.e., patch #6) for now, and focus on the XFS use case only?

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 1175 bytes --]

On 13 Feb 2024, at 17:21, David Hildenbrand wrote:

> On 13.02.24 22:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> Hi all,
>>
>> File folio supports any order and multi-size THP is upstreamed[1], so both
>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>> only splits a huge page to order-0 pages, but splitting to orders higher than
>> 0 is going to better utilize large folios. In addition, Large Block
>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>> splitting a large folio to any lower order folios and uses it during file
>> folio truncate operations.
>>
>> For Patch 6, Hugh did not like my approach to minimize the number of
>> folios for truncate[3]. I would like to get more feedback, especially
>> from FS people, on it to decide whether to keep it or not.
>
> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?

Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
Now we have multi-size THP and XFS use cases, it can be dropped.

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 2114 bytes --]

On 13 Feb 2024, at 17:19, Zi Yan wrote:

> On 13 Feb 2024, at 17:15, Zi Yan wrote:
>
>> On 13 Feb 2024, at 17:05, Luis Chamberlain wrote:
>>
>>> On Tue, Feb 13, 2024 at 04:55:18PM -0500, Zi Yan wrote:
>>>> From: Zi Yan <ziy@nvidia.com>
>>>> Order-1 folio is not supported because _deferred_list, which is used by
>>>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>>>> has subpage 0 and 1.
>>>
>>> The LBS patches has the patch from Matthew which enables and allowed us
>>> to successfully test order 1. So this restriction could be dropped if
>>> that gets merged.
>>
>> OK. But it only applies to file-backed folios IIUC. Anonymous folios still
>> cannot be split to order-1.
>
> Something like this would lift the restriction:

Actually this, since folio_prep_large_rmappable() is changed by that patch:

diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 0564b007cbd1..05eeeafaa9dc 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2852,7 +2852,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
        clear_compound_head(page_tail);
        if (new_order) {
                prep_compound_page(page_tail, new_order);
-               folio_prep_large_rmappable(page_folio(page_tail));
+               new_folio = folio_prep_large_rmappable(new_folio);
        }

        /* Finally unfreeze refcount. Additional reference from page cache. */
@@ -3045,9 +3045,9 @@ int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
        VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
        VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);

-       /* Cannot split THP to order-1 (no order-1 THPs) */
-       if (new_order == 1) {
-               VM_WARN_ONCE(1, "Cannot split to order-1 folio");
+       /* Cannot split anonymous folios to order-1 (no order-1 anon folios) */
+       if (new_order == 1 && folio_test_anon(folio)) {
+               VM_WARN_ONCE(1, "Cannot split to order-1 anonymous folio");
                return -EINVAL;
        }

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 1/7] mm/memcg: use order instead of nr in split_page_memcg()

[David Hildenbrand]

On 13.02.24 22:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> We do not have non power of two pages, using nr is error prone if nr
> is not power-of-two. Use page order instead.
> 
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---
>   include/linux/memcontrol.h | 4 ++--
>   mm/huge_memory.c           | 3 ++-
>   mm/memcontrol.c            | 3 ++-
>   mm/page_alloc.c            | 4 ++--
>   4 files changed, 8 insertions(+), 6 deletions(-)
> 
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 4e4caeaea404..173bbb53c1ec 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -1163,7 +1163,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm,
>   	rcu_read_unlock();
>   }
>   
> -void split_page_memcg(struct page *head, unsigned int nr);
> +void split_page_memcg(struct page *head, int order);
>   
>   unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
>   						gfp_t gfp_mask,
> @@ -1621,7 +1621,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx)
>   {
>   }
>   
> -static inline void split_page_memcg(struct page *head, unsigned int nr)
> +static inline void split_page_memcg(struct page *head, int order)
>   {
>   }
>   
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 016e20bd813e..0cd5fba0923c 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2877,9 +2877,10 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>   	unsigned long offset = 0;
>   	unsigned int nr = thp_nr_pages(head);
>   	int i, nr_dropped = 0;
> +	int order = folio_order(folio);

You could calculate "nr" from "order" here, removing the usage of 
thp_nr_pages().

>   
>   	/* complete memcg works before add pages to LRU */
> -	split_page_memcg(head, nr);
> +	split_page_memcg(head, order);
>   
>   	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {

Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 2/7] mm/page_owner: use order instead of nr in split_page_owner()

[David Hildenbrand]

On 13.02.24 22:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> We do not have non power of two pages, using nr is error prone if nr
> is not power-of-two. Use page order instead.
> 
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---
>   include/linux/page_owner.h | 8 ++++----
>   mm/huge_memory.c           | 2 +-
>   mm/page_alloc.c            | 4 ++--
>   mm/page_owner.c            | 3 ++-
>   4 files changed, 9 insertions(+), 8 deletions(-)
> 
> diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
> index 119a0c9d2a8b..d7878523adfc 100644
> --- a/include/linux/page_owner.h
> +++ b/include/linux/page_owner.h
> @@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops;
>   extern void __reset_page_owner(struct page *page, unsigned short order);
>   extern void __set_page_owner(struct page *page,
>   			unsigned short order, gfp_t gfp_mask);
> -extern void __split_page_owner(struct page *page, unsigned int nr);
> +extern void __split_page_owner(struct page *page, int order);
>   extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
>   extern void __set_page_owner_migrate_reason(struct page *page, int reason);
>   extern void __dump_page_owner(const struct page *page);
> @@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page,
>   		__set_page_owner(page, order, gfp_mask);
>   }
>   
> -static inline void split_page_owner(struct page *page, unsigned int nr)
> +static inline void split_page_owner(struct page *page, int order)
>   {
>   	if (static_branch_unlikely(&page_owner_inited))
> -		__split_page_owner(page, nr);
> +		__split_page_owner(page, order);
>   }
>   static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
>   {
> @@ -60,7 +60,7 @@ static inline void set_page_owner(struct page *page,
>   {
>   }
>   static inline void split_page_owner(struct page *page,
> -			unsigned short order)
> +			int order)

I assume this will fit into a single line now.



Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 3/7] mm: memcg: make memcg huge page split support any order split.

[David Hildenbrand]

On 13.02.24 22:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> It sets memcg information for the pages after the split. A new parameter
> new_order is added to tell the order of subpages in the new page, always 0
> for now. It prepares for upcoming changes to support split huge page to
> any lower order.
> 
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---

Nothing jumped at me.

Acked-by: David Hildenbrand <david@redhat.com>

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 4/7] mm: page_owner: add support for splitting to any order in split page_owner.

[David Hildenbrand]

On 13.02.24 22:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> It adds a new_order parameter to set new page order in page owner.
> It prepares for upcoming changes to support split huge page to any
> lower order.
> 
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---
>   include/linux/page_owner.h | 10 +++++-----
>   mm/huge_memory.c           |  2 +-
>   mm/page_alloc.c            |  4 ++--
>   mm/page_owner.c            |  9 +++++----
>   4 files changed, 13 insertions(+), 12 deletions(-)
> 
> diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
> index d7878523adfc..a784ba69f67f 100644
> --- a/include/linux/page_owner.h
> +++ b/include/linux/page_owner.h
> @@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops;
>   extern void __reset_page_owner(struct page *page, unsigned short order);
>   extern void __set_page_owner(struct page *page,
>   			unsigned short order, gfp_t gfp_mask);
> -extern void __split_page_owner(struct page *page, int order);
> +extern void __split_page_owner(struct page *page, int old_order, int new_order);
>   extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
>   extern void __set_page_owner_migrate_reason(struct page *page, int reason);
>   extern void __dump_page_owner(const struct page *page);
> @@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page,
>   		__set_page_owner(page, order, gfp_mask);
>   }
>   
> -static inline void split_page_owner(struct page *page, int order)
> +static inline void split_page_owner(struct page *page, int old_order, int new_order)
>   {
>   	if (static_branch_unlikely(&page_owner_inited))
> -		__split_page_owner(page, order);
> +		__split_page_owner(page, old_order, new_order);
>   }
>   static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
>   {
> @@ -56,11 +56,11 @@ static inline void reset_page_owner(struct page *page, unsigned short order)
>   {
>   }
>   static inline void set_page_owner(struct page *page,
> -			unsigned int order, gfp_t gfp_mask)
> +			unsigned short order, gfp_t gfp_mask)
>   {
>   }
>   static inline void split_page_owner(struct page *page,
> -			int order)
> +			int old_order, int new_order)
>   {
>   }
>   static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio)
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index 3d30eccd3a7f..ad7133c97428 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2919,7 +2919,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>   	unlock_page_lruvec(lruvec);
>   	/* Caller disabled irqs, so they are still disabled here */
>   
> -	split_page_owner(head, order);
> +	split_page_owner(head, order, 0);
>   
>   	/* See comment in __split_huge_page_tail() */
>   	if (PageAnon(head)) {
> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
> index 9d4dd41d0647..e0f107b21c98 100644
> --- a/mm/page_alloc.c
> +++ b/mm/page_alloc.c
> @@ -2652,7 +2652,7 @@ void split_page(struct page *page, unsigned int order)
>   
>   	for (i = 1; i < (1 << order); i++)
>   		set_page_refcounted(page + i);
> -	split_page_owner(page, order);
> +	split_page_owner(page, order, 0);
>   	split_page_memcg(page, order, 0);
>   }
>   EXPORT_SYMBOL_GPL(split_page);
> @@ -4837,7 +4837,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order,
>   		struct page *page = virt_to_page((void *)addr);
>   		struct page *last = page + nr;
>   
> -		split_page_owner(page, order);
> +		split_page_owner(page, order, 0);
>   		split_page_memcg(page, order, 0);
>   		while (page < --last)
>   			set_page_refcounted(last);
> diff --git a/mm/page_owner.c b/mm/page_owner.c
> index 1319e402c2cf..ebbffa0501db 100644
> --- a/mm/page_owner.c
> +++ b/mm/page_owner.c
> @@ -292,19 +292,20 @@ void __set_page_owner_migrate_reason(struct page *page, int reason)
>   	page_ext_put(page_ext);
>   }
>   
> -void __split_page_owner(struct page *page, int order)
> +void __split_page_owner(struct page *page, int old_order, int new_order)
>   {
>   	int i;
>   	struct page_ext *page_ext = page_ext_get(page);
>   	struct page_owner *page_owner;
> -	unsigned int nr = 1 << order;
> +	unsigned int old_nr = 1 << old_order;
> +	unsigned int new_nr = 1 << new_order;
>   
>   	if (unlikely(!page_ext))
>   		return;
>   
> -	for (i = 0; i < nr; i++) {
> +	for (i = 0; i < old_nr; i += new_nr) {
>   		page_owner = get_page_owner(page_ext);
> -		page_owner->order = 0;
> +		page_owner->order = new_order;
>   		page_ext = page_ext_next(page_ext);

Staring at __set_page_owner_handle(), we do set all 1<<order page_exts 
(corresponding to 1<<order "struct page"s) to have ->order set.

Wouldn't you have to do the same here?

for (i = 0; i < 1 << old_order; i++) {
	page_owner = get_page_owner(page_ext);
	page_owner->order = new_order;
	page_ext = page_ext_next(page_ext);
}

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Ryan Roberts]

On 13/02/2024 21:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> To split a THP to any lower order (except order-1) pages, we need to
> reform THPs on subpages at given order and add page refcount based on the
> new page order. Also we need to reinitialize page_deferred_list after
> removing the page from the split_queue, otherwise a subsequent split will
> see list corruption when checking the page_deferred_list again.
> 
> It has many uses, like minimizing the number of pages after
> truncating a huge pagecache page. For anonymous THPs, we can only split
> them to order-0 like before until we add support for any size anonymous
> THPs.

multi-size THP is now upstream. Not sure if this comment still makes sense.
Still its not completely clear to me how you would integrate this new machinery
and decide what non-zero order to split anon THP to?

> 
> Order-1 folio is not supported because _deferred_list, which is used by
> partially mapped folios, is stored in subpage 2 and an order-1 folio only
> has subpage 0 and 1.
> 
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---
>  include/linux/huge_mm.h |  21 +++++---
>  mm/huge_memory.c        | 114 +++++++++++++++++++++++++++++++---------
>  2 files changed, 101 insertions(+), 34 deletions(-)
> 
> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
> index 5adb86af35fc..de0c89105076 100644
> --- a/include/linux/huge_mm.h
> +++ b/include/linux/huge_mm.h
> @@ -265,10 +265,11 @@ unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
>  
>  void folio_prep_large_rmappable(struct folio *folio);
>  bool can_split_folio(struct folio *folio, int *pextra_pins);
> -int split_huge_page_to_list(struct page *page, struct list_head *list);
> +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
> +		unsigned int new_order);
>  static inline int split_huge_page(struct page *page)
>  {
> -	return split_huge_page_to_list(page, NULL);
> +	return split_huge_page_to_list_to_order(page, NULL, 0);
>  }
>  void deferred_split_folio(struct folio *folio);
>  
> @@ -422,7 +423,8 @@ can_split_folio(struct folio *folio, int *pextra_pins)
>  	return false;
>  }
>  static inline int
> -split_huge_page_to_list(struct page *page, struct list_head *list)
> +split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
> +		unsigned int new_order)
>  {
>  	return 0;
>  }
> @@ -519,17 +521,20 @@ static inline bool thp_migration_supported(void)
>  }
>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>  
> -static inline int split_folio_to_list(struct folio *folio,
> -		struct list_head *list)
> +static inline int split_folio_to_list_to_order(struct folio *folio,
> +		struct list_head *list, int new_order)
>  {
> -	return split_huge_page_to_list(&folio->page, list);
> +	return split_huge_page_to_list_to_order(&folio->page, list, new_order);
>  }
>  
> -static inline int split_folio(struct folio *folio)
> +static inline int split_folio_to_order(struct folio *folio, int new_order)
>  {
> -	return split_folio_to_list(folio, NULL);
> +	return split_folio_to_list_to_order(folio, NULL, new_order);
>  }
>  
> +#define split_folio_to_list(f, l) split_folio_to_list_to_order(f, l, 0)
> +#define split_folio(f) split_folio_to_order(f, 0)
> +
>  /*
>   * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
>   * limitations in the implementation like arm64 MTE can override this to
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index ad7133c97428..d0e555a8ea98 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2718,11 +2718,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
>  
>  static void unmap_folio(struct folio *folio)
>  {
> -	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD |
> -		TTU_SYNC | TTU_BATCH_FLUSH;
> +	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC |
> +		TTU_BATCH_FLUSH;
>  
>  	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
>  
> +	if (folio_test_pmd_mappable(folio))
> +		ttu_flags |= TTU_SPLIT_HUGE_PMD;

Should we split this change out? I think it makes sense independent of this series?

> +
>  	/*
>  	 * Anon pages need migration entries to preserve them, but file
>  	 * pages can simply be left unmapped, then faulted back on demand.
> @@ -2756,7 +2759,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>  		struct lruvec *lruvec, struct list_head *list)
>  {
>  	VM_BUG_ON_PAGE(!PageHead(head), head);
> -	VM_BUG_ON_PAGE(PageCompound(tail), head);
>  	VM_BUG_ON_PAGE(PageLRU(tail), head);
>  	lockdep_assert_held(&lruvec->lru_lock);
>  
> @@ -2777,7 +2779,8 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>  }
>  
>  static void __split_huge_page_tail(struct folio *folio, int tail,
> -		struct lruvec *lruvec, struct list_head *list)
> +		struct lruvec *lruvec, struct list_head *list,
> +		unsigned int new_order)
>  {
>  	struct page *head = &folio->page;
>  	struct page *page_tail = head + tail;
> @@ -2847,10 +2850,15 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>  	 * which needs correct compound_head().
>  	 */
>  	clear_compound_head(page_tail);
> +	if (new_order) {
> +		prep_compound_page(page_tail, new_order);
> +		folio_prep_large_rmappable(page_folio(page_tail));
> +	}
>  
>  	/* Finally unfreeze refcount. Additional reference from page cache. */
> -	page_ref_unfreeze(page_tail, 1 + (!folio_test_anon(folio) ||
> -					  folio_test_swapcache(folio)));
> +	page_ref_unfreeze(page_tail,
> +		1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ?
> +			     folio_nr_pages(page_folio(page_tail)) : 0));
>  
>  	if (folio_test_young(folio))
>  		folio_set_young(new_folio);
> @@ -2868,7 +2876,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>  }
>  
>  static void __split_huge_page(struct page *page, struct list_head *list,
> -		pgoff_t end)
> +		pgoff_t end, unsigned int new_order)
>  {
>  	struct folio *folio = page_folio(page);
>  	struct page *head = &folio->page;
> @@ -2877,10 +2885,11 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>  	unsigned long offset = 0;
>  	unsigned int nr = thp_nr_pages(head);
>  	int i, nr_dropped = 0;
> +	unsigned int new_nr = 1 << new_order;
>  	int order = folio_order(folio);
>  
>  	/* complete memcg works before add pages to LRU */
> -	split_page_memcg(head, order, 0);
> +	split_page_memcg(head, order, new_order);
>  
>  	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
>  		offset = swp_offset(folio->swap);
> @@ -2893,8 +2902,8 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>  
>  	ClearPageHasHWPoisoned(head);
>  
> -	for (i = nr - 1; i >= 1; i--) {
> -		__split_huge_page_tail(folio, i, lruvec, list);
> +	for (i = nr - new_nr; i >= new_nr; i -= new_nr) {
> +		__split_huge_page_tail(folio, i, lruvec, list, new_order);
>  		/* Some pages can be beyond EOF: drop them from page cache */
>  		if (head[i].index >= end) {
>  			struct folio *tail = page_folio(head + i);
> @@ -2910,29 +2919,41 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>  			__xa_store(&head->mapping->i_pages, head[i].index,
>  					head + i, 0);
>  		} else if (swap_cache) {
> +			/*
> +			 * split anonymous THPs (including swapped out ones) to
> +			 * non-zero order not supported
> +			 */
> +			VM_WARN_ONCE(new_order,
> +				"Split swap-cached anon folio to non-0 order not supported");

Why isn't it supported? Even if it's not supported, is this level the right
place to enforce these kinds of policy decisions? I wonder if we should be
leaving that to the higher level to decide?

Thanks,
Ryan

>  			__xa_store(&swap_cache->i_pages, offset + i,
>  					head + i, 0);
>  		}
>  	}
>  
> -	ClearPageCompound(head);
> +	if (!new_order)
> +		ClearPageCompound(head);
> +	else {
> +		struct folio *new_folio = (struct folio *)head;
> +
> +		folio_set_order(new_folio, new_order);
> +	}
>  	unlock_page_lruvec(lruvec);
>  	/* Caller disabled irqs, so they are still disabled here */
>  
> -	split_page_owner(head, order, 0);
> +	split_page_owner(head, order, new_order);
>  
>  	/* See comment in __split_huge_page_tail() */
>  	if (PageAnon(head)) {
>  		/* Additional pin to swap cache */
>  		if (PageSwapCache(head)) {
> -			page_ref_add(head, 2);
> +			page_ref_add(head, 1 + new_nr);
>  			xa_unlock(&swap_cache->i_pages);
>  		} else {
>  			page_ref_inc(head);
>  		}
>  	} else {
>  		/* Additional pin to page cache */
> -		page_ref_add(head, 2);
> +		page_ref_add(head, 1 + new_nr);
>  		xa_unlock(&head->mapping->i_pages);
>  	}
>  	local_irq_enable();
> @@ -2944,7 +2965,15 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>  	if (folio_test_swapcache(folio))
>  		split_swap_cluster(folio->swap);
>  
> -	for (i = 0; i < nr; i++) {
> +	/*
> +	 * set page to its compound_head when split to non order-0 pages, so
> +	 * we can skip unlocking it below, since PG_locked is transferred to
> +	 * the compound_head of the page and the caller will unlock it.
> +	 */
> +	if (new_order)
> +		page = compound_head(page);
> +
> +	for (i = 0; i < nr; i += new_nr) {
>  		struct page *subpage = head + i;
>  		if (subpage == page)
>  			continue;
> @@ -2978,29 +3007,35 @@ bool can_split_folio(struct folio *folio, int *pextra_pins)
>  }
>  
>  /*
> - * This function splits huge page into normal pages. @page can point to any
> - * subpage of huge page to split. Split doesn't change the position of @page.
> + * This function splits huge page into pages in @new_order. @page can point to
> + * any subpage of huge page to split. Split doesn't change the position of
> + * @page.
> + *
> + * NOTE: order-1 folio is not supported because _deferred_list, which is used
> + * by partially mapped folios, is stored in subpage 2 and an order-1 folio
> + * only has subpage 0 and 1.
>   *
>   * Only caller must hold pin on the @page, otherwise split fails with -EBUSY.
>   * The huge page must be locked.
>   *
>   * If @list is null, tail pages will be added to LRU list, otherwise, to @list.
>   *
> - * Both head page and tail pages will inherit mapping, flags, and so on from
> - * the hugepage.
> + * Pages in new_order will inherit mapping, flags, and so on from the hugepage.
>   *
> - * GUP pin and PG_locked transferred to @page. Rest subpages can be freed if
> - * they are not mapped.
> + * GUP pin and PG_locked transferred to @page or the compound page @page belongs
> + * to. Rest subpages can be freed if they are not mapped.
>   *
>   * Returns 0 if the hugepage is split successfully.
>   * Returns -EBUSY if the page is pinned or if anon_vma disappeared from under
>   * us.
>   */
> -int split_huge_page_to_list(struct page *page, struct list_head *list)
> +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
> +				     unsigned int new_order)
>  {
>  	struct folio *folio = page_folio(page);
>  	struct deferred_split *ds_queue = get_deferred_split_queue(folio);
> -	XA_STATE(xas, &folio->mapping->i_pages, folio->index);
> +	/* reset xarray order to new order after split */
> +	XA_STATE_ORDER(xas, &folio->mapping->i_pages, folio->index, new_order);
>  	struct anon_vma *anon_vma = NULL;
>  	struct address_space *mapping = NULL;
>  	int extra_pins, ret;
> @@ -3010,6 +3045,26 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
>  	VM_BUG_ON_FOLIO(!folio_test_locked(folio), folio);
>  	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
>  
> +	/* Cannot split THP to order-1 (no order-1 THPs) */
> +	if (new_order == 1) {
> +		VM_WARN_ONCE(1, "Cannot split to order-1 folio");
> +		return -EINVAL;
> +	}
> +
> +	if (new_order) {
> +		/* Split shmem folio to non-zero order not supported */
> +		if (shmem_mapping(folio->mapping)) {
> +			VM_WARN_ONCE(1, "Split shmem folio to non-0 order not support");
> +			return -EINVAL;
> +		}
> +		/* No split if the file system does not support large folio */
> +		if (!mapping_large_folio_support(folio->mapping)) {
> +			VM_WARN_ONCE(1, "Split file folio to non-0 order not support");
> +			return -EINVAL;
> +		}
> +	}
> +
> +
>  	is_hzp = is_huge_zero_page(&folio->page);
>  	if (is_hzp) {
>  		pr_warn_ratelimited("Called split_huge_page for huge zero page\n");
> @@ -3105,14 +3160,21 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
>  	if (folio_ref_freeze(folio, 1 + extra_pins)) {
>  		if (!list_empty(&folio->_deferred_list)) {
>  			ds_queue->split_queue_len--;
> -			list_del(&folio->_deferred_list);
> +			/*
> +			 * Reinitialize page_deferred_list after removing the
> +			 * page from the split_queue, otherwise a subsequent
> +			 * split will see list corruption when checking the
> +			 * page_deferred_list.
> +			 */
> +			list_del_init(&folio->_deferred_list);
>  		}
>  		spin_unlock(&ds_queue->split_queue_lock);
>  		if (mapping) {
>  			int nr = folio_nr_pages(folio);
>  
>  			xas_split(&xas, folio, folio_order(folio));
> -			if (folio_test_pmd_mappable(folio)) {
> +			if (folio_test_pmd_mappable(folio) &&
> +			    new_order < HPAGE_PMD_ORDER) {
>  				if (folio_test_swapbacked(folio)) {
>  					__lruvec_stat_mod_folio(folio,
>  							NR_SHMEM_THPS, -nr);
> @@ -3124,7 +3186,7 @@ int split_huge_page_to_list(struct page *page, struct list_head *list)
>  			}
>  		}
>  
> -		__split_huge_page(page, list, end);
> +		__split_huge_page(page, list, end, new_order);
>  		ret = 0;
>  	} else {
>  		spin_unlock(&ds_queue->split_queue_lock);

Re: [PATCH v4 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

[Ryan Roberts]

On 13/02/2024 21:55, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> To minimize the number of pages after a huge page truncation, we do not
> need to split it all the way down to order-0. The huge page has at most
> three parts, the part before offset, the part to be truncated, the part
> remaining at the end. Find the greatest common divisor of them to
> calculate the new page order from it, so we can split the huge
> page to this order and keep the remaining pages as large and as few as
> possible.
> 
> Signed-off-by: Zi Yan <ziy@nvidia.com>
> ---
>  mm/truncate.c | 21 +++++++++++++++++++--
>  1 file changed, 19 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/truncate.c b/mm/truncate.c
> index 725b150e47ac..49ddbbf7a617 100644
> --- a/mm/truncate.c
> +++ b/mm/truncate.c
> @@ -21,6 +21,7 @@
>  #include <linux/task_io_accounting_ops.h>
>  #include <linux/shmem_fs.h>
>  #include <linux/rmap.h>
> +#include <linux/gcd.h>
>  #include "internal.h"
>  
>  /*
> @@ -210,7 +211,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio)
>  bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>  {
>  	loff_t pos = folio_pos(folio);
> -	unsigned int offset, length;
> +	unsigned int offset, length, remaining;
> +	unsigned int new_order = folio_order(folio);
>  
>  	if (pos < start)
>  		offset = start - pos;
> @@ -221,6 +223,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>  		length = length - offset;
>  	else
>  		length = end + 1 - pos - offset;
> +	remaining = folio_size(folio) - offset - length;
>  
>  	folio_wait_writeback(folio);
>  	if (length == folio_size(folio)) {
> @@ -235,11 +238,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>  	 */
>  	folio_zero_range(folio, offset, length);
>  
> +	/*
> +	 * Use the greatest common divisor of offset, length, and remaining
> +	 * as the smallest page size and compute the new order from it. So we
> +	 * can truncate a subpage as large as possible. Round up gcd to
> +	 * PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0.
> +	 */
> +	new_order = ilog2(round_up(gcd(gcd(offset, length), remaining),
> +				   PAGE_SIZE) / PAGE_SIZE);

Given you have up to 2 regions remaining, isn't it possible that you want a
different order for both those regions (or even multiple orders within the same
region)? I guess you just choose gcd for simplicity?

> +
> +	/* order-1 THP not supported, downgrade to order-0 */
> +	if (new_order == 1)
> +		new_order = 0;

I guess this would need to change if supporting order-1 file folios?

> +
> +
>  	if (folio_has_private(folio))
>  		folio_invalidate(folio, offset, length);
>  	if (!folio_test_large(folio))
>  		return true;
> -	if (split_folio(folio) == 0)
> +	if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0)

I know you are discussing removing this patch, but since you created
split_folio_to_order() wouldn't that be better here?

Thanks,
Ryan


>  		return true;
>  	if (folio_test_dirty(folio))
>  		return false;

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Ryan Roberts]

On 13/02/2024 22:31, Zi Yan wrote:
> On 13 Feb 2024, at 17:21, David Hildenbrand wrote:
> 
>> On 13.02.24 22:55, Zi Yan wrote:
>>> From: Zi Yan <ziy@nvidia.com>
>>>
>>> Hi all,
>>>
>>> File folio supports any order and multi-size THP is upstreamed[1], so both
>>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>>> only splits a huge page to order-0 pages, but splitting to orders higher than
>>> 0 is going to better utilize large folios. In addition, Large Block
>>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>>> splitting a large folio to any lower order folios and uses it during file
>>> folio truncate operations.
>>>
>>> For Patch 6, Hugh did not like my approach to minimize the number of
>>> folios for truncate[3]. I would like to get more feedback, especially
>>> from FS people, on it to decide whether to keep it or not.
>>
>> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?
> 
> Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
> Now we have multi-size THP and XFS use cases, it can be dropped.

What are your plans for how to determine when to split THP and to what order? I
don't see anything in this series that would split anon THP to non-zero order?

We have talked about using hints from user space in the past (e.g.  mremap,
munmap, madvise, etc). But chrome has a use case where it temporarily mprotects
a single (4K) page as part of garbage collection (IIRC). If you eagerly split on
that hint, you will have lost the benefits of the large folio when it later
mprotects back to the original setting.

I guess David will suggest this would be a good use case for the khugepaged-lite
machanism we have been talking about. I dunno - it seems wasteful to split then
collapse again.

Or perhaps you're considering doing something clever in deferred split?

> 
> --
> Best Regards,
> Yan, Zi

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[David Hildenbrand]

On 14.02.24 11:50, Ryan Roberts wrote:
> On 13/02/2024 22:31, Zi Yan wrote:
>> On 13 Feb 2024, at 17:21, David Hildenbrand wrote:
>>
>>> On 13.02.24 22:55, Zi Yan wrote:
>>>> From: Zi Yan <ziy@nvidia.com>
>>>>
>>>> Hi all,
>>>>
>>>> File folio supports any order and multi-size THP is upstreamed[1], so both
>>>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>>>> only splits a huge page to order-0 pages, but splitting to orders higher than
>>>> 0 is going to better utilize large folios. In addition, Large Block
>>>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>>>> splitting a large folio to any lower order folios and uses it during file
>>>> folio truncate operations.
>>>>
>>>> For Patch 6, Hugh did not like my approach to minimize the number of
>>>> folios for truncate[3]. I would like to get more feedback, especially
>>>> from FS people, on it to decide whether to keep it or not.
>>>
>>> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?
>>
>> Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
>> Now we have multi-size THP and XFS use cases, it can be dropped.
> 
> What are your plans for how to determine when to split THP and to what order? I
> don't see anything in this series that would split anon THP to non-zero order?
> 
> We have talked about using hints from user space in the past (e.g.  mremap,
> munmap, madvise, etc). But chrome has a use case where it temporarily mprotects
> a single (4K) page as part of garbage collection (IIRC). If you eagerly split on
> that hint, you will have lost the benefits of the large folio when it later
> mprotects back to the original setting.

Not only that, splitting will make some of these operations more 
expensive, possibly with no actual benefit.

> 
> I guess David will suggest this would be a good use case for the khugepaged-lite
> machanism we have been talking about. I dunno - it seems wasteful to split then
> collapse again.

I agree. mprotect() and even madvise(), ... might not be good candidates 
for splitting. mremap() likely is, if the folio is mapped exclusively. 
MADV_DONTNEED/munmap()/mlock() might be good candidates (again, if 
mapped exclusively). This will need a lot of thought I'm afraid (as you 
say, deferred splitting is another example).

-- 
Cheers,

David / dhildenb

Re: [PATCH v4 1/7] mm/memcg: use order instead of nr in split_page_memcg()

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 2146 bytes --]

On 14 Feb 2024, at 4:12, David Hildenbrand wrote:

> On 13.02.24 22:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> We do not have non power of two pages, using nr is error prone if nr
>> is not power-of-two. Use page order instead.
>>
>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>> ---
>>   include/linux/memcontrol.h | 4 ++--
>>   mm/huge_memory.c           | 3 ++-
>>   mm/memcontrol.c            | 3 ++-
>>   mm/page_alloc.c            | 4 ++--
>>   4 files changed, 8 insertions(+), 6 deletions(-)
>>
>> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
>> index 4e4caeaea404..173bbb53c1ec 100644
>> --- a/include/linux/memcontrol.h
>> +++ b/include/linux/memcontrol.h
>> @@ -1163,7 +1163,7 @@ static inline void memcg_memory_event_mm(struct mm_struct *mm,
>>   	rcu_read_unlock();
>>   }
>>  -void split_page_memcg(struct page *head, unsigned int nr);
>> +void split_page_memcg(struct page *head, int order);
>>    unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
>>   						gfp_t gfp_mask,
>> @@ -1621,7 +1621,7 @@ void count_memcg_event_mm(struct mm_struct *mm, enum vm_event_item idx)
>>   {
>>   }
>>  -static inline void split_page_memcg(struct page *head, unsigned int nr)
>> +static inline void split_page_memcg(struct page *head, int order)
>>   {
>>   }
>>  diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index 016e20bd813e..0cd5fba0923c 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2877,9 +2877,10 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>   	unsigned long offset = 0;
>>   	unsigned int nr = thp_nr_pages(head);
>>   	int i, nr_dropped = 0;
>> +	int order = folio_order(folio);
>
> You could calculate "nr" from "order" here, removing the usage of thp_nr_pages().

Sure.

>
>>    	/* complete memcg works before add pages to LRU */
>> -	split_page_memcg(head, nr);
>> +	split_page_memcg(head, order);
>>    	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
>
> Acked-by: David Hildenbrand <david@redhat.com>

Thanks.

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 2/7] mm/page_owner: use order instead of nr in split_page_owner()

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 2163 bytes --]

On 14 Feb 2024, at 4:14, David Hildenbrand wrote:

> On 13.02.24 22:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> We do not have non power of two pages, using nr is error prone if nr
>> is not power-of-two. Use page order instead.
>>
>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>> ---
>>   include/linux/page_owner.h | 8 ++++----
>>   mm/huge_memory.c           | 2 +-
>>   mm/page_alloc.c            | 4 ++--
>>   mm/page_owner.c            | 3 ++-
>>   4 files changed, 9 insertions(+), 8 deletions(-)
>>
>> diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
>> index 119a0c9d2a8b..d7878523adfc 100644
>> --- a/include/linux/page_owner.h
>> +++ b/include/linux/page_owner.h
>> @@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops;
>>   extern void __reset_page_owner(struct page *page, unsigned short order);
>>   extern void __set_page_owner(struct page *page,
>>   			unsigned short order, gfp_t gfp_mask);
>> -extern void __split_page_owner(struct page *page, unsigned int nr);
>> +extern void __split_page_owner(struct page *page, int order);
>>   extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
>>   extern void __set_page_owner_migrate_reason(struct page *page, int reason);
>>   extern void __dump_page_owner(const struct page *page);
>> @@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page,
>>   		__set_page_owner(page, order, gfp_mask);
>>   }
>>  -static inline void split_page_owner(struct page *page, unsigned int nr)
>> +static inline void split_page_owner(struct page *page, int order)
>>   {
>>   	if (static_branch_unlikely(&page_owner_inited))
>> -		__split_page_owner(page, nr);
>> +		__split_page_owner(page, order);
>>   }
>>   static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
>>   {
>> @@ -60,7 +60,7 @@ static inline void set_page_owner(struct page *page,
>>   {
>>   }
>>   static inline void split_page_owner(struct page *page,
>> -			unsigned short order)
>> +			int order)
>
> I assume this will fit into a single line now.

Sure.
>
> Acked-by: David Hildenbrand <david@redhat.com>

Thanks.

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 4/7] mm: page_owner: add support for splitting to any order in split page_owner.

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 5283 bytes --]

On 14 Feb 2024, at 4:34, David Hildenbrand wrote:

> On 13.02.24 22:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> It adds a new_order parameter to set new page order in page owner.
>> It prepares for upcoming changes to support split huge page to any
>> lower order.
>>
>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>> ---
>>   include/linux/page_owner.h | 10 +++++-----
>>   mm/huge_memory.c           |  2 +-
>>   mm/page_alloc.c            |  4 ++--
>>   mm/page_owner.c            |  9 +++++----
>>   4 files changed, 13 insertions(+), 12 deletions(-)
>>
>> diff --git a/include/linux/page_owner.h b/include/linux/page_owner.h
>> index d7878523adfc..a784ba69f67f 100644
>> --- a/include/linux/page_owner.h
>> +++ b/include/linux/page_owner.h
>> @@ -11,7 +11,7 @@ extern struct page_ext_operations page_owner_ops;
>>   extern void __reset_page_owner(struct page *page, unsigned short order);
>>   extern void __set_page_owner(struct page *page,
>>   			unsigned short order, gfp_t gfp_mask);
>> -extern void __split_page_owner(struct page *page, int order);
>> +extern void __split_page_owner(struct page *page, int old_order, int new_order);
>>   extern void __folio_copy_owner(struct folio *newfolio, struct folio *old);
>>   extern void __set_page_owner_migrate_reason(struct page *page, int reason);
>>   extern void __dump_page_owner(const struct page *page);
>> @@ -31,10 +31,10 @@ static inline void set_page_owner(struct page *page,
>>   		__set_page_owner(page, order, gfp_mask);
>>   }
>>  -static inline void split_page_owner(struct page *page, int order)
>> +static inline void split_page_owner(struct page *page, int old_order, int new_order)
>>   {
>>   	if (static_branch_unlikely(&page_owner_inited))
>> -		__split_page_owner(page, order);
>> +		__split_page_owner(page, old_order, new_order);
>>   }
>>   static inline void folio_copy_owner(struct folio *newfolio, struct folio *old)
>>   {
>> @@ -56,11 +56,11 @@ static inline void reset_page_owner(struct page *page, unsigned short order)
>>   {
>>   }
>>   static inline void set_page_owner(struct page *page,
>> -			unsigned int order, gfp_t gfp_mask)
>> +			unsigned short order, gfp_t gfp_mask)
>>   {
>>   }
>>   static inline void split_page_owner(struct page *page,
>> -			int order)
>> +			int old_order, int new_order)
>>   {
>>   }
>>   static inline void folio_copy_owner(struct folio *newfolio, struct folio *folio)
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index 3d30eccd3a7f..ad7133c97428 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2919,7 +2919,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>   	unlock_page_lruvec(lruvec);
>>   	/* Caller disabled irqs, so they are still disabled here */
>>  -	split_page_owner(head, order);
>> +	split_page_owner(head, order, 0);
>>    	/* See comment in __split_huge_page_tail() */
>>   	if (PageAnon(head)) {
>> diff --git a/mm/page_alloc.c b/mm/page_alloc.c
>> index 9d4dd41d0647..e0f107b21c98 100644
>> --- a/mm/page_alloc.c
>> +++ b/mm/page_alloc.c
>> @@ -2652,7 +2652,7 @@ void split_page(struct page *page, unsigned int order)
>>    	for (i = 1; i < (1 << order); i++)
>>   		set_page_refcounted(page + i);
>> -	split_page_owner(page, order);
>> +	split_page_owner(page, order, 0);
>>   	split_page_memcg(page, order, 0);
>>   }
>>   EXPORT_SYMBOL_GPL(split_page);
>> @@ -4837,7 +4837,7 @@ static void *make_alloc_exact(unsigned long addr, unsigned int order,
>>   		struct page *page = virt_to_page((void *)addr);
>>   		struct page *last = page + nr;
>>  -		split_page_owner(page, order);
>> +		split_page_owner(page, order, 0);
>>   		split_page_memcg(page, order, 0);
>>   		while (page < --last)
>>   			set_page_refcounted(last);
>> diff --git a/mm/page_owner.c b/mm/page_owner.c
>> index 1319e402c2cf..ebbffa0501db 100644
>> --- a/mm/page_owner.c
>> +++ b/mm/page_owner.c
>> @@ -292,19 +292,20 @@ void __set_page_owner_migrate_reason(struct page *page, int reason)
>>   	page_ext_put(page_ext);
>>   }
>>  -void __split_page_owner(struct page *page, int order)
>> +void __split_page_owner(struct page *page, int old_order, int new_order)
>>   {
>>   	int i;
>>   	struct page_ext *page_ext = page_ext_get(page);
>>   	struct page_owner *page_owner;
>> -	unsigned int nr = 1 << order;
>> +	unsigned int old_nr = 1 << old_order;
>> +	unsigned int new_nr = 1 << new_order;
>>    	if (unlikely(!page_ext))
>>   		return;
>>  -	for (i = 0; i < nr; i++) {
>> +	for (i = 0; i < old_nr; i += new_nr) {
>>   		page_owner = get_page_owner(page_ext);
>> -		page_owner->order = 0;
>> +		page_owner->order = new_order;
>>   		page_ext = page_ext_next(page_ext);
>
> Staring at __set_page_owner_handle(), we do set all 1<<order page_exts (corresponding to 1<<order "struct page"s) to have ->order set.
>
> Wouldn't you have to do the same here?
>
> for (i = 0; i < 1 << old_order; i++) {
> 	page_owner = get_page_owner(page_ext);
> 	page_owner->order = new_order;
> 	page_ext = page_ext_next(page_ext);
> }

You are right. So page owner is per struct page. I misunderstood the code.
Thank you for pointing this out. Will fix it.


--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 8894 bytes --]

On 14 Feb 2024, at 5:38, Ryan Roberts wrote:

> On 13/02/2024 21:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> To split a THP to any lower order (except order-1) pages, we need to
>> reform THPs on subpages at given order and add page refcount based on the
>> new page order. Also we need to reinitialize page_deferred_list after
>> removing the page from the split_queue, otherwise a subsequent split will
>> see list corruption when checking the page_deferred_list again.
>>
>> It has many uses, like minimizing the number of pages after
>> truncating a huge pagecache page. For anonymous THPs, we can only split
>> them to order-0 like before until we add support for any size anonymous
>> THPs.
>
> multi-size THP is now upstream. Not sure if this comment still makes sense.
Will change it to reflect the fact that multi-size THP is already upstream.

> Still its not completely clear to me how you would integrate this new machinery
> and decide what non-zero order to split anon THP to?

Originally, it was developed along with my 1GB THP support. So it was intended
to split order-18 to order-9. But for now, like you and David said in the cover
letter email thread, we might not want to use it for anonymous large folios
until we find a necessary use case.

>>
>> Order-1 folio is not supported because _deferred_list, which is used by
>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>> has subpage 0 and 1.
>>
>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>> ---
>>  include/linux/huge_mm.h |  21 +++++---
>>  mm/huge_memory.c        | 114 +++++++++++++++++++++++++++++++---------
>>  2 files changed, 101 insertions(+), 34 deletions(-)
>>
>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>> index 5adb86af35fc..de0c89105076 100644
>> --- a/include/linux/huge_mm.h
>> +++ b/include/linux/huge_mm.h
>> @@ -265,10 +265,11 @@ unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
>>
>>  void folio_prep_large_rmappable(struct folio *folio);
>>  bool can_split_folio(struct folio *folio, int *pextra_pins);
>> -int split_huge_page_to_list(struct page *page, struct list_head *list);
>> +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>> +		unsigned int new_order);
>>  static inline int split_huge_page(struct page *page)
>>  {
>> -	return split_huge_page_to_list(page, NULL);
>> +	return split_huge_page_to_list_to_order(page, NULL, 0);
>>  }
>>  void deferred_split_folio(struct folio *folio);
>>
>> @@ -422,7 +423,8 @@ can_split_folio(struct folio *folio, int *pextra_pins)
>>  	return false;
>>  }
>>  static inline int
>> -split_huge_page_to_list(struct page *page, struct list_head *list)
>> +split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>> +		unsigned int new_order)
>>  {
>>  	return 0;
>>  }
>> @@ -519,17 +521,20 @@ static inline bool thp_migration_supported(void)
>>  }
>>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>>
>> -static inline int split_folio_to_list(struct folio *folio,
>> -		struct list_head *list)
>> +static inline int split_folio_to_list_to_order(struct folio *folio,
>> +		struct list_head *list, int new_order)
>>  {
>> -	return split_huge_page_to_list(&folio->page, list);
>> +	return split_huge_page_to_list_to_order(&folio->page, list, new_order);
>>  }
>>
>> -static inline int split_folio(struct folio *folio)
>> +static inline int split_folio_to_order(struct folio *folio, int new_order)
>>  {
>> -	return split_folio_to_list(folio, NULL);
>> +	return split_folio_to_list_to_order(folio, NULL, new_order);
>>  }
>>
>> +#define split_folio_to_list(f, l) split_folio_to_list_to_order(f, l, 0)
>> +#define split_folio(f) split_folio_to_order(f, 0)
>> +
>>  /*
>>   * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
>>   * limitations in the implementation like arm64 MTE can override this to
>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>> index ad7133c97428..d0e555a8ea98 100644
>> --- a/mm/huge_memory.c
>> +++ b/mm/huge_memory.c
>> @@ -2718,11 +2718,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
>>
>>  static void unmap_folio(struct folio *folio)
>>  {
>> -	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD |
>> -		TTU_SYNC | TTU_BATCH_FLUSH;
>> +	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC |
>> +		TTU_BATCH_FLUSH;
>>
>>  	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
>>
>> +	if (folio_test_pmd_mappable(folio))
>> +		ttu_flags |= TTU_SPLIT_HUGE_PMD;
>
> Should we split this change out? I think it makes sense independent of this series?
>

Sure. Since multi-size THP is upstream, this avoid unnecessary code path if
the THP is not PMD-mapped.

>> +
>>  	/*
>>  	 * Anon pages need migration entries to preserve them, but file
>>  	 * pages can simply be left unmapped, then faulted back on demand.
>> @@ -2756,7 +2759,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>  		struct lruvec *lruvec, struct list_head *list)
>>  {
>>  	VM_BUG_ON_PAGE(!PageHead(head), head);
>> -	VM_BUG_ON_PAGE(PageCompound(tail), head);
>>  	VM_BUG_ON_PAGE(PageLRU(tail), head);
>>  	lockdep_assert_held(&lruvec->lru_lock);
>>
>> @@ -2777,7 +2779,8 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>  }
>>
>>  static void __split_huge_page_tail(struct folio *folio, int tail,
>> -		struct lruvec *lruvec, struct list_head *list)
>> +		struct lruvec *lruvec, struct list_head *list,
>> +		unsigned int new_order)
>>  {
>>  	struct page *head = &folio->page;
>>  	struct page *page_tail = head + tail;
>> @@ -2847,10 +2850,15 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>  	 * which needs correct compound_head().
>>  	 */
>>  	clear_compound_head(page_tail);
>> +	if (new_order) {
>> +		prep_compound_page(page_tail, new_order);
>> +		folio_prep_large_rmappable(page_folio(page_tail));
>> +	}
>>
>>  	/* Finally unfreeze refcount. Additional reference from page cache. */
>> -	page_ref_unfreeze(page_tail, 1 + (!folio_test_anon(folio) ||
>> -					  folio_test_swapcache(folio)));
>> +	page_ref_unfreeze(page_tail,
>> +		1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ?
>> +			     folio_nr_pages(page_folio(page_tail)) : 0));
>>
>>  	if (folio_test_young(folio))
>>  		folio_set_young(new_folio);
>> @@ -2868,7 +2876,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>  }
>>
>>  static void __split_huge_page(struct page *page, struct list_head *list,
>> -		pgoff_t end)
>> +		pgoff_t end, unsigned int new_order)
>>  {
>>  	struct folio *folio = page_folio(page);
>>  	struct page *head = &folio->page;
>> @@ -2877,10 +2885,11 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>  	unsigned long offset = 0;
>>  	unsigned int nr = thp_nr_pages(head);
>>  	int i, nr_dropped = 0;
>> +	unsigned int new_nr = 1 << new_order;
>>  	int order = folio_order(folio);
>>
>>  	/* complete memcg works before add pages to LRU */
>> -	split_page_memcg(head, order, 0);
>> +	split_page_memcg(head, order, new_order);
>>
>>  	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
>>  		offset = swp_offset(folio->swap);
>> @@ -2893,8 +2902,8 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>
>>  	ClearPageHasHWPoisoned(head);
>>
>> -	for (i = nr - 1; i >= 1; i--) {
>> -		__split_huge_page_tail(folio, i, lruvec, list);
>> +	for (i = nr - new_nr; i >= new_nr; i -= new_nr) {
>> +		__split_huge_page_tail(folio, i, lruvec, list, new_order);
>>  		/* Some pages can be beyond EOF: drop them from page cache */
>>  		if (head[i].index >= end) {
>>  			struct folio *tail = page_folio(head + i);
>> @@ -2910,29 +2919,41 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>  			__xa_store(&head->mapping->i_pages, head[i].index,
>>  					head + i, 0);
>>  		} else if (swap_cache) {
>> +			/*
>> +			 * split anonymous THPs (including swapped out ones) to
>> +			 * non-zero order not supported
>> +			 */
>> +			VM_WARN_ONCE(new_order,
>> +				"Split swap-cached anon folio to non-0 order not supported");
>
> Why isn't it supported? Even if it's not supported, is this level the right
> place to enforce these kinds of policy decisions? I wonder if we should be
> leaving that to the higher level to decide?

Is the swap-out small-size THP without splitting merged? This needs that patchset.
You are right that a warning here is not appropriate. I will fail the splitting
if the folio is swapcached and going to be split into >0 order.

>>  			__xa_store(&swap_cache->i_pages, offset + i,
>>  					head + i, 0);
>>  		}
>>  	}
>>


--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 3869 bytes --]

On 14 Feb 2024, at 5:43, Ryan Roberts wrote:

> On 13/02/2024 21:55, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> To minimize the number of pages after a huge page truncation, we do not
>> need to split it all the way down to order-0. The huge page has at most
>> three parts, the part before offset, the part to be truncated, the part
>> remaining at the end. Find the greatest common divisor of them to
>> calculate the new page order from it, so we can split the huge
>> page to this order and keep the remaining pages as large and as few as
>> possible.
>>
>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>> ---
>>  mm/truncate.c | 21 +++++++++++++++++++--
>>  1 file changed, 19 insertions(+), 2 deletions(-)
>>
>> diff --git a/mm/truncate.c b/mm/truncate.c
>> index 725b150e47ac..49ddbbf7a617 100644
>> --- a/mm/truncate.c
>> +++ b/mm/truncate.c
>> @@ -21,6 +21,7 @@
>>  #include <linux/task_io_accounting_ops.h>
>>  #include <linux/shmem_fs.h>
>>  #include <linux/rmap.h>
>> +#include <linux/gcd.h>
>>  #include "internal.h"
>>
>>  /*
>> @@ -210,7 +211,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio)
>>  bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>>  {
>>  	loff_t pos = folio_pos(folio);
>> -	unsigned int offset, length;
>> +	unsigned int offset, length, remaining;
>> +	unsigned int new_order = folio_order(folio);
>>
>>  	if (pos < start)
>>  		offset = start - pos;
>> @@ -221,6 +223,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>>  		length = length - offset;
>>  	else
>>  		length = end + 1 - pos - offset;
>> +	remaining = folio_size(folio) - offset - length;
>>
>>  	folio_wait_writeback(folio);
>>  	if (length == folio_size(folio)) {
>> @@ -235,11 +238,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>>  	 */
>>  	folio_zero_range(folio, offset, length);
>>
>> +	/*
>> +	 * Use the greatest common divisor of offset, length, and remaining
>> +	 * as the smallest page size and compute the new order from it. So we
>> +	 * can truncate a subpage as large as possible. Round up gcd to
>> +	 * PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0.
>> +	 */
>> +	new_order = ilog2(round_up(gcd(gcd(offset, length), remaining),
>> +				   PAGE_SIZE) / PAGE_SIZE);
>
> Given you have up to 2 regions remaining, isn't it possible that you want a
> different order for both those regions (or even multiple orders within the same
> region)? I guess you just choose gcd for simplicity?

Right. You raise the same concern as Hugh[1]. I am minimizing the call of
split_huge_page_to_list_to_order() and you and Hugh want to minimize the
number of folios after the split. Yours will give better outcome after split,
but requires either multiple calls or a more sophisticated implementation
of page split[2]. We probably can revisit this once splitting to any order
gets wider use.

>> +
>> +	/* order-1 THP not supported, downgrade to order-0 */
>> +	if (new_order == 1)
>> +		new_order = 0;
>
> I guess this would need to change if supporting order-1 file folios?

Right.

>> +
>> +
>>  	if (folio_has_private(folio))
>>  		folio_invalidate(folio, offset, length);
>>  	if (!folio_test_large(folio))
>>  		return true;
>> -	if (split_folio(folio) == 0)
>> +	if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0)
>
> I know you are discussing removing this patch, but since you created
> split_folio_to_order() wouldn't that be better here?

Sure. Will change the patch locally.

[1] https://lore.kernel.org/linux-mm/9dd96da-efa2-5123-20d4-4992136ef3ad@google.com/
[2] https://lore.kernel.org/linux-mm/0AC0520E-1BD2-497E-A7ED-05394400BFC9@nvidia.com/

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Ryan Roberts]

On 14/02/2024 16:11, Zi Yan wrote:
> On 14 Feb 2024, at 5:38, Ryan Roberts wrote:
> 
>> On 13/02/2024 21:55, Zi Yan wrote:
>>> From: Zi Yan <ziy@nvidia.com>
>>>
>>> To split a THP to any lower order (except order-1) pages, we need to
>>> reform THPs on subpages at given order and add page refcount based on the
>>> new page order. Also we need to reinitialize page_deferred_list after
>>> removing the page from the split_queue, otherwise a subsequent split will
>>> see list corruption when checking the page_deferred_list again.
>>>
>>> It has many uses, like minimizing the number of pages after
>>> truncating a huge pagecache page. For anonymous THPs, we can only split
>>> them to order-0 like before until we add support for any size anonymous
>>> THPs.
>>
>> multi-size THP is now upstream. Not sure if this comment still makes sense.
> Will change it to reflect the fact that multi-size THP is already upstream.
> 
>> Still its not completely clear to me how you would integrate this new machinery
>> and decide what non-zero order to split anon THP to?
> 
> Originally, it was developed along with my 1GB THP support. So it was intended
> to split order-18 to order-9. But for now, like you and David said in the cover
> letter email thread, we might not want to use it for anonymous large folios
> until we find a necessary use case.
> 
>>>
>>> Order-1 folio is not supported because _deferred_list, which is used by
>>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>>> has subpage 0 and 1.
>>>
>>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>>> ---
>>>  include/linux/huge_mm.h |  21 +++++---
>>>  mm/huge_memory.c        | 114 +++++++++++++++++++++++++++++++---------
>>>  2 files changed, 101 insertions(+), 34 deletions(-)
>>>
>>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>>> index 5adb86af35fc..de0c89105076 100644
>>> --- a/include/linux/huge_mm.h
>>> +++ b/include/linux/huge_mm.h
>>> @@ -265,10 +265,11 @@ unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
>>>
>>>  void folio_prep_large_rmappable(struct folio *folio);
>>>  bool can_split_folio(struct folio *folio, int *pextra_pins);
>>> -int split_huge_page_to_list(struct page *page, struct list_head *list);
>>> +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>>> +		unsigned int new_order);
>>>  static inline int split_huge_page(struct page *page)
>>>  {
>>> -	return split_huge_page_to_list(page, NULL);
>>> +	return split_huge_page_to_list_to_order(page, NULL, 0);
>>>  }
>>>  void deferred_split_folio(struct folio *folio);
>>>
>>> @@ -422,7 +423,8 @@ can_split_folio(struct folio *folio, int *pextra_pins)
>>>  	return false;
>>>  }
>>>  static inline int
>>> -split_huge_page_to_list(struct page *page, struct list_head *list)
>>> +split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>>> +		unsigned int new_order)
>>>  {
>>>  	return 0;
>>>  }
>>> @@ -519,17 +521,20 @@ static inline bool thp_migration_supported(void)
>>>  }
>>>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>>>
>>> -static inline int split_folio_to_list(struct folio *folio,
>>> -		struct list_head *list)
>>> +static inline int split_folio_to_list_to_order(struct folio *folio,
>>> +		struct list_head *list, int new_order)
>>>  {
>>> -	return split_huge_page_to_list(&folio->page, list);
>>> +	return split_huge_page_to_list_to_order(&folio->page, list, new_order);
>>>  }
>>>
>>> -static inline int split_folio(struct folio *folio)
>>> +static inline int split_folio_to_order(struct folio *folio, int new_order)
>>>  {
>>> -	return split_folio_to_list(folio, NULL);
>>> +	return split_folio_to_list_to_order(folio, NULL, new_order);
>>>  }
>>>
>>> +#define split_folio_to_list(f, l) split_folio_to_list_to_order(f, l, 0)
>>> +#define split_folio(f) split_folio_to_order(f, 0)
>>> +
>>>  /*
>>>   * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
>>>   * limitations in the implementation like arm64 MTE can override this to
>>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>>> index ad7133c97428..d0e555a8ea98 100644
>>> --- a/mm/huge_memory.c
>>> +++ b/mm/huge_memory.c
>>> @@ -2718,11 +2718,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
>>>
>>>  static void unmap_folio(struct folio *folio)
>>>  {
>>> -	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD |
>>> -		TTU_SYNC | TTU_BATCH_FLUSH;
>>> +	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC |
>>> +		TTU_BATCH_FLUSH;
>>>
>>>  	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
>>>
>>> +	if (folio_test_pmd_mappable(folio))
>>> +		ttu_flags |= TTU_SPLIT_HUGE_PMD;
>>
>> Should we split this change out? I think it makes sense independent of this series?
>>
> 
> Sure. Since multi-size THP is upstream, this avoid unnecessary code path if
> the THP is not PMD-mapped.
> 
>>> +
>>>  	/*
>>>  	 * Anon pages need migration entries to preserve them, but file
>>>  	 * pages can simply be left unmapped, then faulted back on demand.
>>> @@ -2756,7 +2759,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>>  		struct lruvec *lruvec, struct list_head *list)
>>>  {
>>>  	VM_BUG_ON_PAGE(!PageHead(head), head);
>>> -	VM_BUG_ON_PAGE(PageCompound(tail), head);
>>>  	VM_BUG_ON_PAGE(PageLRU(tail), head);
>>>  	lockdep_assert_held(&lruvec->lru_lock);
>>>
>>> @@ -2777,7 +2779,8 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>>  }
>>>
>>>  static void __split_huge_page_tail(struct folio *folio, int tail,
>>> -		struct lruvec *lruvec, struct list_head *list)
>>> +		struct lruvec *lruvec, struct list_head *list,
>>> +		unsigned int new_order)
>>>  {
>>>  	struct page *head = &folio->page;
>>>  	struct page *page_tail = head + tail;
>>> @@ -2847,10 +2850,15 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>>  	 * which needs correct compound_head().
>>>  	 */
>>>  	clear_compound_head(page_tail);
>>> +	if (new_order) {
>>> +		prep_compound_page(page_tail, new_order);
>>> +		folio_prep_large_rmappable(page_folio(page_tail));
>>> +	}
>>>
>>>  	/* Finally unfreeze refcount. Additional reference from page cache. */
>>> -	page_ref_unfreeze(page_tail, 1 + (!folio_test_anon(folio) ||
>>> -					  folio_test_swapcache(folio)));
>>> +	page_ref_unfreeze(page_tail,
>>> +		1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ?
>>> +			     folio_nr_pages(page_folio(page_tail)) : 0));
>>>
>>>  	if (folio_test_young(folio))
>>>  		folio_set_young(new_folio);
>>> @@ -2868,7 +2876,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>>  }
>>>
>>>  static void __split_huge_page(struct page *page, struct list_head *list,
>>> -		pgoff_t end)
>>> +		pgoff_t end, unsigned int new_order)
>>>  {
>>>  	struct folio *folio = page_folio(page);
>>>  	struct page *head = &folio->page;
>>> @@ -2877,10 +2885,11 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>  	unsigned long offset = 0;
>>>  	unsigned int nr = thp_nr_pages(head);
>>>  	int i, nr_dropped = 0;
>>> +	unsigned int new_nr = 1 << new_order;
>>>  	int order = folio_order(folio);
>>>
>>>  	/* complete memcg works before add pages to LRU */
>>> -	split_page_memcg(head, order, 0);
>>> +	split_page_memcg(head, order, new_order);
>>>
>>>  	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
>>>  		offset = swp_offset(folio->swap);
>>> @@ -2893,8 +2902,8 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>
>>>  	ClearPageHasHWPoisoned(head);
>>>
>>> -	for (i = nr - 1; i >= 1; i--) {
>>> -		__split_huge_page_tail(folio, i, lruvec, list);
>>> +	for (i = nr - new_nr; i >= new_nr; i -= new_nr) {
>>> +		__split_huge_page_tail(folio, i, lruvec, list, new_order);
>>>  		/* Some pages can be beyond EOF: drop them from page cache */
>>>  		if (head[i].index >= end) {
>>>  			struct folio *tail = page_folio(head + i);
>>> @@ -2910,29 +2919,41 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>  			__xa_store(&head->mapping->i_pages, head[i].index,
>>>  					head + i, 0);
>>>  		} else if (swap_cache) {
>>> +			/*
>>> +			 * split anonymous THPs (including swapped out ones) to
>>> +			 * non-zero order not supported
>>> +			 */
>>> +			VM_WARN_ONCE(new_order,
>>> +				"Split swap-cached anon folio to non-0 order not supported");
>>
>> Why isn't it supported? Even if it's not supported, is this level the right
>> place to enforce these kinds of policy decisions? I wonder if we should be
>> leaving that to the higher level to decide?
> 
> Is the swap-out small-size THP without splitting merged? This needs that patchset.

No not yet. I have to respin it. Its on my todo list.

I'm not sure I understand the dependency though?

> You are right that a warning here is not appropriate. I will fail the splitting
> if the folio is swapcached and going to be split into >0 order.
> 
>>>  			__xa_store(&swap_cache->i_pages, offset + i,
>>>  					head + i, 0);
>>>  		}
>>>  	}
>>>
> 
> 
> --
> Best Regards,
> Yan, Zi

Re: [PATCH v4 6/7] mm: truncate: split huge page cache page to a non-zero order if possible.

[Ryan Roberts]

On 14/02/2024 16:19, Zi Yan wrote:
> On 14 Feb 2024, at 5:43, Ryan Roberts wrote:
> 
>> On 13/02/2024 21:55, Zi Yan wrote:
>>> From: Zi Yan <ziy@nvidia.com>
>>>
>>> To minimize the number of pages after a huge page truncation, we do not
>>> need to split it all the way down to order-0. The huge page has at most
>>> three parts, the part before offset, the part to be truncated, the part
>>> remaining at the end. Find the greatest common divisor of them to
>>> calculate the new page order from it, so we can split the huge
>>> page to this order and keep the remaining pages as large and as few as
>>> possible.
>>>
>>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>>> ---
>>>  mm/truncate.c | 21 +++++++++++++++++++--
>>>  1 file changed, 19 insertions(+), 2 deletions(-)
>>>
>>> diff --git a/mm/truncate.c b/mm/truncate.c
>>> index 725b150e47ac..49ddbbf7a617 100644
>>> --- a/mm/truncate.c
>>> +++ b/mm/truncate.c
>>> @@ -21,6 +21,7 @@
>>>  #include <linux/task_io_accounting_ops.h>
>>>  #include <linux/shmem_fs.h>
>>>  #include <linux/rmap.h>
>>> +#include <linux/gcd.h>
>>>  #include "internal.h"
>>>
>>>  /*
>>> @@ -210,7 +211,8 @@ int truncate_inode_folio(struct address_space *mapping, struct folio *folio)
>>>  bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>>>  {
>>>  	loff_t pos = folio_pos(folio);
>>> -	unsigned int offset, length;
>>> +	unsigned int offset, length, remaining;
>>> +	unsigned int new_order = folio_order(folio);
>>>
>>>  	if (pos < start)
>>>  		offset = start - pos;
>>> @@ -221,6 +223,7 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>>>  		length = length - offset;
>>>  	else
>>>  		length = end + 1 - pos - offset;
>>> +	remaining = folio_size(folio) - offset - length;
>>>
>>>  	folio_wait_writeback(folio);
>>>  	if (length == folio_size(folio)) {
>>> @@ -235,11 +238,25 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>>>  	 */
>>>  	folio_zero_range(folio, offset, length);
>>>
>>> +	/*
>>> +	 * Use the greatest common divisor of offset, length, and remaining
>>> +	 * as the smallest page size and compute the new order from it. So we
>>> +	 * can truncate a subpage as large as possible. Round up gcd to
>>> +	 * PAGE_SIZE, otherwise ilog2 can give -1 when gcd/PAGE_SIZE is 0.
>>> +	 */
>>> +	new_order = ilog2(round_up(gcd(gcd(offset, length), remaining),
>>> +				   PAGE_SIZE) / PAGE_SIZE);
>>
>> Given you have up to 2 regions remaining, isn't it possible that you want a
>> different order for both those regions (or even multiple orders within the same
>> region)? I guess you just choose gcd for simplicity?
> 
> Right. You raise the same concern as Hugh[1]. I am minimizing the call of
> split_huge_page_to_list_to_order() and you and Hugh want to minimize the
> number of folios after the split. Yours will give better outcome after split,
> but requires either multiple calls or a more sophisticated implementation
> of page split[2]. We probably can revisit this once splitting to any order
> gets wider use.

Yeah, fair enough. Sorry hadn't read Hugh's original feedback.

> 
>>> +
>>> +	/* order-1 THP not supported, downgrade to order-0 */
>>> +	if (new_order == 1)
>>> +		new_order = 0;
>>
>> I guess this would need to change if supporting order-1 file folios?
> 
> Right.
> 
>>> +
>>> +
>>>  	if (folio_has_private(folio))
>>>  		folio_invalidate(folio, offset, length);
>>>  	if (!folio_test_large(folio))
>>>  		return true;
>>> -	if (split_folio(folio) == 0)
>>> +	if (split_huge_page_to_list_to_order(&folio->page, NULL, new_order) == 0)
>>
>> I know you are discussing removing this patch, but since you created
>> split_folio_to_order() wouldn't that be better here?
> 
> Sure. Will change the patch locally.
> 
> [1] https://lore.kernel.org/linux-mm/9dd96da-efa2-5123-20d4-4992136ef3ad@google.com/
> [2] https://lore.kernel.org/linux-mm/0AC0520E-1BD2-497E-A7ED-05394400BFC9@nvidia.com/
> 
> --
> Best Regards,
> Yan, Zi

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 9804 bytes --]

On 14 Feb 2024, at 11:22, Ryan Roberts wrote:

> On 14/02/2024 16:11, Zi Yan wrote:
>> On 14 Feb 2024, at 5:38, Ryan Roberts wrote:
>>
>>> On 13/02/2024 21:55, Zi Yan wrote:
>>>> From: Zi Yan <ziy@nvidia.com>
>>>>
>>>> To split a THP to any lower order (except order-1) pages, we need to
>>>> reform THPs on subpages at given order and add page refcount based on the
>>>> new page order. Also we need to reinitialize page_deferred_list after
>>>> removing the page from the split_queue, otherwise a subsequent split will
>>>> see list corruption when checking the page_deferred_list again.
>>>>
>>>> It has many uses, like minimizing the number of pages after
>>>> truncating a huge pagecache page. For anonymous THPs, we can only split
>>>> them to order-0 like before until we add support for any size anonymous
>>>> THPs.
>>>
>>> multi-size THP is now upstream. Not sure if this comment still makes sense.
>> Will change it to reflect the fact that multi-size THP is already upstream.
>>
>>> Still its not completely clear to me how you would integrate this new machinery
>>> and decide what non-zero order to split anon THP to?
>>
>> Originally, it was developed along with my 1GB THP support. So it was intended
>> to split order-18 to order-9. But for now, like you and David said in the cover
>> letter email thread, we might not want to use it for anonymous large folios
>> until we find a necessary use case.
>>
>>>>
>>>> Order-1 folio is not supported because _deferred_list, which is used by
>>>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>>>> has subpage 0 and 1.
>>>>
>>>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>>>> ---
>>>>  include/linux/huge_mm.h |  21 +++++---
>>>>  mm/huge_memory.c        | 114 +++++++++++++++++++++++++++++++---------
>>>>  2 files changed, 101 insertions(+), 34 deletions(-)
>>>>
>>>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>>>> index 5adb86af35fc..de0c89105076 100644
>>>> --- a/include/linux/huge_mm.h
>>>> +++ b/include/linux/huge_mm.h
>>>> @@ -265,10 +265,11 @@ unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
>>>>
>>>>  void folio_prep_large_rmappable(struct folio *folio);
>>>>  bool can_split_folio(struct folio *folio, int *pextra_pins);
>>>> -int split_huge_page_to_list(struct page *page, struct list_head *list);
>>>> +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>>>> +		unsigned int new_order);
>>>>  static inline int split_huge_page(struct page *page)
>>>>  {
>>>> -	return split_huge_page_to_list(page, NULL);
>>>> +	return split_huge_page_to_list_to_order(page, NULL, 0);
>>>>  }
>>>>  void deferred_split_folio(struct folio *folio);
>>>>
>>>> @@ -422,7 +423,8 @@ can_split_folio(struct folio *folio, int *pextra_pins)
>>>>  	return false;
>>>>  }
>>>>  static inline int
>>>> -split_huge_page_to_list(struct page *page, struct list_head *list)
>>>> +split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>>>> +		unsigned int new_order)
>>>>  {
>>>>  	return 0;
>>>>  }
>>>> @@ -519,17 +521,20 @@ static inline bool thp_migration_supported(void)
>>>>  }
>>>>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>>>>
>>>> -static inline int split_folio_to_list(struct folio *folio,
>>>> -		struct list_head *list)
>>>> +static inline int split_folio_to_list_to_order(struct folio *folio,
>>>> +		struct list_head *list, int new_order)
>>>>  {
>>>> -	return split_huge_page_to_list(&folio->page, list);
>>>> +	return split_huge_page_to_list_to_order(&folio->page, list, new_order);
>>>>  }
>>>>
>>>> -static inline int split_folio(struct folio *folio)
>>>> +static inline int split_folio_to_order(struct folio *folio, int new_order)
>>>>  {
>>>> -	return split_folio_to_list(folio, NULL);
>>>> +	return split_folio_to_list_to_order(folio, NULL, new_order);
>>>>  }
>>>>
>>>> +#define split_folio_to_list(f, l) split_folio_to_list_to_order(f, l, 0)
>>>> +#define split_folio(f) split_folio_to_order(f, 0)
>>>> +
>>>>  /*
>>>>   * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
>>>>   * limitations in the implementation like arm64 MTE can override this to
>>>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>>>> index ad7133c97428..d0e555a8ea98 100644
>>>> --- a/mm/huge_memory.c
>>>> +++ b/mm/huge_memory.c
>>>> @@ -2718,11 +2718,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
>>>>
>>>>  static void unmap_folio(struct folio *folio)
>>>>  {
>>>> -	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD |
>>>> -		TTU_SYNC | TTU_BATCH_FLUSH;
>>>> +	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC |
>>>> +		TTU_BATCH_FLUSH;
>>>>
>>>>  	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
>>>>
>>>> +	if (folio_test_pmd_mappable(folio))
>>>> +		ttu_flags |= TTU_SPLIT_HUGE_PMD;
>>>
>>> Should we split this change out? I think it makes sense independent of this series?
>>>
>>
>> Sure. Since multi-size THP is upstream, this avoid unnecessary code path if
>> the THP is not PMD-mapped.
>>
>>>> +
>>>>  	/*
>>>>  	 * Anon pages need migration entries to preserve them, but file
>>>>  	 * pages can simply be left unmapped, then faulted back on demand.
>>>> @@ -2756,7 +2759,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>>>  		struct lruvec *lruvec, struct list_head *list)
>>>>  {
>>>>  	VM_BUG_ON_PAGE(!PageHead(head), head);
>>>> -	VM_BUG_ON_PAGE(PageCompound(tail), head);
>>>>  	VM_BUG_ON_PAGE(PageLRU(tail), head);
>>>>  	lockdep_assert_held(&lruvec->lru_lock);
>>>>
>>>> @@ -2777,7 +2779,8 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>>>  }
>>>>
>>>>  static void __split_huge_page_tail(struct folio *folio, int tail,
>>>> -		struct lruvec *lruvec, struct list_head *list)
>>>> +		struct lruvec *lruvec, struct list_head *list,
>>>> +		unsigned int new_order)
>>>>  {
>>>>  	struct page *head = &folio->page;
>>>>  	struct page *page_tail = head + tail;
>>>> @@ -2847,10 +2850,15 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>>>  	 * which needs correct compound_head().
>>>>  	 */
>>>>  	clear_compound_head(page_tail);
>>>> +	if (new_order) {
>>>> +		prep_compound_page(page_tail, new_order);
>>>> +		folio_prep_large_rmappable(page_folio(page_tail));
>>>> +	}
>>>>
>>>>  	/* Finally unfreeze refcount. Additional reference from page cache. */
>>>> -	page_ref_unfreeze(page_tail, 1 + (!folio_test_anon(folio) ||
>>>> -					  folio_test_swapcache(folio)));
>>>> +	page_ref_unfreeze(page_tail,
>>>> +		1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ?
>>>> +			     folio_nr_pages(page_folio(page_tail)) : 0));
>>>>
>>>>  	if (folio_test_young(folio))
>>>>  		folio_set_young(new_folio);
>>>> @@ -2868,7 +2876,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>>>  }
>>>>
>>>>  static void __split_huge_page(struct page *page, struct list_head *list,
>>>> -		pgoff_t end)
>>>> +		pgoff_t end, unsigned int new_order)
>>>>  {
>>>>  	struct folio *folio = page_folio(page);
>>>>  	struct page *head = &folio->page;
>>>> @@ -2877,10 +2885,11 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>>  	unsigned long offset = 0;
>>>>  	unsigned int nr = thp_nr_pages(head);
>>>>  	int i, nr_dropped = 0;
>>>> +	unsigned int new_nr = 1 << new_order;
>>>>  	int order = folio_order(folio);
>>>>
>>>>  	/* complete memcg works before add pages to LRU */
>>>> -	split_page_memcg(head, order, 0);
>>>> +	split_page_memcg(head, order, new_order);
>>>>
>>>>  	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
>>>>  		offset = swp_offset(folio->swap);
>>>> @@ -2893,8 +2902,8 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>>
>>>>  	ClearPageHasHWPoisoned(head);
>>>>
>>>> -	for (i = nr - 1; i >= 1; i--) {
>>>> -		__split_huge_page_tail(folio, i, lruvec, list);
>>>> +	for (i = nr - new_nr; i >= new_nr; i -= new_nr) {
>>>> +		__split_huge_page_tail(folio, i, lruvec, list, new_order);
>>>>  		/* Some pages can be beyond EOF: drop them from page cache */
>>>>  		if (head[i].index >= end) {
>>>>  			struct folio *tail = page_folio(head + i);
>>>> @@ -2910,29 +2919,41 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>>  			__xa_store(&head->mapping->i_pages, head[i].index,
>>>>  					head + i, 0);
>>>>  		} else if (swap_cache) {
>>>> +			/*
>>>> +			 * split anonymous THPs (including swapped out ones) to
>>>> +			 * non-zero order not supported
>>>> +			 */
>>>> +			VM_WARN_ONCE(new_order,
>>>> +				"Split swap-cached anon folio to non-0 order not supported");
>>>
>>> Why isn't it supported? Even if it's not supported, is this level the right
>>> place to enforce these kinds of policy decisions? I wonder if we should be
>>> leaving that to the higher level to decide?
>>
>> Is the swap-out small-size THP without splitting merged? This needs that patchset.
>
> No not yet. I have to respin it. Its on my todo list.
>
> I'm not sure I understand the dependency though?

IIUC, swap cache only supports one cluster size, HPAGE_PMD_NR, so splitting
a PMD-size swapcached folio will need to split a cluster to smaller ones, which
needs your patchset support. Let me know if I get it wrong.

>
>> You are right that a warning here is not appropriate. I will fail the splitting
>> if the folio is swapcached and going to be split into >0 order.
>>
>>>>  			__xa_store(&swap_cache->i_pages, offset + i,
>>>>  					head + i, 0);
>>>>  		}
>>>>  	}
>>>>
>>
>>
>> --
>> Best Regards,
>> Yan, Zi


--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 3023 bytes --]

On 14 Feb 2024, at 5:55, David Hildenbrand wrote:

> On 14.02.24 11:50, Ryan Roberts wrote:
>> On 13/02/2024 22:31, Zi Yan wrote:
>>> On 13 Feb 2024, at 17:21, David Hildenbrand wrote:
>>>
>>>> On 13.02.24 22:55, Zi Yan wrote:
>>>>> From: Zi Yan <ziy@nvidia.com>
>>>>>
>>>>> Hi all,
>>>>>
>>>>> File folio supports any order and multi-size THP is upstreamed[1], so both
>>>>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>>>>> only splits a huge page to order-0 pages, but splitting to orders higher than
>>>>> 0 is going to better utilize large folios. In addition, Large Block
>>>>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>>>>> splitting a large folio to any lower order folios and uses it during file
>>>>> folio truncate operations.
>>>>>
>>>>> For Patch 6, Hugh did not like my approach to minimize the number of
>>>>> folios for truncate[3]. I would like to get more feedback, especially
>>>>> from FS people, on it to decide whether to keep it or not.
>>>>
>>>> I'm curious, would it make sense to exclude the "more" controversial parts (i.e., patch #6) for now, and focus on the XFS use case only?
>>>
>>> Sure. Patch 6 was there to make use of split_huge_page_to_list_to_order().
>>> Now we have multi-size THP and XFS use cases, it can be dropped.
>>
>> What are your plans for how to determine when to split THP and to what order? I
>> don't see anything in this series that would split anon THP to non-zero order?
>>
>> We have talked about using hints from user space in the past (e.g.  mremap,
>> munmap, madvise, etc). But chrome has a use case where it temporarily mprotects
>> a single (4K) page as part of garbage collection (IIRC). If you eagerly split on
>> that hint, you will have lost the benefits of the large folio when it later
>> mprotects back to the original setting.
>
> Not only that, splitting will make some of these operations more expensive, possibly with no actual benefit.
>
>>
>> I guess David will suggest this would be a good use case for the khugepaged-lite
>> machanism we have been talking about. I dunno - it seems wasteful to split then
>> collapse again.
>
> I agree. mprotect() and even madvise(), ... might not be good candidates for splitting. mremap() likely is, if the folio is mapped exclusively. MADV_DONTNEED/munmap()/mlock() might be good candidates (again, if mapped exclusively). This will need a lot of thought I'm afraid (as you say, deferred splitting is another example).

My initial use was for splitting 1GB THP to 2MB THP, but 1GB THP is not upstream
yet. So for now, this might only be used by XFS. For anonymous large folios,
we will use this when we find a justified use case. What I can think of is
when a PMD-mapped THP happens to be split and the resulting order can be a HW/SW
favored order, like 64KB or 32KB (to be able to use contig PTE), we split
to that order, otherwise, we still split to order-0.

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 5/7] mm: thp: split huge page to any lower order pages (except order-1).

[Ryan Roberts]

On 14/02/2024 16:28, Zi Yan wrote:
> On 14 Feb 2024, at 11:22, Ryan Roberts wrote:
> 
>> On 14/02/2024 16:11, Zi Yan wrote:
>>> On 14 Feb 2024, at 5:38, Ryan Roberts wrote:
>>>
>>>> On 13/02/2024 21:55, Zi Yan wrote:
>>>>> From: Zi Yan <ziy@nvidia.com>
>>>>>
>>>>> To split a THP to any lower order (except order-1) pages, we need to
>>>>> reform THPs on subpages at given order and add page refcount based on the
>>>>> new page order. Also we need to reinitialize page_deferred_list after
>>>>> removing the page from the split_queue, otherwise a subsequent split will
>>>>> see list corruption when checking the page_deferred_list again.
>>>>>
>>>>> It has many uses, like minimizing the number of pages after
>>>>> truncating a huge pagecache page. For anonymous THPs, we can only split
>>>>> them to order-0 like before until we add support for any size anonymous
>>>>> THPs.
>>>>
>>>> multi-size THP is now upstream. Not sure if this comment still makes sense.
>>> Will change it to reflect the fact that multi-size THP is already upstream.
>>>
>>>> Still its not completely clear to me how you would integrate this new machinery
>>>> and decide what non-zero order to split anon THP to?
>>>
>>> Originally, it was developed along with my 1GB THP support. So it was intended
>>> to split order-18 to order-9. But for now, like you and David said in the cover
>>> letter email thread, we might not want to use it for anonymous large folios
>>> until we find a necessary use case.
>>>
>>>>>
>>>>> Order-1 folio is not supported because _deferred_list, which is used by
>>>>> partially mapped folios, is stored in subpage 2 and an order-1 folio only
>>>>> has subpage 0 and 1.
>>>>>
>>>>> Signed-off-by: Zi Yan <ziy@nvidia.com>
>>>>> ---
>>>>>  include/linux/huge_mm.h |  21 +++++---
>>>>>  mm/huge_memory.c        | 114 +++++++++++++++++++++++++++++++---------
>>>>>  2 files changed, 101 insertions(+), 34 deletions(-)
>>>>>
>>>>> diff --git a/include/linux/huge_mm.h b/include/linux/huge_mm.h
>>>>> index 5adb86af35fc..de0c89105076 100644
>>>>> --- a/include/linux/huge_mm.h
>>>>> +++ b/include/linux/huge_mm.h
>>>>> @@ -265,10 +265,11 @@ unsigned long thp_get_unmapped_area(struct file *filp, unsigned long addr,
>>>>>
>>>>>  void folio_prep_large_rmappable(struct folio *folio);
>>>>>  bool can_split_folio(struct folio *folio, int *pextra_pins);
>>>>> -int split_huge_page_to_list(struct page *page, struct list_head *list);
>>>>> +int split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>>>>> +		unsigned int new_order);
>>>>>  static inline int split_huge_page(struct page *page)
>>>>>  {
>>>>> -	return split_huge_page_to_list(page, NULL);
>>>>> +	return split_huge_page_to_list_to_order(page, NULL, 0);
>>>>>  }
>>>>>  void deferred_split_folio(struct folio *folio);
>>>>>
>>>>> @@ -422,7 +423,8 @@ can_split_folio(struct folio *folio, int *pextra_pins)
>>>>>  	return false;
>>>>>  }
>>>>>  static inline int
>>>>> -split_huge_page_to_list(struct page *page, struct list_head *list)
>>>>> +split_huge_page_to_list_to_order(struct page *page, struct list_head *list,
>>>>> +		unsigned int new_order)
>>>>>  {
>>>>>  	return 0;
>>>>>  }
>>>>> @@ -519,17 +521,20 @@ static inline bool thp_migration_supported(void)
>>>>>  }
>>>>>  #endif /* CONFIG_TRANSPARENT_HUGEPAGE */
>>>>>
>>>>> -static inline int split_folio_to_list(struct folio *folio,
>>>>> -		struct list_head *list)
>>>>> +static inline int split_folio_to_list_to_order(struct folio *folio,
>>>>> +		struct list_head *list, int new_order)
>>>>>  {
>>>>> -	return split_huge_page_to_list(&folio->page, list);
>>>>> +	return split_huge_page_to_list_to_order(&folio->page, list, new_order);
>>>>>  }
>>>>>
>>>>> -static inline int split_folio(struct folio *folio)
>>>>> +static inline int split_folio_to_order(struct folio *folio, int new_order)
>>>>>  {
>>>>> -	return split_folio_to_list(folio, NULL);
>>>>> +	return split_folio_to_list_to_order(folio, NULL, new_order);
>>>>>  }
>>>>>
>>>>> +#define split_folio_to_list(f, l) split_folio_to_list_to_order(f, l, 0)
>>>>> +#define split_folio(f) split_folio_to_order(f, 0)
>>>>> +
>>>>>  /*
>>>>>   * archs that select ARCH_WANTS_THP_SWAP but don't support THP_SWP due to
>>>>>   * limitations in the implementation like arm64 MTE can override this to
>>>>> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
>>>>> index ad7133c97428..d0e555a8ea98 100644
>>>>> --- a/mm/huge_memory.c
>>>>> +++ b/mm/huge_memory.c
>>>>> @@ -2718,11 +2718,14 @@ void vma_adjust_trans_huge(struct vm_area_struct *vma,
>>>>>
>>>>>  static void unmap_folio(struct folio *folio)
>>>>>  {
>>>>> -	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SPLIT_HUGE_PMD |
>>>>> -		TTU_SYNC | TTU_BATCH_FLUSH;
>>>>> +	enum ttu_flags ttu_flags = TTU_RMAP_LOCKED | TTU_SYNC |
>>>>> +		TTU_BATCH_FLUSH;
>>>>>
>>>>>  	VM_BUG_ON_FOLIO(!folio_test_large(folio), folio);
>>>>>
>>>>> +	if (folio_test_pmd_mappable(folio))
>>>>> +		ttu_flags |= TTU_SPLIT_HUGE_PMD;
>>>>
>>>> Should we split this change out? I think it makes sense independent of this series?
>>>>
>>>
>>> Sure. Since multi-size THP is upstream, this avoid unnecessary code path if
>>> the THP is not PMD-mapped.
>>>
>>>>> +
>>>>>  	/*
>>>>>  	 * Anon pages need migration entries to preserve them, but file
>>>>>  	 * pages can simply be left unmapped, then faulted back on demand.
>>>>> @@ -2756,7 +2759,6 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>>>>  		struct lruvec *lruvec, struct list_head *list)
>>>>>  {
>>>>>  	VM_BUG_ON_PAGE(!PageHead(head), head);
>>>>> -	VM_BUG_ON_PAGE(PageCompound(tail), head);
>>>>>  	VM_BUG_ON_PAGE(PageLRU(tail), head);
>>>>>  	lockdep_assert_held(&lruvec->lru_lock);
>>>>>
>>>>> @@ -2777,7 +2779,8 @@ static void lru_add_page_tail(struct page *head, struct page *tail,
>>>>>  }
>>>>>
>>>>>  static void __split_huge_page_tail(struct folio *folio, int tail,
>>>>> -		struct lruvec *lruvec, struct list_head *list)
>>>>> +		struct lruvec *lruvec, struct list_head *list,
>>>>> +		unsigned int new_order)
>>>>>  {
>>>>>  	struct page *head = &folio->page;
>>>>>  	struct page *page_tail = head + tail;
>>>>> @@ -2847,10 +2850,15 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>>>>  	 * which needs correct compound_head().
>>>>>  	 */
>>>>>  	clear_compound_head(page_tail);
>>>>> +	if (new_order) {
>>>>> +		prep_compound_page(page_tail, new_order);
>>>>> +		folio_prep_large_rmappable(page_folio(page_tail));
>>>>> +	}
>>>>>
>>>>>  	/* Finally unfreeze refcount. Additional reference from page cache. */
>>>>> -	page_ref_unfreeze(page_tail, 1 + (!folio_test_anon(folio) ||
>>>>> -					  folio_test_swapcache(folio)));
>>>>> +	page_ref_unfreeze(page_tail,
>>>>> +		1 + ((!folio_test_anon(folio) || folio_test_swapcache(folio)) ?
>>>>> +			     folio_nr_pages(page_folio(page_tail)) : 0));
>>>>>
>>>>>  	if (folio_test_young(folio))
>>>>>  		folio_set_young(new_folio);
>>>>> @@ -2868,7 +2876,7 @@ static void __split_huge_page_tail(struct folio *folio, int tail,
>>>>>  }
>>>>>
>>>>>  static void __split_huge_page(struct page *page, struct list_head *list,
>>>>> -		pgoff_t end)
>>>>> +		pgoff_t end, unsigned int new_order)
>>>>>  {
>>>>>  	struct folio *folio = page_folio(page);
>>>>>  	struct page *head = &folio->page;
>>>>> @@ -2877,10 +2885,11 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>>>  	unsigned long offset = 0;
>>>>>  	unsigned int nr = thp_nr_pages(head);
>>>>>  	int i, nr_dropped = 0;
>>>>> +	unsigned int new_nr = 1 << new_order;
>>>>>  	int order = folio_order(folio);
>>>>>
>>>>>  	/* complete memcg works before add pages to LRU */
>>>>> -	split_page_memcg(head, order, 0);
>>>>> +	split_page_memcg(head, order, new_order);
>>>>>
>>>>>  	if (folio_test_anon(folio) && folio_test_swapcache(folio)) {
>>>>>  		offset = swp_offset(folio->swap);
>>>>> @@ -2893,8 +2902,8 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>>>
>>>>>  	ClearPageHasHWPoisoned(head);
>>>>>
>>>>> -	for (i = nr - 1; i >= 1; i--) {
>>>>> -		__split_huge_page_tail(folio, i, lruvec, list);
>>>>> +	for (i = nr - new_nr; i >= new_nr; i -= new_nr) {
>>>>> +		__split_huge_page_tail(folio, i, lruvec, list, new_order);
>>>>>  		/* Some pages can be beyond EOF: drop them from page cache */
>>>>>  		if (head[i].index >= end) {
>>>>>  			struct folio *tail = page_folio(head + i);
>>>>> @@ -2910,29 +2919,41 @@ static void __split_huge_page(struct page *page, struct list_head *list,
>>>>>  			__xa_store(&head->mapping->i_pages, head[i].index,
>>>>>  					head + i, 0);
>>>>>  		} else if (swap_cache) {
>>>>> +			/*
>>>>> +			 * split anonymous THPs (including swapped out ones) to
>>>>> +			 * non-zero order not supported
>>>>> +			 */
>>>>> +			VM_WARN_ONCE(new_order,
>>>>> +				"Split swap-cached anon folio to non-0 order not supported");
>>>>
>>>> Why isn't it supported? Even if it's not supported, is this level the right
>>>> place to enforce these kinds of policy decisions? I wonder if we should be
>>>> leaving that to the higher level to decide?
>>>
>>> Is the swap-out small-size THP without splitting merged? This needs that patchset.
>>
>> No not yet. I have to respin it. Its on my todo list.
>>
>> I'm not sure I understand the dependency though?
> 
> IIUC, swap cache only supports one cluster size, HPAGE_PMD_NR, so splitting
> a PMD-size swapcached folio will need to split a cluster to smaller ones, which
> needs your patchset support. Let me know if I get it wrong.

Ahh yeah, sorry, obvious now that you've spelled it out - thanks!

> 
>>
>>> You are right that a warning here is not appropriate. I will fail the splitting
>>> if the folio is swapcached and going to be split into >0 order.
>>>
>>>>>  			__xa_store(&swap_cache->i_pages, offset + i,
>>>>>  					head + i, 0);
>>>>>  		}
>>>>>  	}
>>>>>
>>>
>>>
>>> --
>>> Best Regards,
>>> Yan, Zi
> 
> 
> --
> Best Regards,
> Yan, Zi

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 745 bytes --]

Hi Pankaj,

On 13 Feb 2024, at 16:55, Zi Yan wrote:

> From: Zi Yan <ziy@nvidia.com>
>
> Hi all,
>
> File folio supports any order and multi-size THP is upstreamed[1], so both
> file and anonymous folios can be >0 order. Currently, split_huge_page()
> only splits a huge page to order-0 pages, but splitting to orders higher than
> 0 is going to better utilize large folios. In addition, Large Block
> Sizes in XFS support would benefit from it[2]. This patchset adds support for

Just talked to Matthew about his order-1 pagecache folio, I am planning to
grab that into this one, so that I can remove the restriction in my patches
and you guys do not need to do that in your patchset. Let me know if it works
for you.

--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Pankaj Raghav (Samsung)]

On Wed, Feb 14, 2024 at 12:18:14PM -0500, Zi Yan wrote:
> Hi Pankaj,
> 
> On 13 Feb 2024, at 16:55, Zi Yan wrote:
> 
> > From: Zi Yan <ziy@nvidia.com>
> >
> > Hi all,
> >
> > File folio supports any order and multi-size THP is upstreamed[1], so both
> > file and anonymous folios can be >0 order. Currently, split_huge_page()
> > only splits a huge page to order-0 pages, but splitting to orders higher than
> > 0 is going to better utilize large folios. In addition, Large Block
> > Sizes in XFS support would benefit from it[2]. This patchset adds support for
> 
> Just talked to Matthew about his order-1 pagecache folio, I am planning to
> grab that into this one, so that I can remove the restriction in my patches
> and you guys do not need to do that in your patchset. Let me know if it works
> for you.
> 

Cool! Sounds good to me. I generally base my baseline based on -rcs. So
I might include it while sending for reviews until 6.8. I will remove
that patch once this gets in for the 6.9 merge window.

Thanks.
--
Pankaj

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Pankaj Raghav (Samsung)]

Hi Zi Yan,

On Tue, Feb 13, 2024 at 04:55:13PM -0500, Zi Yan wrote:
> From: Zi Yan <ziy@nvidia.com>
> 
> Hi all,
> 
> File folio supports any order and multi-size THP is upstreamed[1], so both
> file and anonymous folios can be >0 order. Currently, split_huge_page()
> only splits a huge page to order-0 pages, but splitting to orders higher than
> 0 is going to better utilize large folios. In addition, Large Block
> Sizes in XFS support would benefit from it[2]. This patchset adds support for
> splitting a large folio to any lower order folios and uses it during file
> folio truncate operations.

I added your patches on top of my patches, but removed patch 6 and I
added this instead:

diff --git a/mm/truncate.c b/mm/truncate.c
index 725b150e47ac..dd07e2e327a8 100644
--- a/mm/truncate.c
+++ b/mm/truncate.c
@@ -239,7 +239,8 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
                folio_invalidate(folio, offset, length);
        if (!folio_test_large(folio))
                return true;
-       if (split_folio(folio) == 0)
+       if (split_folio_to_order(folio,
+                                mapping_min_folio_order(folio->mapping)) == 0)
                return true;
        if (folio_test_dirty(folio))
                return false;

I ran genric/476 fstest[1] with SOAK_DURATION set to 360 seconds. This
test uses fstress to do a lot of writes, truncate operations, etc. I ran
this on XFS with **64k block size on a 4k page size system**.

I recorded the vm event for split page and this was the result I got:

Before your patches:
root@debian:~/xfstests# cat /proc/vmstat  | grep split
thp_split_page 0
thp_split_page_failed 5819

After your patches:
root@debian:~/xfstests# cat /proc/vmstat  | grep split
thp_split_page 5846
thp_split_page_failed 20

Your patch series definitely helps with splitting the folios while still
maintaining the min_folio_order that LBS requires.

We are still discussing how to quantify this benefit in terms of some
metric with this support. If you have some ideas here, let me know.

I will run the whole xfstests tonight to check for any regressions.

--
Pankaj

[1] https://github.com/kdave/xfstests/blob/master/tests/generic/476

Re: [PATCH v4 0/7] Split a folio to any lower order folios

[Zi Yan]

[-- Attachment #1: Type: text/plain, Size: 3019 bytes --]

On 16 Feb 2024, at 5:06, Pankaj Raghav (Samsung) wrote:

> Hi Zi Yan,
>
> On Tue, Feb 13, 2024 at 04:55:13PM -0500, Zi Yan wrote:
>> From: Zi Yan <ziy@nvidia.com>
>>
>> Hi all,
>>
>> File folio supports any order and multi-size THP is upstreamed[1], so both
>> file and anonymous folios can be >0 order. Currently, split_huge_page()
>> only splits a huge page to order-0 pages, but splitting to orders higher than
>> 0 is going to better utilize large folios. In addition, Large Block
>> Sizes in XFS support would benefit from it[2]. This patchset adds support for
>> splitting a large folio to any lower order folios and uses it during file
>> folio truncate operations.
>
> I added your patches on top of my patches, but removed patch 6 and I
> added this instead:
>
> diff --git a/mm/truncate.c b/mm/truncate.c
> index 725b150e47ac..dd07e2e327a8 100644
> --- a/mm/truncate.c
> +++ b/mm/truncate.c
> @@ -239,7 +239,8 @@ bool truncate_inode_partial_folio(struct folio *folio, loff_t start, loff_t end)
>                 folio_invalidate(folio, offset, length);
>         if (!folio_test_large(folio))
>                 return true;
> -       if (split_folio(folio) == 0)
> +       if (split_folio_to_order(folio,
> +                                mapping_min_folio_order(folio->mapping)) == 0)
>                 return true;
>         if (folio_test_dirty(folio))
>                 return false;
>
> I ran genric/476 fstest[1] with SOAK_DURATION set to 360 seconds. This
> test uses fstress to do a lot of writes, truncate operations, etc. I ran
> this on XFS with **64k block size on a 4k page size system**.
>
> I recorded the vm event for split page and this was the result I got:
>
> Before your patches:
> root@debian:~/xfstests# cat /proc/vmstat  | grep split
> thp_split_page 0
> thp_split_page_failed 5819
>
> After your patches:
> root@debian:~/xfstests# cat /proc/vmstat  | grep split
> thp_split_page 5846
> thp_split_page_failed 20
>
> Your patch series definitely helps with splitting the folios while still
> maintaining the min_folio_order that LBS requires.

Sounds great! Thanks for testing.

>
> We are still discussing how to quantify this benefit in terms of some
> metric with this support. If you have some ideas here, let me know.

From my understanding, the benefit will come from that page cache folio
size is bigger with LBS (plus this patchset) after truncate. I assume any
benchmark testing read/write throughput after truncate operations might
be helpful.

>
> I will run the whole xfstests tonight to check for any regressions.

Can you use the update patches from: https://github.com/x-y-z/linux-1gb-thp/tree/split_thp_to_any_order_v5-mm-everything-2024-02-16-01-35? It contains
changes and fixes based on the feedback from this version. I am planning
to send this new version out soon.

>
> --
> Pankaj
>
> [1] https://github.com/kdave/xfstests/blob/master/tests/generic/476


--
Best Regards,
Yan, Zi

[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 854 bytes --]