[PATCH v13b 00/18] Convert memcg to folios

[PATCH v13b 00/18] Convert memcg to folios

[Matthew Wilcox (Oracle)]

This is the part of the v13 patch series which deals with converting
(most of) the memcg interfaces to work with folios instead of pages.
A few interfaces are not changed due to having exclusively or many
users which are not going to be converted to folios soon.  Eventually,
all memcg interfaces should be converted to folios as all accounting of
memory is done on and with the head page.

A few of these patches ran into trouble with the build bots, and those
problems have been corrected.  I imagine that posting this as a patch
series independently of v13a will cause the build bots to report errors
as I don't know how to tell them that this series depends on that series.

Matthew Wilcox (Oracle) (18):
  mm: Add folio_nid()
  mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()
  mm/memcg: Use the node id in mem_cgroup_update_tree()
  mm/memcg: Remove soft_limit_tree_node()
  mm/memcg: Convert memcg_check_events to take a node ID
  mm/memcg: Add folio_memcg() and related functions
  mm/memcg: Convert commit_charge() to take a folio
  mm/memcg: Convert mem_cgroup_charge() to take a folio
  mm/memcg: Convert uncharge_page() to uncharge_folio()
  mm/memcg: Convert mem_cgroup_uncharge() to take a folio
  mm/memcg: Convert mem_cgroup_migrate() to take folios
  mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio
  mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
  mm/memcg: Convert mem_cgroup_move_account() to use a folio
  mm/memcg: Add folio_lruvec()
  mm/memcg: Add folio_lruvec_lock() and similar functions
  mm/memcg: Add folio_lruvec_relock_irq() and
    folio_lruvec_relock_irqsave()
  mm/workingset: Convert workingset_activation to take a folio

 include/linux/memcontrol.h       | 223 ++++++++++++---------
 include/linux/mm.h               |   5 +
 include/linux/swap.h             |   2 +-
 include/trace/events/writeback.h |   8 +-
 kernel/events/uprobes.c          |   3 +-
 mm/compaction.c                  |   4 +-
 mm/filemap.c                     |   8 +-
 mm/huge_memory.c                 |   7 +-
 mm/khugepaged.c                  |   8 +-
 mm/ksm.c                         |   3 +-
 mm/memcontrol.c                  | 323 +++++++++++++++----------------
 mm/memory-failure.c              |   2 +-
 mm/memory.c                      |   9 +-
 mm/memremap.c                    |   2 +-
 mm/migrate.c                     |   6 +-
 mm/mlock.c                       |   3 +-
 mm/page_alloc.c                  |   2 +-
 mm/rmap.c                        |   2 +-
 mm/shmem.c                       |   7 +-
 mm/swap.c                        |  26 ++-
 mm/userfaultfd.c                 |   2 +-
 mm/vmscan.c                      |   8 +-
 mm/workingset.c                  |  10 +-
 23 files changed, 358 insertions(+), 315 deletions(-)

-- 
2.30.2

[PATCH v13 01/18] mm: Add folio_nid()

[Matthew Wilcox (Oracle)]

This is the folio equivalent of page_to_nid().

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/mm.h | 5 +++++
 1 file changed, 5 insertions(+)

diff --git a/include/linux/mm.h b/include/linux/mm.h
index a856c078e040..80f27eb151ba 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1427,6 +1427,11 @@ static inline int page_to_nid(const struct page *page)
 }
 #endif
 
+static inline int folio_nid(const struct folio *folio)
+{
+	return page_to_nid(&folio->page);
+}
+
 #ifdef CONFIG_NUMA_BALANCING
 static inline int cpu_pid_to_cpupid(int cpu, int pid)
 {
-- 
2.30.2

[PATCH v13 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()

[Matthew Wilcox (Oracle)]

The last use of 'page' was removed by commit 468c398233da ("mm:
memcontrol: switch to native NR_ANON_THPS counter"), so we can now remove
the parameter from the function.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
---
 mm/memcontrol.c | 11 +++++------
 1 file changed, 5 insertions(+), 6 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index ae1f5d0cb581..ee892daecb8b 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -831,7 +831,6 @@ static unsigned long memcg_events_local(struct mem_cgroup *memcg, int event)
 }
 
 static void mem_cgroup_charge_statistics(struct mem_cgroup *memcg,
-					 struct page *page,
 					 int nr_pages)
 {
 	/* pagein of a big page is an event. So, ignore page size */
@@ -5692,9 +5691,9 @@ static int mem_cgroup_move_account(struct page *page,
 	ret = 0;
 
 	local_irq_disable();
-	mem_cgroup_charge_statistics(to, page, nr_pages);
+	mem_cgroup_charge_statistics(to, nr_pages);
 	memcg_check_events(to, page);
-	mem_cgroup_charge_statistics(from, page, -nr_pages);
+	mem_cgroup_charge_statistics(from, -nr_pages);
 	memcg_check_events(from, page);
 	local_irq_enable();
 out_unlock:
@@ -6715,7 +6714,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 	commit_charge(page, memcg);
 
 	local_irq_disable();
-	mem_cgroup_charge_statistics(memcg, page, nr_pages);
+	mem_cgroup_charge_statistics(memcg, nr_pages);
 	memcg_check_events(memcg, page);
 	local_irq_enable();
 out:
@@ -7006,7 +7005,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	commit_charge(newpage, memcg);
 
 	local_irq_save(flags);
-	mem_cgroup_charge_statistics(memcg, newpage, nr_pages);
+	mem_cgroup_charge_statistics(memcg, nr_pages);
 	memcg_check_events(memcg, newpage);
 	local_irq_restore(flags);
 }
@@ -7236,7 +7235,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
 	 * only synchronisation we have for updating the per-CPU variables.
 	 */
 	VM_BUG_ON(!irqs_disabled());
-	mem_cgroup_charge_statistics(memcg, page, -nr_entries);
+	mem_cgroup_charge_statistics(memcg, -nr_entries);
 	memcg_check_events(memcg, page);
 
 	css_put(&memcg->css);
-- 
2.30.2

[PATCH v13 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()

[Matthew Wilcox (Oracle)]

By using the node id in mem_cgroup_update_tree(), we can delete
soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo().  Saves 42
bytes of kernel text on my config.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 mm/memcontrol.c | 24 ++++--------------------
 1 file changed, 4 insertions(+), 20 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index ee892daecb8b..d57ff5c5d330 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -451,28 +451,12 @@ ino_t page_cgroup_ino(struct page *page)
 	return ino;
 }
 
-static struct mem_cgroup_per_node *
-mem_cgroup_page_nodeinfo(struct mem_cgroup *memcg, struct page *page)
-{
-	int nid = page_to_nid(page);
-
-	return memcg->nodeinfo[nid];
-}
-
 static struct mem_cgroup_tree_per_node *
 soft_limit_tree_node(int nid)
 {
 	return soft_limit_tree.rb_tree_per_node[nid];
 }
 
-static struct mem_cgroup_tree_per_node *
-soft_limit_tree_from_page(struct page *page)
-{
-	int nid = page_to_nid(page);
-
-	return soft_limit_tree.rb_tree_per_node[nid];
-}
-
 static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
 					 struct mem_cgroup_tree_per_node *mctz,
 					 unsigned long new_usage_in_excess)
@@ -543,13 +527,13 @@ static unsigned long soft_limit_excess(struct mem_cgroup *memcg)
 	return excess;
 }
 
-static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
+static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
 {
 	unsigned long excess;
 	struct mem_cgroup_per_node *mz;
 	struct mem_cgroup_tree_per_node *mctz;
 
-	mctz = soft_limit_tree_from_page(page);
+	mctz = soft_limit_tree_node(nid);
 	if (!mctz)
 		return;
 	/*
@@ -557,7 +541,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
 	 * because their event counter is not touched.
 	 */
 	for (; memcg; memcg = parent_mem_cgroup(memcg)) {
-		mz = mem_cgroup_page_nodeinfo(memcg, page);
+		mz = memcg->nodeinfo[nid];
 		excess = soft_limit_excess(memcg);
 		/*
 		 * We have to update the tree if mz is on RB-tree or
@@ -884,7 +868,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
 						MEM_CGROUP_TARGET_SOFTLIMIT);
 		mem_cgroup_threshold(memcg);
 		if (unlikely(do_softlimit))
-			mem_cgroup_update_tree(memcg, page);
+			mem_cgroup_update_tree(memcg, page_to_nid(page));
 	}
 }
 
-- 
2.30.2

[PATCH v13 04/18] mm/memcg: Remove soft_limit_tree_node()

[Matthew Wilcox (Oracle)]

Opencode this one-line function in its three callers.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 mm/memcontrol.c | 12 +++---------
 1 file changed, 3 insertions(+), 9 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index d57ff5c5d330..f70e33d691aa 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -451,12 +451,6 @@ ino_t page_cgroup_ino(struct page *page)
 	return ino;
 }
 
-static struct mem_cgroup_tree_per_node *
-soft_limit_tree_node(int nid)
-{
-	return soft_limit_tree.rb_tree_per_node[nid];
-}
-
 static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
 					 struct mem_cgroup_tree_per_node *mctz,
 					 unsigned long new_usage_in_excess)
@@ -533,7 +527,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
 	struct mem_cgroup_per_node *mz;
 	struct mem_cgroup_tree_per_node *mctz;
 
-	mctz = soft_limit_tree_node(nid);
+	mctz = soft_limit_tree.rb_tree_per_node[nid];
 	if (!mctz)
 		return;
 	/*
@@ -572,7 +566,7 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
 
 	for_each_node(nid) {
 		mz = memcg->nodeinfo[nid];
-		mctz = soft_limit_tree_node(nid);
+		mctz = soft_limit_tree.rb_tree_per_node[nid];
 		if (mctz)
 			mem_cgroup_remove_exceeded(mz, mctz);
 	}
@@ -3420,7 +3414,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 	if (order > 0)
 		return 0;
 
-	mctz = soft_limit_tree_node(pgdat->node_id);
+	mctz = soft_limit_tree.rb_tree_per_node[pgdat->node_id];
 
 	/*
 	 * Do not even bother to check the largest node if the root
-- 
2.30.2

[PATCH v13 05/18] mm/memcg: Convert memcg_check_events to take a node ID

[Matthew Wilcox (Oracle)]

memcg_check_events only uses the page's nid, so call page_to_nid in the
callers to make the interface easier to understand.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 mm/memcontrol.c | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index f70e33d691aa..1a049bfa0e0a 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -851,7 +851,7 @@ static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
  * Check events in order.
  *
  */
-static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
+static void memcg_check_events(struct mem_cgroup *memcg, int nid)
 {
 	/* threshold event is triggered in finer grain than soft limit */
 	if (unlikely(mem_cgroup_event_ratelimit(memcg,
@@ -862,7 +862,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
 						MEM_CGROUP_TARGET_SOFTLIMIT);
 		mem_cgroup_threshold(memcg);
 		if (unlikely(do_softlimit))
-			mem_cgroup_update_tree(memcg, page_to_nid(page));
+			mem_cgroup_update_tree(memcg, nid);
 	}
 }
 
@@ -5578,7 +5578,7 @@ static int mem_cgroup_move_account(struct page *page,
 	struct lruvec *from_vec, *to_vec;
 	struct pglist_data *pgdat;
 	unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
-	int ret;
+	int nid, ret;
 
 	VM_BUG_ON(from == to);
 	VM_BUG_ON_PAGE(PageLRU(page), page);
@@ -5667,12 +5667,13 @@ static int mem_cgroup_move_account(struct page *page,
 	__unlock_page_memcg(from);
 
 	ret = 0;
+	nid = page_to_nid(page);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(to, nr_pages);
-	memcg_check_events(to, page);
+	memcg_check_events(to, nid);
 	mem_cgroup_charge_statistics(from, -nr_pages);
-	memcg_check_events(from, page);
+	memcg_check_events(from, nid);
 	local_irq_enable();
 out_unlock:
 	unlock_page(page);
@@ -6693,7 +6694,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, page);
+	memcg_check_events(memcg, page_to_nid(page));
 	local_irq_enable();
 out:
 	return ret;
@@ -6801,7 +6802,7 @@ struct uncharge_gather {
 	unsigned long nr_memory;
 	unsigned long pgpgout;
 	unsigned long nr_kmem;
-	struct page *dummy_page;
+	int nid;
 };
 
 static inline void uncharge_gather_clear(struct uncharge_gather *ug)
@@ -6825,7 +6826,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 	local_irq_save(flags);
 	__count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout);
 	__this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory);
-	memcg_check_events(ug->memcg, ug->dummy_page);
+	memcg_check_events(ug->memcg, ug->nid);
 	local_irq_restore(flags);
 
 	/* drop reference from uncharge_page */
@@ -6866,7 +6867,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 			uncharge_gather_clear(ug);
 		}
 		ug->memcg = memcg;
-		ug->dummy_page = page;
+		ug->nid = page_to_nid(page);
 
 		/* pairs with css_put in uncharge_batch */
 		css_get(&memcg->css);
@@ -6984,7 +6985,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 
 	local_irq_save(flags);
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, newpage);
+	memcg_check_events(memcg, page_to_nid(newpage));
 	local_irq_restore(flags);
 }
 
@@ -7214,7 +7215,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
 	 */
 	VM_BUG_ON(!irqs_disabled());
 	mem_cgroup_charge_statistics(memcg, -nr_entries);
-	memcg_check_events(memcg, page);
+	memcg_check_events(memcg, page_to_nid(page));
 
 	css_put(&memcg->css);
 }
-- 
2.30.2

[PATCH v13 06/18] mm/memcg: Add folio_memcg() and related functions

[Matthew Wilcox (Oracle)]

memcg information is only stored in the head page, so the memcg
subsystem needs to assure that all accesses are to the head page.
The first step is converting page_memcg() to folio_memcg().

The callers of page_memcg() and PageMemcgKmem() are not yet ready to be
converted to use folios, so retain them as wrappers around folio_memcg()
and folio_memcg_kmem().  They will be converted in a later patch set.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/memcontrol.h | 109 ++++++++++++++++++++++---------------
 mm/memcontrol.c            |  21 ++++---
 2 files changed, 77 insertions(+), 53 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index bfe5c486f4ad..f40c4e0b0431 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -372,6 +372,7 @@ enum page_memcg_data_flags {
 #define MEMCG_DATA_FLAGS_MASK (__NR_MEMCG_DATA_FLAGS - 1)
 
 static inline bool PageMemcgKmem(struct page *page);
+static inline bool folio_memcg_kmem(struct folio *folio);
 
 /*
  * After the initialization objcg->memcg is always pointing at
@@ -386,73 +387,77 @@ static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
 }
 
 /*
- * __page_memcg - get the memory cgroup associated with a non-kmem page
- * @page: a pointer to the page struct
+ * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages or
- * kmem pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios or
+ * kmem folios.
  */
-static inline struct mem_cgroup *__page_memcg(struct page *page)
+static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
 {
-	unsigned long memcg_data = page->memcg_data;
+	unsigned long memcg_data = folio->memcg_data;
 
-	VM_BUG_ON_PAGE(PageSlab(page), page);
-	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
-	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
+	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
+	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
+	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
 
 	return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
 }
 
 /*
- * __page_objcg - get the object cgroup associated with a kmem page
- * @page: a pointer to the page struct
+ * __folio_objcg - get the object cgroup associated with a kmem folio.
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the object cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the object cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper object cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages or
- * LRU pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios or
+ * LRU folios.
  */
-static inline struct obj_cgroup *__page_objcg(struct page *page)
+static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
 {
-	unsigned long memcg_data = page->memcg_data;
+	unsigned long memcg_data = folio->memcg_data;
 
-	VM_BUG_ON_PAGE(PageSlab(page), page);
-	VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
-	VM_BUG_ON_PAGE(!(memcg_data & MEMCG_DATA_KMEM), page);
+	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
+	VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
+	VM_BUG_ON_FOLIO(!(memcg_data & MEMCG_DATA_KMEM), folio);
 
 	return (struct obj_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
 }
 
 /*
- * page_memcg - get the memory cgroup associated with a page
- * @page: a pointer to the page struct
+ * folio_memcg - Get the memory cgroup associated with a folio.
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios.
  *
- * For a non-kmem page any of the following ensures page and memcg binding
+ * For a non-kmem folio any of the following ensures folio and memcg binding
  * stability:
  *
- * - the page lock
+ * - the folio lock
  * - LRU isolation
  * - lock_page_memcg()
  * - exclusive reference
  *
- * For a kmem page a caller should hold an rcu read lock to protect memcg
- * associated with a kmem page from being released.
+ * For a kmem folio a caller should hold an rcu read lock to protect memcg
+ * associated with a kmem folio from being released.
  */
+static inline struct mem_cgroup *folio_memcg(struct folio *folio)
+{
+	if (folio_memcg_kmem(folio))
+		return obj_cgroup_memcg(__folio_objcg(folio));
+	return __folio_memcg(folio);
+}
+
 static inline struct mem_cgroup *page_memcg(struct page *page)
 {
-	if (PageMemcgKmem(page))
-		return obj_cgroup_memcg(__page_objcg(page));
-	else
-		return __page_memcg(page);
+	return folio_memcg(page_folio(page));
 }
 
 /*
@@ -525,17 +530,18 @@ static inline struct mem_cgroup *page_memcg_check(struct page *page)
 
 #ifdef CONFIG_MEMCG_KMEM
 /*
- * PageMemcgKmem - check if the page has MemcgKmem flag set
- * @page: a pointer to the page struct
+ * folio_memcg_kmem - Check if the folio has the memcg_kmem flag set.
+ * @folio: Pointer to the folio.
  *
- * Checks if the page has MemcgKmem flag set. The caller must ensure that
- * the page has an associated memory cgroup. It's not safe to call this function
- * against some types of pages, e.g. slab pages.
+ * Checks if the folio has MemcgKmem flag set. The caller must ensure
+ * that the folio has an associated memory cgroup. It's not safe to call
+ * this function against some types of folios, e.g. slab folios.
  */
-static inline bool PageMemcgKmem(struct page *page)
+static inline bool folio_memcg_kmem(struct folio *folio)
 {
-	VM_BUG_ON_PAGE(page->memcg_data & MEMCG_DATA_OBJCGS, page);
-	return page->memcg_data & MEMCG_DATA_KMEM;
+	VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
+	VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
+	return folio->memcg_data & MEMCG_DATA_KMEM;
 }
 
 /*
@@ -579,7 +585,7 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
 }
 
 #else
-static inline bool PageMemcgKmem(struct page *page)
+static inline bool folio_memcg_kmem(struct folio *folio)
 {
 	return false;
 }
@@ -595,6 +601,11 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
 }
 #endif
 
+static inline bool PageMemcgKmem(struct page *page)
+{
+	return folio_memcg_kmem(page_folio(page));
+}
+
 static __always_inline bool memcg_stat_item_in_bytes(int idx)
 {
 	if (idx == MEMCG_PERCPU_B)
@@ -1106,6 +1117,11 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
 #define MEM_CGROUP_ID_SHIFT	0
 #define MEM_CGROUP_ID_MAX	0
 
+static inline struct mem_cgroup *folio_memcg(struct folio *folio)
+{
+	return NULL;
+}
+
 static inline struct mem_cgroup *page_memcg(struct page *page)
 {
 	return NULL;
@@ -1122,6 +1138,11 @@ static inline struct mem_cgroup *page_memcg_check(struct page *page)
 	return NULL;
 }
 
+static inline bool folio_memcg_kmem(struct folio *folio)
+{
+	return false;
+}
+
 static inline bool PageMemcgKmem(struct page *page)
 {
 	return false;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 1a049bfa0e0a..f0f781dde37a 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -3050,15 +3050,16 @@ int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order)
  */
 void __memcg_kmem_uncharge_page(struct page *page, int order)
 {
+	struct folio *folio = page_folio(page);
 	struct obj_cgroup *objcg;
 	unsigned int nr_pages = 1 << order;
 
-	if (!PageMemcgKmem(page))
+	if (!folio_memcg_kmem(folio))
 		return;
 
-	objcg = __page_objcg(page);
+	objcg = __folio_objcg(folio);
 	obj_cgroup_uncharge_pages(objcg, nr_pages);
-	page->memcg_data = 0;
+	folio->memcg_data = 0;
 	obj_cgroup_put(objcg);
 }
 
@@ -3290,17 +3291,18 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size)
  */
 void split_page_memcg(struct page *head, unsigned int nr)
 {
-	struct mem_cgroup *memcg = page_memcg(head);
+	struct folio *folio = page_folio(head);
+	struct mem_cgroup *memcg = folio_memcg(folio);
 	int i;
 
 	if (mem_cgroup_disabled() || !memcg)
 		return;
 
 	for (i = 1; i < nr; i++)
-		head[i].memcg_data = head->memcg_data;
+		folio_page(folio, i)->memcg_data = folio->memcg_data;
 
-	if (PageMemcgKmem(head))
-		obj_cgroup_get_many(__page_objcg(head), nr - 1);
+	if (folio_memcg_kmem(folio))
+		obj_cgroup_get_many(__folio_objcg(folio), nr - 1);
 	else
 		css_get_many(&memcg->css, nr - 1);
 }
@@ -6835,6 +6837,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 
 static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 {
+	struct folio *folio = page_folio(page);
 	unsigned long nr_pages;
 	struct mem_cgroup *memcg;
 	struct obj_cgroup *objcg;
@@ -6848,14 +6851,14 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 	 * exclusive access to the page.
 	 */
 	if (use_objcg) {
-		objcg = __page_objcg(page);
+		objcg = __folio_objcg(folio);
 		/*
 		 * This get matches the put at the end of the function and
 		 * kmem pages do not hold memcg references anymore.
 		 */
 		memcg = get_mem_cgroup_from_objcg(objcg);
 	} else {
-		memcg = __page_memcg(page);
+		memcg = __folio_memcg(folio);
 	}
 
 	if (!memcg)
-- 
2.30.2

[PATCH v13 07/18] mm/memcg: Convert commit_charge() to take a folio

[Matthew Wilcox (Oracle)]

The memcg_data is only set on the head page, so enforce that by
typing it as a folio.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Michal Hocko <mhocko@suse.com>
---
 mm/memcontrol.c | 27 +++++++++++++--------------
 1 file changed, 13 insertions(+), 14 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index f0f781dde37a..f64869c0e06e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2769,9 +2769,9 @@ static void cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages)
 }
 #endif
 
-static void commit_charge(struct page *page, struct mem_cgroup *memcg)
+static void commit_charge(struct folio *folio, struct mem_cgroup *memcg)
 {
-	VM_BUG_ON_PAGE(page_memcg(page), page);
+	VM_BUG_ON_FOLIO(folio_memcg(folio), folio);
 	/*
 	 * Any of the following ensures page's memcg stability:
 	 *
@@ -2780,7 +2780,7 @@ static void commit_charge(struct page *page, struct mem_cgroup *memcg)
 	 * - lock_page_memcg()
 	 * - exclusive reference
 	 */
-	page->memcg_data = (unsigned long)memcg;
+	folio->memcg_data = (unsigned long)memcg;
 }
 
 static struct mem_cgroup *get_mem_cgroup_from_objcg(struct obj_cgroup *objcg)
@@ -6684,7 +6684,8 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 			       gfp_t gfp)
 {
-	unsigned int nr_pages = thp_nr_pages(page);
+	struct folio *folio = page_folio(page);
+	unsigned int nr_pages = folio_nr_pages(folio);
 	int ret;
 
 	ret = try_charge(memcg, gfp, nr_pages);
@@ -6692,7 +6693,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 		goto out;
 
 	css_get(&memcg->css);
-	commit_charge(page, memcg);
+	commit_charge(folio, memcg);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, nr_pages);
@@ -6952,21 +6953,21 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
  */
 void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 {
+	struct folio *newfolio = page_folio(newpage);
 	struct mem_cgroup *memcg;
-	unsigned int nr_pages;
+	unsigned int nr_pages = folio_nr_pages(newfolio);
 	unsigned long flags;
 
 	VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
-	VM_BUG_ON_PAGE(!PageLocked(newpage), newpage);
-	VM_BUG_ON_PAGE(PageAnon(oldpage) != PageAnon(newpage), newpage);
-	VM_BUG_ON_PAGE(PageTransHuge(oldpage) != PageTransHuge(newpage),
-		       newpage);
+	VM_BUG_ON_FOLIO(!folio_locked(newfolio), newfolio);
+	VM_BUG_ON_FOLIO(PageAnon(oldpage) != folio_anon(newfolio), newfolio);
+	VM_BUG_ON_FOLIO(compound_nr(oldpage) != nr_pages, newfolio);
 
 	if (mem_cgroup_disabled())
 		return;
 
 	/* Page cache replacement: new page already charged? */
-	if (page_memcg(newpage))
+	if (folio_memcg(newfolio))
 		return;
 
 	memcg = page_memcg(oldpage);
@@ -6975,8 +6976,6 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 		return;
 
 	/* Force-charge the new page. The old one will be freed soon */
-	nr_pages = thp_nr_pages(newpage);
-
 	if (!mem_cgroup_is_root(memcg)) {
 		page_counter_charge(&memcg->memory, nr_pages);
 		if (do_memsw_account())
@@ -6984,7 +6983,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	}
 
 	css_get(&memcg->css);
-	commit_charge(newpage, memcg);
+	commit_charge(newfolio, memcg);
 
 	local_irq_save(flags);
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-- 
2.30.2

[PATCH v13 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio

[Matthew Wilcox (Oracle)]

Convert all callers of mem_cgroup_charge() to call page_folio() on the
page they're currently passing in.  Many of them will be converted to
use folios themselves soon.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/memcontrol.h |  6 +++---
 kernel/events/uprobes.c    |  3 ++-
 mm/filemap.c               |  2 +-
 mm/huge_memory.c           |  2 +-
 mm/khugepaged.c            |  4 ++--
 mm/ksm.c                   |  3 ++-
 mm/memcontrol.c            | 26 +++++++++++++-------------
 mm/memory.c                |  9 +++++----
 mm/migrate.c               |  2 +-
 mm/shmem.c                 |  2 +-
 mm/userfaultfd.c           |  2 +-
 11 files changed, 32 insertions(+), 29 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index f40c4e0b0431..0c3b386660a8 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -704,7 +704,7 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *memcg)
 		page_counter_read(&memcg->memory);
 }
 
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask);
+int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
 int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 				  gfp_t gfp, swp_entry_t entry);
 void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
@@ -1190,8 +1190,8 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *memcg)
 	return false;
 }
 
-static inline int mem_cgroup_charge(struct page *page, struct mm_struct *mm,
-				    gfp_t gfp_mask)
+static inline int mem_cgroup_charge(struct folio *folio,
+		struct mm_struct *mm, gfp_t gfp)
 {
 	return 0;
 }
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index af24dc3febbe..6357c3580d07 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -167,7 +167,8 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr,
 				addr + PAGE_SIZE);
 
 	if (new_page) {
-		err = mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL);
+		err = mem_cgroup_charge(page_folio(new_page), vma->vm_mm,
+					GFP_KERNEL);
 		if (err)
 			return err;
 	}
diff --git a/mm/filemap.c b/mm/filemap.c
index 8e6c69db5559..44498bfe7b45 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -872,7 +872,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
 	page->index = offset;
 
 	if (!huge) {
-		error = mem_cgroup_charge(page, NULL, gfp);
+		error = mem_cgroup_charge(page_folio(page), NULL, gfp);
 		if (error)
 			goto error;
 		charged = true;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index afff3ac87067..ecb1fb1f5f3e 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -603,7 +603,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,
 
 	VM_BUG_ON_PAGE(!PageCompound(page), page);
 
-	if (mem_cgroup_charge(page, vma->vm_mm, gfp)) {
+	if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
 		put_page(page);
 		count_vm_event(THP_FAULT_FALLBACK);
 		count_vm_event(THP_FAULT_FALLBACK_CHARGE);
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index b0412be08fa2..8f6d7fdea9f4 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1087,7 +1087,7 @@ static void collapse_huge_page(struct mm_struct *mm,
 		goto out_nolock;
 	}
 
-	if (unlikely(mem_cgroup_charge(new_page, mm, gfp))) {
+	if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
 		result = SCAN_CGROUP_CHARGE_FAIL;
 		goto out_nolock;
 	}
@@ -1658,7 +1658,7 @@ static void collapse_file(struct mm_struct *mm,
 		goto out;
 	}
 
-	if (unlikely(mem_cgroup_charge(new_page, mm, gfp))) {
+	if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
 		result = SCAN_CGROUP_CHARGE_FAIL;
 		goto out;
 	}
diff --git a/mm/ksm.c b/mm/ksm.c
index 3fa9bc8a67cf..23d36b59f997 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -2580,7 +2580,8 @@ struct page *ksm_might_need_to_copy(struct page *page,
 		return page;		/* let do_swap_page report the error */
 
 	new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, address);
-	if (new_page && mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL)) {
+	if (new_page &&
+	    mem_cgroup_charge(page_folio(new_page), vma->vm_mm, GFP_KERNEL)) {
 		put_page(new_page);
 		new_page = NULL;
 	}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index f64869c0e06e..ebad42c55f76 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6681,10 +6681,9 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
 			atomic_long_read(&parent->memory.children_low_usage)));
 }
 
-static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
+static int __mem_cgroup_charge(struct folio *folio, struct mem_cgroup *memcg,
 			       gfp_t gfp)
 {
-	struct folio *folio = page_folio(page);
 	unsigned int nr_pages = folio_nr_pages(folio);
 	int ret;
 
@@ -6697,27 +6696,27 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, page_to_nid(page));
+	memcg_check_events(memcg, folio_nid(folio));
 	local_irq_enable();
 out:
 	return ret;
 }
 
 /**
- * mem_cgroup_charge - charge a newly allocated page to a cgroup
- * @page: page to charge
- * @mm: mm context of the victim
- * @gfp_mask: reclaim mode
+ * mem_cgroup_charge - Charge a newly allocated folio to a cgroup.
+ * @folio: Folio to charge.
+ * @mm: mm context of the allocating task.
+ * @gfp: reclaim mode
  *
- * Try to charge @page to the memcg that @mm belongs to, reclaiming
- * pages according to @gfp_mask if necessary. if @mm is NULL, try to
+ * Try to charge @folio to the memcg that @mm belongs to, reclaiming
+ * pages according to @gfp if necessary.  If @mm is NULL, try to
  * charge to the active memcg.
  *
- * Do not use this for pages allocated for swapin.
+ * Do not use this for folios allocated for swapin.
  *
  * Returns 0 on success. Otherwise, an error code is returned.
  */
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
+int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
 {
 	struct mem_cgroup *memcg;
 	int ret;
@@ -6726,7 +6725,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
 		return 0;
 
 	memcg = get_mem_cgroup_from_mm(mm);
-	ret = __mem_cgroup_charge(page, memcg, gfp_mask);
+	ret = __mem_cgroup_charge(folio, memcg, gfp);
 	css_put(&memcg->css);
 
 	return ret;
@@ -6747,6 +6746,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
 int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 				  gfp_t gfp, swp_entry_t entry)
 {
+	struct folio *folio = page_folio(page);
 	struct mem_cgroup *memcg;
 	unsigned short id;
 	int ret;
@@ -6761,7 +6761,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 		memcg = get_mem_cgroup_from_mm(mm);
 	rcu_read_unlock();
 
-	ret = __mem_cgroup_charge(page, memcg, gfp);
+	ret = __mem_cgroup_charge(folio, memcg, gfp);
 
 	css_put(&memcg->css);
 	return ret;
diff --git a/mm/memory.c b/mm/memory.c
index 2f111f9b3dbc..614418e26e2c 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -990,7 +990,7 @@ page_copy_prealloc(struct mm_struct *src_mm, struct vm_area_struct *vma,
 	if (!new_page)
 		return NULL;
 
-	if (mem_cgroup_charge(new_page, src_mm, GFP_KERNEL)) {
+	if (mem_cgroup_charge(page_folio(new_page), src_mm, GFP_KERNEL)) {
 		put_page(new_page);
 		return NULL;
 	}
@@ -3019,7 +3019,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
 		}
 	}
 
-	if (mem_cgroup_charge(new_page, mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(new_page), mm, GFP_KERNEL))
 		goto oom_free_new;
 	cgroup_throttle_swaprate(new_page, GFP_KERNEL);
 
@@ -3768,7 +3768,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
 	if (!page)
 		goto oom;
 
-	if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
 		goto oom_free_page;
 	cgroup_throttle_swaprate(page, GFP_KERNEL);
 
@@ -4183,7 +4183,8 @@ static vm_fault_t do_cow_fault(struct vm_fault *vmf)
 	if (!vmf->cow_page)
 		return VM_FAULT_OOM;
 
-	if (mem_cgroup_charge(vmf->cow_page, vma->vm_mm, GFP_KERNEL)) {
+	if (mem_cgroup_charge(page_folio(vmf->cow_page), vma->vm_mm,
+				GFP_KERNEL)) {
 		put_page(vmf->cow_page);
 		return VM_FAULT_OOM;
 	}
diff --git a/mm/migrate.c b/mm/migrate.c
index 23cbd9de030b..01c05d7f9d6a 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2811,7 +2811,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
 
 	if (unlikely(anon_vma_prepare(vma)))
 		goto abort;
-	if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
 		goto abort;
 
 	/*
diff --git a/mm/shmem.c b/mm/shmem.c
index 70d9ce294bb4..3931fed5c8d8 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -685,7 +685,7 @@ static int shmem_add_to_page_cache(struct page *page,
 	page->index = index;
 
 	if (!PageSwapCache(page)) {
-		error = mem_cgroup_charge(page, charge_mm, gfp);
+		error = mem_cgroup_charge(page_folio(page), charge_mm, gfp);
 		if (error) {
 			if (PageTransHuge(page)) {
 				count_vm_event(THP_FILE_FALLBACK);
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 0e2132834bc7..5d0f55f3c0ed 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -164,7 +164,7 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm,
 	__SetPageUptodate(page);
 
 	ret = -ENOMEM;
-	if (mem_cgroup_charge(page, dst_mm, GFP_KERNEL))
+	if (mem_cgroup_charge(page_folio(page), dst_mm, GFP_KERNEL))
 		goto out_release;
 
 	ret = mfill_atomic_install_pte(dst_mm, dst_pmd, dst_vma, dst_addr,
-- 
2.30.2

[PATCH v13 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio()

[Matthew Wilcox (Oracle)]

Use a folio rather than a page to ensure that we're only operating on
base or head pages, and not tail pages.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 mm/memcontrol.c | 29 ++++++++++++++---------------
 1 file changed, 14 insertions(+), 15 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index ebad42c55f76..2436ad3841d8 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6832,24 +6832,23 @@ static void uncharge_batch(const struct uncharge_gather *ug)
 	memcg_check_events(ug->memcg, ug->nid);
 	local_irq_restore(flags);
 
-	/* drop reference from uncharge_page */
+	/* drop reference from uncharge_folio */
 	css_put(&ug->memcg->css);
 }
 
-static void uncharge_page(struct page *page, struct uncharge_gather *ug)
+static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 {
-	struct folio *folio = page_folio(page);
 	unsigned long nr_pages;
 	struct mem_cgroup *memcg;
 	struct obj_cgroup *objcg;
-	bool use_objcg = PageMemcgKmem(page);
+	bool use_objcg = folio_memcg_kmem(folio);
 
-	VM_BUG_ON_PAGE(PageLRU(page), page);
+	VM_BUG_ON_FOLIO(folio_lru(folio), folio);
 
 	/*
 	 * Nobody should be changing or seriously looking at
-	 * page memcg or objcg at this point, we have fully
-	 * exclusive access to the page.
+	 * folio memcg or objcg at this point, we have fully
+	 * exclusive access to the folio.
 	 */
 	if (use_objcg) {
 		objcg = __folio_objcg(folio);
@@ -6871,19 +6870,19 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 			uncharge_gather_clear(ug);
 		}
 		ug->memcg = memcg;
-		ug->nid = page_to_nid(page);
+		ug->nid = folio_nid(folio);
 
 		/* pairs with css_put in uncharge_batch */
 		css_get(&memcg->css);
 	}
 
-	nr_pages = compound_nr(page);
+	nr_pages = folio_nr_pages(folio);
 
 	if (use_objcg) {
 		ug->nr_memory += nr_pages;
 		ug->nr_kmem += nr_pages;
 
-		page->memcg_data = 0;
+		folio->memcg_data = 0;
 		obj_cgroup_put(objcg);
 	} else {
 		/* LRU pages aren't accounted at the root level */
@@ -6891,7 +6890,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
 			ug->nr_memory += nr_pages;
 		ug->pgpgout++;
 
-		page->memcg_data = 0;
+		folio->memcg_data = 0;
 	}
 
 	css_put(&memcg->css);
@@ -6915,7 +6914,7 @@ void mem_cgroup_uncharge(struct page *page)
 		return;
 
 	uncharge_gather_clear(&ug);
-	uncharge_page(page, &ug);
+	uncharge_folio(page_folio(page), &ug);
 	uncharge_batch(&ug);
 }
 
@@ -6929,14 +6928,14 @@ void mem_cgroup_uncharge(struct page *page)
 void mem_cgroup_uncharge_list(struct list_head *page_list)
 {
 	struct uncharge_gather ug;
-	struct page *page;
+	struct folio *folio;
 
 	if (mem_cgroup_disabled())
 		return;
 
 	uncharge_gather_clear(&ug);
-	list_for_each_entry(page, page_list, lru)
-		uncharge_page(page, &ug);
+	list_for_each_entry(folio, page_list, lru)
+		uncharge_folio(folio, &ug);
 	if (ug.memcg)
 		uncharge_batch(&ug);
 }
-- 
2.30.2

[PATCH v13 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio

[Matthew Wilcox (Oracle)]

Convert all the callers to call page_folio().  Most of them were already
using a head page, but a few of them I can't prove were, so this may
actually fix a bug.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/memcontrol.h |  4 ++--
 mm/filemap.c               |  2 +-
 mm/khugepaged.c            |  4 ++--
 mm/memcontrol.c            | 14 +++++++-------
 mm/memory-failure.c        |  2 +-
 mm/memremap.c              |  2 +-
 mm/page_alloc.c            |  2 +-
 mm/swap.c                  |  2 +-
 8 files changed, 16 insertions(+), 16 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 0c3b386660a8..876465323b06 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -709,7 +709,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
 				  gfp_t gfp, swp_entry_t entry);
 void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
 
-void mem_cgroup_uncharge(struct page *page);
+void mem_cgroup_uncharge(struct folio *folio);
 void mem_cgroup_uncharge_list(struct list_head *page_list);
 
 void mem_cgroup_migrate(struct page *oldpage, struct page *newpage);
@@ -1206,7 +1206,7 @@ static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry)
 {
 }
 
-static inline void mem_cgroup_uncharge(struct page *page)
+static inline void mem_cgroup_uncharge(struct folio *folio)
 {
 }
 
diff --git a/mm/filemap.c b/mm/filemap.c
index 44498bfe7b45..b8be62793316 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -923,7 +923,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
 	if (xas_error(&xas)) {
 		error = xas_error(&xas);
 		if (charged)
-			mem_cgroup_uncharge(page);
+			mem_cgroup_uncharge(page_folio(page));
 		goto error;
 	}
 
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 8f6d7fdea9f4..6b9c98ddcd09 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1211,7 +1211,7 @@ static void collapse_huge_page(struct mm_struct *mm,
 	mmap_write_unlock(mm);
 out_nolock:
 	if (!IS_ERR_OR_NULL(*hpage))
-		mem_cgroup_uncharge(*hpage);
+		mem_cgroup_uncharge(page_folio(*hpage));
 	trace_mm_collapse_huge_page(mm, isolated, result);
 	return;
 }
@@ -1975,7 +1975,7 @@ static void collapse_file(struct mm_struct *mm,
 out:
 	VM_BUG_ON(!list_empty(&pagelist));
 	if (!IS_ERR_OR_NULL(*hpage))
-		mem_cgroup_uncharge(*hpage);
+		mem_cgroup_uncharge(page_folio(*hpage));
 	/* TODO: tracepoints */
 }
 
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 2436ad3841d8..c787a87a54ff 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6897,24 +6897,24 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
 }
 
 /**
- * mem_cgroup_uncharge - uncharge a page
- * @page: page to uncharge
+ * mem_cgroup_uncharge - Uncharge a folio.
+ * @folio: Folio to uncharge.
  *
- * Uncharge a page previously charged with mem_cgroup_charge().
+ * Uncharge a folio previously charged with folio_charge_cgroup().
  */
-void mem_cgroup_uncharge(struct page *page)
+void mem_cgroup_uncharge(struct folio *folio)
 {
 	struct uncharge_gather ug;
 
 	if (mem_cgroup_disabled())
 		return;
 
-	/* Don't touch page->lru of any random page, pre-check: */
-	if (!page_memcg(page))
+	/* Don't touch folio->lru of any random page, pre-check: */
+	if (!folio_memcg(folio))
 		return;
 
 	uncharge_gather_clear(&ug);
-	uncharge_folio(page_folio(page), &ug);
+	uncharge_folio(folio, &ug);
 	uncharge_batch(&ug);
 }
 
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index eefd823deb67..9ae7a57a4cc0 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -763,7 +763,7 @@ static int delete_from_lru_cache(struct page *p)
 		 * Poisoned page might never drop its ref count to 0 so we have
 		 * to uncharge it manually from its memcg.
 		 */
-		mem_cgroup_uncharge(p);
+		mem_cgroup_uncharge(page_folio(p));
 
 		/*
 		 * drop the page count elevated by isolate_lru_page()
diff --git a/mm/memremap.c b/mm/memremap.c
index 15a074ffb8d7..6eac40f9f62a 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -508,7 +508,7 @@ void free_devmap_managed_page(struct page *page)
 
 	__ClearPageWaiters(page);
 
-	mem_cgroup_uncharge(page);
+	mem_cgroup_uncharge(page_folio(page));
 
 	/*
 	 * When a device_private page is freed, the page->mapping field
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 3b97e17806be..d72a0d9d4184 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -726,7 +726,7 @@ static inline void free_the_page(struct page *page, unsigned int order)
 
 void free_compound_page(struct page *page)
 {
-	mem_cgroup_uncharge(page);
+	mem_cgroup_uncharge(page_folio(page));
 	free_the_page(page, compound_order(page));
 }
 
diff --git a/mm/swap.c b/mm/swap.c
index 6d4696eb2d43..b28c76a2e955 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -94,7 +94,7 @@ static void __page_cache_release(struct page *page)
 static void __put_single_page(struct page *page)
 {
 	__page_cache_release(page);
-	mem_cgroup_uncharge(page);
+	mem_cgroup_uncharge(page_folio(page));
 	free_unref_page(page, 0);
 }
 
-- 
2.30.2

[PATCH v13 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios

[Matthew Wilcox (Oracle)]

Convert all callers of mem_cgroup_migrate() to call page_folio() first.
They all look like they're using head pages already, but this proves it.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/memcontrol.h |  4 ++--
 mm/filemap.c               |  4 +++-
 mm/memcontrol.c            | 35 +++++++++++++++++------------------
 mm/migrate.c               |  4 +++-
 mm/shmem.c                 |  5 ++++-
 5 files changed, 29 insertions(+), 23 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 876465323b06..46bad4a50e19 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -712,7 +712,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
 void mem_cgroup_uncharge(struct folio *folio);
 void mem_cgroup_uncharge_list(struct list_head *page_list);
 
-void mem_cgroup_migrate(struct page *oldpage, struct page *newpage);
+void mem_cgroup_migrate(struct folio *old, struct folio *new);
 
 /**
  * mem_cgroup_lruvec - get the lru list vector for a memcg & node
@@ -1214,7 +1214,7 @@ static inline void mem_cgroup_uncharge_list(struct list_head *page_list)
 {
 }
 
-static inline void mem_cgroup_migrate(struct page *old, struct page *new)
+static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
 {
 }
 
diff --git a/mm/filemap.c b/mm/filemap.c
index b8be62793316..cc21da4157f1 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -817,6 +817,8 @@ EXPORT_SYMBOL(file_write_and_wait_range);
  */
 void replace_page_cache_page(struct page *old, struct page *new)
 {
+	struct folio *fold = page_folio(old);
+	struct folio *fnew = page_folio(new);
 	struct address_space *mapping = old->mapping;
 	void (*freepage)(struct page *) = mapping->a_ops->freepage;
 	pgoff_t offset = old->index;
@@ -831,7 +833,7 @@ void replace_page_cache_page(struct page *old, struct page *new)
 	new->mapping = mapping;
 	new->index = offset;
 
-	mem_cgroup_migrate(old, new);
+	mem_cgroup_migrate(fold, fnew);
 
 	xas_lock_irqsave(&xas, flags);
 	xas_store(&xas, new);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index c787a87a54ff..06fc88d75a04 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6941,36 +6941,35 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
 }
 
 /**
- * mem_cgroup_migrate - charge a page's replacement
- * @oldpage: currently circulating page
- * @newpage: replacement page
+ * mem_cgroup_migrate - Charge a folio's replacement.
+ * @old: Currently circulating folio.
+ * @new: Replacement folio.
  *
- * Charge @newpage as a replacement page for @oldpage. @oldpage will
+ * Charge @new as a replacement folio for @old. @old will
  * be uncharged upon free.
  *
- * Both pages must be locked, @newpage->mapping must be set up.
+ * Both folios must be locked, @new->mapping must be set up.
  */
-void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
+void mem_cgroup_migrate(struct folio *old, struct folio *new)
 {
-	struct folio *newfolio = page_folio(newpage);
 	struct mem_cgroup *memcg;
-	unsigned int nr_pages = folio_nr_pages(newfolio);
+	unsigned int nr_pages = folio_nr_pages(new);
 	unsigned long flags;
 
-	VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
-	VM_BUG_ON_FOLIO(!folio_locked(newfolio), newfolio);
-	VM_BUG_ON_FOLIO(PageAnon(oldpage) != folio_anon(newfolio), newfolio);
-	VM_BUG_ON_FOLIO(compound_nr(oldpage) != nr_pages, newfolio);
+	VM_BUG_ON_FOLIO(!folio_locked(old), old);
+	VM_BUG_ON_FOLIO(!folio_locked(new), new);
+	VM_BUG_ON_FOLIO(folio_anon(old) != folio_anon(new), new);
+	VM_BUG_ON_FOLIO(folio_nr_pages(old) != nr_pages, new);
 
 	if (mem_cgroup_disabled())
 		return;
 
-	/* Page cache replacement: new page already charged? */
-	if (folio_memcg(newfolio))
+	/* Page cache replacement: new folio already charged? */
+	if (folio_memcg(new))
 		return;
 
-	memcg = page_memcg(oldpage);
-	VM_WARN_ON_ONCE_PAGE(!memcg, oldpage);
+	memcg = folio_memcg(old);
+	VM_WARN_ON_ONCE_FOLIO(!memcg, old);
 	if (!memcg)
 		return;
 
@@ -6982,11 +6981,11 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
 	}
 
 	css_get(&memcg->css);
-	commit_charge(newfolio, memcg);
+	commit_charge(new, memcg);
 
 	local_irq_save(flags);
 	mem_cgroup_charge_statistics(memcg, nr_pages);
-	memcg_check_events(memcg, page_to_nid(newpage));
+	memcg_check_events(memcg, folio_nid(new));
 	local_irq_restore(flags);
 }
 
diff --git a/mm/migrate.c b/mm/migrate.c
index 01c05d7f9d6a..d8df117dca7e 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -589,6 +589,8 @@ void copy_huge_page(struct page *dst, struct page *src)
  */
 void migrate_page_states(struct page *newpage, struct page *page)
 {
+	struct folio *folio = page_folio(page);
+	struct folio *newfolio = page_folio(newpage);
 	int cpupid;
 
 	if (PageError(page))
@@ -656,7 +658,7 @@ void migrate_page_states(struct page *newpage, struct page *page)
 	copy_page_owner(page, newpage);
 
 	if (!PageHuge(page))
-		mem_cgroup_migrate(page, newpage);
+		mem_cgroup_migrate(folio, newfolio);
 }
 EXPORT_SYMBOL(migrate_page_states);
 
diff --git a/mm/shmem.c b/mm/shmem.c
index 3931fed5c8d8..2fd75b4d4974 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1619,6 +1619,7 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
 				struct shmem_inode_info *info, pgoff_t index)
 {
 	struct page *oldpage, *newpage;
+	struct folio *old, *new;
 	struct address_space *swap_mapping;
 	swp_entry_t entry;
 	pgoff_t swap_index;
@@ -1655,7 +1656,9 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
 	xa_lock_irq(&swap_mapping->i_pages);
 	error = shmem_replace_entry(swap_mapping, swap_index, oldpage, newpage);
 	if (!error) {
-		mem_cgroup_migrate(oldpage, newpage);
+		old = page_folio(oldpage);
+		new = page_folio(newpage);
+		mem_cgroup_migrate(old, new);
 		__inc_lruvec_page_state(newpage, NR_FILE_PAGES);
 		__dec_lruvec_page_state(oldpage, NR_FILE_PAGES);
 	}
-- 
2.30.2

[PATCH v13 12/18] mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio

[Matthew Wilcox (Oracle)]

The page was only being used for the memcg and to gather trace
information, so this is a simple conversion.  The only caller of
mem_cgroup_track_foreign_dirty() will be converted to folios in a later
patch, so doing this now makes that patch simpler.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/memcontrol.h       | 7 ++++---
 include/trace/events/writeback.h | 8 ++++----
 mm/memcontrol.c                  | 6 +++---
 3 files changed, 11 insertions(+), 10 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 46bad4a50e19..a82939df13dc 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1560,17 +1560,18 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,
 			 unsigned long *pheadroom, unsigned long *pdirty,
 			 unsigned long *pwriteback);
 
-void mem_cgroup_track_foreign_dirty_slowpath(struct page *page,
+void mem_cgroup_track_foreign_dirty_slowpath(struct folio *folio,
 					     struct bdi_writeback *wb);
 
 static inline void mem_cgroup_track_foreign_dirty(struct page *page,
 						  struct bdi_writeback *wb)
 {
+	struct folio *folio = page_folio(page);
 	if (mem_cgroup_disabled())
 		return;
 
-	if (unlikely(&page_memcg(page)->css != wb->memcg_css))
-		mem_cgroup_track_foreign_dirty_slowpath(page, wb);
+	if (unlikely(&folio_memcg(folio)->css != wb->memcg_css))
+		mem_cgroup_track_foreign_dirty_slowpath(folio, wb);
 }
 
 void mem_cgroup_flush_foreign(struct bdi_writeback *wb);
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index 840d1ba84cf5..297871ca0004 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -236,9 +236,9 @@ TRACE_EVENT(inode_switch_wbs,
 
 TRACE_EVENT(track_foreign_dirty,
 
-	TP_PROTO(struct page *page, struct bdi_writeback *wb),
+	TP_PROTO(struct folio *folio, struct bdi_writeback *wb),
 
-	TP_ARGS(page, wb),
+	TP_ARGS(folio, wb),
 
 	TP_STRUCT__entry(
 		__array(char,		name, 32)
@@ -250,7 +250,7 @@ TRACE_EVENT(track_foreign_dirty,
 	),
 
 	TP_fast_assign(
-		struct address_space *mapping = page_mapping(page);
+		struct address_space *mapping = folio_mapping(folio);
 		struct inode *inode = mapping ? mapping->host : NULL;
 
 		strscpy_pad(__entry->name, bdi_dev_name(wb->bdi), 32);
@@ -258,7 +258,7 @@ TRACE_EVENT(track_foreign_dirty,
 		__entry->ino		= inode ? inode->i_ino : 0;
 		__entry->memcg_id	= wb->memcg_css->id;
 		__entry->cgroup_ino	= __trace_wb_assign_cgroup(wb);
-		__entry->page_cgroup_ino = cgroup_ino(page_memcg(page)->css.cgroup);
+		__entry->page_cgroup_ino = cgroup_ino(folio_memcg(folio)->css.cgroup);
 	),
 
 	TP_printk("bdi %s[%llu]: ino=%lu memcg_id=%u cgroup_ino=%lu page_cgroup_ino=%lu",
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 06fc88d75a04..ffa9a9b2ad76 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -4571,17 +4571,17 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,
  * As being wrong occasionally doesn't matter, updates and accesses to the
  * records are lockless and racy.
  */
-void mem_cgroup_track_foreign_dirty_slowpath(struct page *page,
+void mem_cgroup_track_foreign_dirty_slowpath(struct folio *folio,
 					     struct bdi_writeback *wb)
 {
-	struct mem_cgroup *memcg = page_memcg(page);
+	struct mem_cgroup *memcg = folio_memcg(folio);
 	struct memcg_cgwb_frn *frn;
 	u64 now = get_jiffies_64();
 	u64 oldest_at = now;
 	int oldest = -1;
 	int i;
 
-	trace_track_foreign_dirty(page, wb);
+	trace_track_foreign_dirty(folio, wb);
 
 	/*
 	 * Pick the slot to use.  If there is already a slot for @wb, keep
-- 
2.30.2

[PATCH v13 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Matthew Wilcox (Oracle)]

These are the folio equivalents of lock_page_memcg() and
unlock_page_memcg().

lock_page_memcg() and unlock_page_memcg() have too many callers to be
easily replaced in a single patch, so reimplement them as wrappers for
now to be cleaned up later when enough callers have been converted to
use folios.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/memcontrol.h | 10 +++++++++
 mm/memcontrol.c            | 45 ++++++++++++++++++++++++--------------
 2 files changed, 39 insertions(+), 16 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index a82939df13dc..be8664e027d5 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -950,6 +950,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
 extern bool cgroup_memory_noswap;
 #endif
 
+void folio_memcg_lock(struct folio *folio);
+void folio_memcg_unlock(struct folio *folio);
 void lock_page_memcg(struct page *page);
 void unlock_page_memcg(struct page *page);
 
@@ -1367,6 +1369,14 @@ static inline void unlock_page_memcg(struct page *page)
 {
 }
 
+static inline void folio_memcg_lock(struct folio *folio)
+{
+}
+
+static inline void folio_memcg_unlock(struct folio *folio)
+{
+}
+
 static inline void mem_cgroup_handle_over_high(void)
 {
 }
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index ffa9a9b2ad76..cff267c59f4d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1965,18 +1965,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg)
 }
 
 /**
- * lock_page_memcg - lock a page and memcg binding
- * @page: the page
+ * folio_memcg_lock - Bind a folio to its memcg.
+ * @folio: The folio.
  *
- * This function protects unlocked LRU pages from being moved to
+ * This function prevents unlocked LRU folios from being moved to
  * another cgroup.
  *
- * It ensures lifetime of the locked memcg. Caller is responsible
- * for the lifetime of the page.
+ * It ensures lifetime of the bound memcg.  The caller is responsible
+ * for the lifetime of the folio.
  */
-void lock_page_memcg(struct page *page)
+void folio_memcg_lock(struct folio *folio)
 {
-	struct page *head = compound_head(page); /* rmap on tail pages */
 	struct mem_cgroup *memcg;
 	unsigned long flags;
 
@@ -1990,7 +1989,7 @@ void lock_page_memcg(struct page *page)
 	if (mem_cgroup_disabled())
 		return;
 again:
-	memcg = page_memcg(head);
+	memcg = folio_memcg(folio);
 	if (unlikely(!memcg))
 		return;
 
@@ -2004,7 +2003,7 @@ void lock_page_memcg(struct page *page)
 		return;
 
 	spin_lock_irqsave(&memcg->move_lock, flags);
-	if (memcg != page_memcg(head)) {
+	if (memcg != folio_memcg(folio)) {
 		spin_unlock_irqrestore(&memcg->move_lock, flags);
 		goto again;
 	}
@@ -2018,9 +2017,15 @@ void lock_page_memcg(struct page *page)
 	memcg->move_lock_task = current;
 	memcg->move_lock_flags = flags;
 }
+EXPORT_SYMBOL(folio_memcg_lock);
+
+void lock_page_memcg(struct page *page)
+{
+	folio_memcg_lock(page_folio(page));
+}
 EXPORT_SYMBOL(lock_page_memcg);
 
-static void __unlock_page_memcg(struct mem_cgroup *memcg)
+static void __folio_memcg_unlock(struct mem_cgroup *memcg)
 {
 	if (memcg && memcg->move_lock_task == current) {
 		unsigned long flags = memcg->move_lock_flags;
@@ -2035,14 +2040,22 @@ static void __unlock_page_memcg(struct mem_cgroup *memcg)
 }
 
 /**
- * unlock_page_memcg - unlock a page and memcg binding
- * @page: the page
+ * folio_memcg_unlock - Release the binding between a folio and its memcg.
+ * @folio: The folio.
+ *
+ * This releases the binding created by folio_memcg_lock().  This does
+ * not change the accounting of this folio to its memcg, but it does
+ * permit others to change it.
  */
-void unlock_page_memcg(struct page *page)
+void folio_memcg_unlock(struct folio *folio)
 {
-	struct page *head = compound_head(page);
+	__folio_memcg_unlock(folio_memcg(folio));
+}
+EXPORT_SYMBOL(folio_memcg_unlock);
 
-	__unlock_page_memcg(page_memcg(head));
+void unlock_page_memcg(struct page *page)
+{
+	folio_memcg_unlock(page_folio(page));
 }
 EXPORT_SYMBOL(unlock_page_memcg);
 
@@ -5666,7 +5679,7 @@ static int mem_cgroup_move_account(struct page *page,
 
 	page->memcg_data = (unsigned long)to;
 
-	__unlock_page_memcg(from);
+	__folio_memcg_unlock(from);
 
 	ret = 0;
 	nid = page_to_nid(page);
-- 
2.30.2

[PATCH v13 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Matthew Wilcox (Oracle)]

This saves dozens of bytes of text by eliminating a lot of calls to
compound_head().

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 mm/memcontrol.c | 37 +++++++++++++++++++------------------
 1 file changed, 19 insertions(+), 18 deletions(-)

diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index cff267c59f4d..63cebdd48c27 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5590,38 +5590,39 @@ static int mem_cgroup_move_account(struct page *page,
 				   struct mem_cgroup *from,
 				   struct mem_cgroup *to)
 {
+	struct folio *folio = page_folio(page);
 	struct lruvec *from_vec, *to_vec;
 	struct pglist_data *pgdat;
-	unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
+	unsigned int nr_pages = compound ? folio_nr_pages(folio) : 1;
 	int nid, ret;
 
 	VM_BUG_ON(from == to);
-	VM_BUG_ON_PAGE(PageLRU(page), page);
-	VM_BUG_ON(compound && !PageTransHuge(page));
+	VM_BUG_ON_FOLIO(folio_lru(folio), folio);
+	VM_BUG_ON(compound && !folio_multi(folio));
 
 	/*
 	 * Prevent mem_cgroup_migrate() from looking at
 	 * page's memory cgroup of its source page while we change it.
 	 */
 	ret = -EBUSY;
-	if (!trylock_page(page))
+	if (!folio_trylock(folio))
 		goto out;
 
 	ret = -EINVAL;
-	if (page_memcg(page) != from)
+	if (folio_memcg(folio) != from)
 		goto out_unlock;
 
-	pgdat = page_pgdat(page);
+	pgdat = folio_pgdat(folio);
 	from_vec = mem_cgroup_lruvec(from, pgdat);
 	to_vec = mem_cgroup_lruvec(to, pgdat);
 
-	lock_page_memcg(page);
+	folio_memcg_lock(folio);
 
-	if (PageAnon(page)) {
-		if (page_mapped(page)) {
+	if (folio_anon(folio)) {
+		if (folio_mapped(folio)) {
 			__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
 			__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
-			if (PageTransHuge(page)) {
+			if (folio_transhuge(folio)) {
 				__mod_lruvec_state(from_vec, NR_ANON_THPS,
 						   -nr_pages);
 				__mod_lruvec_state(to_vec, NR_ANON_THPS,
@@ -5632,18 +5633,18 @@ static int mem_cgroup_move_account(struct page *page,
 		__mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages);
 		__mod_lruvec_state(to_vec, NR_FILE_PAGES, nr_pages);
 
-		if (PageSwapBacked(page)) {
+		if (folio_swapbacked(folio)) {
 			__mod_lruvec_state(from_vec, NR_SHMEM, -nr_pages);
 			__mod_lruvec_state(to_vec, NR_SHMEM, nr_pages);
 		}
 
-		if (page_mapped(page)) {
+		if (folio_mapped(folio)) {
 			__mod_lruvec_state(from_vec, NR_FILE_MAPPED, -nr_pages);
 			__mod_lruvec_state(to_vec, NR_FILE_MAPPED, nr_pages);
 		}
 
-		if (PageDirty(page)) {
-			struct address_space *mapping = page_mapping(page);
+		if (folio_dirty(folio)) {
+			struct address_space *mapping = folio_mapping(folio);
 
 			if (mapping_can_writeback(mapping)) {
 				__mod_lruvec_state(from_vec, NR_FILE_DIRTY,
@@ -5654,7 +5655,7 @@ static int mem_cgroup_move_account(struct page *page,
 		}
 	}
 
-	if (PageWriteback(page)) {
+	if (folio_writeback(folio)) {
 		__mod_lruvec_state(from_vec, NR_WRITEBACK, -nr_pages);
 		__mod_lruvec_state(to_vec, NR_WRITEBACK, nr_pages);
 	}
@@ -5677,12 +5678,12 @@ static int mem_cgroup_move_account(struct page *page,
 	css_get(&to->css);
 	css_put(&from->css);
 
-	page->memcg_data = (unsigned long)to;
+	folio->memcg_data = (unsigned long)to;
 
 	__folio_memcg_unlock(from);
 
 	ret = 0;
-	nid = page_to_nid(page);
+	nid = folio_nid(folio);
 
 	local_irq_disable();
 	mem_cgroup_charge_statistics(to, nr_pages);
@@ -5691,7 +5692,7 @@ static int mem_cgroup_move_account(struct page *page,
 	memcg_check_events(from, nid);
 	local_irq_enable();
 out_unlock:
-	unlock_page(page);
+	folio_unlock(folio);
 out:
 	return ret;
 }
-- 
2.30.2

[PATCH v13 15/18] mm/memcg: Add folio_lruvec()

[Matthew Wilcox (Oracle)]

This replaces mem_cgroup_page_lruvec().  All callers converted.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/memcontrol.h | 20 +++++++++-----------
 mm/compaction.c            |  2 +-
 mm/memcontrol.c            |  9 ++++++---
 mm/swap.c                  |  3 ++-
 mm/workingset.c            |  3 ++-
 5 files changed, 20 insertions(+), 17 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index be8664e027d5..d4af898a1294 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -751,18 +751,17 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
 }
 
 /**
- * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page
- * @page: the page
+ * folio_lruvec - return lruvec for isolating/putting an LRU folio
+ * @folio: Pointer to the folio.
  *
- * This function relies on page->mem_cgroup being stable.
+ * This function relies on folio->mem_cgroup being stable.
  */
-static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+static inline struct lruvec *folio_lruvec(struct folio *folio)
 {
-	pg_data_t *pgdat = page_pgdat(page);
-	struct mem_cgroup *memcg = page_memcg(page);
+	struct mem_cgroup *memcg = folio_memcg(folio);
 
-	VM_WARN_ON_ONCE_PAGE(!memcg && !mem_cgroup_disabled(), page);
-	return mem_cgroup_lruvec(memcg, pgdat);
+	VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
+	return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
 }
 
 struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
@@ -1226,10 +1225,9 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
 	return &pgdat->__lruvec;
 }
 
-static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+static inline struct lruvec *folio_lruvec(struct folio *folio)
 {
-	pg_data_t *pgdat = page_pgdat(page);
-
+	struct pglist_data *pgdat = folio_pgdat(folio);
 	return &pgdat->__lruvec;
 }
 
diff --git a/mm/compaction.c b/mm/compaction.c
index 621508e0ecd5..a88f7b893f80 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1028,7 +1028,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 		if (!TestClearPageLRU(page))
 			goto isolate_fail_put;
 
-		lruvec = mem_cgroup_page_lruvec(page);
+		lruvec = folio_lruvec(page_folio(page));
 
 		/* If we already hold the lock, we can skip some rechecking */
 		if (lruvec != locked) {
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 63cebdd48c27..3152a0e1ba6f 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1186,9 +1186,10 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
  */
 struct lruvec *lock_page_lruvec(struct page *page)
 {
+	struct folio *folio = page_folio(page);
 	struct lruvec *lruvec;
 
-	lruvec = mem_cgroup_page_lruvec(page);
+	lruvec = folio_lruvec(folio);
 	spin_lock(&lruvec->lru_lock);
 
 	lruvec_memcg_debug(lruvec, page);
@@ -1198,9 +1199,10 @@ struct lruvec *lock_page_lruvec(struct page *page)
 
 struct lruvec *lock_page_lruvec_irq(struct page *page)
 {
+	struct folio *folio = page_folio(page);
 	struct lruvec *lruvec;
 
-	lruvec = mem_cgroup_page_lruvec(page);
+	lruvec = folio_lruvec(folio);
 	spin_lock_irq(&lruvec->lru_lock);
 
 	lruvec_memcg_debug(lruvec, page);
@@ -1210,9 +1212,10 @@ struct lruvec *lock_page_lruvec_irq(struct page *page)
 
 struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags)
 {
+	struct folio *folio = page_folio(page);
 	struct lruvec *lruvec;
 
-	lruvec = mem_cgroup_page_lruvec(page);
+	lruvec = folio_lruvec(folio);
 	spin_lock_irqsave(&lruvec->lru_lock, *flags);
 
 	lruvec_memcg_debug(lruvec, page);
diff --git a/mm/swap.c b/mm/swap.c
index b28c76a2e955..d5136cac4267 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -315,7 +315,8 @@ void lru_note_cost(struct lruvec *lruvec, bool file, unsigned int nr_pages)
 
 void lru_note_cost_page(struct page *page)
 {
-	lru_note_cost(mem_cgroup_page_lruvec(page),
+	struct folio *folio = page_folio(page);
+	lru_note_cost(folio_lruvec(folio),
 		      page_is_file_lru(page), thp_nr_pages(page));
 }
 
diff --git a/mm/workingset.c b/mm/workingset.c
index 5ba3e42446fa..e62c0f2084a2 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -396,6 +396,7 @@ void workingset_refault(struct page *page, void *shadow)
  */
 void workingset_activation(struct page *page)
 {
+	struct folio *folio = page_folio(page);
 	struct mem_cgroup *memcg;
 	struct lruvec *lruvec;
 
@@ -410,7 +411,7 @@ void workingset_activation(struct page *page)
 	memcg = page_memcg_rcu(page);
 	if (!mem_cgroup_disabled() && !memcg)
 		goto out;
-	lruvec = mem_cgroup_page_lruvec(page);
+	lruvec = folio_lruvec(folio);
 	workingset_age_nonresident(lruvec, thp_nr_pages(page));
 out:
 	rcu_read_unlock();
-- 
2.30.2

[PATCH v13 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions

[Matthew Wilcox (Oracle)]

These are the folio equivalents of lock_page_lruvec() and similar
functions.  Also convert lruvec_memcg_debug() to take a folio.

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/memcontrol.h | 32 ++++++++++++++-----------
 mm/compaction.c            |  2 +-
 mm/huge_memory.c           |  5 ++--
 mm/memcontrol.c            | 48 ++++++++++++++++----------------------
 mm/rmap.c                  |  2 +-
 mm/swap.c                  |  8 ++++---
 mm/vmscan.c                |  3 ++-
 7 files changed, 50 insertions(+), 50 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index d4af898a1294..57b1bf457f51 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -768,15 +768,16 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
 
 struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
 
-struct lruvec *lock_page_lruvec(struct page *page);
-struct lruvec *lock_page_lruvec_irq(struct page *page);
-struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+struct lruvec *folio_lruvec_lock(struct folio *folio);
+struct lruvec *folio_lruvec_lock_irq(struct folio *folio);
+struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
 						unsigned long *flags);
 
 #ifdef CONFIG_DEBUG_VM
-void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page);
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio);
 #else
-static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+static inline
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
 {
 }
 #endif
@@ -1231,7 +1232,8 @@ static inline struct lruvec *folio_lruvec(struct folio *folio)
 	return &pgdat->__lruvec;
 }
 
-static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+static inline
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
 {
 }
 
@@ -1261,26 +1263,26 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg)
 {
 }
 
-static inline struct lruvec *lock_page_lruvec(struct page *page)
+static inline struct lruvec *folio_lruvec_lock(struct folio *folio)
 {
-	struct pglist_data *pgdat = page_pgdat(page);
+	struct pglist_data *pgdat = folio_pgdat(folio);
 
 	spin_lock(&pgdat->__lruvec.lru_lock);
 	return &pgdat->__lruvec;
 }
 
-static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
+static inline struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
 {
-	struct pglist_data *pgdat = page_pgdat(page);
+	struct pglist_data *pgdat = folio_pgdat(folio);
 
 	spin_lock_irq(&pgdat->__lruvec.lru_lock);
 	return &pgdat->__lruvec;
 }
 
-static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
 		unsigned long *flagsp)
 {
-	struct pglist_data *pgdat = page_pgdat(page);
+	struct pglist_data *pgdat = folio_pgdat(folio);
 
 	spin_lock_irqsave(&pgdat->__lruvec.lru_lock, *flagsp);
 	return &pgdat->__lruvec;
@@ -1537,6 +1539,7 @@ static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec)
 static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
 		struct lruvec *locked_lruvec)
 {
+	struct folio *folio = page_folio(page);
 	if (locked_lruvec) {
 		if (page_matches_lruvec(page, locked_lruvec))
 			return locked_lruvec;
@@ -1544,13 +1547,14 @@ static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
 		unlock_page_lruvec_irq(locked_lruvec);
 	}
 
-	return lock_page_lruvec_irq(page);
+	return folio_lruvec_lock_irq(folio);
 }
 
 /* Don't lock again iff page's lruvec locked */
 static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
 		struct lruvec *locked_lruvec, unsigned long *flags)
 {
+	struct folio *folio = page_folio(page);
 	if (locked_lruvec) {
 		if (page_matches_lruvec(page, locked_lruvec))
 			return locked_lruvec;
@@ -1558,7 +1562,7 @@ static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
 		unlock_page_lruvec_irqrestore(locked_lruvec, *flags);
 	}
 
-	return lock_page_lruvec_irqsave(page, flags);
+	return folio_lruvec_lock_irqsave(folio, flags);
 }
 
 #ifdef CONFIG_CGROUP_WRITEBACK
diff --git a/mm/compaction.c b/mm/compaction.c
index a88f7b893f80..6f77577be248 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1038,7 +1038,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
 			compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
 			locked = lruvec;
 
-			lruvec_memcg_debug(lruvec, page);
+			lruvec_memcg_debug(lruvec, page_folio(page));
 
 			/* Try get exclusive access under lock */
 			if (!skip_updated) {
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index ecb1fb1f5f3e..763bf687ca92 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2431,7 +2431,8 @@ static void __split_huge_page_tail(struct page *head, int tail,
 static void __split_huge_page(struct page *page, struct list_head *list,
 		pgoff_t end)
 {
-	struct page *head = compound_head(page);
+	struct folio *folio = page_folio(page);
+	struct page *head = &folio->page;
 	struct lruvec *lruvec;
 	struct address_space *swap_cache = NULL;
 	unsigned long offset = 0;
@@ -2450,7 +2451,7 @@ static void __split_huge_page(struct page *page, struct list_head *list,
 	}
 
 	/* lock lru list/PageCompound, ref frozen by page_ref_freeze */
-	lruvec = lock_page_lruvec(head);
+	lruvec = folio_lruvec_lock(folio);
 
 	for (i = nr - 1; i >= 1; i--) {
 		__split_huge_page_tail(head, i, lruvec, list);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 3152a0e1ba6f..08add9e110ee 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1158,67 +1158,59 @@ int mem_cgroup_scan_tasks(struct mem_cgroup *memcg,
 }
 
 #ifdef CONFIG_DEBUG_VM
-void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
 {
 	struct mem_cgroup *memcg;
 
 	if (mem_cgroup_disabled())
 		return;
 
-	memcg = page_memcg(page);
+	memcg = folio_memcg(folio);
 
 	if (!memcg)
-		VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != root_mem_cgroup, page);
+		VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != root_mem_cgroup, folio);
 	else
-		VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != memcg, page);
+		VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != memcg, folio);
 }
 #endif
 
 /**
- * lock_page_lruvec - lock and return lruvec for a given page.
- * @page: the page
+ * folio_lruvec_lock - lock and return lruvec for a given folio.
+ * @folio: Pointer to the folio.
  *
  * These functions are safe to use under any of the following conditions:
- * - page locked
- * - PageLRU cleared
- * - lock_page_memcg()
- * - page->_refcount is zero
+ * - folio locked
+ * - folio_lru cleared
+ * - folio_memcg_lock()
+ * - folio frozen (refcount of 0)
  */
-struct lruvec *lock_page_lruvec(struct page *page)
+struct lruvec *folio_lruvec_lock(struct folio *folio)
 {
-	struct folio *folio = page_folio(page);
-	struct lruvec *lruvec;
+	struct lruvec *lruvec = folio_lruvec(folio);
 
-	lruvec = folio_lruvec(folio);
 	spin_lock(&lruvec->lru_lock);
-
-	lruvec_memcg_debug(lruvec, page);
+	lruvec_memcg_debug(lruvec, folio);
 
 	return lruvec;
 }
 
-struct lruvec *lock_page_lruvec_irq(struct page *page)
+struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
 {
-	struct folio *folio = page_folio(page);
-	struct lruvec *lruvec;
+	struct lruvec *lruvec = folio_lruvec(folio);
 
-	lruvec = folio_lruvec(folio);
 	spin_lock_irq(&lruvec->lru_lock);
-
-	lruvec_memcg_debug(lruvec, page);
+	lruvec_memcg_debug(lruvec, folio);
 
 	return lruvec;
 }
 
-struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags)
+struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
+		unsigned long *flags)
 {
-	struct folio *folio = page_folio(page);
-	struct lruvec *lruvec;
+	struct lruvec *lruvec = folio_lruvec(folio);
 
-	lruvec = folio_lruvec(folio);
 	spin_lock_irqsave(&lruvec->lru_lock, *flags);
-
-	lruvec_memcg_debug(lruvec, page);
+	lruvec_memcg_debug(lruvec, folio);
 
 	return lruvec;
 }
diff --git a/mm/rmap.c b/mm/rmap.c
index 795f9d5f8386..b416af486812 100644
--- a/mm/rmap.c
+++ b/mm/rmap.c
@@ -33,7 +33,7 @@
  *                 mapping->private_lock (in __set_page_dirty_buffers)
  *                   lock_page_memcg move_lock (in __set_page_dirty_buffers)
  *                     i_pages lock (widely used)
- *                       lruvec->lru_lock (in lock_page_lruvec_irq)
+ *                       lruvec->lru_lock (in folio_lruvec_lock_irq)
  *                 inode->i_lock (in set_page_dirty's __mark_inode_dirty)
  *                 bdi.wb->list_lock (in set_page_dirty's __mark_inode_dirty)
  *                   sb_lock (within inode_lock in fs/fs-writeback.c)
diff --git a/mm/swap.c b/mm/swap.c
index d5136cac4267..a82812caf409 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -80,10 +80,11 @@ static DEFINE_PER_CPU(struct lru_pvecs, lru_pvecs) = {
 static void __page_cache_release(struct page *page)
 {
 	if (PageLRU(page)) {
+		struct folio *folio = page_folio(page);
 		struct lruvec *lruvec;
 		unsigned long flags;
 
-		lruvec = lock_page_lruvec_irqsave(page, &flags);
+		lruvec = folio_lruvec_lock_irqsave(folio, &flags);
 		del_page_from_lru_list(page, lruvec);
 		__clear_page_lru_flags(page);
 		unlock_page_lruvec_irqrestore(lruvec, flags);
@@ -372,11 +373,12 @@ static inline void activate_page_drain(int cpu)
 
 static void activate_page(struct page *page)
 {
+	struct folio *folio = page_folio(page);
 	struct lruvec *lruvec;
 
-	page = compound_head(page);
+	page = &folio->page;
 	if (TestClearPageLRU(page)) {
-		lruvec = lock_page_lruvec_irq(page);
+		lruvec = folio_lruvec_lock_irq(folio);
 		__activate_page(page, lruvec);
 		unlock_page_lruvec_irq(lruvec);
 		SetPageLRU(page);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 4620df62f0ff..0d48306d37dc 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -1965,6 +1965,7 @@ static unsigned long isolate_lru_pages(unsigned long nr_to_scan,
  */
 int isolate_lru_page(struct page *page)
 {
+	struct folio *folio = page_folio(page);
 	int ret = -EBUSY;
 
 	VM_BUG_ON_PAGE(!page_count(page), page);
@@ -1974,7 +1975,7 @@ int isolate_lru_page(struct page *page)
 		struct lruvec *lruvec;
 
 		get_page(page);
-		lruvec = lock_page_lruvec_irq(page);
+		lruvec = folio_lruvec_lock_irq(folio);
 		del_page_from_lru_list(page, lruvec);
 		unlock_page_lruvec_irq(lruvec);
 		ret = 0;
-- 
2.30.2

[PATCH v13 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave()

[Matthew Wilcox (Oracle)]

These are the folio equivalents of relock_page_lruvec_irq() and
folio_lruvec_relock_irqsave().  Also convert page_matches_lruvec()
to folio_matches_lruvec().

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
 include/linux/memcontrol.h | 17 ++++++++---------
 mm/mlock.c                 |  3 ++-
 mm/swap.c                  | 11 +++++++----
 mm/vmscan.c                |  5 +++--
 4 files changed, 20 insertions(+), 16 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 57b1bf457f51..469e5473bdfb 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1529,19 +1529,19 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec,
 }
 
 /* Test requires a stable page->memcg binding, see page_memcg() */
-static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec)
+static inline bool folio_matches_lruvec(struct folio *folio,
+		struct lruvec *lruvec)
 {
-	return lruvec_pgdat(lruvec) == page_pgdat(page) &&
-	       lruvec_memcg(lruvec) == page_memcg(page);
+	return lruvec_pgdat(lruvec) == folio_pgdat(folio) &&
+	       lruvec_memcg(lruvec) == folio_memcg(folio);
 }
 
 /* Don't lock again iff page's lruvec locked */
-static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
+static inline struct lruvec *folio_lruvec_relock_irq(struct folio *folio,
 		struct lruvec *locked_lruvec)
 {
-	struct folio *folio = page_folio(page);
 	if (locked_lruvec) {
-		if (page_matches_lruvec(page, locked_lruvec))
+		if (folio_matches_lruvec(folio, locked_lruvec))
 			return locked_lruvec;
 
 		unlock_page_lruvec_irq(locked_lruvec);
@@ -1551,12 +1551,11 @@ static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
 }
 
 /* Don't lock again iff page's lruvec locked */
-static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
+static inline struct lruvec *folio_lruvec_relock_irqsave(struct folio *folio,
 		struct lruvec *locked_lruvec, unsigned long *flags)
 {
-	struct folio *folio = page_folio(page);
 	if (locked_lruvec) {
-		if (page_matches_lruvec(page, locked_lruvec))
+		if (folio_matches_lruvec(folio, locked_lruvec))
 			return locked_lruvec;
 
 		unlock_page_lruvec_irqrestore(locked_lruvec, *flags);
diff --git a/mm/mlock.c b/mm/mlock.c
index 16d2ee160d43..e263d62ae2d0 100644
--- a/mm/mlock.c
+++ b/mm/mlock.c
@@ -271,6 +271,7 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone)
 	/* Phase 1: page isolation */
 	for (i = 0; i < nr; i++) {
 		struct page *page = pvec->pages[i];
+		struct folio *folio = page_folio(page);
 
 		if (TestClearPageMlocked(page)) {
 			/*
@@ -278,7 +279,7 @@ static void __munlock_pagevec(struct pagevec *pvec, struct zone *zone)
 			 * so we can spare the get_page() here.
 			 */
 			if (TestClearPageLRU(page)) {
-				lruvec = relock_page_lruvec_irq(page, lruvec);
+				lruvec = folio_lruvec_relock_irq(folio, lruvec);
 				del_page_from_lru_list(page, lruvec);
 				continue;
 			} else
diff --git a/mm/swap.c b/mm/swap.c
index a82812caf409..42222653e6ef 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -211,12 +211,13 @@ static void pagevec_lru_move_fn(struct pagevec *pvec,
 
 	for (i = 0; i < pagevec_count(pvec); i++) {
 		struct page *page = pvec->pages[i];
+		struct folio *folio = page_folio(page);
 
 		/* block memcg migration during page moving between lru */
 		if (!TestClearPageLRU(page))
 			continue;
 
-		lruvec = relock_page_lruvec_irqsave(page, lruvec, &flags);
+		lruvec = folio_lruvec_relock_irqsave(folio, lruvec, &flags);
 		(*move_fn)(page, lruvec);
 
 		SetPageLRU(page);
@@ -907,6 +908,7 @@ void release_pages(struct page **pages, int nr)
 
 	for (i = 0; i < nr; i++) {
 		struct page *page = pages[i];
+		struct folio *folio = page_folio(page);
 
 		/*
 		 * Make sure the IRQ-safe lock-holding time does not get
@@ -918,7 +920,7 @@ void release_pages(struct page **pages, int nr)
 			lruvec = NULL;
 		}
 
-		page = compound_head(page);
+		page = &folio->page;
 		if (is_huge_zero_page(page))
 			continue;
 
@@ -957,7 +959,7 @@ void release_pages(struct page **pages, int nr)
 		if (PageLRU(page)) {
 			struct lruvec *prev_lruvec = lruvec;
 
-			lruvec = relock_page_lruvec_irqsave(page, lruvec,
+			lruvec = folio_lruvec_relock_irqsave(folio, lruvec,
 									&flags);
 			if (prev_lruvec != lruvec)
 				lock_batch = 0;
@@ -1061,8 +1063,9 @@ void __pagevec_lru_add(struct pagevec *pvec)
 
 	for (i = 0; i < pagevec_count(pvec); i++) {
 		struct page *page = pvec->pages[i];
+		struct folio *folio = page_folio(page);
 
-		lruvec = relock_page_lruvec_irqsave(page, lruvec, &flags);
+		lruvec = folio_lruvec_relock_irqsave(folio, lruvec, &flags);
 		__pagevec_lru_add_fn(page, lruvec);
 	}
 	if (lruvec)
diff --git a/mm/vmscan.c b/mm/vmscan.c
index 0d48306d37dc..7a2f25b904d9 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2075,7 +2075,7 @@ static unsigned int move_pages_to_lru(struct lruvec *lruvec,
 		 * All pages were isolated from the same lruvec (and isolation
 		 * inhibits memcg migration).
 		 */
-		VM_BUG_ON_PAGE(!page_matches_lruvec(page, lruvec), page);
+		VM_BUG_ON_PAGE(!folio_matches_lruvec(page_folio(page), lruvec), page);
 		add_page_to_lru_list(page, lruvec);
 		nr_pages = thp_nr_pages(page);
 		nr_moved += nr_pages;
@@ -4514,6 +4514,7 @@ void check_move_unevictable_pages(struct pagevec *pvec)
 
 	for (i = 0; i < pvec->nr; i++) {
 		struct page *page = pvec->pages[i];
+		struct folio *folio = page_folio(page);
 		int nr_pages;
 
 		if (PageTransTail(page))
@@ -4526,7 +4527,7 @@ void check_move_unevictable_pages(struct pagevec *pvec)
 		if (!TestClearPageLRU(page))
 			continue;
 
-		lruvec = relock_page_lruvec_irq(page, lruvec);
+		lruvec = folio_lruvec_relock_irq(folio, lruvec);
 		if (page_evictable(page) && PageUnevictable(page)) {
 			del_page_from_lru_list(page, lruvec);
 			ClearPageUnevictable(page);
-- 
2.30.2

[PATCH v13 18/18] mm/workingset: Convert workingset_activation to take a folio

[Matthew Wilcox (Oracle)]

This function already assumed it was being passed a head page.  No real
change here, except that thp_nr_pages() compiles away on kernels with
THP compiled out while folio_nr_pages() is always present.  Also convert
page_memcg_rcu() to folio_memcg_rcu().

Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
 include/linux/memcontrol.h | 18 +++++++++---------
 include/linux/swap.h       |  2 +-
 mm/swap.c                  |  2 +-
 mm/workingset.c            | 11 ++++-------
 4 files changed, 15 insertions(+), 18 deletions(-)

diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 469e5473bdfb..42fec3af745d 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -461,19 +461,19 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
 }
 
 /*
- * page_memcg_rcu - locklessly get the memory cgroup associated with a page
- * @page: a pointer to the page struct
+ * folio_memcg_rcu - Locklessly get the memory cgroup associated with a folio.
+ * @folio: Pointer to the folio.
  *
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
  * proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios.
  */
-static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
+static inline struct mem_cgroup *folio_memcg_rcu(struct folio *folio)
 {
-	unsigned long memcg_data = READ_ONCE(page->memcg_data);
+	unsigned long memcg_data = READ_ONCE(folio->memcg_data);
 
-	VM_BUG_ON_PAGE(PageSlab(page), page);
+	VM_BUG_ON_FOLIO(folio_slab(folio), folio);
 	WARN_ON_ONCE(!rcu_read_lock_held());
 
 	if (memcg_data & MEMCG_DATA_KMEM) {
@@ -1129,7 +1129,7 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
 	return NULL;
 }
 
-static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
+static inline struct mem_cgroup *folio_memcg_rcu(struct folio *folio)
 {
 	WARN_ON_ONCE(!rcu_read_lock_held());
 	return NULL;
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 8394716a002b..989d8f78c256 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -330,7 +330,7 @@ static inline swp_entry_t folio_swap_entry(struct folio *folio)
 void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_pages);
 void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg);
 void workingset_refault(struct page *page, void *shadow);
-void workingset_activation(struct page *page);
+void workingset_activation(struct folio *folio);
 
 /* Only track the nodes of mappings with shadow entries */
 void workingset_update_node(struct xa_node *node);
diff --git a/mm/swap.c b/mm/swap.c
index 42222653e6ef..5c681c01e3fa 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -451,7 +451,7 @@ void mark_page_accessed(struct page *page)
 		else
 			__lru_cache_activate_page(page);
 		ClearPageReferenced(page);
-		workingset_activation(page);
+		workingset_activation(page_folio(page));
 	}
 	if (page_is_idle(page))
 		clear_page_idle(page);
diff --git a/mm/workingset.c b/mm/workingset.c
index e62c0f2084a2..39bb60d50217 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -392,13 +392,11 @@ void workingset_refault(struct page *page, void *shadow)
 
 /**
  * workingset_activation - note a page activation
- * @page: page that is being activated
+ * @folio: Folio that is being activated.
  */
-void workingset_activation(struct page *page)
+void workingset_activation(struct folio *folio)
 {
-	struct folio *folio = page_folio(page);
 	struct mem_cgroup *memcg;
-	struct lruvec *lruvec;
 
 	rcu_read_lock();
 	/*
@@ -408,11 +406,10 @@ void workingset_activation(struct page *page)
 	 * XXX: See workingset_refault() - this should return
 	 * root_mem_cgroup even for !CONFIG_MEMCG.
 	 */
-	memcg = page_memcg_rcu(page);
+	memcg = folio_memcg_rcu(folio);
 	if (!mem_cgroup_disabled() && !memcg)
 		goto out;
-	lruvec = folio_lruvec(folio);
-	workingset_age_nonresident(lruvec, thp_nr_pages(page));
+	workingset_age_nonresident(folio_lruvec(folio), folio_nr_pages(folio));
 out:
 	rcu_read_unlock();
 }
-- 
2.30.2

Re: [PATCH v13 06/18] mm/memcg: Add folio_memcg() and related functions

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 35437 bytes --]

Hi "Matthew,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.14-rc1 next-20210712]
[cannot apply to hnaz-linux-mm/master tip/perf/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Convert-memcg-to-folios/20210713-035650
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git e73f0f0ee7541171d89f2e2491130c7771ba58d3
config: i386-randconfig-p002-20210712 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/90cfa1fbae827b338bee737e073ffab84823bfed
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Convert-memcg-to-folios/20210713-035650
        git checkout 90cfa1fbae827b338bee737e073ffab84823bfed
        # save the attached .config to linux build tree
        make W=1 ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/kallsyms.h:13,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/mm.h:1380:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1380 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
>> include/linux/mm.h:1382:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1382 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                            ^~~~~
   include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_memcg':
   include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
     401 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |  ^~~~~~~~~~~~~~~
         |  VM_BUG_ON_MM
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
>> include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:455:23: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     455 |  return __folio_memcg(folio);
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
>> include/linux/memcontrol.h:460:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:460:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:540:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
>> include/linux/memcontrol.h:540:20: error: conflicting types for 'folio_memcg_kmem'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   In file included from include/asm-generic/atomic-instrumented.h:20,
                    from include/linux/atomic.h:81,
                    from include/linux/crypto.h:15,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/memcontrol.h: In function 'folio_memcg_kmem':
   include/linux/memcontrol.h:542:35: error: dereferencing pointer to incomplete type 'struct folio'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |                                   ^~
   include/linux/build_bug.h:30:63: note: in definition of macro 'BUILD_BUG_ON_INVALID'
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                                               ^
   include/linux/memcontrol.h:542:2: note: in expansion of macro 'VM_BUG_ON_PGFLAGS'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |  ^~~~~~~~~~~~~~~~~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:606:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     606 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:540:51: note: expected 'struct folio *' but argument is of type 'int'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   arch/x86/kernel/asm-offsets.c: At top level:
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
--
   In file included from include/linux/kallsyms.h:13,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/mm.h:1380:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1380 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
>> include/linux/mm.h:1382:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1382 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                            ^~~~~
   include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_memcg':
   include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
     401 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |  ^~~~~~~~~~~~~~~
         |  VM_BUG_ON_MM
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
>> include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:455:23: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     455 |  return __folio_memcg(folio);
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
>> include/linux/memcontrol.h:460:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:460:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:540:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
>> include/linux/memcontrol.h:540:20: error: conflicting types for 'folio_memcg_kmem'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   In file included from include/asm-generic/atomic-instrumented.h:20,
                    from include/linux/atomic.h:81,
                    from include/linux/crypto.h:15,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/memcontrol.h: In function 'folio_memcg_kmem':
   include/linux/memcontrol.h:542:35: error: dereferencing pointer to incomplete type 'struct folio'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |                                   ^~
   include/linux/build_bug.h:30:63: note: in definition of macro 'BUILD_BUG_ON_INVALID'
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                                               ^
   include/linux/memcontrol.h:542:2: note: in expansion of macro 'VM_BUG_ON_PGFLAGS'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |  ^~~~~~~~~~~~~~~~~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:606:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     606 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:540:51: note: expected 'struct folio *' but argument is of type 'int'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   arch/x86/kernel/asm-offsets.c: At top level:
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1212: prepare0] Error 2
   make[1]: Target 'modules_prepare' not remade because of errors.
   make: *** [Makefile:220: __sub-make] Error 2
   make: Target 'modules_prepare' not remade because of errors.
--
   In file included from include/linux/kallsyms.h:13,
                    from include/linux/bpf.h:20,
                    from include/linux/bpf-cgroup.h:5,
                    from include/linux/cgroup-defs.h:22,
                    from include/linux/cgroup.h:28,
                    from include/linux/memcontrol.h:13,
                    from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/mm.h:1380:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1380 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
>> include/linux/mm.h:1382:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1382 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                            ^~~~~
   include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_memcg':
   include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
     401 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |  ^~~~~~~~~~~~~~~
         |  VM_BUG_ON_MM
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
     403 |  VM_BUG_ON_FOLIO(folio_slab(folio), folio);
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
>> include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:455:23: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     455 |  return __folio_memcg(folio);
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
>> include/linux/memcontrol.h:460:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:460:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:540:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
>> include/linux/memcontrol.h:540:20: error: conflicting types for 'folio_memcg_kmem'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   In file included from include/asm-generic/atomic-instrumented.h:20,
                    from include/linux/atomic.h:81,
                    from include/linux/crypto.h:15,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/memcontrol.h: In function 'folio_memcg_kmem':
   include/linux/memcontrol.h:542:35: error: dereferencing pointer to incomplete type 'struct folio'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |                                   ^~
   include/linux/build_bug.h:30:63: note: in definition of macro 'BUILD_BUG_ON_INVALID'
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                                               ^
   include/linux/memcontrol.h:542:2: note: in expansion of macro 'VM_BUG_ON_PGFLAGS'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |  ^~~~~~~~~~~~~~~~~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:606:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     606 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:540:51: note: expected 'struct folio *' but argument is of type 'int'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   arch/x86/kernel/asm-offsets.c: At top level:
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1212: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:220: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.


vim +403 include/linux/memcontrol.h

   388	
   389	/*
   390	 * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
   391	 * @folio: Pointer to the folio.
   392	 *
   393	 * Returns a pointer to the memory cgroup associated with the folio,
   394	 * or NULL. This function assumes that the folio is known to have a
   395	 * proper memory cgroup pointer. It's not safe to call this function
   396	 * against some type of folios, e.g. slab folios or ex-slab folios or
   397	 * kmem folios.
   398	 */
   399	static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
   400	{
   401		unsigned long memcg_data = folio->memcg_data;
   402	
 > 403		VM_BUG_ON_FOLIO(folio_slab(folio), folio);
   404		VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
   405		VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
   406	
   407		return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   408	}
   409	
   410	/*
   411	 * __folio_objcg - get the object cgroup associated with a kmem folio.
   412	 * @folio: Pointer to the folio.
   413	 *
   414	 * Returns a pointer to the object cgroup associated with the folio,
   415	 * or NULL. This function assumes that the folio is known to have a
   416	 * proper object cgroup pointer. It's not safe to call this function
   417	 * against some type of folios, e.g. slab folios or ex-slab folios or
   418	 * LRU folios.
   419	 */
   420	static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
   421	{
   422		unsigned long memcg_data = folio->memcg_data;
   423	
   424		VM_BUG_ON_FOLIO(folio_slab(folio), folio);
   425		VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
   426		VM_BUG_ON_FOLIO(!(memcg_data & MEMCG_DATA_KMEM), folio);
   427	
   428		return (struct obj_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   429	}
   430	
   431	/*
   432	 * folio_memcg - Get the memory cgroup associated with a folio.
   433	 * @folio: Pointer to the folio.
   434	 *
   435	 * Returns a pointer to the memory cgroup associated with the folio,
   436	 * or NULL. This function assumes that the folio is known to have a
   437	 * proper memory cgroup pointer. It's not safe to call this function
   438	 * against some type of folios, e.g. slab folios or ex-slab folios.
   439	 *
   440	 * For a non-kmem folio any of the following ensures folio and memcg binding
   441	 * stability:
   442	 *
   443	 * - the folio lock
   444	 * - LRU isolation
   445	 * - lock_page_memcg()
   446	 * - exclusive reference
   447	 *
   448	 * For a kmem folio a caller should hold an rcu read lock to protect memcg
   449	 * associated with a kmem folio from being released.
   450	 */
   451	static inline struct mem_cgroup *folio_memcg(struct folio *folio)
   452	{
 > 453		if (folio_memcg_kmem(folio))
 > 454			return obj_cgroup_memcg(__folio_objcg(folio));
 > 455		return __folio_memcg(folio);
   456	}
   457	
   458	static inline struct mem_cgroup *page_memcg(struct page *page)
   459	{
 > 460		return folio_memcg(page_folio(page));
   461	}
   462	
   463	/*
   464	 * page_memcg_rcu - locklessly get the memory cgroup associated with a page
   465	 * @page: a pointer to the page struct
   466	 *
   467	 * Returns a pointer to the memory cgroup associated with the page,
   468	 * or NULL. This function assumes that the page is known to have a
   469	 * proper memory cgroup pointer. It's not safe to call this function
   470	 * against some type of pages, e.g. slab pages or ex-slab pages.
   471	 */
   472	static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
   473	{
   474		unsigned long memcg_data = READ_ONCE(page->memcg_data);
   475	
   476		VM_BUG_ON_PAGE(PageSlab(page), page);
   477		WARN_ON_ONCE(!rcu_read_lock_held());
   478	
   479		if (memcg_data & MEMCG_DATA_KMEM) {
   480			struct obj_cgroup *objcg;
   481	
   482			objcg = (void *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   483			return obj_cgroup_memcg(objcg);
   484		}
   485	
   486		return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   487	}
   488	
   489	/*
   490	 * page_memcg_check - get the memory cgroup associated with a page
   491	 * @page: a pointer to the page struct
   492	 *
   493	 * Returns a pointer to the memory cgroup associated with the page,
   494	 * or NULL. This function unlike page_memcg() can take any page
   495	 * as an argument. It has to be used in cases when it's not known if a page
   496	 * has an associated memory cgroup pointer or an object cgroups vector or
   497	 * an object cgroup.
   498	 *
   499	 * For a non-kmem page any of the following ensures page and memcg binding
   500	 * stability:
   501	 *
   502	 * - the page lock
   503	 * - LRU isolation
   504	 * - lock_page_memcg()
   505	 * - exclusive reference
   506	 *
   507	 * For a kmem page a caller should hold an rcu read lock to protect memcg
   508	 * associated with a kmem page from being released.
   509	 */
   510	static inline struct mem_cgroup *page_memcg_check(struct page *page)
   511	{
   512		/*
   513		 * Because page->memcg_data might be changed asynchronously
   514		 * for slab pages, READ_ONCE() should be used here.
   515		 */
   516		unsigned long memcg_data = READ_ONCE(page->memcg_data);
   517	
   518		if (memcg_data & MEMCG_DATA_OBJCGS)
   519			return NULL;
   520	
   521		if (memcg_data & MEMCG_DATA_KMEM) {
   522			struct obj_cgroup *objcg;
   523	
   524			objcg = (void *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   525			return obj_cgroup_memcg(objcg);
   526		}
   527	
   528		return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
   529	}
   530	
   531	#ifdef CONFIG_MEMCG_KMEM
   532	/*
   533	 * folio_memcg_kmem - Check if the folio has the memcg_kmem flag set.
   534	 * @folio: Pointer to the folio.
   535	 *
   536	 * Checks if the folio has MemcgKmem flag set. The caller must ensure
   537	 * that the folio has an associated memory cgroup. It's not safe to call
   538	 * this function against some types of folios, e.g. slab folios.
   539	 */
 > 540	static inline bool folio_memcg_kmem(struct folio *folio)
   541	{
   542		VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
   543		VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
   544		return folio->memcg_data & MEMCG_DATA_KMEM;
   545	}
   546	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 38001 bytes --]

Re: [PATCH v13 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 14409 bytes --]

Hi "Matthew,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.14-rc1 next-20210712]
[cannot apply to hnaz-linux-mm/master tip/perf/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Convert-memcg-to-folios/20210713-035650
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git e73f0f0ee7541171d89f2e2491130c7771ba58d3
config: nds32-allnoconfig (attached as .config)
compiler: nds32le-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
        wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
        chmod +x ~/bin/make.cross
        # https://github.com/0day-ci/linux/commit/3aa23c53058c0abac2b7fd5d8c80f9b458a2665f
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Convert-memcg-to-folios/20210713-035650
        git checkout 3aa23c53058c0abac2b7fd5d8c80f9b458a2665f
        # save the attached .config to linux build tree
        COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=nds32 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

   In file included from include/linux/dax.h:6,
                    from mm/filemap.c:15:
   include/linux/mm.h:1380:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1380 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
   include/linux/mm.h:1382:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1382 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from mm/filemap.c:23:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1120:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1120 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h:1141:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1141 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1193:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                            ^~~~~
   mm/filemap.c: In function '__add_to_page_cache_locked':
>> mm/filemap.c:875:29: error: implicit declaration of function 'page_folio'; did you mean 'page_endio'? [-Werror=implicit-function-declaration]
     875 |   error = mem_cgroup_charge(page_folio(page), NULL, gfp);
         |                             ^~~~~~~~~~
         |                             page_endio
   mm/filemap.c:875:29: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
     875 |   error = mem_cgroup_charge(page_folio(page), NULL, gfp);
         |                             ^~~~~~~~~~~~~~~~
         |                             |
         |                             int
   In file included from include/linux/swap.h:9,
                    from mm/filemap.c:23:
   include/linux/memcontrol.h:1193:51: note: expected 'struct folio *' but argument is of type 'int'
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                     ~~~~~~~~~~~~~~^~~~~
   cc1: some warnings being treated as errors
--
   In file included from include/linux/security.h:33,
                    from include/linux/fs_context.h:14,
                    from include/linux/fs_parser.h:11,
                    from include/linux/ramfs.h:5,
                    from mm/shmem.c:28:
   include/linux/mm.h:1380:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1380 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
   include/linux/mm.h:1382:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1382 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from mm/shmem.c:35:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1120:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1120 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h:1141:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1141 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1193:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                            ^~~~~
   mm/shmem.c: In function 'shmem_add_to_page_cache':
>> mm/shmem.c:688:29: error: implicit declaration of function 'page_folio'; did you mean 'page_endio'? [-Werror=implicit-function-declaration]
     688 |   error = mem_cgroup_charge(page_folio(page), charge_mm, gfp);
         |                             ^~~~~~~~~~
         |                             page_endio
   mm/shmem.c:688:29: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
     688 |   error = mem_cgroup_charge(page_folio(page), charge_mm, gfp);
         |                             ^~~~~~~~~~~~~~~~
         |                             |
         |                             int
   In file included from include/linux/swap.h:9,
                    from mm/shmem.c:35:
   include/linux/memcontrol.h:1193:51: note: expected 'struct folio *' but argument is of type 'int'
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                     ~~~~~~~~~~~~~~^~~~~
   cc1: some warnings being treated as errors
--
   In file included from mm/memory.c:43:
   include/linux/mm.h:1380:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1380 | static inline int folio_nid(const struct folio *folio)
         |                                          ^~~~~
   include/linux/mm.h: In function 'folio_nid':
   include/linux/mm.h:1382:27: error: dereferencing pointer to incomplete type 'const struct folio'
    1382 |  return page_to_nid(&folio->page);
         |                           ^~
   In file included from include/linux/swap.h:9,
                    from mm/memory.c:50:
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:1120:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1120 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h:1141:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1141 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:1193:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                            ^~~~~
   mm/memory.c: In function 'page_copy_prealloc':
>> mm/memory.c:993:24: error: implicit declaration of function 'page_folio'; did you mean 'page_endio'? [-Werror=implicit-function-declaration]
     993 |  if (mem_cgroup_charge(page_folio(new_page), src_mm, GFP_KERNEL)) {
         |                        ^~~~~~~~~~
         |                        page_endio
   mm/memory.c:993:24: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
     993 |  if (mem_cgroup_charge(page_folio(new_page), src_mm, GFP_KERNEL)) {
         |                        ^~~~~~~~~~~~~~~~~~~~
         |                        |
         |                        int
   In file included from include/linux/swap.h:9,
                    from mm/memory.c:50:
   include/linux/memcontrol.h:1193:51: note: expected 'struct folio *' but argument is of type 'int'
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                     ~~~~~~~~~~~~~~^~~~~
   mm/memory.c: In function 'wp_page_copy':
   mm/memory.c:3022:24: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
    3022 |  if (mem_cgroup_charge(page_folio(new_page), mm, GFP_KERNEL))
         |                        ^~~~~~~~~~~~~~~~~~~~
         |                        |
         |                        int
   In file included from include/linux/swap.h:9,
                    from mm/memory.c:50:
   include/linux/memcontrol.h:1193:51: note: expected 'struct folio *' but argument is of type 'int'
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                     ~~~~~~~~~~~~~~^~~~~
   mm/memory.c: In function 'do_anonymous_page':
   mm/memory.c:3771:24: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
    3771 |  if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
         |                        ^~~~~~~~~~~~~~~~
         |                        |
         |                        int
   In file included from include/linux/swap.h:9,
                    from mm/memory.c:50:
   include/linux/memcontrol.h:1193:51: note: expected 'struct folio *' but argument is of type 'int'
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                     ~~~~~~~~~~~~~~^~~~~
   mm/memory.c: In function 'do_cow_fault':
   mm/memory.c:4186:24: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
    4186 |  if (mem_cgroup_charge(page_folio(vmf->cow_page), vma->vm_mm,
         |                        ^~~~~~~~~~~~~~~~~~~~~~~~~
         |                        |
         |                        int
   In file included from include/linux/swap.h:9,
                    from mm/memory.c:50:
   include/linux/memcontrol.h:1193:51: note: expected 'struct folio *' but argument is of type 'int'
    1193 | static inline int mem_cgroup_charge(struct folio *folio,
         |                                     ~~~~~~~~~~~~~~^~~~~
   cc1: some warnings being treated as errors


vim +875 mm/filemap.c

   855	
   856	noinline int __add_to_page_cache_locked(struct page *page,
   857						struct address_space *mapping,
   858						pgoff_t offset, gfp_t gfp,
   859						void **shadowp)
   860	{
   861		XA_STATE(xas, &mapping->i_pages, offset);
   862		int huge = PageHuge(page);
   863		int error;
   864		bool charged = false;
   865	
   866		VM_BUG_ON_PAGE(!PageLocked(page), page);
   867		VM_BUG_ON_PAGE(PageSwapBacked(page), page);
   868		mapping_set_update(&xas, mapping);
   869	
   870		get_page(page);
   871		page->mapping = mapping;
   872		page->index = offset;
   873	
   874		if (!huge) {
 > 875			error = mem_cgroup_charge(page_folio(page), NULL, gfp);
   876			if (error)
   877				goto error;
   878			charged = true;
   879		}
   880	
   881		gfp &= GFP_RECLAIM_MASK;
   882	
   883		do {
   884			unsigned int order = xa_get_order(xas.xa, xas.xa_index);
   885			void *entry, *old = NULL;
   886	
   887			if (order > thp_order(page))
   888				xas_split_alloc(&xas, xa_load(xas.xa, xas.xa_index),
   889						order, gfp);
   890			xas_lock_irq(&xas);
   891			xas_for_each_conflict(&xas, entry) {
   892				old = entry;
   893				if (!xa_is_value(entry)) {
   894					xas_set_err(&xas, -EEXIST);
   895					goto unlock;
   896				}
   897			}
   898	
   899			if (old) {
   900				if (shadowp)
   901					*shadowp = old;
   902				/* entry may have been split before we acquired lock */
   903				order = xa_get_order(xas.xa, xas.xa_index);
   904				if (order > thp_order(page)) {
   905					xas_split(&xas, old, order);
   906					xas_reset(&xas);
   907				}
   908			}
   909	
   910			xas_store(&xas, page);
   911			if (xas_error(&xas))
   912				goto unlock;
   913	
   914			mapping->nrpages++;
   915	
   916			/* hugetlb pages do not participate in page cache accounting */
   917			if (!huge)
   918				__inc_lruvec_page_state(page, NR_FILE_PAGES);
   919	unlock:
   920			xas_unlock_irq(&xas);
   921		} while (xas_nomem(&xas, gfp));
   922	
   923		if (xas_error(&xas)) {
   924			error = xas_error(&xas);
   925			if (charged)
   926				mem_cgroup_uncharge(page);
   927			goto error;
   928		}
   929	
   930		trace_mm_filemap_add_to_page_cache(page);
   931		return 0;
   932	error:
   933		page->mapping = NULL;
   934		/* Leave page->index set: truncation relies upon it */
   935		put_page(page);
   936		return error;
   937	}
   938	ALLOW_ERROR_INJECTION(__add_to_page_cache_locked, ERRNO);
   939	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 5948 bytes --]

Re: [PATCH v13 15/18] mm/memcg: Add folio_lruvec()

[kernel test robot]

[-- Attachment #1: Type: text/plain, Size: 32991 bytes --]

Hi "Matthew,

Thank you for the patch! Yet something to improve:

[auto build test ERROR on linus/master]
[also build test ERROR on v5.14-rc1 next-20210712]
[cannot apply to hnaz-linux-mm/master tip/perf/core linux/master]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]

url:    https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Convert-memcg-to-folios/20210713-035650
base:   https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git e73f0f0ee7541171d89f2e2491130c7771ba58d3
config: i386-randconfig-p002-20210712 (attached as .config)
compiler: gcc-9 (Debian 9.3.0-22) 9.3.0
reproduce (this is a W=1 build):
        # https://github.com/0day-ci/linux/commit/01c63778e430697015122448ac6344d1dc2f10e4
        git remote add linux-review https://github.com/0day-ci/linux
        git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Convert-memcg-to-folios/20210713-035650
        git checkout 01c63778e430697015122448ac6344d1dc2f10e4
        # save the attached .config to linux build tree
        make W=1 ARCH=i386 

If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>

All errors (new ones prefixed by >>):

         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
   include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:455:23: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     455 |  return __folio_memcg(folio);
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
   include/linux/memcontrol.h:460:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:460:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:540:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:540:20: error: conflicting types for 'folio_memcg_kmem'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   In file included from include/asm-generic/atomic-instrumented.h:20,
                    from include/linux/atomic.h:81,
                    from include/linux/crypto.h:15,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/memcontrol.h: In function 'folio_memcg_kmem':
   include/linux/memcontrol.h:542:35: error: dereferencing pointer to incomplete type 'struct folio'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |                                   ^~
   include/linux/build_bug.h:30:63: note: in definition of macro 'BUILD_BUG_ON_INVALID'
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                                               ^
   include/linux/memcontrol.h:542:2: note: in expansion of macro 'VM_BUG_ON_PGFLAGS'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |  ^~~~~~~~~~~~~~~~~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:606:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     606 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:540:51: note: expected 'struct folio *' but argument is of type 'int'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:707:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     707 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                              ^~~~~
   include/linux/memcontrol.h:712:33: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     712 | void mem_cgroup_uncharge(struct folio *folio);
         |                                 ^~~~~
   include/linux/memcontrol.h:715:32: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     715 | void mem_cgroup_migrate(struct folio *old, struct folio *new);
         |                                ^~~~~
   include/linux/memcontrol.h:759:50: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     759 | static inline struct lruvec *folio_lruvec(struct folio *folio)
         |                                                  ^~~~~
   include/linux/memcontrol.h: In function 'folio_lruvec':
>> include/linux/memcontrol.h:761:41: error: passing argument 1 of 'folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     761 |  struct mem_cgroup *memcg = folio_memcg(folio);
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'struct folio *'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:763:2: error: implicit declaration of function 'VM_WARN_ON_ONCE_FOLIO'; did you mean 'VM_WARN_ON_ONCE_PAGE'? [-Werror=implicit-function-declaration]
     763 |  VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
         |  ^~~~~~~~~~~~~~~~~~~~~
         |  VM_WARN_ON_ONCE_PAGE
>> include/linux/memcontrol.h:764:34: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
     764 |  return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
         |                                  ^~~~~~~~~~~
         |                                  folio_nid
   include/linux/memcontrol.h:764:34: warning: passing argument 2 of 'mem_cgroup_lruvec' makes pointer from integer without a cast [-Wint-conversion]
     764 |  return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
         |                                  ^~~~~~~~~~~~~~~~~~
         |                                  |
         |                                  int
   include/linux/memcontrol.h:727:33: note: expected 'struct pglist_data *' but argument is of type 'int'
     727 |             struct pglist_data *pgdat)
         |             ~~~~~~~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:952:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     952 | void folio_memcg_lock(struct folio *folio);
         |                              ^~~~~
   include/linux/memcontrol.h:953:32: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     953 | void folio_memcg_unlock(struct folio *folio);
         |                                ^~~~~
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
--
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
   include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:455:23: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     455 |  return __folio_memcg(folio);
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
   include/linux/memcontrol.h:460:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:460:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:540:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:540:20: error: conflicting types for 'folio_memcg_kmem'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   In file included from include/asm-generic/atomic-instrumented.h:20,
                    from include/linux/atomic.h:81,
                    from include/linux/crypto.h:15,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/memcontrol.h: In function 'folio_memcg_kmem':
   include/linux/memcontrol.h:542:35: error: dereferencing pointer to incomplete type 'struct folio'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |                                   ^~
   include/linux/build_bug.h:30:63: note: in definition of macro 'BUILD_BUG_ON_INVALID'
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                                               ^
   include/linux/memcontrol.h:542:2: note: in expansion of macro 'VM_BUG_ON_PGFLAGS'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |  ^~~~~~~~~~~~~~~~~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:606:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     606 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:540:51: note: expected 'struct folio *' but argument is of type 'int'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:707:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     707 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                              ^~~~~
   include/linux/memcontrol.h:712:33: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     712 | void mem_cgroup_uncharge(struct folio *folio);
         |                                 ^~~~~
   include/linux/memcontrol.h:715:32: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     715 | void mem_cgroup_migrate(struct folio *old, struct folio *new);
         |                                ^~~~~
   include/linux/memcontrol.h:759:50: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     759 | static inline struct lruvec *folio_lruvec(struct folio *folio)
         |                                                  ^~~~~
   include/linux/memcontrol.h: In function 'folio_lruvec':
>> include/linux/memcontrol.h:761:41: error: passing argument 1 of 'folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     761 |  struct mem_cgroup *memcg = folio_memcg(folio);
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'struct folio *'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:763:2: error: implicit declaration of function 'VM_WARN_ON_ONCE_FOLIO'; did you mean 'VM_WARN_ON_ONCE_PAGE'? [-Werror=implicit-function-declaration]
     763 |  VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
         |  ^~~~~~~~~~~~~~~~~~~~~
         |  VM_WARN_ON_ONCE_PAGE
>> include/linux/memcontrol.h:764:34: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
     764 |  return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
         |                                  ^~~~~~~~~~~
         |                                  folio_nid
   include/linux/memcontrol.h:764:34: warning: passing argument 2 of 'mem_cgroup_lruvec' makes pointer from integer without a cast [-Wint-conversion]
     764 |  return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
         |                                  ^~~~~~~~~~~~~~~~~~
         |                                  |
         |                                  int
   include/linux/memcontrol.h:727:33: note: expected 'struct pglist_data *' but argument is of type 'int'
     727 |             struct pglist_data *pgdat)
         |             ~~~~~~~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:952:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     952 | void folio_memcg_lock(struct folio *folio);
         |                              ^~~~~
   include/linux/memcontrol.h:953:32: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     953 | void folio_memcg_unlock(struct folio *folio);
         |                                ^~~~~
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1212: prepare0] Error 2
   make[1]: Target 'modules_prepare' not remade because of errors.
   make: *** [Makefile:220: __sub-make] Error 2
   make: Target 'modules_prepare' not remade because of errors.
--
         |                  ^~~~~~~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                       ^~~~~
   include/linux/memcontrol.h: In function '__folio_objcg':
   include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
     422 |  unsigned long memcg_data = folio->memcg_data;
         |                                  ^~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                                     ^~~~~
   include/linux/memcontrol.h: In function 'folio_memcg':
   include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
     453 |  if (folio_memcg_kmem(folio))
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     454 |   return obj_cgroup_memcg(__folio_objcg(folio));
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h:455:23: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     455 |  return __folio_memcg(folio);
         |                       ^~~~~
         |                       |
         |                       struct folio *
   include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
     399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
         |                                                ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: In function 'page_memcg':
   include/linux/memcontrol.h:460:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~
   include/linux/memcontrol.h:460:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
     460 |  return folio_memcg(page_folio(page));
         |                     ^~~~~~~~~~~~~~~~
         |                     |
         |                     int
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:540:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                            ^~~~~
   include/linux/memcontrol.h:540:20: error: conflicting types for 'folio_memcg_kmem'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                    ^~~~~~~~~~~~~~~~
   include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   In file included from include/asm-generic/atomic-instrumented.h:20,
                    from include/linux/atomic.h:81,
                    from include/linux/crypto.h:15,
                    from arch/x86/kernel/asm-offsets.c:9:
   include/linux/memcontrol.h: In function 'folio_memcg_kmem':
   include/linux/memcontrol.h:542:35: error: dereferencing pointer to incomplete type 'struct folio'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |                                   ^~
   include/linux/build_bug.h:30:63: note: in definition of macro 'BUILD_BUG_ON_INVALID'
      30 | #define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
         |                                                               ^
   include/linux/memcontrol.h:542:2: note: in expansion of macro 'VM_BUG_ON_PGFLAGS'
     542 |  VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
         |  ^~~~~~~~~~~~~~~~~
   In file included from include/linux/swap.h:9,
                    from include/linux/suspend.h:5,
                    from arch/x86/kernel/asm-offsets.c:13:
   include/linux/memcontrol.h: In function 'PageMemcgKmem':
   include/linux/memcontrol.h:606:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
     606 |  return folio_memcg_kmem(page_folio(page));
         |                          ^~~~~~~~~~~~~~~~
         |                          |
         |                          int
   include/linux/memcontrol.h:540:51: note: expected 'struct folio *' but argument is of type 'int'
     540 | static inline bool folio_memcg_kmem(struct folio *folio)
         |                                     ~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:707:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     707 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
         |                              ^~~~~
   include/linux/memcontrol.h:712:33: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     712 | void mem_cgroup_uncharge(struct folio *folio);
         |                                 ^~~~~
   include/linux/memcontrol.h:715:32: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     715 | void mem_cgroup_migrate(struct folio *old, struct folio *new);
         |                                ^~~~~
   include/linux/memcontrol.h:759:50: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     759 | static inline struct lruvec *folio_lruvec(struct folio *folio)
         |                                                  ^~~~~
   include/linux/memcontrol.h: In function 'folio_lruvec':
>> include/linux/memcontrol.h:761:41: error: passing argument 1 of 'folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
     761 |  struct mem_cgroup *memcg = folio_memcg(folio);
         |                                         ^~~~~
         |                                         |
         |                                         struct folio *
   include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'struct folio *'
     451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
         |                                              ~~~~~~~~~~~~~~^~~~~
>> include/linux/memcontrol.h:763:2: error: implicit declaration of function 'VM_WARN_ON_ONCE_FOLIO'; did you mean 'VM_WARN_ON_ONCE_PAGE'? [-Werror=implicit-function-declaration]
     763 |  VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
         |  ^~~~~~~~~~~~~~~~~~~~~
         |  VM_WARN_ON_ONCE_PAGE
>> include/linux/memcontrol.h:764:34: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
     764 |  return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
         |                                  ^~~~~~~~~~~
         |                                  folio_nid
   include/linux/memcontrol.h:764:34: warning: passing argument 2 of 'mem_cgroup_lruvec' makes pointer from integer without a cast [-Wint-conversion]
     764 |  return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
         |                                  ^~~~~~~~~~~~~~~~~~
         |                                  |
         |                                  int
   include/linux/memcontrol.h:727:33: note: expected 'struct pglist_data *' but argument is of type 'int'
     727 |             struct pglist_data *pgdat)
         |             ~~~~~~~~~~~~~~~~~~~~^~~~~
   include/linux/memcontrol.h: At top level:
   include/linux/memcontrol.h:952:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     952 | void folio_memcg_lock(struct folio *folio);
         |                              ^~~~~
   include/linux/memcontrol.h:953:32: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
     953 | void folio_memcg_unlock(struct folio *folio);
         |                                ^~~~~
   include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
     375 | static inline bool folio_memcg_kmem(struct folio *folio);
         |                    ^~~~~~~~~~~~~~~~
   cc1: some warnings being treated as errors
   make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
   make[2]: Target '__build' not remade because of errors.
   make[1]: *** [Makefile:1212: prepare0] Error 2
   make[1]: Target 'prepare' not remade because of errors.
   make: *** [Makefile:220: __sub-make] Error 2
   make: Target 'prepare' not remade because of errors.


vim +/folio_memcg +761 include/linux/memcontrol.h

   752	
   753	/**
   754	 * folio_lruvec - return lruvec for isolating/putting an LRU folio
   755	 * @folio: Pointer to the folio.
   756	 *
   757	 * This function relies on folio->mem_cgroup being stable.
   758	 */
   759	static inline struct lruvec *folio_lruvec(struct folio *folio)
   760	{
 > 761		struct mem_cgroup *memcg = folio_memcg(folio);
   762	
 > 763		VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
 > 764		return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
   765	}
   766	

---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org

[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 38001 bytes --]

Re: [PATCH v13 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()

[Christoph Hellwig]

On Mon, Jul 12, 2021 at 08:45:46PM +0100, Matthew Wilcox (Oracle) wrote:
> These are the folio equivalents of lock_page_memcg() and
> unlock_page_memcg().
> 
> lock_page_memcg() and unlock_page_memcg() have too many callers to be
> easily replaced in a single patch, so reimplement them as wrappers for
> now to be cleaned up later when enough callers have been converted to
> use folios.
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v13 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio

[Christoph Hellwig]

On Mon, Jul 12, 2021 at 08:45:47PM +0100, Matthew Wilcox (Oracle) wrote:
> This saves dozens of bytes of text by eliminating a lot of calls to
> compound_head().
> 
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v13 15/18] mm/memcg: Add folio_lruvec()

[Christoph Hellwig]

On Mon, Jul 12, 2021 at 08:45:48PM +0100, Matthew Wilcox (Oracle) wrote:
> This replaces mem_cgroup_page_lruvec().  All callers converted.

Looks good, especially as it doesn't have much to do with
cgroups except as an implementation detail.  Shouldn't this
function also move out of memcontrol.h eventually?

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v13 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions

[Christoph Hellwig]

On Mon, Jul 12, 2021 at 08:45:49PM +0100, Matthew Wilcox (Oracle) wrote:
> These are the folio equivalents of lock_page_lruvec() and similar
> functions.  Also convert lruvec_memcg_debug() to take a folio.

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>

Re: [PATCH v13 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave()

[Christoph Hellwig]

On Mon, Jul 12, 2021 at 08:45:50PM +0100, Matthew Wilcox (Oracle) wrote:
> These are the folio equivalents of relock_page_lruvec_irq() and
> folio_lruvec_relock_irqsave().  Also convert page_matches_lruvec()
> to folio_matches_lruvec().

Looks good,

Reviewed-by: Christoph Hellwig <hch@lst.de>