* [PATCH v3 00/18] Folio conversion of memcg
@ 2021-06-30 4:00 Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 01/18] mm: Add folio_nid() Matthew Wilcox (Oracle)
` (18 more replies)
0 siblings, 19 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
After Michel's comments on "Folio-enabling the page cache", I thought it
best to split out the memcg patches from the rest of the page cache folio
patches and redo them to focus on correctness (ie always passing a folio).
This is fundamentally for review rather than application. I've rebased
on Linus' current tree, which includes the recent patchbomb from akpm.
That upstream version won't boot on any system I have available, and I'm
not comfortable asking for patches to be applied unless I can actually
try them. That said, these patches were fine on top of 5.13.
There are still a few functions which take pages, but they rely on other
conversions happening first, which in turn rely on this set of patches,
so I think this is a good place to stop, with the understanding that
there will be more patches later.
Some of the commit logs may be garbled ... I haven't fully taken into
account all of Muchun's recent rework. All of this work is visible in
context in a git tree here:
https://git.infradead.org/users/willy/pagecache.git/shortlog/refs/heads/folio
Matthew Wilcox (Oracle) (18):
mm: Add folio_nid()
mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()
mm/memcg: Use the node id in mem_cgroup_update_tree()
mm/memcg: Remove soft_limit_tree_node()
mm/memcg: Convert memcg_check_events to take a node ID
mm/memcg: Add folio_memcg() and related functions
mm/memcg: Convert commit_charge() to take a folio
mm/memcg: Convert mem_cgroup_charge() to take a folio
mm/memcg: Convert uncharge_page() to uncharge_folio()
mm/memcg: Convert mem_cgroup_uncharge() to take a folio
mm/memcg: Convert mem_cgroup_migrate() to take folios
mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio
mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
mm/memcg: Convert mem_cgroup_move_account() to use a folio
mm/memcg: Add mem_cgroup_folio_lruvec()
mm/memcg: Add folio_lruvec_lock() and similar functions
mm/memcg: Add folio_lruvec_relock_irq() and
folio_lruvec_relock_irqsave()
mm/workingset: Convert workingset_activation to take a folio
include/linux/memcontrol.h | 249 +++++++++++++++----------
include/linux/mm.h | 5 +
include/linux/swap.h | 2 +-
include/trace/events/writeback.h | 8 +-
kernel/events/uprobes.c | 3 +-
mm/compaction.c | 2 +-
mm/filemap.c | 8 +-
mm/huge_memory.c | 2 +-
mm/khugepaged.c | 8 +-
mm/ksm.c | 3 +-
mm/memcontrol.c | 311 +++++++++++++++----------------
mm/memory-failure.c | 2 +-
mm/memory.c | 9 +-
mm/memremap.c | 2 +-
mm/migrate.c | 6 +-
mm/page_alloc.c | 2 +-
mm/shmem.c | 7 +-
mm/swap.c | 4 +-
mm/userfaultfd.c | 2 +-
mm/vmscan.c | 2 +-
mm/workingset.c | 10 +-
21 files changed, 355 insertions(+), 292 deletions(-)
--
2.30.2
^ permalink raw reply [flat|nested] 59+ messages in thread
* [PATCH v3 01/18] mm: Add folio_nid()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-07-01 6:56 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics() Matthew Wilcox (Oracle)
` (17 subsequent siblings)
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
This is the folio equivalent of page_to_nid().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/mm.h | 5 +++++
1 file changed, 5 insertions(+)
diff --git a/include/linux/mm.h b/include/linux/mm.h
index 7ad3c8a115ae..48d2ccd9f8c3 100644
--- a/include/linux/mm.h
+++ b/include/linux/mm.h
@@ -1429,6 +1429,11 @@ static inline int page_to_nid(const struct page *page)
}
#endif
+static inline int folio_nid(const struct folio *folio)
+{
+ return page_to_nid(&folio->page);
+}
+
#ifdef CONFIG_NUMA_BALANCING
static inline int cpu_pid_to_cpupid(int cpu, int pid)
{
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 01/18] mm: Add folio_nid() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 14:17 ` Johannes Weiner
2021-06-30 4:00 ` [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree() Matthew Wilcox (Oracle)
` (16 subsequent siblings)
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov,
Christoph Hellwig, Michal Hocko
The last use of 'page' was removed by commit 468c398233da ("mm:
memcontrol: switch to native NR_ANON_THPS counter"), so we can now remove
the parameter from the function.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Michal Hocko <mhocko@suse.com>
---
mm/memcontrol.c | 11 +++++------
1 file changed, 5 insertions(+), 6 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 4ee243ce6135..25cad0fb7d4e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -826,7 +826,6 @@ static unsigned long memcg_events_local(struct mem_cgroup *memcg, int event)
}
static void mem_cgroup_charge_statistics(struct mem_cgroup *memcg,
- struct page *page,
int nr_pages)
{
/* pagein of a big page is an event. So, ignore page size */
@@ -5687,9 +5686,9 @@ static int mem_cgroup_move_account(struct page *page,
ret = 0;
local_irq_disable();
- mem_cgroup_charge_statistics(to, page, nr_pages);
+ mem_cgroup_charge_statistics(to, nr_pages);
memcg_check_events(to, page);
- mem_cgroup_charge_statistics(from, page, -nr_pages);
+ mem_cgroup_charge_statistics(from, -nr_pages);
memcg_check_events(from, page);
local_irq_enable();
out_unlock:
@@ -6710,7 +6709,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
commit_charge(page, memcg);
local_irq_disable();
- mem_cgroup_charge_statistics(memcg, page, nr_pages);
+ mem_cgroup_charge_statistics(memcg, nr_pages);
memcg_check_events(memcg, page);
local_irq_enable();
out:
@@ -7001,7 +7000,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
commit_charge(newpage, memcg);
local_irq_save(flags);
- mem_cgroup_charge_statistics(memcg, newpage, nr_pages);
+ mem_cgroup_charge_statistics(memcg, nr_pages);
memcg_check_events(memcg, newpage);
local_irq_restore(flags);
}
@@ -7231,7 +7230,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
* only synchronisation we have for updating the per-CPU variables.
*/
VM_BUG_ON(!irqs_disabled());
- mem_cgroup_charge_statistics(memcg, page, -nr_entries);
+ mem_cgroup_charge_statistics(memcg, -nr_entries);
memcg_check_events(memcg, page);
css_put(&memcg->css);
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 01/18] mm: Add folio_nid() Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 6:55 ` Michal Hocko
` (2 more replies)
2021-06-30 4:00 ` [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node() Matthew Wilcox (Oracle)
` (15 subsequent siblings)
18 siblings, 3 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
By using the node id in mem_cgroup_update_tree(), we can delete
soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo(). Saves 42
bytes of kernel text on my config.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
mm/memcontrol.c | 24 ++++--------------------
1 file changed, 4 insertions(+), 20 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 25cad0fb7d4e..29b28a050707 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -446,28 +446,12 @@ ino_t page_cgroup_ino(struct page *page)
return ino;
}
-static struct mem_cgroup_per_node *
-mem_cgroup_page_nodeinfo(struct mem_cgroup *memcg, struct page *page)
-{
- int nid = page_to_nid(page);
-
- return memcg->nodeinfo[nid];
-}
-
static struct mem_cgroup_tree_per_node *
soft_limit_tree_node(int nid)
{
return soft_limit_tree.rb_tree_per_node[nid];
}
-static struct mem_cgroup_tree_per_node *
-soft_limit_tree_from_page(struct page *page)
-{
- int nid = page_to_nid(page);
-
- return soft_limit_tree.rb_tree_per_node[nid];
-}
-
static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
struct mem_cgroup_tree_per_node *mctz,
unsigned long new_usage_in_excess)
@@ -538,13 +522,13 @@ static unsigned long soft_limit_excess(struct mem_cgroup *memcg)
return excess;
}
-static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
+static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
{
unsigned long excess;
struct mem_cgroup_per_node *mz;
struct mem_cgroup_tree_per_node *mctz;
- mctz = soft_limit_tree_from_page(page);
+ mctz = soft_limit_tree_node(nid);
if (!mctz)
return;
/*
@@ -552,7 +536,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
* because their event counter is not touched.
*/
for (; memcg; memcg = parent_mem_cgroup(memcg)) {
- mz = mem_cgroup_page_nodeinfo(memcg, page);
+ mz = memcg->nodeinfo[nid];
excess = soft_limit_excess(memcg);
/*
* We have to update the tree if mz is on RB-tree or
@@ -879,7 +863,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
MEM_CGROUP_TARGET_SOFTLIMIT);
mem_cgroup_threshold(memcg);
if (unlikely(do_softlimit))
- mem_cgroup_update_tree(memcg, page);
+ mem_cgroup_update_tree(memcg, page_to_nid(page));
}
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (2 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 6:56 ` Michal Hocko
` (2 more replies)
2021-06-30 4:00 ` [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID Matthew Wilcox (Oracle)
` (14 subsequent siblings)
18 siblings, 3 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
Opencode this one-line function in its three callers.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
mm/memcontrol.c | 12 +++---------
1 file changed, 3 insertions(+), 9 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 29b28a050707..29fdb70dca42 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -446,12 +446,6 @@ ino_t page_cgroup_ino(struct page *page)
return ino;
}
-static struct mem_cgroup_tree_per_node *
-soft_limit_tree_node(int nid)
-{
- return soft_limit_tree.rb_tree_per_node[nid];
-}
-
static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
struct mem_cgroup_tree_per_node *mctz,
unsigned long new_usage_in_excess)
@@ -528,7 +522,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
struct mem_cgroup_per_node *mz;
struct mem_cgroup_tree_per_node *mctz;
- mctz = soft_limit_tree_node(nid);
+ mctz = soft_limit_tree.rb_tree_per_node[nid];
if (!mctz)
return;
/*
@@ -567,7 +561,7 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
for_each_node(nid) {
mz = memcg->nodeinfo[nid];
- mctz = soft_limit_tree_node(nid);
+ mctz = soft_limit_tree.rb_tree_per_node[nid];
if (mctz)
mem_cgroup_remove_exceeded(mz, mctz);
}
@@ -3415,7 +3409,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
if (order > 0)
return 0;
- mctz = soft_limit_tree_node(pgdat->node_id);
+ mctz = soft_limit_tree.rb_tree_per_node[pgdat->node_id];
/*
* Do not even bother to check the largest node if the root
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (3 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 6:58 ` Michal Hocko
2021-07-01 7:09 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions Matthew Wilcox (Oracle)
` (13 subsequent siblings)
18 siblings, 2 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
memcg_check_events only uses the page's nid, so call page_to_nid in the
callers to make the folio conversion easier.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
mm/memcontrol.c | 23 ++++++++++++-----------
1 file changed, 12 insertions(+), 11 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 29fdb70dca42..5d143d46a8a4 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -846,7 +846,7 @@ static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
* Check events in order.
*
*/
-static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
+static void memcg_check_events(struct mem_cgroup *memcg, int nid)
{
/* threshold event is triggered in finer grain than soft limit */
if (unlikely(mem_cgroup_event_ratelimit(memcg,
@@ -857,7 +857,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
MEM_CGROUP_TARGET_SOFTLIMIT);
mem_cgroup_threshold(memcg);
if (unlikely(do_softlimit))
- mem_cgroup_update_tree(memcg, page_to_nid(page));
+ mem_cgroup_update_tree(memcg, nid);
}
}
@@ -5573,7 +5573,7 @@ static int mem_cgroup_move_account(struct page *page,
struct lruvec *from_vec, *to_vec;
struct pglist_data *pgdat;
unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
- int ret;
+ int nid, ret;
VM_BUG_ON(from == to);
VM_BUG_ON_PAGE(PageLRU(page), page);
@@ -5662,12 +5662,13 @@ static int mem_cgroup_move_account(struct page *page,
__unlock_page_memcg(from);
ret = 0;
+ nid = page_to_nid(page);
local_irq_disable();
mem_cgroup_charge_statistics(to, nr_pages);
- memcg_check_events(to, page);
+ memcg_check_events(to, nid);
mem_cgroup_charge_statistics(from, -nr_pages);
- memcg_check_events(from, page);
+ memcg_check_events(from, nid);
local_irq_enable();
out_unlock:
unlock_page(page);
@@ -6688,7 +6689,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
local_irq_disable();
mem_cgroup_charge_statistics(memcg, nr_pages);
- memcg_check_events(memcg, page);
+ memcg_check_events(memcg, page_to_nid(page));
local_irq_enable();
out:
return ret;
@@ -6796,7 +6797,7 @@ struct uncharge_gather {
unsigned long nr_memory;
unsigned long pgpgout;
unsigned long nr_kmem;
- struct page *dummy_page;
+ int nid;
};
static inline void uncharge_gather_clear(struct uncharge_gather *ug)
@@ -6820,7 +6821,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
local_irq_save(flags);
__count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout);
__this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory);
- memcg_check_events(ug->memcg, ug->dummy_page);
+ memcg_check_events(ug->memcg, ug->nid);
local_irq_restore(flags);
/* drop reference from uncharge_page */
@@ -6861,7 +6862,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
uncharge_gather_clear(ug);
}
ug->memcg = memcg;
- ug->dummy_page = page;
+ ug->nid = page_to_nid(page);
/* pairs with css_put in uncharge_batch */
css_get(&memcg->css);
@@ -6979,7 +6980,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
local_irq_save(flags);
mem_cgroup_charge_statistics(memcg, nr_pages);
- memcg_check_events(memcg, newpage);
+ memcg_check_events(memcg, page_to_nid(newpage));
local_irq_restore(flags);
}
@@ -7209,7 +7210,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
*/
VM_BUG_ON(!irqs_disabled());
mem_cgroup_charge_statistics(memcg, -nr_entries);
- memcg_check_events(memcg, page);
+ memcg_check_events(memcg, page_to_nid(page));
css_put(&memcg->css);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (4 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 6:53 ` kernel test robot
2021-07-01 7:12 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 07/18] mm/memcg: Convert commit_charge() to take a folio Matthew Wilcox (Oracle)
` (12 subsequent siblings)
18 siblings, 2 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
memcg information is only stored in the head page, so the memcg
subsystem needs to assure that all accesses are to the head page.
The first step is converting page_memcg() to folio_memcg().
Retain page_memcg() as a wrapper around folio_memcg() and PageMemcgKmem()
as a wrapper around folio_memcg_kmem() but convert __page_memcg() to
__folio_memcg() and __page_objcg() to __folio_objcg().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 105 +++++++++++++++++++++----------------
mm/memcontrol.c | 21 ++++----
2 files changed, 73 insertions(+), 53 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 6d66037be646..92689fb2dab4 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -372,6 +372,7 @@ enum page_memcg_data_flags {
#define MEMCG_DATA_FLAGS_MASK (__NR_MEMCG_DATA_FLAGS - 1)
static inline bool PageMemcgKmem(struct page *page);
+static inline bool folio_memcg_kmem(struct folio *folio);
/*
* After the initialization objcg->memcg is always pointing at
@@ -386,73 +387,78 @@ static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
}
/*
- * __page_memcg - get the memory cgroup associated with a non-kmem page
- * @page: a pointer to the page struct
+ * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
+ * @folio: Pointer to the folio.
*
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
* proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages or
- * kmem pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios or
+ * kmem folios.
*/
-static inline struct mem_cgroup *__page_memcg(struct page *page)
+static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
{
- unsigned long memcg_data = page->memcg_data;
+ unsigned long memcg_data = folio->memcg_data;
- VM_BUG_ON_PAGE(PageSlab(page), page);
- VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
- VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
+ VM_BUG_ON_FOLIO(folio_slab(folio), folio);
+ VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
+ VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
}
/*
- * __page_objcg - get the object cgroup associated with a kmem page
- * @page: a pointer to the page struct
+ * __folio_objcg - get the object cgroup associated with a kmem folio.
+ * @folio: Pointer to the folio.
*
- * Returns a pointer to the object cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the object cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
* proper object cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages or
- * LRU pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios or
+ * LRU folios.
*/
-static inline struct obj_cgroup *__page_objcg(struct page *page)
+static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
{
- unsigned long memcg_data = page->memcg_data;
+ unsigned long memcg_data = folio->memcg_data;
- VM_BUG_ON_PAGE(PageSlab(page), page);
- VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
- VM_BUG_ON_PAGE(!(memcg_data & MEMCG_DATA_KMEM), page);
+ VM_BUG_ON_FOLIO(folio_slab(folio), folio);
+ VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
+ VM_BUG_ON_FOLIO(!(memcg_data & MEMCG_DATA_KMEM), folio);
return (struct obj_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
}
/*
- * page_memcg - get the memory cgroup associated with a page
- * @page: a pointer to the page struct
+ * folio_memcg - Get the memory cgroup associated with a folio.
+ * @folio: Pointer to the folio.
*
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
* proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios.
*
- * For a non-kmem page any of the following ensures page and memcg binding
+ * For a non-kmem folio any of the following ensures folio and memcg binding
* stability:
*
- * - the page lock
+ * - the folio lock
* - LRU isolation
* - lock_page_memcg()
* - exclusive reference
*
- * For a kmem page a caller should hold an rcu read lock to protect memcg
- * associated with a kmem page from being released.
+ * For a kmem folio a caller should hold an rcu read lock to protect memcg
+ * associated with a kmem folio from being released.
*/
-static inline struct mem_cgroup *page_memcg(struct page *page)
+static inline struct mem_cgroup *folio_memcg(struct folio *folio)
{
- if (PageMemcgKmem(page))
- return obj_cgroup_memcg(__page_objcg(page));
+ if (folio_memcg_kmem(folio))
+ return obj_cgroup_memcg(__folio_objcg(folio));
else
- return __page_memcg(page);
+ return __folio_memcg(folio);
+}
+
+static inline struct mem_cgroup *page_memcg(struct page *page)
+{
+ return folio_memcg(page_folio(page));
}
/*
@@ -525,17 +531,18 @@ static inline struct mem_cgroup *page_memcg_check(struct page *page)
#ifdef CONFIG_MEMCG_KMEM
/*
- * PageMemcgKmem - check if the page has MemcgKmem flag set
- * @page: a pointer to the page struct
+ * folio_memcg_kmem - Check if the folio has the memcg_kmem flag set.
+ * @folio: Pointer to the folio.
*
- * Checks if the page has MemcgKmem flag set. The caller must ensure that
- * the page has an associated memory cgroup. It's not safe to call this function
- * against some types of pages, e.g. slab pages.
+ * Checks if the folio has MemcgKmem flag set. The caller must ensure
+ * that the folio has an associated memory cgroup. It's not safe to call
+ * this function against some types of folios, e.g. slab folios.
*/
-static inline bool PageMemcgKmem(struct page *page)
+static inline bool folio_memcg_kmem(struct folio *folio)
{
- VM_BUG_ON_PAGE(page->memcg_data & MEMCG_DATA_OBJCGS, page);
- return page->memcg_data & MEMCG_DATA_KMEM;
+ VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
+ VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
+ return folio->memcg_data & MEMCG_DATA_KMEM;
}
/*
@@ -579,7 +586,7 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
}
#else
-static inline bool PageMemcgKmem(struct page *page)
+static inline bool folio_memcg_kmem(struct folio *folio)
{
return false;
}
@@ -595,6 +602,11 @@ static inline struct obj_cgroup **page_objcgs_check(struct page *page)
}
#endif
+static inline bool PageMemcgKmem(struct page *page)
+{
+ return folio_memcg_kmem(page_folio(page));
+}
+
static __always_inline bool memcg_stat_item_in_bytes(int idx)
{
if (idx == MEMCG_PERCPU_B)
@@ -1122,6 +1134,11 @@ static inline struct mem_cgroup *page_memcg_check(struct page *page)
return NULL;
}
+static inline bool folio_memcg_kmem(struct folio *folio)
+{
+ return false;
+}
+
static inline bool PageMemcgKmem(struct page *page)
{
return false;
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 5d143d46a8a4..f369bbaf584b 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -3045,15 +3045,16 @@ int __memcg_kmem_charge_page(struct page *page, gfp_t gfp, int order)
*/
void __memcg_kmem_uncharge_page(struct page *page, int order)
{
+ struct folio *folio = page_folio(page);
struct obj_cgroup *objcg;
unsigned int nr_pages = 1 << order;
- if (!PageMemcgKmem(page))
+ if (!folio_memcg_kmem(folio))
return;
- objcg = __page_objcg(page);
+ objcg = __folio_objcg(folio);
obj_cgroup_uncharge_pages(objcg, nr_pages);
- page->memcg_data = 0;
+ folio->memcg_data = 0;
obj_cgroup_put(objcg);
}
@@ -3285,17 +3286,18 @@ void obj_cgroup_uncharge(struct obj_cgroup *objcg, size_t size)
*/
void split_page_memcg(struct page *head, unsigned int nr)
{
- struct mem_cgroup *memcg = page_memcg(head);
+ struct folio *folio = page_folio(head);
+ struct mem_cgroup *memcg = folio_memcg(folio);
int i;
if (mem_cgroup_disabled() || !memcg)
return;
for (i = 1; i < nr; i++)
- head[i].memcg_data = head->memcg_data;
+ folio_page(folio, i)->memcg_data = folio->memcg_data;
- if (PageMemcgKmem(head))
- obj_cgroup_get_many(__page_objcg(head), nr - 1);
+ if (folio_memcg_kmem(folio))
+ obj_cgroup_get_many(__folio_objcg(folio), nr - 1);
else
css_get_many(&memcg->css, nr - 1);
}
@@ -6830,6 +6832,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
static void uncharge_page(struct page *page, struct uncharge_gather *ug)
{
+ struct folio *folio = page_folio(page);
unsigned long nr_pages;
struct mem_cgroup *memcg;
struct obj_cgroup *objcg;
@@ -6843,14 +6846,14 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
* exclusive access to the page.
*/
if (use_objcg) {
- objcg = __page_objcg(page);
+ objcg = __folio_objcg(folio);
/*
* This get matches the put at the end of the function and
* kmem pages do not hold memcg references anymore.
*/
memcg = get_mem_cgroup_from_objcg(objcg);
} else {
- memcg = __page_memcg(page);
+ memcg = __folio_memcg(folio);
}
if (!memcg)
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 07/18] mm/memcg: Convert commit_charge() to take a folio
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (5 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() " Matthew Wilcox (Oracle)
` (11 subsequent siblings)
18 siblings, 0 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov,
Christoph Hellwig, Michal Hocko
The memcg_data is only set on the head page, so enforce that by
typing it as a folio.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
Acked-by: Michal Hocko <mhocko@suse.com>
---
mm/memcontrol.c | 27 +++++++++++++--------------
1 file changed, 13 insertions(+), 14 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index f369bbaf584b..727bd578ca7d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -2764,9 +2764,9 @@ static void cancel_charge(struct mem_cgroup *memcg, unsigned int nr_pages)
}
#endif
-static void commit_charge(struct page *page, struct mem_cgroup *memcg)
+static void commit_charge(struct folio *folio, struct mem_cgroup *memcg)
{
- VM_BUG_ON_PAGE(page_memcg(page), page);
+ VM_BUG_ON_FOLIO(folio_memcg(folio), folio);
/*
* Any of the following ensures page's memcg stability:
*
@@ -2775,7 +2775,7 @@ static void commit_charge(struct page *page, struct mem_cgroup *memcg)
* - lock_page_memcg()
* - exclusive reference
*/
- page->memcg_data = (unsigned long)memcg;
+ folio->memcg_data = (unsigned long)memcg;
}
static struct mem_cgroup *get_mem_cgroup_from_objcg(struct obj_cgroup *objcg)
@@ -6679,7 +6679,8 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
gfp_t gfp)
{
- unsigned int nr_pages = thp_nr_pages(page);
+ struct folio *folio = page_folio(page);
+ unsigned int nr_pages = folio_nr_pages(folio);
int ret;
ret = try_charge(memcg, gfp, nr_pages);
@@ -6687,7 +6688,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
goto out;
css_get(&memcg->css);
- commit_charge(page, memcg);
+ commit_charge(folio, memcg);
local_irq_disable();
mem_cgroup_charge_statistics(memcg, nr_pages);
@@ -6947,21 +6948,21 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
*/
void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
{
+ struct folio *newfolio = page_folio(newpage);
struct mem_cgroup *memcg;
- unsigned int nr_pages;
+ unsigned int nr_pages = folio_nr_pages(newfolio);
unsigned long flags;
VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
- VM_BUG_ON_PAGE(!PageLocked(newpage), newpage);
- VM_BUG_ON_PAGE(PageAnon(oldpage) != PageAnon(newpage), newpage);
- VM_BUG_ON_PAGE(PageTransHuge(oldpage) != PageTransHuge(newpage),
- newpage);
+ VM_BUG_ON_FOLIO(!folio_locked(newfolio), newfolio);
+ VM_BUG_ON_FOLIO(PageAnon(oldpage) != folio_anon(newfolio), newfolio);
+ VM_BUG_ON_FOLIO(compound_nr(oldpage) != nr_pages, newfolio);
if (mem_cgroup_disabled())
return;
/* Page cache replacement: new page already charged? */
- if (page_memcg(newpage))
+ if (folio_memcg(newfolio))
return;
memcg = page_memcg(oldpage);
@@ -6970,8 +6971,6 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
return;
/* Force-charge the new page. The old one will be freed soon */
- nr_pages = thp_nr_pages(newpage);
-
if (!mem_cgroup_is_root(memcg)) {
page_counter_charge(&memcg->memory, nr_pages);
if (do_memsw_account())
@@ -6979,7 +6978,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
}
css_get(&memcg->css);
- commit_charge(newpage, memcg);
+ commit_charge(newfolio, memcg);
local_irq_save(flags);
mem_cgroup_charge_statistics(memcg, nr_pages);
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (6 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 07/18] mm/memcg: Convert commit_charge() to take a folio Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 7:17 ` kernel test robot
2021-07-01 7:13 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio() Matthew Wilcox (Oracle)
` (10 subsequent siblings)
18 siblings, 2 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
Convert all callers of mem_cgroup_charge() to call page_folio() on the
page they're currently passing in. Many of them will be converted to
use folios themselves soon.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 6 +++---
kernel/events/uprobes.c | 3 ++-
mm/filemap.c | 2 +-
mm/huge_memory.c | 2 +-
mm/khugepaged.c | 4 ++--
mm/ksm.c | 3 ++-
mm/memcontrol.c | 26 +++++++++++++-------------
mm/memory.c | 9 +++++----
mm/migrate.c | 2 +-
mm/shmem.c | 2 +-
mm/userfaultfd.c | 2 +-
11 files changed, 32 insertions(+), 29 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 92689fb2dab4..90d48b0e3191 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -705,7 +705,7 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *memcg)
page_counter_read(&memcg->memory);
}
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask);
+int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
gfp_t gfp, swp_entry_t entry);
void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
@@ -1186,8 +1186,8 @@ static inline bool mem_cgroup_below_min(struct mem_cgroup *memcg)
return false;
}
-static inline int mem_cgroup_charge(struct page *page, struct mm_struct *mm,
- gfp_t gfp_mask)
+static inline int mem_cgroup_charge(struct folio *folio,
+ struct mm_struct *mm, gfp_t gfp)
{
return 0;
}
diff --git a/kernel/events/uprobes.c b/kernel/events/uprobes.c
index af24dc3febbe..6357c3580d07 100644
--- a/kernel/events/uprobes.c
+++ b/kernel/events/uprobes.c
@@ -167,7 +167,8 @@ static int __replace_page(struct vm_area_struct *vma, unsigned long addr,
addr + PAGE_SIZE);
if (new_page) {
- err = mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL);
+ err = mem_cgroup_charge(page_folio(new_page), vma->vm_mm,
+ GFP_KERNEL);
if (err)
return err;
}
diff --git a/mm/filemap.c b/mm/filemap.c
index 3fb48130f980..9600bca84162 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -872,7 +872,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
page->index = offset;
if (!huge) {
- error = mem_cgroup_charge(page, NULL, gfp);
+ error = mem_cgroup_charge(page_folio(page), NULL, gfp);
if (error)
goto error;
charged = true;
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index 6d2a0119fc58..662f92bb1953 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -594,7 +594,7 @@ static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,
VM_BUG_ON_PAGE(!PageCompound(page), page);
- if (mem_cgroup_charge(page, vma->vm_mm, gfp)) {
+ if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
put_page(page);
count_vm_event(THP_FAULT_FALLBACK);
count_vm_event(THP_FAULT_FALLBACK_CHARGE);
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 6c0185fdd815..0daa21fbdd71 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1088,7 +1088,7 @@ static void collapse_huge_page(struct mm_struct *mm,
goto out_nolock;
}
- if (unlikely(mem_cgroup_charge(new_page, mm, gfp))) {
+ if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
result = SCAN_CGROUP_CHARGE_FAIL;
goto out_nolock;
}
@@ -1659,7 +1659,7 @@ static void collapse_file(struct mm_struct *mm,
goto out;
}
- if (unlikely(mem_cgroup_charge(new_page, mm, gfp))) {
+ if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
result = SCAN_CGROUP_CHARGE_FAIL;
goto out;
}
diff --git a/mm/ksm.c b/mm/ksm.c
index 3fa9bc8a67cf..23d36b59f997 100644
--- a/mm/ksm.c
+++ b/mm/ksm.c
@@ -2580,7 +2580,8 @@ struct page *ksm_might_need_to_copy(struct page *page,
return page; /* let do_swap_page report the error */
new_page = alloc_page_vma(GFP_HIGHUSER_MOVABLE, vma, address);
- if (new_page && mem_cgroup_charge(new_page, vma->vm_mm, GFP_KERNEL)) {
+ if (new_page &&
+ mem_cgroup_charge(page_folio(new_page), vma->vm_mm, GFP_KERNEL)) {
put_page(new_page);
new_page = NULL;
}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 727bd578ca7d..1dc02ecd3257 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6676,10 +6676,9 @@ void mem_cgroup_calculate_protection(struct mem_cgroup *root,
atomic_long_read(&parent->memory.children_low_usage)));
}
-static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
+static int __mem_cgroup_charge(struct folio *folio, struct mem_cgroup *memcg,
gfp_t gfp)
{
- struct folio *folio = page_folio(page);
unsigned int nr_pages = folio_nr_pages(folio);
int ret;
@@ -6692,27 +6691,27 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
local_irq_disable();
mem_cgroup_charge_statistics(memcg, nr_pages);
- memcg_check_events(memcg, page_to_nid(page));
+ memcg_check_events(memcg, folio_nid(folio));
local_irq_enable();
out:
return ret;
}
/**
- * mem_cgroup_charge - charge a newly allocated page to a cgroup
- * @page: page to charge
- * @mm: mm context of the victim
- * @gfp_mask: reclaim mode
+ * mem_cgroup_charge - Charge a newly allocated folio to a cgroup.
+ * @folio: Folio to charge.
+ * @mm: mm context of the allocating task.
+ * @gfp: reclaim mode
*
- * Try to charge @page to the memcg that @mm belongs to, reclaiming
- * pages according to @gfp_mask if necessary. if @mm is NULL, try to
+ * Try to charge @folio to the memcg that @mm belongs to, reclaiming
+ * pages according to @gfp if necessary. If @mm is NULL, try to
* charge to the active memcg.
*
- * Do not use this for pages allocated for swapin.
+ * Do not use this for folios allocated for swapin.
*
* Returns 0 on success. Otherwise, an error code is returned.
*/
-int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
+int mem_cgroup_charge(struct folio *folio, struct mm_struct *mm, gfp_t gfp)
{
struct mem_cgroup *memcg;
int ret;
@@ -6721,7 +6720,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
return 0;
memcg = get_mem_cgroup_from_mm(mm);
- ret = __mem_cgroup_charge(page, memcg, gfp_mask);
+ ret = __mem_cgroup_charge(folio, memcg, gfp);
css_put(&memcg->css);
return ret;
@@ -6742,6 +6741,7 @@ int mem_cgroup_charge(struct page *page, struct mm_struct *mm, gfp_t gfp_mask)
int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
gfp_t gfp, swp_entry_t entry)
{
+ struct folio *folio = page_folio(page);
struct mem_cgroup *memcg;
unsigned short id;
int ret;
@@ -6756,7 +6756,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
memcg = get_mem_cgroup_from_mm(mm);
rcu_read_unlock();
- ret = __mem_cgroup_charge(page, memcg, gfp);
+ ret = __mem_cgroup_charge(folio, memcg, gfp);
css_put(&memcg->css);
return ret;
diff --git a/mm/memory.c b/mm/memory.c
index afff080caf8b..2e7a568e2c15 100644
--- a/mm/memory.c
+++ b/mm/memory.c
@@ -915,7 +915,7 @@ page_copy_prealloc(struct mm_struct *src_mm, struct vm_area_struct *vma,
if (!new_page)
return NULL;
- if (mem_cgroup_charge(new_page, src_mm, GFP_KERNEL)) {
+ if (mem_cgroup_charge(page_folio(new_page), src_mm, GFP_KERNEL)) {
put_page(new_page);
return NULL;
}
@@ -2922,7 +2922,7 @@ static vm_fault_t wp_page_copy(struct vm_fault *vmf)
}
}
- if (mem_cgroup_charge(new_page, mm, GFP_KERNEL))
+ if (mem_cgroup_charge(page_folio(new_page), mm, GFP_KERNEL))
goto oom_free_new;
cgroup_throttle_swaprate(new_page, GFP_KERNEL);
@@ -3640,7 +3640,7 @@ static vm_fault_t do_anonymous_page(struct vm_fault *vmf)
if (!page)
goto oom;
- if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+ if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
goto oom_free_page;
cgroup_throttle_swaprate(page, GFP_KERNEL);
@@ -4053,7 +4053,8 @@ static vm_fault_t do_cow_fault(struct vm_fault *vmf)
if (!vmf->cow_page)
return VM_FAULT_OOM;
- if (mem_cgroup_charge(vmf->cow_page, vma->vm_mm, GFP_KERNEL)) {
+ if (mem_cgroup_charge(page_folio(vmf->cow_page), vma->vm_mm,
+ GFP_KERNEL)) {
put_page(vmf->cow_page);
return VM_FAULT_OOM;
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 380ca57b9031..94efe09bb2a0 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -2914,7 +2914,7 @@ static void migrate_vma_insert_page(struct migrate_vma *migrate,
if (unlikely(anon_vma_prepare(vma)))
goto abort;
- if (mem_cgroup_charge(page, vma->vm_mm, GFP_KERNEL))
+ if (mem_cgroup_charge(page_folio(page), vma->vm_mm, GFP_KERNEL))
goto abort;
/*
diff --git a/mm/shmem.c b/mm/shmem.c
index 6268b9b4e41a..3cc5ddd5cc6b 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -685,7 +685,7 @@ static int shmem_add_to_page_cache(struct page *page,
page->index = index;
if (!PageSwapCache(page)) {
- error = mem_cgroup_charge(page, charge_mm, gfp);
+ error = mem_cgroup_charge(page_folio(page), charge_mm, gfp);
if (error) {
if (PageTransHuge(page)) {
count_vm_event(THP_FILE_FALLBACK);
diff --git a/mm/userfaultfd.c b/mm/userfaultfd.c
index 63a73e164d55..bc8a2c3e5aea 100644
--- a/mm/userfaultfd.c
+++ b/mm/userfaultfd.c
@@ -96,7 +96,7 @@ static int mcopy_atomic_pte(struct mm_struct *dst_mm,
__SetPageUptodate(page);
ret = -ENOMEM;
- if (mem_cgroup_charge(page, dst_mm, GFP_KERNEL))
+ if (mem_cgroup_charge(page_folio(page), dst_mm, GFP_KERNEL))
goto out_release;
_dst_pte = pte_mkdirty(mk_pte(page, dst_vma->vm_page_prot));
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (7 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() " Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-07-01 7:15 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio Matthew Wilcox (Oracle)
` (9 subsequent siblings)
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
Use a folio rather than a page to ensure that we're only operating on
base or head pages, and not tail pages.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
mm/memcontrol.c | 29 ++++++++++++++---------------
1 file changed, 14 insertions(+), 15 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 1dc02ecd3257..21b791935957 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6827,24 +6827,23 @@ static void uncharge_batch(const struct uncharge_gather *ug)
memcg_check_events(ug->memcg, ug->nid);
local_irq_restore(flags);
- /* drop reference from uncharge_page */
+ /* drop reference from uncharge_folio */
css_put(&ug->memcg->css);
}
-static void uncharge_page(struct page *page, struct uncharge_gather *ug)
+static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
{
- struct folio *folio = page_folio(page);
unsigned long nr_pages;
struct mem_cgroup *memcg;
struct obj_cgroup *objcg;
- bool use_objcg = PageMemcgKmem(page);
+ bool use_objcg = folio_memcg_kmem(folio);
- VM_BUG_ON_PAGE(PageLRU(page), page);
+ VM_BUG_ON_FOLIO(folio_lru(folio), folio);
/*
* Nobody should be changing or seriously looking at
- * page memcg or objcg at this point, we have fully
- * exclusive access to the page.
+ * folio memcg or objcg at this point, we have fully
+ * exclusive access to the folio.
*/
if (use_objcg) {
objcg = __folio_objcg(folio);
@@ -6866,19 +6865,19 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
uncharge_gather_clear(ug);
}
ug->memcg = memcg;
- ug->nid = page_to_nid(page);
+ ug->nid = folio_nid(folio);
/* pairs with css_put in uncharge_batch */
css_get(&memcg->css);
}
- nr_pages = compound_nr(page);
+ nr_pages = folio_nr_pages(folio);
if (use_objcg) {
ug->nr_memory += nr_pages;
ug->nr_kmem += nr_pages;
- page->memcg_data = 0;
+ folio->memcg_data = 0;
obj_cgroup_put(objcg);
} else {
/* LRU pages aren't accounted at the root level */
@@ -6886,7 +6885,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
ug->nr_memory += nr_pages;
ug->pgpgout++;
- page->memcg_data = 0;
+ folio->memcg_data = 0;
}
css_put(&memcg->css);
@@ -6910,7 +6909,7 @@ void mem_cgroup_uncharge(struct page *page)
return;
uncharge_gather_clear(&ug);
- uncharge_page(page, &ug);
+ uncharge_folio(page_folio(page), &ug);
uncharge_batch(&ug);
}
@@ -6924,14 +6923,14 @@ void mem_cgroup_uncharge(struct page *page)
void mem_cgroup_uncharge_list(struct list_head *page_list)
{
struct uncharge_gather ug;
- struct page *page;
+ struct folio *folio;
if (mem_cgroup_disabled())
return;
uncharge_gather_clear(&ug);
- list_for_each_entry(page, page_list, lru)
- uncharge_page(page, &ug);
+ list_for_each_entry(folio, page_list, lru)
+ uncharge_folio(folio, &ug);
if (ug.memcg)
uncharge_batch(&ug);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (8 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:46 ` kernel test robot
2021-07-01 7:17 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios Matthew Wilcox (Oracle)
` (8 subsequent siblings)
18 siblings, 2 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
Convert all the callers to call page_folio().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 4 ++--
mm/filemap.c | 2 +-
mm/khugepaged.c | 4 ++--
mm/memcontrol.c | 14 +++++++-------
mm/memory-failure.c | 2 +-
mm/memremap.c | 2 +-
mm/page_alloc.c | 2 +-
mm/swap.c | 2 +-
8 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 90d48b0e3191..d6386a2b9d7a 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -710,7 +710,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
gfp_t gfp, swp_entry_t entry);
void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
-void mem_cgroup_uncharge(struct page *page);
+void mem_cgroup_uncharge(struct folio *);
void mem_cgroup_uncharge_list(struct list_head *page_list);
void mem_cgroup_migrate(struct page *oldpage, struct page *newpage);
@@ -1202,7 +1202,7 @@ static inline void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry)
{
}
-static inline void mem_cgroup_uncharge(struct page *page)
+static inline void mem_cgroup_uncharge(struct folio *folio)
{
}
diff --git a/mm/filemap.c b/mm/filemap.c
index 9600bca84162..0008ada132c4 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -923,7 +923,7 @@ noinline int __add_to_page_cache_locked(struct page *page,
if (xas_error(&xas)) {
error = xas_error(&xas);
if (charged)
- mem_cgroup_uncharge(page);
+ mem_cgroup_uncharge(page_folio(page));
goto error;
}
diff --git a/mm/khugepaged.c b/mm/khugepaged.c
index 0daa21fbdd71..988a230c7a41 100644
--- a/mm/khugepaged.c
+++ b/mm/khugepaged.c
@@ -1212,7 +1212,7 @@ static void collapse_huge_page(struct mm_struct *mm,
mmap_write_unlock(mm);
out_nolock:
if (!IS_ERR_OR_NULL(*hpage))
- mem_cgroup_uncharge(*hpage);
+ mem_cgroup_uncharge(page_folio(*hpage));
trace_mm_collapse_huge_page(mm, isolated, result);
return;
}
@@ -1963,7 +1963,7 @@ static void collapse_file(struct mm_struct *mm,
out:
VM_BUG_ON(!list_empty(&pagelist));
if (!IS_ERR_OR_NULL(*hpage))
- mem_cgroup_uncharge(*hpage);
+ mem_cgroup_uncharge(page_folio(*hpage));
/* TODO: tracepoints */
}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 21b791935957..90a53f554371 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6892,24 +6892,24 @@ static void uncharge_folio(struct folio *folio, struct uncharge_gather *ug)
}
/**
- * mem_cgroup_uncharge - uncharge a page
- * @page: page to uncharge
+ * mem_cgroup_uncharge - Uncharge a folio.
+ * @folio: Folio to uncharge.
*
- * Uncharge a page previously charged with mem_cgroup_charge().
+ * Uncharge a folio previously charged with folio_charge_cgroup().
*/
-void mem_cgroup_uncharge(struct page *page)
+void mem_cgroup_uncharge(struct folio *folio)
{
struct uncharge_gather ug;
if (mem_cgroup_disabled())
return;
- /* Don't touch page->lru of any random page, pre-check: */
- if (!page_memcg(page))
+ /* Don't touch folio->lru of any random page, pre-check: */
+ if (!folio_memcg(folio))
return;
uncharge_gather_clear(&ug);
- uncharge_folio(page_folio(page), &ug);
+ uncharge_folio(folio, &ug);
uncharge_batch(&ug);
}
diff --git a/mm/memory-failure.c b/mm/memory-failure.c
index e5a1531f7f4e..7ada5959b5ad 100644
--- a/mm/memory-failure.c
+++ b/mm/memory-failure.c
@@ -750,7 +750,7 @@ static int delete_from_lru_cache(struct page *p)
* Poisoned page might never drop its ref count to 0 so we have
* to uncharge it manually from its memcg.
*/
- mem_cgroup_uncharge(p);
+ mem_cgroup_uncharge(page_folio(p));
/*
* drop the page count elevated by isolate_lru_page()
diff --git a/mm/memremap.c b/mm/memremap.c
index 15a074ffb8d7..6eac40f9f62a 100644
--- a/mm/memremap.c
+++ b/mm/memremap.c
@@ -508,7 +508,7 @@ void free_devmap_managed_page(struct page *page)
__ClearPageWaiters(page);
- mem_cgroup_uncharge(page);
+ mem_cgroup_uncharge(page_folio(page));
/*
* When a device_private page is freed, the page->mapping field
diff --git a/mm/page_alloc.c b/mm/page_alloc.c
index 0817d88383d5..5a5fcd4f21a8 100644
--- a/mm/page_alloc.c
+++ b/mm/page_alloc.c
@@ -737,7 +737,7 @@ static inline void free_the_page(struct page *page, unsigned int order)
void free_compound_page(struct page *page)
{
- mem_cgroup_uncharge(page);
+ mem_cgroup_uncharge(page_folio(page));
free_the_page(page, compound_order(page));
}
diff --git a/mm/swap.c b/mm/swap.c
index 6954cfebab4f..8ba62a930370 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -94,7 +94,7 @@ static void __page_cache_release(struct page *page)
static void __put_single_page(struct page *page)
{
__page_cache_release(page);
- mem_cgroup_uncharge(page);
+ mem_cgroup_uncharge(page_folio(page));
free_unref_page(page, 0);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (9 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-07-01 7:20 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 12/18] mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio Matthew Wilcox (Oracle)
` (7 subsequent siblings)
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
Convert all callers of mem_cgroup_migrate() to call page_folio() first.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 4 ++--
mm/filemap.c | 4 +++-
mm/memcontrol.c | 35 +++++++++++++++++------------------
mm/migrate.c | 4 +++-
mm/shmem.c | 5 ++++-
5 files changed, 29 insertions(+), 23 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index d6386a2b9d7a..2c57a405acd2 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -713,7 +713,7 @@ void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
void mem_cgroup_uncharge(struct folio *);
void mem_cgroup_uncharge_list(struct list_head *page_list);
-void mem_cgroup_migrate(struct page *oldpage, struct page *newpage);
+void mem_cgroup_migrate(struct folio *old, struct folio *new);
/**
* mem_cgroup_lruvec - get the lru list vector for a memcg & node
@@ -1210,7 +1210,7 @@ static inline void mem_cgroup_uncharge_list(struct list_head *page_list)
{
}
-static inline void mem_cgroup_migrate(struct page *old, struct page *new)
+static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
{
}
diff --git a/mm/filemap.c b/mm/filemap.c
index 0008ada132c4..964f1643dd97 100644
--- a/mm/filemap.c
+++ b/mm/filemap.c
@@ -817,6 +817,8 @@ EXPORT_SYMBOL(file_write_and_wait_range);
*/
void replace_page_cache_page(struct page *old, struct page *new)
{
+ struct folio *fold = page_folio(old);
+ struct folio *fnew = page_folio(new);
struct address_space *mapping = old->mapping;
void (*freepage)(struct page *) = mapping->a_ops->freepage;
pgoff_t offset = old->index;
@@ -831,7 +833,7 @@ void replace_page_cache_page(struct page *old, struct page *new)
new->mapping = mapping;
new->index = offset;
- mem_cgroup_migrate(old, new);
+ mem_cgroup_migrate(fold, fnew);
xas_lock_irqsave(&xas, flags);
xas_store(&xas, new);
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 90a53f554371..4ce2f2eb81d8 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -6936,36 +6936,35 @@ void mem_cgroup_uncharge_list(struct list_head *page_list)
}
/**
- * mem_cgroup_migrate - charge a page's replacement
- * @oldpage: currently circulating page
- * @newpage: replacement page
+ * mem_cgroup_migrate - Charge a folio's replacement.
+ * @old: Currently circulating folio.
+ * @new: Replacement folio.
*
- * Charge @newpage as a replacement page for @oldpage. @oldpage will
+ * Charge @new as a replacement folio for @old. @old will
* be uncharged upon free.
*
- * Both pages must be locked, @newpage->mapping must be set up.
+ * Both folios must be locked, @new->mapping must be set up.
*/
-void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
+void mem_cgroup_migrate(struct folio *old, struct folio *new)
{
- struct folio *newfolio = page_folio(newpage);
struct mem_cgroup *memcg;
- unsigned int nr_pages = folio_nr_pages(newfolio);
+ unsigned int nr_pages = folio_nr_pages(new);
unsigned long flags;
- VM_BUG_ON_PAGE(!PageLocked(oldpage), oldpage);
- VM_BUG_ON_FOLIO(!folio_locked(newfolio), newfolio);
- VM_BUG_ON_FOLIO(PageAnon(oldpage) != folio_anon(newfolio), newfolio);
- VM_BUG_ON_FOLIO(compound_nr(oldpage) != nr_pages, newfolio);
+ VM_BUG_ON_FOLIO(!folio_locked(old), old);
+ VM_BUG_ON_FOLIO(!folio_locked(new), new);
+ VM_BUG_ON_FOLIO(folio_anon(old) != folio_anon(new), new);
+ VM_BUG_ON_FOLIO(folio_nr_pages(old) != nr_pages, new);
if (mem_cgroup_disabled())
return;
- /* Page cache replacement: new page already charged? */
- if (folio_memcg(newfolio))
+ /* Page cache replacement: new folio already charged? */
+ if (folio_memcg(new))
return;
- memcg = page_memcg(oldpage);
- VM_WARN_ON_ONCE_PAGE(!memcg, oldpage);
+ memcg = folio_memcg(old);
+ VM_WARN_ON_ONCE_FOLIO(!memcg, old);
if (!memcg)
return;
@@ -6977,11 +6976,11 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
}
css_get(&memcg->css);
- commit_charge(newfolio, memcg);
+ commit_charge(new, memcg);
local_irq_save(flags);
mem_cgroup_charge_statistics(memcg, nr_pages);
- memcg_check_events(memcg, page_to_nid(newpage));
+ memcg_check_events(memcg, folio_nid(new));
local_irq_restore(flags);
}
diff --git a/mm/migrate.c b/mm/migrate.c
index 94efe09bb2a0..f71e72f9c812 100644
--- a/mm/migrate.c
+++ b/mm/migrate.c
@@ -582,6 +582,8 @@ static void copy_huge_page(struct page *dst, struct page *src)
*/
void migrate_page_states(struct page *newpage, struct page *page)
{
+ struct folio *folio = page_folio(page);
+ struct folio *newfolio = page_folio(newpage);
int cpupid;
if (PageError(page))
@@ -646,7 +648,7 @@ void migrate_page_states(struct page *newpage, struct page *page)
copy_page_owner(page, newpage);
if (!PageHuge(page))
- mem_cgroup_migrate(page, newpage);
+ mem_cgroup_migrate(folio, newfolio);
}
EXPORT_SYMBOL(migrate_page_states);
diff --git a/mm/shmem.c b/mm/shmem.c
index 3cc5ddd5cc6b..1e172fb40fb3 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1619,6 +1619,7 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
struct shmem_inode_info *info, pgoff_t index)
{
struct page *oldpage, *newpage;
+ struct folio *old, *new;
struct address_space *swap_mapping;
swp_entry_t entry;
pgoff_t swap_index;
@@ -1655,7 +1656,9 @@ static int shmem_replace_page(struct page **pagep, gfp_t gfp,
xa_lock_irq(&swap_mapping->i_pages);
error = shmem_replace_entry(swap_mapping, swap_index, oldpage, newpage);
if (!error) {
- mem_cgroup_migrate(oldpage, newpage);
+ old = page_folio(oldpage);
+ new = page_folio(newpage);
+ mem_cgroup_migrate(old, new);
__inc_lruvec_page_state(newpage, NR_FILE_PAGES);
__dec_lruvec_page_state(oldpage, NR_FILE_PAGES);
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 12/18] mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (10 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Matthew Wilcox (Oracle)
` (6 subsequent siblings)
18 siblings, 0 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov,
Christoph Hellwig
The page was only being used for the memcg and to gather trace
information, so this is a simple conversion. The only caller of
mem_cgroup_track_foreign_dirty() will be converted to folios in a later
patch, so doing this now makes that patch simpler.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
include/linux/memcontrol.h | 7 ++++---
include/trace/events/writeback.h | 8 ++++----
mm/memcontrol.c | 6 +++---
3 files changed, 11 insertions(+), 10 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 2c57a405acd2..ef79f9c0b296 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1556,17 +1556,18 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,
unsigned long *pheadroom, unsigned long *pdirty,
unsigned long *pwriteback);
-void mem_cgroup_track_foreign_dirty_slowpath(struct page *page,
+void mem_cgroup_track_foreign_dirty_slowpath(struct folio *folio,
struct bdi_writeback *wb);
static inline void mem_cgroup_track_foreign_dirty(struct page *page,
struct bdi_writeback *wb)
{
+ struct folio *folio = page_folio(page);
if (mem_cgroup_disabled())
return;
- if (unlikely(&page_memcg(page)->css != wb->memcg_css))
- mem_cgroup_track_foreign_dirty_slowpath(page, wb);
+ if (unlikely(&folio_memcg(folio)->css != wb->memcg_css))
+ mem_cgroup_track_foreign_dirty_slowpath(folio, wb);
}
void mem_cgroup_flush_foreign(struct bdi_writeback *wb);
diff --git a/include/trace/events/writeback.h b/include/trace/events/writeback.h
index 1efa463c4979..80b24801bbf7 100644
--- a/include/trace/events/writeback.h
+++ b/include/trace/events/writeback.h
@@ -235,9 +235,9 @@ TRACE_EVENT(inode_switch_wbs,
TRACE_EVENT(track_foreign_dirty,
- TP_PROTO(struct page *page, struct bdi_writeback *wb),
+ TP_PROTO(struct folio *folio, struct bdi_writeback *wb),
- TP_ARGS(page, wb),
+ TP_ARGS(folio, wb),
TP_STRUCT__entry(
__array(char, name, 32)
@@ -249,7 +249,7 @@ TRACE_EVENT(track_foreign_dirty,
),
TP_fast_assign(
- struct address_space *mapping = page_mapping(page);
+ struct address_space *mapping = folio_mapping(folio);
struct inode *inode = mapping ? mapping->host : NULL;
strscpy_pad(__entry->name, bdi_dev_name(wb->bdi), 32);
@@ -257,7 +257,7 @@ TRACE_EVENT(track_foreign_dirty,
__entry->ino = inode ? inode->i_ino : 0;
__entry->memcg_id = wb->memcg_css->id;
__entry->cgroup_ino = __trace_wb_assign_cgroup(wb);
- __entry->page_cgroup_ino = cgroup_ino(page_memcg(page)->css.cgroup);
+ __entry->page_cgroup_ino = cgroup_ino(folio_memcg(folio)->css.cgroup);
),
TP_printk("bdi %s[%llu]: ino=%lu memcg_id=%u cgroup_ino=%lu page_cgroup_ino=%lu",
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 4ce2f2eb81d8..b925bdce0c6e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -4566,17 +4566,17 @@ void mem_cgroup_wb_stats(struct bdi_writeback *wb, unsigned long *pfilepages,
* As being wrong occasionally doesn't matter, updates and accesses to the
* records are lockless and racy.
*/
-void mem_cgroup_track_foreign_dirty_slowpath(struct page *page,
+void mem_cgroup_track_foreign_dirty_slowpath(struct folio *folio,
struct bdi_writeback *wb)
{
- struct mem_cgroup *memcg = page_memcg(page);
+ struct mem_cgroup *memcg = folio_memcg(folio);
struct memcg_cgwb_frn *frn;
u64 now = get_jiffies_64();
u64 oldest_at = now;
int oldest = -1;
int i;
- trace_track_foreign_dirty(page, wb);
+ trace_track_foreign_dirty(folio, wb);
/*
* Pick the slot to use. If there is already a slot for @wb, keep
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (11 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 12/18] mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:32 ` Michal Hocko
2021-07-07 17:08 ` Johannes Weiner
2021-06-30 4:00 ` [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio Matthew Wilcox (Oracle)
` (5 subsequent siblings)
18 siblings, 2 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
These are the folio equivalents of lock_page_memcg() and
unlock_page_memcg(). Reimplement them as wrappers.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 10 +++++++++
mm/memcontrol.c | 45 ++++++++++++++++++++++++--------------
2 files changed, 39 insertions(+), 16 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index ef79f9c0b296..279ea2640365 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -951,6 +951,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
extern bool cgroup_memory_noswap;
#endif
+void folio_memcg_lock(struct folio *folio);
+void folio_memcg_unlock(struct folio *folio);
void lock_page_memcg(struct page *page);
void unlock_page_memcg(struct page *page);
@@ -1363,6 +1365,14 @@ static inline void unlock_page_memcg(struct page *page)
{
}
+static inline void folio_memcg_lock(struct folio *folio)
+{
+}
+
+static inline void folio_memcg_unlock(struct folio *folio)
+{
+}
+
static inline void mem_cgroup_handle_over_high(void)
{
}
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b925bdce0c6e..b94a6122f27d 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1960,18 +1960,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg)
}
/**
- * lock_page_memcg - lock a page and memcg binding
- * @page: the page
+ * folio_memcg_lock - Bind a folio to its memcg.
+ * @folio: The folio.
*
- * This function protects unlocked LRU pages from being moved to
+ * This function prevents unlocked LRU folios from being moved to
* another cgroup.
*
- * It ensures lifetime of the locked memcg. Caller is responsible
- * for the lifetime of the page.
+ * It ensures lifetime of the bound memcg. The caller is responsible
+ * for the lifetime of the folio.
*/
-void lock_page_memcg(struct page *page)
+void folio_memcg_lock(struct folio *folio)
{
- struct page *head = compound_head(page); /* rmap on tail pages */
struct mem_cgroup *memcg;
unsigned long flags;
@@ -1985,7 +1984,7 @@ void lock_page_memcg(struct page *page)
if (mem_cgroup_disabled())
return;
again:
- memcg = page_memcg(head);
+ memcg = folio_memcg(folio);
if (unlikely(!memcg))
return;
@@ -1999,7 +1998,7 @@ void lock_page_memcg(struct page *page)
return;
spin_lock_irqsave(&memcg->move_lock, flags);
- if (memcg != page_memcg(head)) {
+ if (memcg != folio_memcg(folio)) {
spin_unlock_irqrestore(&memcg->move_lock, flags);
goto again;
}
@@ -2013,9 +2012,15 @@ void lock_page_memcg(struct page *page)
memcg->move_lock_task = current;
memcg->move_lock_flags = flags;
}
+EXPORT_SYMBOL(folio_memcg_lock);
+
+void lock_page_memcg(struct page *page)
+{
+ folio_memcg_lock(page_folio(page));
+}
EXPORT_SYMBOL(lock_page_memcg);
-static void __unlock_page_memcg(struct mem_cgroup *memcg)
+static void __memcg_unlock(struct mem_cgroup *memcg)
{
if (memcg && memcg->move_lock_task == current) {
unsigned long flags = memcg->move_lock_flags;
@@ -2030,14 +2035,22 @@ static void __unlock_page_memcg(struct mem_cgroup *memcg)
}
/**
- * unlock_page_memcg - unlock a page and memcg binding
- * @page: the page
+ * folio_memcg_unlock - Release the binding between a folio and its memcg.
+ * @folio: The folio.
+ *
+ * This releases the binding created by folio_memcg_lock(). This does
+ * not change the accounting of this folio to its memcg, but it does
+ * permit others to change it.
*/
-void unlock_page_memcg(struct page *page)
+void folio_memcg_unlock(struct folio *folio)
{
- struct page *head = compound_head(page);
+ __memcg_unlock(folio_memcg(folio));
+}
+EXPORT_SYMBOL(folio_memcg_unlock);
- __unlock_page_memcg(page_memcg(head));
+void unlock_page_memcg(struct page *page)
+{
+ folio_memcg_unlock(page_folio(page));
}
EXPORT_SYMBOL(unlock_page_memcg);
@@ -5661,7 +5674,7 @@ static int mem_cgroup_move_account(struct page *page,
page->memcg_data = (unsigned long)to;
- __unlock_page_memcg(from);
+ __memcg_unlock(from);
ret = 0;
nid = page_to_nid(page);
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (12 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:30 ` Michal Hocko
2021-06-30 4:00 ` [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec() Matthew Wilcox (Oracle)
` (4 subsequent siblings)
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
This saves dozens of bytes of text by eliminating a lot of calls to
compound_head().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
mm/memcontrol.c | 37 +++++++++++++++++++------------------
1 file changed, 19 insertions(+), 18 deletions(-)
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index b94a6122f27d..95795b65ae3e 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -5585,38 +5585,39 @@ static int mem_cgroup_move_account(struct page *page,
struct mem_cgroup *from,
struct mem_cgroup *to)
{
+ struct folio *folio = page_folio(page);
struct lruvec *from_vec, *to_vec;
struct pglist_data *pgdat;
- unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
+ unsigned int nr_pages = compound ? folio_nr_pages(folio) : 1;
int nid, ret;
VM_BUG_ON(from == to);
- VM_BUG_ON_PAGE(PageLRU(page), page);
- VM_BUG_ON(compound && !PageTransHuge(page));
+ VM_BUG_ON_FOLIO(folio_lru(folio), folio);
+ VM_BUG_ON(compound && !folio_multi(folio));
/*
* Prevent mem_cgroup_migrate() from looking at
* page's memory cgroup of its source page while we change it.
*/
ret = -EBUSY;
- if (!trylock_page(page))
+ if (!folio_trylock(folio))
goto out;
ret = -EINVAL;
- if (page_memcg(page) != from)
+ if (folio_memcg(folio) != from)
goto out_unlock;
- pgdat = page_pgdat(page);
+ pgdat = folio_pgdat(folio);
from_vec = mem_cgroup_lruvec(from, pgdat);
to_vec = mem_cgroup_lruvec(to, pgdat);
- lock_page_memcg(page);
+ folio_memcg_lock(folio);
- if (PageAnon(page)) {
- if (page_mapped(page)) {
+ if (folio_anon(folio)) {
+ if (folio_mapped(folio)) {
__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
- if (PageTransHuge(page)) {
+ if (folio_multi(folio)) {
__mod_lruvec_state(from_vec, NR_ANON_THPS,
-nr_pages);
__mod_lruvec_state(to_vec, NR_ANON_THPS,
@@ -5627,18 +5628,18 @@ static int mem_cgroup_move_account(struct page *page,
__mod_lruvec_state(from_vec, NR_FILE_PAGES, -nr_pages);
__mod_lruvec_state(to_vec, NR_FILE_PAGES, nr_pages);
- if (PageSwapBacked(page)) {
+ if (folio_swapbacked(folio)) {
__mod_lruvec_state(from_vec, NR_SHMEM, -nr_pages);
__mod_lruvec_state(to_vec, NR_SHMEM, nr_pages);
}
- if (page_mapped(page)) {
+ if (folio_mapped(folio)) {
__mod_lruvec_state(from_vec, NR_FILE_MAPPED, -nr_pages);
__mod_lruvec_state(to_vec, NR_FILE_MAPPED, nr_pages);
}
- if (PageDirty(page)) {
- struct address_space *mapping = page_mapping(page);
+ if (folio_dirty(folio)) {
+ struct address_space *mapping = folio_mapping(folio);
if (mapping_can_writeback(mapping)) {
__mod_lruvec_state(from_vec, NR_FILE_DIRTY,
@@ -5649,7 +5650,7 @@ static int mem_cgroup_move_account(struct page *page,
}
}
- if (PageWriteback(page)) {
+ if (folio_writeback(folio)) {
__mod_lruvec_state(from_vec, NR_WRITEBACK, -nr_pages);
__mod_lruvec_state(to_vec, NR_WRITEBACK, nr_pages);
}
@@ -5672,12 +5673,12 @@ static int mem_cgroup_move_account(struct page *page,
css_get(&to->css);
css_put(&from->css);
- page->memcg_data = (unsigned long)to;
+ folio->memcg_data = (unsigned long)to;
__memcg_unlock(from);
ret = 0;
- nid = page_to_nid(page);
+ nid = folio_nid(folio);
local_irq_disable();
mem_cgroup_charge_statistics(to, nr_pages);
@@ -5686,7 +5687,7 @@ static int mem_cgroup_move_account(struct page *page,
memcg_check_events(from, nid);
local_irq_enable();
out_unlock:
- unlock_page(page);
+ folio_unlock(folio);
out:
return ret;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (13 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:12 ` kernel test robot
2021-06-30 19:18 ` Matthew Wilcox
2021-06-30 4:00 ` [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions Matthew Wilcox (Oracle)
` (3 subsequent siblings)
18 siblings, 2 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
This is the folio equivalent of mem_cgroup_page_lruvec().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 25 ++++++++++++++-----------
1 file changed, 14 insertions(+), 11 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index 279ea2640365..a7e1ccbc7ed6 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -752,18 +752,17 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
}
/**
- * mem_cgroup_page_lruvec - return lruvec for isolating/putting an LRU page
- * @page: the page
+ * mem_cgroup_folio_lruvec - return lruvec for isolating/putting an LRU folio
+ * @folio: Pointer to the folio.
*
- * This function relies on page->mem_cgroup being stable.
+ * This function relies on folio->mem_cgroup being stable.
*/
-static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
{
- pg_data_t *pgdat = page_pgdat(page);
- struct mem_cgroup *memcg = page_memcg(page);
+ struct mem_cgroup *memcg = folio_memcg(folio);
- VM_WARN_ON_ONCE_PAGE(!memcg && !mem_cgroup_disabled(), page);
- return mem_cgroup_lruvec(memcg, pgdat);
+ VM_WARN_ON_ONCE_FOLIO(!memcg && !mem_cgroup_disabled(), folio);
+ return mem_cgroup_lruvec(memcg, folio_pgdat(folio));
}
struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
@@ -1222,10 +1221,9 @@ static inline struct lruvec *mem_cgroup_lruvec(struct mem_cgroup *memcg,
return &pgdat->__lruvec;
}
-static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
{
- pg_data_t *pgdat = page_pgdat(page);
-
+ struct pglist_data *pgdat = folio_pgdat(folio);
return &pgdat->__lruvec;
}
@@ -1485,6 +1483,11 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
}
#endif /* CONFIG_MEMCG */
+static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
+{
+ return mem_cgroup_folio_lruvec(page_folio(page));
+}
+
static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
{
__mod_lruvec_kmem_state(p, idx, 1);
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (14 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:36 ` Michal Hocko
2021-06-30 4:00 ` [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave() Matthew Wilcox (Oracle)
` (2 subsequent siblings)
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
These are the folio equivalents of lock_page_lruvec() and
similar functions. Retain lock_page_lruvec() as wrappers so we
don't have to convert all these functions twice. Also convert
lruvec_memcg_debug() to take a folio.
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 39 +++++++++++++++++++++++++++-----------
mm/compaction.c | 2 +-
mm/memcontrol.c | 33 ++++++++++++++------------------
3 files changed, 43 insertions(+), 31 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index a7e1ccbc7ed6..b21a77669277 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -769,15 +769,16 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
-struct lruvec *lock_page_lruvec(struct page *page);
-struct lruvec *lock_page_lruvec_irq(struct page *page);
-struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+struct lruvec *folio_lruvec_lock(struct folio *folio);
+struct lruvec *folio_lruvec_lock_irq(struct folio *folio);
+struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
unsigned long *flags);
#ifdef CONFIG_DEBUG_VM
-void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page);
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio);
#else
-static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+static inline
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
{
}
#endif
@@ -1257,26 +1258,26 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg)
{
}
-static inline struct lruvec *lock_page_lruvec(struct page *page)
+static inline struct lruvec *folio_lruvec_lock(struct folio *folio)
{
- struct pglist_data *pgdat = page_pgdat(page);
+ struct pglist_data *pgdat = folio_pgdat(folio);
spin_lock(&pgdat->__lruvec.lru_lock);
return &pgdat->__lruvec;
}
-static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
+static inline struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
{
- struct pglist_data *pgdat = page_pgdat(page);
+ struct pglist_data *pgdat = folio_pgdat(folio);
spin_lock_irq(&pgdat->__lruvec.lru_lock);
return &pgdat->__lruvec;
}
-static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
unsigned long *flagsp)
{
- struct pglist_data *pgdat = page_pgdat(page);
+ struct pglist_data *pgdat = folio_pgdat(folio);
spin_lock_irqsave(&pgdat->__lruvec.lru_lock, *flagsp);
return &pgdat->__lruvec;
@@ -1488,6 +1489,22 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
return mem_cgroup_folio_lruvec(page_folio(page));
}
+static inline struct lruvec *lock_page_lruvec(struct page *page)
+{
+ return folio_lruvec_lock(page_folio(page));
+}
+
+static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
+{
+ return folio_lruvec_lock_irq(page_folio(page));
+}
+
+static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
+ unsigned long *flags)
+{
+ return folio_lruvec_lock_irqsave(page_folio(page), flags);
+}
+
static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
{
__mod_lruvec_kmem_state(p, idx, 1);
diff --git a/mm/compaction.c b/mm/compaction.c
index 3a509fbf2bea..8b0da04b70f2 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -1038,7 +1038,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
locked = lruvec;
- lruvec_memcg_debug(lruvec, page);
+ lruvec_memcg_debug(lruvec, page_folio(page));
/* Try get exclusive access under lock */
if (!skip_updated) {
diff --git a/mm/memcontrol.c b/mm/memcontrol.c
index 95795b65ae3e..23b166917def 100644
--- a/mm/memcontrol.c
+++ b/mm/memcontrol.c
@@ -1153,19 +1153,19 @@ int mem_cgroup_scan_tasks(struct mem_cgroup *memcg,
}
#ifdef CONFIG_DEBUG_VM
-void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
+void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
{
struct mem_cgroup *memcg;
if (mem_cgroup_disabled())
return;
- memcg = page_memcg(page);
+ memcg = folio_memcg(folio);
if (!memcg)
- VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != root_mem_cgroup, page);
+ VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != root_mem_cgroup, folio);
else
- VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != memcg, page);
+ VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != memcg, folio);
}
#endif
@@ -1179,38 +1179,33 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
* - lock_page_memcg()
* - page->_refcount is zero
*/
-struct lruvec *lock_page_lruvec(struct page *page)
+struct lruvec *folio_lruvec_lock(struct folio *folio)
{
- struct lruvec *lruvec;
+ struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
- lruvec = mem_cgroup_page_lruvec(page);
spin_lock(&lruvec->lru_lock);
-
- lruvec_memcg_debug(lruvec, page);
+ lruvec_memcg_debug(lruvec, folio);
return lruvec;
}
-struct lruvec *lock_page_lruvec_irq(struct page *page)
+struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
{
- struct lruvec *lruvec;
+ struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
- lruvec = mem_cgroup_page_lruvec(page);
spin_lock_irq(&lruvec->lru_lock);
-
- lruvec_memcg_debug(lruvec, page);
+ lruvec_memcg_debug(lruvec, folio);
return lruvec;
}
-struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags)
+struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
+ unsigned long *flags)
{
- struct lruvec *lruvec;
+ struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
- lruvec = mem_cgroup_page_lruvec(page);
spin_lock_irqsave(&lruvec->lru_lock, *flags);
-
- lruvec_memcg_debug(lruvec, page);
+ lruvec_memcg_debug(lruvec, folio);
return lruvec;
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave()
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (15 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:39 ` Michal Hocko
2021-06-30 4:00 ` [PATCH v3 18/18] mm/workingset: Convert workingset_activation to take a folio Matthew Wilcox (Oracle)
2021-06-30 8:44 ` [PATCH v3 00/18] Folio conversion of memcg Michal Hocko
18 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle), Johannes Weiner, Michal Hocko, Vladimir Davydov
These are the folio equivalents of relock_page_lruvec_irq() and
folio_lruvec_relock_irqsave(), which are retained as compatibility wrappers.
Also convert lruvec_holds_page_lru_lock() to folio_lruvec_holds_lru_lock().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
---
include/linux/memcontrol.h | 31 ++++++++++++++++++++++---------
mm/vmscan.c | 2 +-
2 files changed, 23 insertions(+), 10 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index b21a77669277..e6b5e8fbf770 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -1545,40 +1545,53 @@ static inline void unlock_page_lruvec_irqrestore(struct lruvec *lruvec,
}
/* Test requires a stable page->memcg binding, see page_memcg() */
-static inline bool page_matches_lruvec(struct page *page, struct lruvec *lruvec)
+static inline bool folio_matches_lruvec(struct folio *folio,
+ struct lruvec *lruvec)
{
- return lruvec_pgdat(lruvec) == page_pgdat(page) &&
- lruvec_memcg(lruvec) == page_memcg(page);
+ return lruvec_pgdat(lruvec) == folio_pgdat(folio) &&
+ lruvec_memcg(lruvec) == folio_memcg(folio);
}
/* Don't lock again iff page's lruvec locked */
-static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
+static inline struct lruvec *folio_lruvec_relock_irq(struct folio *folio,
struct lruvec *locked_lruvec)
{
if (locked_lruvec) {
- if (page_matches_lruvec(page, locked_lruvec))
+ if (folio_matches_lruvec(folio, locked_lruvec))
return locked_lruvec;
unlock_page_lruvec_irq(locked_lruvec);
}
- return lock_page_lruvec_irq(page);
+ return folio_lruvec_lock_irq(folio);
}
/* Don't lock again iff page's lruvec locked */
-static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
+static inline struct lruvec *folio_lruvec_relock_irqsave(struct folio *folio,
struct lruvec *locked_lruvec, unsigned long *flags)
{
if (locked_lruvec) {
- if (page_matches_lruvec(page, locked_lruvec))
+ if (folio_matches_lruvec(folio, locked_lruvec))
return locked_lruvec;
unlock_page_lruvec_irqrestore(locked_lruvec, *flags);
}
- return lock_page_lruvec_irqsave(page, flags);
+ return folio_lruvec_lock_irqsave(folio, flags);
}
+static inline struct lruvec *relock_page_lruvec_irq(struct page *page,
+ struct lruvec *locked_lruvec)
+{
+ return folio_lruvec_relock_irq(page_folio(page), locked_lruvec);
+}
+
+static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
+ struct lruvec *locked_lruvec, unsigned long *flags)
+{
+ return folio_lruvec_relock_irqsave(page_folio(page), locked_lruvec,
+ flags);
+}
#ifdef CONFIG_CGROUP_WRITEBACK
struct wb_domain *mem_cgroup_wb_domain(struct bdi_writeback *wb);
diff --git a/mm/vmscan.c b/mm/vmscan.c
index d7c3cb8688dd..a8d8f4673451 100644
--- a/mm/vmscan.c
+++ b/mm/vmscan.c
@@ -2063,7 +2063,7 @@ static unsigned int move_pages_to_lru(struct lruvec *lruvec,
* All pages were isolated from the same lruvec (and isolation
* inhibits memcg migration).
*/
- VM_BUG_ON_PAGE(!page_matches_lruvec(page, lruvec), page);
+ VM_BUG_ON_PAGE(!folio_matches_lruvec(page_folio(page), lruvec), page);
add_page_to_lru_list(page, lruvec);
nr_pages = thp_nr_pages(page);
nr_moved += nr_pages;
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* [PATCH v3 18/18] mm/workingset: Convert workingset_activation to take a folio
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (16 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave() Matthew Wilcox (Oracle)
@ 2021-06-30 4:00 ` Matthew Wilcox (Oracle)
2021-06-30 8:44 ` [PATCH v3 00/18] Folio conversion of memcg Michal Hocko
18 siblings, 0 replies; 59+ messages in thread
From: Matthew Wilcox (Oracle) @ 2021-06-30 4:00 UTC (permalink / raw)
To: linux-mm, cgroups
Cc: Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov,
Christoph Hellwig
This function already assumed it was being passed a head page. No real
change here, except that thp_nr_pages() compiles away on kernels with
THP compiled out while folio_nr_pages() is always present. Also convert
page_memcg_rcu() to folio_memcg_rcu().
Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Reviewed-by: Christoph Hellwig <hch@lst.de>
---
include/linux/memcontrol.h | 18 +++++++++---------
include/linux/swap.h | 2 +-
mm/swap.c | 2 +-
mm/workingset.c | 10 +++++-----
4 files changed, 16 insertions(+), 16 deletions(-)
diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
index e6b5e8fbf770..be131c28b3bc 100644
--- a/include/linux/memcontrol.h
+++ b/include/linux/memcontrol.h
@@ -462,19 +462,19 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
}
/*
- * page_memcg_rcu - locklessly get the memory cgroup associated with a page
- * @page: a pointer to the page struct
+ * folio_memcg_rcu - Locklessly get the memory cgroup associated with a folio.
+ * @folio: Pointer to the folio.
*
- * Returns a pointer to the memory cgroup associated with the page,
- * or NULL. This function assumes that the page is known to have a
+ * Returns a pointer to the memory cgroup associated with the folio,
+ * or NULL. This function assumes that the folio is known to have a
* proper memory cgroup pointer. It's not safe to call this function
- * against some type of pages, e.g. slab pages or ex-slab pages.
+ * against some type of folios, e.g. slab folios or ex-slab folios.
*/
-static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
+static inline struct mem_cgroup *folio_memcg_rcu(struct folio *folio)
{
- unsigned long memcg_data = READ_ONCE(page->memcg_data);
+ unsigned long memcg_data = READ_ONCE(folio->memcg_data);
- VM_BUG_ON_PAGE(PageSlab(page), page);
+ VM_BUG_ON_FOLIO(folio_slab(folio), folio);
WARN_ON_ONCE(!rcu_read_lock_held());
if (memcg_data & MEMCG_DATA_KMEM) {
@@ -1125,7 +1125,7 @@ static inline struct mem_cgroup *page_memcg(struct page *page)
return NULL;
}
-static inline struct mem_cgroup *page_memcg_rcu(struct page *page)
+static inline struct mem_cgroup *folio_memcg_rcu(struct folio *folio)
{
WARN_ON_ONCE(!rcu_read_lock_held());
return NULL;
diff --git a/include/linux/swap.h b/include/linux/swap.h
index 950dd96007ad..614bbef65777 100644
--- a/include/linux/swap.h
+++ b/include/linux/swap.h
@@ -325,7 +325,7 @@ static inline swp_entry_t folio_swap_entry(struct folio *folio)
void workingset_age_nonresident(struct lruvec *lruvec, unsigned long nr_pages);
void *workingset_eviction(struct page *page, struct mem_cgroup *target_memcg);
void workingset_refault(struct page *page, void *shadow);
-void workingset_activation(struct page *page);
+void workingset_activation(struct folio *folio);
/* Only track the nodes of mappings with shadow entries */
void workingset_update_node(struct xa_node *node);
diff --git a/mm/swap.c b/mm/swap.c
index 8ba62a930370..3c817717af0c 100644
--- a/mm/swap.c
+++ b/mm/swap.c
@@ -447,7 +447,7 @@ void mark_page_accessed(struct page *page)
else
__lru_cache_activate_page(page);
ClearPageReferenced(page);
- workingset_activation(page);
+ workingset_activation(page_folio(page));
}
if (page_is_idle(page))
clear_page_idle(page);
diff --git a/mm/workingset.c b/mm/workingset.c
index 4f7a306ce75a..86e239ec0306 100644
--- a/mm/workingset.c
+++ b/mm/workingset.c
@@ -390,9 +390,9 @@ void workingset_refault(struct page *page, void *shadow)
/**
* workingset_activation - note a page activation
- * @page: page that is being activated
+ * @folio: Folio that is being activated.
*/
-void workingset_activation(struct page *page)
+void workingset_activation(struct folio *folio)
{
struct mem_cgroup *memcg;
struct lruvec *lruvec;
@@ -405,11 +405,11 @@ void workingset_activation(struct page *page)
* XXX: See workingset_refault() - this should return
* root_mem_cgroup even for !CONFIG_MEMCG.
*/
- memcg = page_memcg_rcu(page);
+ memcg = folio_memcg_rcu(folio);
if (!mem_cgroup_disabled() && !memcg)
goto out;
- lruvec = mem_cgroup_page_lruvec(page);
- workingset_age_nonresident(lruvec, thp_nr_pages(page));
+ lruvec = mem_cgroup_folio_lruvec(folio);
+ workingset_age_nonresident(lruvec, folio_nr_pages(folio));
out:
rcu_read_unlock();
}
--
2.30.2
^ permalink raw reply related [flat|nested] 59+ messages in thread
* Re: [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions
2021-06-30 4:00 ` [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions Matthew Wilcox (Oracle)
@ 2021-06-30 6:53 ` kernel test robot
2021-07-01 7:12 ` Christoph Hellwig
1 sibling, 0 replies; 59+ messages in thread
From: kernel test robot @ 2021-06-30 6:53 UTC (permalink / raw)
To: Matthew Wilcox (Oracle), linux-mm, cgroups
Cc: clang-built-linux, kbuild-all, Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov
[-- Attachment #1: Type: text/plain, Size: 31713 bytes --]
Hi "Matthew,
Thank you for the patch! Yet something to improve:
[auto build test ERROR on linus/master]
[also build test ERROR on next-20210629]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: x86_64-randconfig-a003-20210628 (attached as .config)
compiler: clang version 13.0.0 (https://github.com/llvm/llvm-project 8d21d5472501460933e78aead04cf59579025ba4)
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# install x86_64 cross compiling tool for clang build
# apt-get install binutils-x86-64-linux-gnu
# https://github.com/0day-ci/linux/commit/615adf6169006fa1bc5e19d2105ff9a333bef423
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
git checkout 615adf6169006fa1bc5e19d2105ff9a333bef423
# save the attached .config to linux build tree
mkdir build_dir
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=clang make.cross O=build_dir ARCH=x86_64 olddefconfig
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All errors (new ones prefixed by >>):
In file included from arch/x86/kernel/asm-offsets.c:13:
In file included from include/linux/suspend.h:5:
In file included from include/linux/swap.h:9:
In file included from include/linux/memcontrol.h:13:
In file included from include/linux/cgroup.h:28:
In file included from include/linux/cgroup-defs.h:22:
In file included from include/linux/bpf-cgroup.h:5:
In file included from include/linux/bpf.h:20:
In file included from include/linux/kallsyms.h:12:
include/linux/mm.h:1382:42: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline int folio_nid(const struct folio *folio)
^
include/linux/mm.h:1384:27: error: incomplete definition of type 'struct folio'
return page_to_nid(&folio->page);
~~~~~^
include/linux/mm.h:1382:42: note: forward declaration of 'struct folio'
static inline int folio_nid(const struct folio *folio)
^
In file included from arch/x86/kernel/asm-offsets.c:13:
In file included from include/linux/suspend.h:5:
In file included from include/linux/swap.h:9:
include/linux/memcontrol.h:375:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:399:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
>> include/linux/memcontrol.h:401:34: error: incomplete definition of type 'struct folio'
unsigned long memcg_data = folio->memcg_data;
~~~~~^
include/linux/memcontrol.h:399:55: note: forward declaration of 'struct folio'
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:420:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:422:34: error: incomplete definition of type 'struct folio'
unsigned long memcg_data = folio->memcg_data;
~~~~~^
include/linux/memcontrol.h:420:55: note: forward declaration of 'struct folio'
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:424:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:424:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:451:53: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct mem_cgroup *folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:453:23: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
if (folio_memcg_kmem(folio))
^~~~~
include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:454:41: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
return obj_cgroup_memcg(__folio_objcg(folio));
^~~~~
include/linux/memcontrol.h:420:62: note: passing argument to parameter 'folio' here
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:456:24: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
return __folio_memcg(folio);
^~~~~
include/linux/memcontrol.h:399:62: note: passing argument to parameter 'folio' here
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
return folio_memcg(page_folio(page));
^
include/linux/memcontrol.h:461:21: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
return folio_memcg(page_folio(page));
^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:451:60: note: passing argument to parameter 'folio' here
static inline struct mem_cgroup *folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:541:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:541:20: error: conflicting types for 'folio_memcg_kmem'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:375:20: note: previous declaration is here
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:543:35: error: incomplete definition of type 'struct folio'
VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
~~~~~^
include/linux/mmdebug.h:76:60: note: expanded from macro 'VM_BUG_ON_PGFLAGS'
#define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
^~~~
include/linux/build_bug.h:30:63: note: expanded from macro 'BUILD_BUG_ON_INVALID'
#define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:544:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
^
include/linux/memcontrol.h:544:23: error: incomplete definition of type 'struct folio'
VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
~~~~~^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:545:14: error: incomplete definition of type 'struct folio'
return folio->memcg_data & MEMCG_DATA_KMEM;
~~~~~^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:607:26: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
return folio_memcg_kmem(page_folio(page));
^
include/linux/memcontrol.h:607:26: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
return folio_memcg_kmem(page_folio(page));
^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
static inline bool folio_memcg_kmem(struct folio *folio);
^
8 warnings and 17 errors generated.
--
In file included from arch/x86/kernel/asm-offsets.c:13:
In file included from include/linux/suspend.h:5:
In file included from include/linux/swap.h:9:
In file included from include/linux/memcontrol.h:13:
In file included from include/linux/cgroup.h:28:
In file included from include/linux/cgroup-defs.h:22:
In file included from include/linux/bpf-cgroup.h:5:
In file included from include/linux/bpf.h:20:
In file included from include/linux/kallsyms.h:12:
include/linux/mm.h:1382:42: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline int folio_nid(const struct folio *folio)
^
include/linux/mm.h:1384:27: error: incomplete definition of type 'struct folio'
return page_to_nid(&folio->page);
~~~~~^
include/linux/mm.h:1382:42: note: forward declaration of 'struct folio'
static inline int folio_nid(const struct folio *folio)
^
In file included from arch/x86/kernel/asm-offsets.c:13:
In file included from include/linux/suspend.h:5:
In file included from include/linux/swap.h:9:
include/linux/memcontrol.h:375:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:399:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
>> include/linux/memcontrol.h:401:34: error: incomplete definition of type 'struct folio'
unsigned long memcg_data = folio->memcg_data;
~~~~~^
include/linux/memcontrol.h:399:55: note: forward declaration of 'struct folio'
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:420:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:422:34: error: incomplete definition of type 'struct folio'
unsigned long memcg_data = folio->memcg_data;
~~~~~^
include/linux/memcontrol.h:420:55: note: forward declaration of 'struct folio'
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:424:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:424:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:451:53: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct mem_cgroup *folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:453:23: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
if (folio_memcg_kmem(folio))
^~~~~
include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:454:41: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
return obj_cgroup_memcg(__folio_objcg(folio));
^~~~~
include/linux/memcontrol.h:420:62: note: passing argument to parameter 'folio' here
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:456:24: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
return __folio_memcg(folio);
^~~~~
include/linux/memcontrol.h:399:62: note: passing argument to parameter 'folio' here
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
return folio_memcg(page_folio(page));
^
include/linux/memcontrol.h:461:21: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
return folio_memcg(page_folio(page));
^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:451:60: note: passing argument to parameter 'folio' here
static inline struct mem_cgroup *folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:541:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:541:20: error: conflicting types for 'folio_memcg_kmem'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:375:20: note: previous declaration is here
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:543:35: error: incomplete definition of type 'struct folio'
VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
~~~~~^
include/linux/mmdebug.h:76:60: note: expanded from macro 'VM_BUG_ON_PGFLAGS'
#define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
^~~~
include/linux/build_bug.h:30:63: note: expanded from macro 'BUILD_BUG_ON_INVALID'
#define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:544:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
^
include/linux/memcontrol.h:544:23: error: incomplete definition of type 'struct folio'
VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
~~~~~^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:545:14: error: incomplete definition of type 'struct folio'
return folio->memcg_data & MEMCG_DATA_KMEM;
~~~~~^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:607:26: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
return folio_memcg_kmem(page_folio(page));
^
include/linux/memcontrol.h:607:26: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
return folio_memcg_kmem(page_folio(page));
^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
static inline bool folio_memcg_kmem(struct folio *folio);
^
8 warnings and 17 errors generated.
make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [Makefile:1235: prepare0] Error 2
make[1]: Target 'modules_prepare' not remade because of errors.
make: *** [Makefile:215: __sub-make] Error 2
make: Target 'modules_prepare' not remade because of errors.
--
In file included from arch/x86/kernel/asm-offsets.c:13:
In file included from include/linux/suspend.h:5:
In file included from include/linux/swap.h:9:
In file included from include/linux/memcontrol.h:13:
In file included from include/linux/cgroup.h:28:
In file included from include/linux/cgroup-defs.h:22:
In file included from include/linux/bpf-cgroup.h:5:
In file included from include/linux/bpf.h:20:
In file included from include/linux/kallsyms.h:12:
include/linux/mm.h:1382:42: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline int folio_nid(const struct folio *folio)
^
include/linux/mm.h:1384:27: error: incomplete definition of type 'struct folio'
return page_to_nid(&folio->page);
~~~~~^
include/linux/mm.h:1382:42: note: forward declaration of 'struct folio'
static inline int folio_nid(const struct folio *folio)
^
In file included from arch/x86/kernel/asm-offsets.c:13:
In file included from include/linux/suspend.h:5:
In file included from include/linux/swap.h:9:
include/linux/memcontrol.h:375:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:399:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
>> include/linux/memcontrol.h:401:34: error: incomplete definition of type 'struct folio'
unsigned long memcg_data = folio->memcg_data;
~~~~~^
include/linux/memcontrol.h:399:55: note: forward declaration of 'struct folio'
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
>> include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
>> include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:420:55: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:422:34: error: incomplete definition of type 'struct folio'
unsigned long memcg_data = folio->memcg_data;
~~~~~^
include/linux/memcontrol.h:420:55: note: forward declaration of 'struct folio'
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:424:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:424:18: error: implicit declaration of function 'folio_slab' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio_slab(folio), folio);
^
include/linux/memcontrol.h:451:53: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline struct mem_cgroup *folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:453:23: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
if (folio_memcg_kmem(folio))
^~~~~
include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:454:41: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
return obj_cgroup_memcg(__folio_objcg(folio));
^~~~~
include/linux/memcontrol.h:420:62: note: passing argument to parameter 'folio' here
static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
^
include/linux/memcontrol.h:456:24: error: incompatible pointer types passing 'struct folio *' to parameter of type 'struct folio *' [-Werror,-Wincompatible-pointer-types]
return __folio_memcg(folio);
^~~~~
include/linux/memcontrol.h:399:62: note: passing argument to parameter 'folio' here
static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
return folio_memcg(page_folio(page));
^
include/linux/memcontrol.h:461:21: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
return folio_memcg(page_folio(page));
^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:451:60: note: passing argument to parameter 'folio' here
static inline struct mem_cgroup *folio_memcg(struct folio *folio)
^
include/linux/memcontrol.h:541:44: warning: declaration of 'struct folio' will not be visible outside of this function [-Wvisibility]
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:541:20: error: conflicting types for 'folio_memcg_kmem'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:375:20: note: previous declaration is here
static inline bool folio_memcg_kmem(struct folio *folio);
^
include/linux/memcontrol.h:543:35: error: incomplete definition of type 'struct folio'
VM_BUG_ON_PGFLAGS(PageTail(&folio->page), &folio->page);
~~~~~^
include/linux/mmdebug.h:76:60: note: expanded from macro 'VM_BUG_ON_PGFLAGS'
#define VM_BUG_ON_PGFLAGS(cond, page) BUILD_BUG_ON_INVALID(cond)
^~~~
include/linux/build_bug.h:30:63: note: expanded from macro 'BUILD_BUG_ON_INVALID'
#define BUILD_BUG_ON_INVALID(e) ((void)(sizeof((__force long)(e))))
^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:544:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO' [-Werror,-Wimplicit-function-declaration]
VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
^
include/linux/memcontrol.h:544:23: error: incomplete definition of type 'struct folio'
VM_BUG_ON_FOLIO(folio->memcg_data & MEMCG_DATA_OBJCGS, folio);
~~~~~^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:545:14: error: incomplete definition of type 'struct folio'
return folio->memcg_data & MEMCG_DATA_KMEM;
~~~~~^
include/linux/memcontrol.h:541:44: note: forward declaration of 'struct folio'
static inline bool folio_memcg_kmem(struct folio *folio)
^
include/linux/memcontrol.h:607:26: error: implicit declaration of function 'page_folio' [-Werror,-Wimplicit-function-declaration]
return folio_memcg_kmem(page_folio(page));
^
include/linux/memcontrol.h:607:26: warning: incompatible integer to pointer conversion passing 'int' to parameter of type 'struct folio *' [-Wint-conversion]
return folio_memcg_kmem(page_folio(page));
^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:375:51: note: passing argument to parameter 'folio' here
static inline bool folio_memcg_kmem(struct folio *folio);
^
8 warnings and 17 errors generated.
make[2]: *** [scripts/Makefile.build:117: arch/x86/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [Makefile:1235: prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:215: __sub-make] Error 2
make: Target 'prepare' not remade because of errors.
vim +401 include/linux/memcontrol.h
373
374 static inline bool PageMemcgKmem(struct page *page);
> 375 static inline bool folio_memcg_kmem(struct folio *folio);
376
377 /*
378 * After the initialization objcg->memcg is always pointing at
379 * a valid memcg, but can be atomically swapped to the parent memcg.
380 *
381 * The caller must ensure that the returned memcg won't be released:
382 * e.g. acquire the rcu_read_lock or css_set_lock.
383 */
384 static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
385 {
386 return READ_ONCE(objcg->memcg);
387 }
388
389 /*
390 * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
391 * @folio: Pointer to the folio.
392 *
393 * Returns a pointer to the memory cgroup associated with the folio,
394 * or NULL. This function assumes that the folio is known to have a
395 * proper memory cgroup pointer. It's not safe to call this function
396 * against some type of folios, e.g. slab folios or ex-slab folios or
397 * kmem folios.
398 */
> 399 static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
400 {
> 401 unsigned long memcg_data = folio->memcg_data;
402
> 403 VM_BUG_ON_FOLIO(folio_slab(folio), folio);
404 VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
405 VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
406
407 return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
408 }
409
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 49428 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()
2021-06-30 4:00 ` [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree() Matthew Wilcox (Oracle)
@ 2021-06-30 6:55 ` Michal Hocko
2021-06-30 14:18 ` Johannes Weiner
2021-07-01 6:57 ` Christoph Hellwig
2 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 6:55 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:19, Matthew Wilcox wrote:
> By using the node id in mem_cgroup_update_tree(), we can delete
> soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo(). Saves 42
> bytes of kernel text on my config.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Thanks!
> ---
> mm/memcontrol.c | 24 ++++--------------------
> 1 file changed, 4 insertions(+), 20 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 25cad0fb7d4e..29b28a050707 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -446,28 +446,12 @@ ino_t page_cgroup_ino(struct page *page)
> return ino;
> }
>
> -static struct mem_cgroup_per_node *
> -mem_cgroup_page_nodeinfo(struct mem_cgroup *memcg, struct page *page)
> -{
> - int nid = page_to_nid(page);
> -
> - return memcg->nodeinfo[nid];
> -}
> -
> static struct mem_cgroup_tree_per_node *
> soft_limit_tree_node(int nid)
> {
> return soft_limit_tree.rb_tree_per_node[nid];
> }
>
> -static struct mem_cgroup_tree_per_node *
> -soft_limit_tree_from_page(struct page *page)
> -{
> - int nid = page_to_nid(page);
> -
> - return soft_limit_tree.rb_tree_per_node[nid];
> -}
> -
> static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
> struct mem_cgroup_tree_per_node *mctz,
> unsigned long new_usage_in_excess)
> @@ -538,13 +522,13 @@ static unsigned long soft_limit_excess(struct mem_cgroup *memcg)
> return excess;
> }
>
> -static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
> +static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
> {
> unsigned long excess;
> struct mem_cgroup_per_node *mz;
> struct mem_cgroup_tree_per_node *mctz;
>
> - mctz = soft_limit_tree_from_page(page);
> + mctz = soft_limit_tree_node(nid);
> if (!mctz)
> return;
> /*
> @@ -552,7 +536,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, struct page *page)
> * because their event counter is not touched.
> */
> for (; memcg; memcg = parent_mem_cgroup(memcg)) {
> - mz = mem_cgroup_page_nodeinfo(memcg, page);
> + mz = memcg->nodeinfo[nid];
> excess = soft_limit_excess(memcg);
> /*
> * We have to update the tree if mz is on RB-tree or
> @@ -879,7 +863,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
> MEM_CGROUP_TARGET_SOFTLIMIT);
> mem_cgroup_threshold(memcg);
> if (unlikely(do_softlimit))
> - mem_cgroup_update_tree(memcg, page);
> + mem_cgroup_update_tree(memcg, page_to_nid(page));
> }
> }
>
> --
> 2.30.2
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()
2021-06-30 4:00 ` [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node() Matthew Wilcox (Oracle)
@ 2021-06-30 6:56 ` Michal Hocko
2021-06-30 14:19 ` Johannes Weiner
2021-07-01 7:09 ` Christoph Hellwig
2 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 6:56 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:20, Matthew Wilcox wrote:
> Opencode this one-line function in its three callers.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Michal Hocko <mhocko@suse.com>
> ---
> mm/memcontrol.c | 12 +++---------
> 1 file changed, 3 insertions(+), 9 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 29b28a050707..29fdb70dca42 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -446,12 +446,6 @@ ino_t page_cgroup_ino(struct page *page)
> return ino;
> }
>
> -static struct mem_cgroup_tree_per_node *
> -soft_limit_tree_node(int nid)
> -{
> - return soft_limit_tree.rb_tree_per_node[nid];
> -}
> -
> static void __mem_cgroup_insert_exceeded(struct mem_cgroup_per_node *mz,
> struct mem_cgroup_tree_per_node *mctz,
> unsigned long new_usage_in_excess)
> @@ -528,7 +522,7 @@ static void mem_cgroup_update_tree(struct mem_cgroup *memcg, int nid)
> struct mem_cgroup_per_node *mz;
> struct mem_cgroup_tree_per_node *mctz;
>
> - mctz = soft_limit_tree_node(nid);
> + mctz = soft_limit_tree.rb_tree_per_node[nid];
> if (!mctz)
> return;
> /*
> @@ -567,7 +561,7 @@ static void mem_cgroup_remove_from_trees(struct mem_cgroup *memcg)
>
> for_each_node(nid) {
> mz = memcg->nodeinfo[nid];
> - mctz = soft_limit_tree_node(nid);
> + mctz = soft_limit_tree.rb_tree_per_node[nid];
> if (mctz)
> mem_cgroup_remove_exceeded(mz, mctz);
> }
> @@ -3415,7 +3409,7 @@ unsigned long mem_cgroup_soft_limit_reclaim(pg_data_t *pgdat, int order,
> if (order > 0)
> return 0;
>
> - mctz = soft_limit_tree_node(pgdat->node_id);
> + mctz = soft_limit_tree.rb_tree_per_node[pgdat->node_id];
>
> /*
> * Do not even bother to check the largest node if the root
> --
> 2.30.2
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID
2021-06-30 4:00 ` [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID Matthew Wilcox (Oracle)
@ 2021-06-30 6:58 ` Michal Hocko
2021-06-30 6:59 ` Michal Hocko
2021-07-01 7:09 ` Christoph Hellwig
1 sibling, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 6:58 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:21, Matthew Wilcox wrote:
> memcg_check_events only uses the page's nid, so call page_to_nid in the
> callers to make the folio conversion easier.
It will also make the interface slightly easier to follow as there
shouldn't be any real reason to take the page for these events.
So this is a good cleanup in general.
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Michal Hocko <mhocko@suse.com>
Thanks.
> ---
> mm/memcontrol.c | 23 ++++++++++++-----------
> 1 file changed, 12 insertions(+), 11 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 29fdb70dca42..5d143d46a8a4 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -846,7 +846,7 @@ static bool mem_cgroup_event_ratelimit(struct mem_cgroup *memcg,
> * Check events in order.
> *
> */
> -static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
> +static void memcg_check_events(struct mem_cgroup *memcg, int nid)
> {
> /* threshold event is triggered in finer grain than soft limit */
> if (unlikely(mem_cgroup_event_ratelimit(memcg,
> @@ -857,7 +857,7 @@ static void memcg_check_events(struct mem_cgroup *memcg, struct page *page)
> MEM_CGROUP_TARGET_SOFTLIMIT);
> mem_cgroup_threshold(memcg);
> if (unlikely(do_softlimit))
> - mem_cgroup_update_tree(memcg, page_to_nid(page));
> + mem_cgroup_update_tree(memcg, nid);
> }
> }
>
> @@ -5573,7 +5573,7 @@ static int mem_cgroup_move_account(struct page *page,
> struct lruvec *from_vec, *to_vec;
> struct pglist_data *pgdat;
> unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
> - int ret;
> + int nid, ret;
>
> VM_BUG_ON(from == to);
> VM_BUG_ON_PAGE(PageLRU(page), page);
> @@ -5662,12 +5662,13 @@ static int mem_cgroup_move_account(struct page *page,
> __unlock_page_memcg(from);
>
> ret = 0;
> + nid = page_to_nid(page);
>
> local_irq_disable();
> mem_cgroup_charge_statistics(to, nr_pages);
> - memcg_check_events(to, page);
> + memcg_check_events(to, nid);
> mem_cgroup_charge_statistics(from, -nr_pages);
> - memcg_check_events(from, page);
> + memcg_check_events(from, nid);
> local_irq_enable();
> out_unlock:
> unlock_page(page);
> @@ -6688,7 +6689,7 @@ static int __mem_cgroup_charge(struct page *page, struct mem_cgroup *memcg,
>
> local_irq_disable();
> mem_cgroup_charge_statistics(memcg, nr_pages);
> - memcg_check_events(memcg, page);
> + memcg_check_events(memcg, page_to_nid(page));
> local_irq_enable();
> out:
> return ret;
> @@ -6796,7 +6797,7 @@ struct uncharge_gather {
> unsigned long nr_memory;
> unsigned long pgpgout;
> unsigned long nr_kmem;
> - struct page *dummy_page;
> + int nid;
> };
>
> static inline void uncharge_gather_clear(struct uncharge_gather *ug)
> @@ -6820,7 +6821,7 @@ static void uncharge_batch(const struct uncharge_gather *ug)
> local_irq_save(flags);
> __count_memcg_events(ug->memcg, PGPGOUT, ug->pgpgout);
> __this_cpu_add(ug->memcg->vmstats_percpu->nr_page_events, ug->nr_memory);
> - memcg_check_events(ug->memcg, ug->dummy_page);
> + memcg_check_events(ug->memcg, ug->nid);
> local_irq_restore(flags);
>
> /* drop reference from uncharge_page */
> @@ -6861,7 +6862,7 @@ static void uncharge_page(struct page *page, struct uncharge_gather *ug)
> uncharge_gather_clear(ug);
> }
> ug->memcg = memcg;
> - ug->dummy_page = page;
> + ug->nid = page_to_nid(page);
>
> /* pairs with css_put in uncharge_batch */
> css_get(&memcg->css);
> @@ -6979,7 +6980,7 @@ void mem_cgroup_migrate(struct page *oldpage, struct page *newpage)
>
> local_irq_save(flags);
> mem_cgroup_charge_statistics(memcg, nr_pages);
> - memcg_check_events(memcg, newpage);
> + memcg_check_events(memcg, page_to_nid(newpage));
> local_irq_restore(flags);
> }
>
> @@ -7209,7 +7210,7 @@ void mem_cgroup_swapout(struct page *page, swp_entry_t entry)
> */
> VM_BUG_ON(!irqs_disabled());
> mem_cgroup_charge_statistics(memcg, -nr_entries);
> - memcg_check_events(memcg, page);
> + memcg_check_events(memcg, page_to_nid(page));
>
> css_put(&memcg->css);
> }
> --
> 2.30.2
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID
2021-06-30 6:58 ` Michal Hocko
@ 2021-06-30 6:59 ` Michal Hocko
0 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 6:59 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 08:58:20, Michal Hocko wrote:
> On Wed 30-06-21 05:00:21, Matthew Wilcox wrote:
> > memcg_check_events only uses the page's nid, so call page_to_nid in the
> > callers to make the folio conversion easier.
>
> It will also make the interface slightly easier to follow as there
> shouldn't be any real reason to take the page for these events.
> So this is a good cleanup in general.
>
> > Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
>
> Acked-by: Michal Hocko <mhocko@suse.com>
Btw. patches 2-5 seem good enough to go to Andrew even without the rest
so that you do not have to carry them along with the rest which is quite
large I can imagine.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio
2021-06-30 4:00 ` [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() " Matthew Wilcox (Oracle)
@ 2021-06-30 7:17 ` kernel test robot
2021-07-01 7:13 ` Christoph Hellwig
1 sibling, 0 replies; 59+ messages in thread
From: kernel test robot @ 2021-06-30 7:17 UTC (permalink / raw)
To: Matthew Wilcox (Oracle), linux-mm, cgroups
Cc: kbuild-all, Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov
[-- Attachment #1: Type: text/plain, Size: 21064 bytes --]
Hi "Matthew,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on next-20210629]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: sparc64-randconfig-r002-20210628 (attached as .config)
compiler: sparc64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/014d3a00c94acec84ed571a521e10b9fe77a95cc
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
git checkout 014d3a00c94acec84ed571a521e10b9fe77a95cc
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=sparc64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
include/linux/mm.h:1382:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1382 | static inline int folio_nid(const struct folio *folio)
| ^~~~~
include/linux/mm.h: In function 'folio_nid':
include/linux/mm.h:1384:27: error: dereferencing pointer to incomplete type 'const struct folio'
1384 | return page_to_nid(&folio->page);
| ^~
In file included from include/linux/rmap.h:12,
from mm/huge_memory.c:16:
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~
include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function '__folio_memcg':
include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
401 | unsigned long memcg_data = folio->memcg_data;
| ^~
include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
403 | VM_BUG_ON_FOLIO(folio_slab(folio), folio);
| ^~~~~~~~~~~~~~~
| VM_BUG_ON_MM
include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
403 | VM_BUG_ON_FOLIO(folio_slab(folio), folio);
| ^~~~~~~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function '__folio_objcg':
include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
422 | unsigned long memcg_data = folio->memcg_data;
| ^~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'folio_memcg':
include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
453 | if (folio_memcg_kmem(folio))
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
454 | return obj_cgroup_memcg(__folio_objcg(folio));
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h:456:24: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
456 | return __folio_memcg(folio);
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: In function 'page_memcg':
include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
461 | return folio_memcg(page_folio(page));
| ^~~~~~~~~~
include/linux/memcontrol.h:461:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
461 | return folio_memcg(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:589:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:589:20: error: conflicting types for 'folio_memcg_kmem'
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~~~~~~~~~~~~
include/linux/memcontrol.h: In function 'PageMemcgKmem':
include/linux/memcontrol.h:607:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
607 | return folio_memcg_kmem(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:589:51: note: expected 'struct folio *' but argument is of type 'int'
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:708:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
| ^~~~~
mm/huge_memory.c: In function '__do_huge_pmd_anonymous_page':
>> mm/huge_memory.c:597:24: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
597 | if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
| ^~~~~~~~~~~~~~~~
| |
| int
In file included from include/linux/rmap.h:12,
from mm/huge_memory.c:16:
include/linux/memcontrol.h:708:23: note: expected 'struct folio *' but argument is of type 'int'
708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
| ^~~~~~~~~~~~~~
mm/huge_memory.c: At top level:
include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
--
1384 | return page_to_nid(&folio->page);
| ^~
In file included from include/linux/rmap.h:12,
from mm/khugepaged.c:9:
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:375:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~
include/linux/memcontrol.h:399:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function '__folio_memcg':
include/linux/memcontrol.h:401:34: error: dereferencing pointer to incomplete type 'struct folio'
401 | unsigned long memcg_data = folio->memcg_data;
| ^~
include/linux/memcontrol.h:403:2: error: implicit declaration of function 'VM_BUG_ON_FOLIO'; did you mean 'VM_BUG_ON_MM'? [-Werror=implicit-function-declaration]
403 | VM_BUG_ON_FOLIO(folio_slab(folio), folio);
| ^~~~~~~~~~~~~~~
| VM_BUG_ON_MM
include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
403 | VM_BUG_ON_FOLIO(folio_slab(folio), folio);
| ^~~~~~~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function '__folio_objcg':
include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
422 | unsigned long memcg_data = folio->memcg_data;
| ^~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'folio_memcg':
include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
453 | if (folio_memcg_kmem(folio))
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
454 | return obj_cgroup_memcg(__folio_objcg(folio));
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h:456:24: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
456 | return __folio_memcg(folio);
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: In function 'page_memcg':
include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
461 | return folio_memcg(page_folio(page));
| ^~~~~~~~~~
include/linux/memcontrol.h:461:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
461 | return folio_memcg(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:589:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:589:20: error: conflicting types for 'folio_memcg_kmem'
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~~~~~~~~~~~~
include/linux/memcontrol.h: In function 'PageMemcgKmem':
include/linux/memcontrol.h:607:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
607 | return folio_memcg_kmem(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:589:51: note: expected 'struct folio *' but argument is of type 'int'
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:708:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
| ^~~~~
In file included from arch/sparc/include/asm/bug.h:6,
from include/linux/bug.h:5,
from include/linux/mmdebug.h:5,
from include/linux/mm.h:9,
from mm/khugepaged.c:4:
mm/khugepaged.c: In function 'collapse_huge_page':
>> mm/khugepaged.c:1091:33: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
1091 | if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
| ^~~~~~~~~~~~~~~~~~~~
| |
| int
include/linux/compiler.h:78:42: note: in definition of macro 'unlikely'
78 | # define unlikely(x) __builtin_expect(!!(x), 0)
| ^
In file included from include/linux/rmap.h:12,
from mm/khugepaged.c:9:
include/linux/memcontrol.h:708:23: note: expected 'struct folio *' but argument is of type 'int'
708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
| ^~~~~~~~~~~~~~
mm/khugepaged.c: At top level:
include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/mem_cgroup_charge +597 mm/huge_memory.c
586
587 static vm_fault_t __do_huge_pmd_anonymous_page(struct vm_fault *vmf,
588 struct page *page, gfp_t gfp)
589 {
590 struct vm_area_struct *vma = vmf->vma;
591 pgtable_t pgtable;
592 unsigned long haddr = vmf->address & HPAGE_PMD_MASK;
593 vm_fault_t ret = 0;
594
595 VM_BUG_ON_PAGE(!PageCompound(page), page);
596
> 597 if (mem_cgroup_charge(page_folio(page), vma->vm_mm, gfp)) {
598 put_page(page);
599 count_vm_event(THP_FAULT_FALLBACK);
600 count_vm_event(THP_FAULT_FALLBACK_CHARGE);
601 return VM_FAULT_FALLBACK;
602 }
603 cgroup_throttle_swaprate(page, gfp);
604
605 pgtable = pte_alloc_one(vma->vm_mm);
606 if (unlikely(!pgtable)) {
607 ret = VM_FAULT_OOM;
608 goto release;
609 }
610
611 clear_huge_page(page, vmf->address, HPAGE_PMD_NR);
612 /*
613 * The memory barrier inside __SetPageUptodate makes sure that
614 * clear_huge_page writes become visible before the set_pmd_at()
615 * write.
616 */
617 __SetPageUptodate(page);
618
619 vmf->ptl = pmd_lock(vma->vm_mm, vmf->pmd);
620 if (unlikely(!pmd_none(*vmf->pmd))) {
621 goto unlock_release;
622 } else {
623 pmd_t entry;
624
625 ret = check_stable_address_space(vma->vm_mm);
626 if (ret)
627 goto unlock_release;
628
629 /* Deliver the page fault to userland */
630 if (userfaultfd_missing(vma)) {
631 spin_unlock(vmf->ptl);
632 put_page(page);
633 pte_free(vma->vm_mm, pgtable);
634 ret = handle_userfault(vmf, VM_UFFD_MISSING);
635 VM_BUG_ON(ret & VM_FAULT_FALLBACK);
636 return ret;
637 }
638
639 entry = mk_huge_pmd(page, vma->vm_page_prot);
640 entry = maybe_pmd_mkwrite(pmd_mkdirty(entry), vma);
641 page_add_new_anon_rmap(page, vma, haddr, true);
642 lru_cache_add_inactive_or_unevictable(page, vma);
643 pgtable_trans_huge_deposit(vma->vm_mm, vmf->pmd, pgtable);
644 set_pmd_at(vma->vm_mm, haddr, vmf->pmd, entry);
645 update_mmu_cache_pmd(vma, vmf->address, vmf->pmd);
646 add_mm_counter(vma->vm_mm, MM_ANONPAGES, HPAGE_PMD_NR);
647 mm_inc_nr_ptes(vma->vm_mm);
648 spin_unlock(vmf->ptl);
649 count_vm_event(THP_FAULT_ALLOC);
650 count_memcg_event_mm(vma->vm_mm, THP_FAULT_ALLOC);
651 }
652
653 return 0;
654 unlock_release:
655 spin_unlock(vmf->ptl);
656 release:
657 if (pgtable)
658 pte_free(vma->vm_mm, pgtable);
659 put_page(page);
660 return ret;
661
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26865 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()
2021-06-30 4:00 ` [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec() Matthew Wilcox (Oracle)
@ 2021-06-30 8:12 ` kernel test robot
2021-06-30 19:18 ` Matthew Wilcox
1 sibling, 0 replies; 59+ messages in thread
From: kernel test robot @ 2021-06-30 8:12 UTC (permalink / raw)
To: Matthew Wilcox (Oracle), linux-mm, cgroups
Cc: kbuild-all, Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov
[-- Attachment #1: Type: text/plain, Size: 11115 bytes --]
Hi "Matthew,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on next-20210629]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: arm64-randconfig-c024-20210630 (attached as .config)
compiler: aarch64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/c7ce93681b922cb2b709109c21e8206ef623e0be
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
git checkout c7ce93681b922cb2b709109c21e8206ef623e0be
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=arm64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
In file included from arch/arm64/kernel/asm-offsets.c:12:
include/linux/mm.h:1382:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1382 | static inline int folio_nid(const struct folio *folio)
| ^~~~~
include/linux/mm.h: In function 'folio_nid':
include/linux/mm.h:1384:27: error: dereferencing pointer to incomplete type 'const struct folio'
1384 | return page_to_nid(&folio->page);
| ^~
In file included from include/linux/swap.h:9,
from include/linux/suspend.h:5,
from arch/arm64/kernel/asm-offsets.c:16:
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:1138:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1138 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:1190:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1190 | static inline int mem_cgroup_charge(struct folio *folio,
| ^~~~~
include/linux/memcontrol.h:1206:47: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1206 | static inline void mem_cgroup_uncharge(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:1214:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1214 | static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
| ^~~~~
include/linux/memcontrol.h:1224:61: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'mem_cgroup_folio_lruvec':
include/linux/memcontrol.h:1226:30: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
1226 | struct pglist_data *pgdat = folio_pgdat(folio);
| ^~~~~~~~~~~
| folio_nid
>> include/linux/memcontrol.h:1226:30: warning: initialization of 'struct pglist_data *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:1366:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1366 | static inline void folio_memcg_lock(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:1370:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1370 | static inline void folio_memcg_unlock(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'mem_cgroup_page_lruvec':
include/linux/memcontrol.h:1488:33: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
1488 | return mem_cgroup_folio_lruvec(page_folio(page));
| ^~~~~~~~~~
include/linux/memcontrol.h:1488:33: warning: passing argument 1 of 'mem_cgroup_folio_lruvec' makes pointer from integer without a cast [-Wint-conversion]
1488 | return mem_cgroup_folio_lruvec(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:1224:68: note: expected 'struct folio *' but argument is of type 'int'
1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
cc1: some warnings being treated as errors
--
In file included from arch/arm64/kernel/asm-offsets.c:12:
include/linux/mm.h:1382:42: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1382 | static inline int folio_nid(const struct folio *folio)
| ^~~~~
include/linux/mm.h: In function 'folio_nid':
include/linux/mm.h:1384:27: error: dereferencing pointer to incomplete type 'const struct folio'
1384 | return page_to_nid(&folio->page);
| ^~
In file included from include/linux/swap.h:9,
from include/linux/suspend.h:5,
from arch/arm64/kernel/asm-offsets.c:16:
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:1138:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1138 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:1190:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1190 | static inline int mem_cgroup_charge(struct folio *folio,
| ^~~~~
include/linux/memcontrol.h:1206:47: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1206 | static inline void mem_cgroup_uncharge(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:1214:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1214 | static inline void mem_cgroup_migrate(struct folio *old, struct folio *new)
| ^~~~~
include/linux/memcontrol.h:1224:61: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'mem_cgroup_folio_lruvec':
include/linux/memcontrol.h:1226:30: error: implicit declaration of function 'folio_pgdat'; did you mean 'folio_nid'? [-Werror=implicit-function-declaration]
1226 | struct pglist_data *pgdat = folio_pgdat(folio);
| ^~~~~~~~~~~
| folio_nid
>> include/linux/memcontrol.h:1226:30: warning: initialization of 'struct pglist_data *' from 'int' makes pointer from integer without a cast [-Wint-conversion]
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:1366:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1366 | static inline void folio_memcg_lock(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:1370:46: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
1370 | static inline void folio_memcg_unlock(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'mem_cgroup_page_lruvec':
include/linux/memcontrol.h:1488:33: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
1488 | return mem_cgroup_folio_lruvec(page_folio(page));
| ^~~~~~~~~~
include/linux/memcontrol.h:1488:33: warning: passing argument 1 of 'mem_cgroup_folio_lruvec' makes pointer from integer without a cast [-Wint-conversion]
1488 | return mem_cgroup_folio_lruvec(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:1224:68: note: expected 'struct folio *' but argument is of type 'int'
1224 | static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
cc1: some warnings being treated as errors
make[2]: *** [scripts/Makefile.build:117: arch/arm64/kernel/asm-offsets.s] Error 1
make[2]: Target '__build' not remade because of errors.
make[1]: *** [Makefile:1235: prepare0] Error 2
make[1]: Target 'prepare' not remade because of errors.
make: *** [Makefile:215: __sub-make] Error 2
make: Target 'prepare' not remade because of errors.
vim +1226 include/linux/memcontrol.h
1223
1224 static inline struct lruvec *mem_cgroup_folio_lruvec(struct folio *folio)
1225 {
> 1226 struct pglist_data *pgdat = folio_pgdat(folio);
1227 return &pgdat->__lruvec;
1228 }
1229
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 41649 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 4:00 ` [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio Matthew Wilcox (Oracle)
@ 2021-06-30 8:30 ` Michal Hocko
2021-06-30 11:22 ` Matthew Wilcox
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 8:30 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:30, Matthew Wilcox wrote:
> This saves dozens of bytes of text by eliminating a lot of calls to
> compound_head().
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
> mm/memcontrol.c | 37 +++++++++++++++++++------------------
> 1 file changed, 19 insertions(+), 18 deletions(-)
>
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index b94a6122f27d..95795b65ae3e 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -5585,38 +5585,39 @@ static int mem_cgroup_move_account(struct page *page,
> struct mem_cgroup *from,
> struct mem_cgroup *to)
> {
> + struct folio *folio = page_folio(page);
> struct lruvec *from_vec, *to_vec;
> struct pglist_data *pgdat;
> - unsigned int nr_pages = compound ? thp_nr_pages(page) : 1;
> + unsigned int nr_pages = compound ? folio_nr_pages(folio) : 1;
> int nid, ret;
>
> VM_BUG_ON(from == to);
> - VM_BUG_ON_PAGE(PageLRU(page), page);
> - VM_BUG_ON(compound && !PageTransHuge(page));
> + VM_BUG_ON_FOLIO(folio_lru(folio), folio);
> + VM_BUG_ON(compound && !folio_multi(folio));
>
> /*
> * Prevent mem_cgroup_migrate() from looking at
> * page's memory cgroup of its source page while we change it.
> */
> ret = -EBUSY;
> - if (!trylock_page(page))
> + if (!folio_trylock(folio))
> goto out;
>
> ret = -EINVAL;
> - if (page_memcg(page) != from)
> + if (folio_memcg(folio) != from)
> goto out_unlock;
>
> - pgdat = page_pgdat(page);
> + pgdat = folio_pgdat(folio);
> from_vec = mem_cgroup_lruvec(from, pgdat);
> to_vec = mem_cgroup_lruvec(to, pgdat);
>
> - lock_page_memcg(page);
> + folio_memcg_lock(folio);
>
> - if (PageAnon(page)) {
> - if (page_mapped(page)) {
> + if (folio_anon(folio)) {
> + if (folio_mapped(folio)) {
> __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
> __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> - if (PageTransHuge(page)) {
> + if (folio_multi(folio)) {
Shouldn't be folio_transhuge? The resulting code is the same but
folio_transhuge is more explicit and matches the THP aspect.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-06-30 4:00 ` [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Matthew Wilcox (Oracle)
@ 2021-06-30 8:32 ` Michal Hocko
2021-07-07 15:10 ` Matthew Wilcox
2021-07-07 17:08 ` Johannes Weiner
1 sibling, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 8:32 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:29, Matthew Wilcox wrote:
> These are the folio equivalents of lock_page_memcg() and
> unlock_page_memcg(). Reimplement them as wrappers.
Is there any reason why you haven't followed the same approach as for
the previous patches. I mean callers can call page_folio and then
lock_page_memcg wrapper shouldn't be really needed.
I do not really want to be annoying here but I have to say that I like
the conversion by previous patches much better than this wrapper
approach as mentioned during the previous review already. If you have
some reasons to stick with this approach for this particular case then
make it explicit in the changelog.
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
> include/linux/memcontrol.h | 10 +++++++++
> mm/memcontrol.c | 45 ++++++++++++++++++++++++--------------
> 2 files changed, 39 insertions(+), 16 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index ef79f9c0b296..279ea2640365 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -951,6 +951,8 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg);
> extern bool cgroup_memory_noswap;
> #endif
>
> +void folio_memcg_lock(struct folio *folio);
> +void folio_memcg_unlock(struct folio *folio);
> void lock_page_memcg(struct page *page);
> void unlock_page_memcg(struct page *page);
>
> @@ -1363,6 +1365,14 @@ static inline void unlock_page_memcg(struct page *page)
> {
> }
>
> +static inline void folio_memcg_lock(struct folio *folio)
> +{
> +}
> +
> +static inline void folio_memcg_unlock(struct folio *folio)
> +{
> +}
> +
> static inline void mem_cgroup_handle_over_high(void)
> {
> }
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index b925bdce0c6e..b94a6122f27d 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1960,18 +1960,17 @@ void mem_cgroup_print_oom_group(struct mem_cgroup *memcg)
> }
>
> /**
> - * lock_page_memcg - lock a page and memcg binding
> - * @page: the page
> + * folio_memcg_lock - Bind a folio to its memcg.
> + * @folio: The folio.
> *
> - * This function protects unlocked LRU pages from being moved to
> + * This function prevents unlocked LRU folios from being moved to
> * another cgroup.
> *
> - * It ensures lifetime of the locked memcg. Caller is responsible
> - * for the lifetime of the page.
> + * It ensures lifetime of the bound memcg. The caller is responsible
> + * for the lifetime of the folio.
> */
> -void lock_page_memcg(struct page *page)
> +void folio_memcg_lock(struct folio *folio)
> {
> - struct page *head = compound_head(page); /* rmap on tail pages */
> struct mem_cgroup *memcg;
> unsigned long flags;
>
> @@ -1985,7 +1984,7 @@ void lock_page_memcg(struct page *page)
> if (mem_cgroup_disabled())
> return;
> again:
> - memcg = page_memcg(head);
> + memcg = folio_memcg(folio);
> if (unlikely(!memcg))
> return;
>
> @@ -1999,7 +1998,7 @@ void lock_page_memcg(struct page *page)
> return;
>
> spin_lock_irqsave(&memcg->move_lock, flags);
> - if (memcg != page_memcg(head)) {
> + if (memcg != folio_memcg(folio)) {
> spin_unlock_irqrestore(&memcg->move_lock, flags);
> goto again;
> }
> @@ -2013,9 +2012,15 @@ void lock_page_memcg(struct page *page)
> memcg->move_lock_task = current;
> memcg->move_lock_flags = flags;
> }
> +EXPORT_SYMBOL(folio_memcg_lock);
> +
> +void lock_page_memcg(struct page *page)
> +{
> + folio_memcg_lock(page_folio(page));
> +}
> EXPORT_SYMBOL(lock_page_memcg);
>
> -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> +static void __memcg_unlock(struct mem_cgroup *memcg)
> {
> if (memcg && memcg->move_lock_task == current) {
> unsigned long flags = memcg->move_lock_flags;
> @@ -2030,14 +2035,22 @@ static void __unlock_page_memcg(struct mem_cgroup *memcg)
> }
>
> /**
> - * unlock_page_memcg - unlock a page and memcg binding
> - * @page: the page
> + * folio_memcg_unlock - Release the binding between a folio and its memcg.
> + * @folio: The folio.
> + *
> + * This releases the binding created by folio_memcg_lock(). This does
> + * not change the accounting of this folio to its memcg, but it does
> + * permit others to change it.
> */
> -void unlock_page_memcg(struct page *page)
> +void folio_memcg_unlock(struct folio *folio)
> {
> - struct page *head = compound_head(page);
> + __memcg_unlock(folio_memcg(folio));
> +}
> +EXPORT_SYMBOL(folio_memcg_unlock);
>
> - __unlock_page_memcg(page_memcg(head));
> +void unlock_page_memcg(struct page *page)
> +{
> + folio_memcg_unlock(page_folio(page));
> }
> EXPORT_SYMBOL(unlock_page_memcg);
>
> @@ -5661,7 +5674,7 @@ static int mem_cgroup_move_account(struct page *page,
>
> page->memcg_data = (unsigned long)to;
>
> - __unlock_page_memcg(from);
> + __memcg_unlock(from);
>
> ret = 0;
> nid = page_to_nid(page);
> --
> 2.30.2
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions
2021-06-30 4:00 ` [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions Matthew Wilcox (Oracle)
@ 2021-06-30 8:36 ` Michal Hocko
0 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 8:36 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:32, Matthew Wilcox wrote:
> These are the folio equivalents of lock_page_lruvec() and
> similar functions. Retain lock_page_lruvec() as wrappers so we
> don't have to convert all these functions twice. Also convert
> lruvec_memcg_debug() to take a folio.
Here again I would rather not go with wrappers. I can see how changing
all the callers (20+) is annoying but they all should be pretty trivial
so if we can avoid that I would go with a direct folio_lruvec_{un}lock
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
> include/linux/memcontrol.h | 39 +++++++++++++++++++++++++++-----------
> mm/compaction.c | 2 +-
> mm/memcontrol.c | 33 ++++++++++++++------------------
> 3 files changed, 43 insertions(+), 31 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index a7e1ccbc7ed6..b21a77669277 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -769,15 +769,16 @@ struct mem_cgroup *mem_cgroup_from_task(struct task_struct *p);
>
> struct mem_cgroup *get_mem_cgroup_from_mm(struct mm_struct *mm);
>
> -struct lruvec *lock_page_lruvec(struct page *page);
> -struct lruvec *lock_page_lruvec_irq(struct page *page);
> -struct lruvec *lock_page_lruvec_irqsave(struct page *page,
> +struct lruvec *folio_lruvec_lock(struct folio *folio);
> +struct lruvec *folio_lruvec_lock_irq(struct folio *folio);
> +struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
> unsigned long *flags);
>
> #ifdef CONFIG_DEBUG_VM
> -void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page);
> +void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio);
> #else
> -static inline void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
> +static inline
> +void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
> {
> }
> #endif
> @@ -1257,26 +1258,26 @@ static inline void mem_cgroup_put(struct mem_cgroup *memcg)
> {
> }
>
> -static inline struct lruvec *lock_page_lruvec(struct page *page)
> +static inline struct lruvec *folio_lruvec_lock(struct folio *folio)
> {
> - struct pglist_data *pgdat = page_pgdat(page);
> + struct pglist_data *pgdat = folio_pgdat(folio);
>
> spin_lock(&pgdat->__lruvec.lru_lock);
> return &pgdat->__lruvec;
> }
>
> -static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
> +static inline struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
> {
> - struct pglist_data *pgdat = page_pgdat(page);
> + struct pglist_data *pgdat = folio_pgdat(folio);
>
> spin_lock_irq(&pgdat->__lruvec.lru_lock);
> return &pgdat->__lruvec;
> }
>
> -static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
> +static inline struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
> unsigned long *flagsp)
> {
> - struct pglist_data *pgdat = page_pgdat(page);
> + struct pglist_data *pgdat = folio_pgdat(folio);
>
> spin_lock_irqsave(&pgdat->__lruvec.lru_lock, *flagsp);
> return &pgdat->__lruvec;
> @@ -1488,6 +1489,22 @@ static inline struct lruvec *mem_cgroup_page_lruvec(struct page *page)
> return mem_cgroup_folio_lruvec(page_folio(page));
> }
>
> +static inline struct lruvec *lock_page_lruvec(struct page *page)
> +{
> + return folio_lruvec_lock(page_folio(page));
> +}
> +
> +static inline struct lruvec *lock_page_lruvec_irq(struct page *page)
> +{
> + return folio_lruvec_lock_irq(page_folio(page));
> +}
> +
> +static inline struct lruvec *lock_page_lruvec_irqsave(struct page *page,
> + unsigned long *flags)
> +{
> + return folio_lruvec_lock_irqsave(page_folio(page), flags);
> +}
> +
> static inline void __inc_lruvec_kmem_state(void *p, enum node_stat_item idx)
> {
> __mod_lruvec_kmem_state(p, idx, 1);
> diff --git a/mm/compaction.c b/mm/compaction.c
> index 3a509fbf2bea..8b0da04b70f2 100644
> --- a/mm/compaction.c
> +++ b/mm/compaction.c
> @@ -1038,7 +1038,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
> locked = lruvec;
>
> - lruvec_memcg_debug(lruvec, page);
> + lruvec_memcg_debug(lruvec, page_folio(page));
>
> /* Try get exclusive access under lock */
> if (!skip_updated) {
> diff --git a/mm/memcontrol.c b/mm/memcontrol.c
> index 95795b65ae3e..23b166917def 100644
> --- a/mm/memcontrol.c
> +++ b/mm/memcontrol.c
> @@ -1153,19 +1153,19 @@ int mem_cgroup_scan_tasks(struct mem_cgroup *memcg,
> }
>
> #ifdef CONFIG_DEBUG_VM
> -void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
> +void lruvec_memcg_debug(struct lruvec *lruvec, struct folio *folio)
> {
> struct mem_cgroup *memcg;
>
> if (mem_cgroup_disabled())
> return;
>
> - memcg = page_memcg(page);
> + memcg = folio_memcg(folio);
>
> if (!memcg)
> - VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != root_mem_cgroup, page);
> + VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != root_mem_cgroup, folio);
> else
> - VM_BUG_ON_PAGE(lruvec_memcg(lruvec) != memcg, page);
> + VM_BUG_ON_FOLIO(lruvec_memcg(lruvec) != memcg, folio);
> }
> #endif
>
> @@ -1179,38 +1179,33 @@ void lruvec_memcg_debug(struct lruvec *lruvec, struct page *page)
> * - lock_page_memcg()
> * - page->_refcount is zero
> */
> -struct lruvec *lock_page_lruvec(struct page *page)
> +struct lruvec *folio_lruvec_lock(struct folio *folio)
> {
> - struct lruvec *lruvec;
> + struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
>
> - lruvec = mem_cgroup_page_lruvec(page);
> spin_lock(&lruvec->lru_lock);
> -
> - lruvec_memcg_debug(lruvec, page);
> + lruvec_memcg_debug(lruvec, folio);
>
> return lruvec;
> }
>
> -struct lruvec *lock_page_lruvec_irq(struct page *page)
> +struct lruvec *folio_lruvec_lock_irq(struct folio *folio)
> {
> - struct lruvec *lruvec;
> + struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
>
> - lruvec = mem_cgroup_page_lruvec(page);
> spin_lock_irq(&lruvec->lru_lock);
> -
> - lruvec_memcg_debug(lruvec, page);
> + lruvec_memcg_debug(lruvec, folio);
>
> return lruvec;
> }
>
> -struct lruvec *lock_page_lruvec_irqsave(struct page *page, unsigned long *flags)
> +struct lruvec *folio_lruvec_lock_irqsave(struct folio *folio,
> + unsigned long *flags)
> {
> - struct lruvec *lruvec;
> + struct lruvec *lruvec = mem_cgroup_folio_lruvec(folio);
>
> - lruvec = mem_cgroup_page_lruvec(page);
> spin_lock_irqsave(&lruvec->lru_lock, *flags);
> -
> - lruvec_memcg_debug(lruvec, page);
> + lruvec_memcg_debug(lruvec, folio);
>
> return lruvec;
> }
> --
> 2.30.2
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave()
2021-06-30 4:00 ` [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave() Matthew Wilcox (Oracle)
@ 2021-06-30 8:39 ` Michal Hocko
0 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 8:39 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:33, Matthew Wilcox wrote:
> These are the folio equivalents of relock_page_lruvec_irq() and
> folio_lruvec_relock_irqsave(), which are retained as compatibility wrappers.
> Also convert lruvec_holds_page_lru_lock() to folio_lruvec_holds_lru_lock().
$ git grep relock_page_lruvec_irqsave
include/linux/memcontrol.h:static inline struct lruvec *relock_page_lruvec_irqsave(struct page *page,
on your tree.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 00/18] Folio conversion of memcg
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
` (17 preceding siblings ...)
2021-06-30 4:00 ` [PATCH v3 18/18] mm/workingset: Convert workingset_activation to take a folio Matthew Wilcox (Oracle)
@ 2021-06-30 8:44 ` Michal Hocko
18 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 8:44 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 05:00:16, Matthew Wilcox wrote:
> After Michel's comments on "Folio-enabling the page cache", I thought it
> best to split out the memcg patches from the rest of the page cache folio
> patches and redo them to focus on correctness (ie always passing a folio).
>
> This is fundamentally for review rather than application. I've rebased
> on Linus' current tree, which includes the recent patchbomb from akpm.
> That upstream version won't boot on any system I have available, and I'm
> not comfortable asking for patches to be applied unless I can actually
> try them. That said, these patches were fine on top of 5.13.
>
> There are still a few functions which take pages, but they rely on other
> conversions happening first, which in turn rely on this set of patches,
> so I think this is a good place to stop, with the understanding that
> there will be more patches later.
Thanks for considering the last review feedback. After looking at this
pile the conversion is really straightforward and reasonably easy to
review. There are still some cases where you opted to go with
compatibility wrappers and I would still like to avoid those as it
somehow obfuscates the conversion IMHO. As I've said previously, though,
I do understand that this a tedious work so I do not want to make it
more annoying unnecessarily for you. The replacement with page_folio for
the replaced API looks better to me. If you have strong reasons (e.g.
due to dependencies etc.) please make it explicit in the respective
changelogs.
Thanks!
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio
2021-06-30 4:00 ` [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio Matthew Wilcox (Oracle)
@ 2021-06-30 8:46 ` kernel test robot
2021-07-01 7:17 ` Christoph Hellwig
1 sibling, 0 replies; 59+ messages in thread
From: kernel test robot @ 2021-06-30 8:46 UTC (permalink / raw)
To: Matthew Wilcox (Oracle), linux-mm, cgroups
Cc: kbuild-all, Matthew Wilcox (Oracle),
Johannes Weiner, Michal Hocko, Vladimir Davydov
[-- Attachment #1: Type: text/plain, Size: 15464 bytes --]
Hi "Matthew,
Thank you for the patch! Perhaps something to improve:
[auto build test WARNING on linus/master]
[also build test WARNING on next-20210630]
[cannot apply to hnaz-linux-mm/master tip/perf/core cgroup/for-next v5.13]
[If your patch is applied to the wrong git tree, kindly drop us a note.
And when submitting patch, we suggest to use '--base' as documented in
https://git-scm.com/docs/git-format-patch]
url: https://github.com/0day-ci/linux/commits/Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
base: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git 007b350a58754a93ca9fe50c498cc27780171153
config: sparc64-randconfig-r002-20210628 (attached as .config)
compiler: sparc64-linux-gcc (GCC) 9.3.0
reproduce (this is a W=1 build):
wget https://raw.githubusercontent.com/intel/lkp-tests/master/sbin/make.cross -O ~/bin/make.cross
chmod +x ~/bin/make.cross
# https://github.com/0day-ci/linux/commit/b527e805e7d7066a8fea14ff4a49f53454c355a1
git remote add linux-review https://github.com/0day-ci/linux
git fetch --no-tags linux-review Matthew-Wilcox-Oracle/Folio-conversion-of-memcg/20210630-121408
git checkout b527e805e7d7066a8fea14ff4a49f53454c355a1
# save the attached .config to linux build tree
COMPILER_INSTALL_PATH=$HOME/0day COMPILER=gcc-9.3.0 make.cross ARCH=sparc64
If you fix the issue, kindly add following tag as appropriate
Reported-by: kernel test robot <lkp@intel.com>
All warnings (new ones prefixed by >>):
403 | VM_BUG_ON_FOLIO(folio_slab(folio), folio);
| ^~~~~~~~~~~~~~~
| VM_BUG_ON_MM
include/linux/memcontrol.h:403:18: error: implicit declaration of function 'folio_slab' [-Werror=implicit-function-declaration]
403 | VM_BUG_ON_FOLIO(folio_slab(folio), folio);
| ^~~~~~~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:420:55: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function '__folio_objcg':
include/linux/memcontrol.h:422:34: error: dereferencing pointer to incomplete type 'struct folio'
422 | unsigned long memcg_data = folio->memcg_data;
| ^~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:451:53: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h: In function 'folio_memcg':
include/linux/memcontrol.h:453:23: error: passing argument 1 of 'folio_memcg_kmem' from incompatible pointer type [-Werror=incompatible-pointer-types]
453 | if (folio_memcg_kmem(folio))
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:375:51: note: expected 'struct folio *' but argument is of type 'struct folio *'
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h:454:41: error: passing argument 1 of '__folio_objcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
454 | return obj_cgroup_memcg(__folio_objcg(folio));
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:420:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
420 | static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h:456:24: error: passing argument 1 of '__folio_memcg' from incompatible pointer type [-Werror=incompatible-pointer-types]
456 | return __folio_memcg(folio);
| ^~~~~
| |
| struct folio *
include/linux/memcontrol.h:399:62: note: expected 'struct folio *' but argument is of type 'struct folio *'
399 | static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: In function 'page_memcg':
include/linux/memcontrol.h:461:21: error: implicit declaration of function 'page_folio' [-Werror=implicit-function-declaration]
461 | return folio_memcg(page_folio(page));
| ^~~~~~~~~~
include/linux/memcontrol.h:461:21: warning: passing argument 1 of 'folio_memcg' makes pointer from integer without a cast [-Wint-conversion]
461 | return folio_memcg(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:451:60: note: expected 'struct folio *' but argument is of type 'int'
451 | static inline struct mem_cgroup *folio_memcg(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:589:44: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~
include/linux/memcontrol.h:589:20: error: conflicting types for 'folio_memcg_kmem'
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ^~~~~~~~~~~~~~~~
include/linux/memcontrol.h:375:20: note: previous declaration of 'folio_memcg_kmem' was here
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~~~~~~~~~~~~
include/linux/memcontrol.h: In function 'PageMemcgKmem':
include/linux/memcontrol.h:607:26: warning: passing argument 1 of 'folio_memcg_kmem' makes pointer from integer without a cast [-Wint-conversion]
607 | return folio_memcg_kmem(page_folio(page));
| ^~~~~~~~~~~~~~~~
| |
| int
include/linux/memcontrol.h:589:51: note: expected 'struct folio *' but argument is of type 'int'
589 | static inline bool folio_memcg_kmem(struct folio *folio)
| ~~~~~~~~~~~~~~^~~~~
include/linux/memcontrol.h: At top level:
include/linux/memcontrol.h:708:30: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
| ^~~~~
include/linux/memcontrol.h:713:33: warning: 'struct folio' declared inside parameter list will not be visible outside of this definition or declaration
713 | void mem_cgroup_uncharge(struct folio *);
| ^~~~~
In file included from arch/sparc/include/asm/bug.h:6,
from include/linux/bug.h:5,
from include/linux/mmdebug.h:5,
from include/linux/mm.h:9,
from mm/khugepaged.c:4:
mm/khugepaged.c: In function 'collapse_huge_page':
mm/khugepaged.c:1091:33: warning: passing argument 1 of 'mem_cgroup_charge' makes pointer from integer without a cast [-Wint-conversion]
1091 | if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
| ^~~~~~~~~~~~~~~~~~~~
| |
| int
include/linux/compiler.h:78:42: note: in definition of macro 'unlikely'
78 | # define unlikely(x) __builtin_expect(!!(x), 0)
| ^
In file included from include/linux/rmap.h:12,
from mm/khugepaged.c:9:
include/linux/memcontrol.h:708:23: note: expected 'struct folio *' but argument is of type 'int'
708 | int mem_cgroup_charge(struct folio *, struct mm_struct *, gfp_t);
| ^~~~~~~~~~~~~~
>> mm/khugepaged.c:1215:23: warning: passing argument 1 of 'mem_cgroup_uncharge' makes pointer from integer without a cast [-Wint-conversion]
1215 | mem_cgroup_uncharge(page_folio(*hpage));
| ^~~~~~~~~~~~~~~~~~
| |
| int
In file included from include/linux/rmap.h:12,
from mm/khugepaged.c:9:
include/linux/memcontrol.h:713:26: note: expected 'struct folio *' but argument is of type 'int'
713 | void mem_cgroup_uncharge(struct folio *);
| ^~~~~~~~~~~~~~
mm/khugepaged.c: At top level:
include/linux/memcontrol.h:375:20: warning: 'folio_memcg_kmem' used but never defined
375 | static inline bool folio_memcg_kmem(struct folio *folio);
| ^~~~~~~~~~~~~~~~
cc1: some warnings being treated as errors
vim +/mem_cgroup_uncharge +1215 mm/khugepaged.c
1056
1057 static void collapse_huge_page(struct mm_struct *mm,
1058 unsigned long address,
1059 struct page **hpage,
1060 int node, int referenced, int unmapped)
1061 {
1062 LIST_HEAD(compound_pagelist);
1063 pmd_t *pmd, _pmd;
1064 pte_t *pte;
1065 pgtable_t pgtable;
1066 struct page *new_page;
1067 spinlock_t *pmd_ptl, *pte_ptl;
1068 int isolated = 0, result = 0;
1069 struct vm_area_struct *vma;
1070 struct mmu_notifier_range range;
1071 gfp_t gfp;
1072
1073 VM_BUG_ON(address & ~HPAGE_PMD_MASK);
1074
1075 /* Only allocate from the target node */
1076 gfp = alloc_hugepage_khugepaged_gfpmask() | __GFP_THISNODE;
1077
1078 /*
1079 * Before allocating the hugepage, release the mmap_lock read lock.
1080 * The allocation can take potentially a long time if it involves
1081 * sync compaction, and we do not need to hold the mmap_lock during
1082 * that. We will recheck the vma after taking it again in write mode.
1083 */
1084 mmap_read_unlock(mm);
1085 new_page = khugepaged_alloc_page(hpage, gfp, node);
1086 if (!new_page) {
1087 result = SCAN_ALLOC_HUGE_PAGE_FAIL;
1088 goto out_nolock;
1089 }
1090
1091 if (unlikely(mem_cgroup_charge(page_folio(new_page), mm, gfp))) {
1092 result = SCAN_CGROUP_CHARGE_FAIL;
1093 goto out_nolock;
1094 }
1095 count_memcg_page_event(new_page, THP_COLLAPSE_ALLOC);
1096
1097 mmap_read_lock(mm);
1098 result = hugepage_vma_revalidate(mm, address, &vma);
1099 if (result) {
1100 mmap_read_unlock(mm);
1101 goto out_nolock;
1102 }
1103
1104 pmd = mm_find_pmd(mm, address);
1105 if (!pmd) {
1106 result = SCAN_PMD_NULL;
1107 mmap_read_unlock(mm);
1108 goto out_nolock;
1109 }
1110
1111 /*
1112 * __collapse_huge_page_swapin always returns with mmap_lock locked.
1113 * If it fails, we release mmap_lock and jump out_nolock.
1114 * Continuing to collapse causes inconsistency.
1115 */
1116 if (unmapped && !__collapse_huge_page_swapin(mm, vma, address,
1117 pmd, referenced)) {
1118 mmap_read_unlock(mm);
1119 goto out_nolock;
1120 }
1121
1122 mmap_read_unlock(mm);
1123 /*
1124 * Prevent all access to pagetables with the exception of
1125 * gup_fast later handled by the ptep_clear_flush and the VM
1126 * handled by the anon_vma lock + PG_lock.
1127 */
1128 mmap_write_lock(mm);
1129 result = hugepage_vma_revalidate(mm, address, &vma);
1130 if (result)
1131 goto out_up_write;
1132 /* check if the pmd is still valid */
1133 if (mm_find_pmd(mm, address) != pmd)
1134 goto out_up_write;
1135
1136 anon_vma_lock_write(vma->anon_vma);
1137
1138 mmu_notifier_range_init(&range, MMU_NOTIFY_CLEAR, 0, NULL, mm,
1139 address, address + HPAGE_PMD_SIZE);
1140 mmu_notifier_invalidate_range_start(&range);
1141
1142 pte = pte_offset_map(pmd, address);
1143 pte_ptl = pte_lockptr(mm, pmd);
1144
1145 pmd_ptl = pmd_lock(mm, pmd); /* probably unnecessary */
1146 /*
1147 * After this gup_fast can't run anymore. This also removes
1148 * any huge TLB entry from the CPU so we won't allow
1149 * huge and small TLB entries for the same virtual address
1150 * to avoid the risk of CPU bugs in that area.
1151 */
1152 _pmd = pmdp_collapse_flush(vma, address, pmd);
1153 spin_unlock(pmd_ptl);
1154 mmu_notifier_invalidate_range_end(&range);
1155
1156 spin_lock(pte_ptl);
1157 isolated = __collapse_huge_page_isolate(vma, address, pte,
1158 &compound_pagelist);
1159 spin_unlock(pte_ptl);
1160
1161 if (unlikely(!isolated)) {
1162 pte_unmap(pte);
1163 spin_lock(pmd_ptl);
1164 BUG_ON(!pmd_none(*pmd));
1165 /*
1166 * We can only use set_pmd_at when establishing
1167 * hugepmds and never for establishing regular pmds that
1168 * points to regular pagetables. Use pmd_populate for that
1169 */
1170 pmd_populate(mm, pmd, pmd_pgtable(_pmd));
1171 spin_unlock(pmd_ptl);
1172 anon_vma_unlock_write(vma->anon_vma);
1173 result = SCAN_FAIL;
1174 goto out_up_write;
1175 }
1176
1177 /*
1178 * All pages are isolated and locked so anon_vma rmap
1179 * can't run anymore.
1180 */
1181 anon_vma_unlock_write(vma->anon_vma);
1182
1183 __collapse_huge_page_copy(pte, new_page, vma, address, pte_ptl,
1184 &compound_pagelist);
1185 pte_unmap(pte);
1186 /*
1187 * spin_lock() below is not the equivalent of smp_wmb(), but
1188 * the smp_wmb() inside __SetPageUptodate() can be reused to
1189 * avoid the copy_huge_page writes to become visible after
1190 * the set_pmd_at() write.
1191 */
1192 __SetPageUptodate(new_page);
1193 pgtable = pmd_pgtable(_pmd);
1194
1195 _pmd = mk_huge_pmd(new_page, vma->vm_page_prot);
1196 _pmd = maybe_pmd_mkwrite(pmd_mkdirty(_pmd), vma);
1197
1198 spin_lock(pmd_ptl);
1199 BUG_ON(!pmd_none(*pmd));
1200 page_add_new_anon_rmap(new_page, vma, address, true);
1201 lru_cache_add_inactive_or_unevictable(new_page, vma);
1202 pgtable_trans_huge_deposit(mm, pmd, pgtable);
1203 set_pmd_at(mm, address, pmd, _pmd);
1204 update_mmu_cache_pmd(vma, address, pmd);
1205 spin_unlock(pmd_ptl);
1206
1207 *hpage = NULL;
1208
1209 khugepaged_pages_collapsed++;
1210 result = SCAN_SUCCEED;
1211 out_up_write:
1212 mmap_write_unlock(mm);
1213 out_nolock:
1214 if (!IS_ERR_OR_NULL(*hpage))
> 1215 mem_cgroup_uncharge(page_folio(*hpage));
1216 trace_mm_collapse_huge_page(mm, isolated, result);
1217 return;
1218 }
1219
---
0-DAY CI Kernel Test Service, Intel Corporation
https://lists.01.org/hyperkitty/list/kbuild-all@lists.01.org
[-- Attachment #2: .config.gz --]
[-- Type: application/gzip, Size: 26865 bytes --]
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 8:30 ` Michal Hocko
@ 2021-06-30 11:22 ` Matthew Wilcox
2021-06-30 12:20 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2021-06-30 11:22 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed, Jun 30, 2021 at 10:30:38AM +0200, Michal Hocko wrote:
> > - if (PageAnon(page)) {
> > - if (page_mapped(page)) {
> > + if (folio_anon(folio)) {
> > + if (folio_mapped(folio)) {
> > __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
> > __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> > - if (PageTransHuge(page)) {
> > + if (folio_multi(folio)) {
>
> Shouldn't be folio_transhuge? The resulting code is the same but
> folio_transhuge is more explicit and matches the THP aspect.
I genuinely don't know. For the benefit of those reading along, the
important part of the context is:
if (folio_mapped(folio)) {
__mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
__mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
if (folio_multi(folio)) {
__mod_lruvec_state(from_vec, NR_ANON_THPS,
-nr_pages);
__mod_lruvec_state(to_vec, NR_ANON_THPS,
nr_pages);
}
}
We need to decide what 'NR_ANON_THPS' means in a folio-based world where
we have folios of all orders. Does it count only the number of pages
in folios >= HPAGE_PMD_SIZE? Or does it count the number of pages in
folios > PAGE_SIZE?
Similar question (and I suspect the same answer) for NR_SHMEM_THPS and
NR_FILE_THPS. Right now, I've been accounting any multi-page folio as
a THP, but I don't have a good sense of what the answer should be.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 11:22 ` Matthew Wilcox
@ 2021-06-30 12:20 ` Michal Hocko
2021-06-30 12:31 ` Matthew Wilcox
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 12:20 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 10:30:38AM +0200, Michal Hocko wrote:
> > > - if (PageAnon(page)) {
> > > - if (page_mapped(page)) {
> > > + if (folio_anon(folio)) {
> > > + if (folio_mapped(folio)) {
> > > __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
> > > __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> > > - if (PageTransHuge(page)) {
> > > + if (folio_multi(folio)) {
> >
> > Shouldn't be folio_transhuge? The resulting code is the same but
> > folio_transhuge is more explicit and matches the THP aspect.
>
> I genuinely don't know. For the benefit of those reading along, the
> important part of the context is:
>
> if (folio_mapped(folio)) {
> __mod_lruvec_state(from_vec, NR_ANON_MAPPED, -nr_pages);
> __mod_lruvec_state(to_vec, NR_ANON_MAPPED, nr_pages);
> if (folio_multi(folio)) {
> __mod_lruvec_state(from_vec, NR_ANON_THPS,
> -nr_pages);
> __mod_lruvec_state(to_vec, NR_ANON_THPS,
> nr_pages);
> }
> }
>
> We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> we have folios of all orders. Does it count only the number of pages
> in folios >= HPAGE_PMD_SIZE? Or does it count the number of pages in
> folios > PAGE_SIZE?
At this stage we only have PMD based, right? I believe it would be
simpler to stick with that at the moment and change that to a more
generic way along with other places which need updating.
Wrt. counters they do count pages so in this case this shouldn't be a
problem. But we do have counters for pmd mappings and that might need
some care.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 12:20 ` Michal Hocko
@ 2021-06-30 12:31 ` Matthew Wilcox
2021-06-30 12:45 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2021-06-30 12:31 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > we have folios of all orders. Does it count only the number of pages
> > in folios >= HPAGE_PMD_SIZE? Or does it count the number of pages in
> > folios > PAGE_SIZE?
>
> At this stage we only have PMD based, right? I believe it would be
> simpler to stick with that at the moment and change that to a more
> generic way along with other places which need updating.
>
> Wrt. counters they do count pages so in this case this shouldn't be a
> problem. But we do have counters for pmd mappings and that might need
> some care.
Looking at how these are reported:
show_val_kb(m, "AnonHugePages: ",
global_node_page_state(NR_ANON_THPS));
show_val_kb(m, "ShmemHugePages: ",
global_node_page_state(NR_SHMEM_THPS));
show_val_kb(m, "ShmemPmdMapped: ",
global_node_page_state(NR_SHMEM_PMDMAPPED));
show_val_kb(m, "FileHugePages: ",
global_node_page_state(NR_FILE_THPS));
show_val_kb(m, "FilePmdMapped: ",
global_node_page_state(NR_FILE_PMDMAPPED));
it specifically refers to 'HugePages', so I think we need to only
count folios with order >= PMD_ORDER. I'll make that change to
folio_transhuge() and use folio_transhuge() here.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 12:31 ` Matthew Wilcox
@ 2021-06-30 12:45 ` Michal Hocko
2021-07-07 15:25 ` Matthew Wilcox
0 siblings, 1 reply; 59+ messages in thread
From: Michal Hocko @ 2021-06-30 12:45 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 30-06-21 13:31:17, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> > On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > > we have folios of all orders. Does it count only the number of pages
> > > in folios >= HPAGE_PMD_SIZE? Or does it count the number of pages in
> > > folios > PAGE_SIZE?
> >
> > At this stage we only have PMD based, right? I believe it would be
> > simpler to stick with that at the moment and change that to a more
> > generic way along with other places which need updating.
> >
> > Wrt. counters they do count pages so in this case this shouldn't be a
> > problem. But we do have counters for pmd mappings and that might need
> > some care.
>
> Looking at how these are reported:
>
> show_val_kb(m, "AnonHugePages: ",
> global_node_page_state(NR_ANON_THPS));
> show_val_kb(m, "ShmemHugePages: ",
> global_node_page_state(NR_SHMEM_THPS));
> show_val_kb(m, "ShmemPmdMapped: ",
> global_node_page_state(NR_SHMEM_PMDMAPPED));
> show_val_kb(m, "FileHugePages: ",
> global_node_page_state(NR_FILE_THPS));
> show_val_kb(m, "FilePmdMapped: ",
> global_node_page_state(NR_FILE_PMDMAPPED));
>
> it specifically refers to 'HugePages', so I think we need to only
> count folios with order >= PMD_ORDER.
Why? The presented value is in kB. It gives us a cumulative number of
transparent large pages. Sure breakdown to respective orders would be
impossible in general but the same would be the case if order > PMD_ORDER.
I am not really sure how useful that information is in practice but that
is a different story.
> I'll make that change to
> folio_transhuge() and use folio_transhuge() here.
Thanks!
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics()
2021-06-30 4:00 ` [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics() Matthew Wilcox (Oracle)
@ 2021-06-30 14:17 ` Johannes Weiner
0 siblings, 0 replies; 59+ messages in thread
From: Johannes Weiner @ 2021-06-30 14:17 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov,
Christoph Hellwig, Michal Hocko
On Wed, Jun 30, 2021 at 05:00:18AM +0100, Matthew Wilcox (Oracle) wrote:
> The last use of 'page' was removed by commit 468c398233da ("mm:
> memcontrol: switch to native NR_ANON_THPS counter"), so we can now remove
> the parameter from the function.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
> Acked-by: Michal Hocko <mhocko@suse.com>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()
2021-06-30 4:00 ` [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree() Matthew Wilcox (Oracle)
2021-06-30 6:55 ` Michal Hocko
@ 2021-06-30 14:18 ` Johannes Weiner
2021-07-01 6:57 ` Christoph Hellwig
2 siblings, 0 replies; 59+ messages in thread
From: Johannes Weiner @ 2021-06-30 14:18 UTC (permalink / raw)
To: Matthew Wilcox (Oracle); +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:19AM +0100, Matthew Wilcox (Oracle) wrote:
> By using the node id in mem_cgroup_update_tree(), we can delete
> soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo(). Saves 42
> bytes of kernel text on my config.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()
2021-06-30 4:00 ` [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node() Matthew Wilcox (Oracle)
2021-06-30 6:56 ` Michal Hocko
@ 2021-06-30 14:19 ` Johannes Weiner
2021-07-01 7:09 ` Christoph Hellwig
2 siblings, 0 replies; 59+ messages in thread
From: Johannes Weiner @ 2021-06-30 14:19 UTC (permalink / raw)
To: Matthew Wilcox (Oracle); +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:20AM +0100, Matthew Wilcox (Oracle) wrote:
> Opencode this one-line function in its three callers.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Acked-by: Johannes Weiner <hannes@cmpxchg.org>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()
2021-06-30 4:00 ` [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec() Matthew Wilcox (Oracle)
2021-06-30 8:12 ` kernel test robot
@ 2021-06-30 19:18 ` Matthew Wilcox
2021-06-30 21:21 ` Johannes Weiner
1 sibling, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2021-06-30 19:18 UTC (permalink / raw)
To: linux-mm, cgroups; +Cc: Johannes Weiner, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:31AM +0100, Matthew Wilcox (Oracle) wrote:
> This is the folio equivalent of mem_cgroup_page_lruvec().
I'm just going through and removing the wrappers.
Why is this function called this? There's an odd mix of
lock_page_memcg()
page_memcg()
count_memcg_page_event()
split_page_memcg()
vs
mem_cgroup_charge()
mem_cgroup_swapin_charge_page()
mem_cgroup_lruvec()
mem_cgroup_from_task()
I'd really like to call this function folio_lruvec(). It happens to
behave differently if the folio is part of a memcg, but conceptually,
lruvecs aren't particularly tied to memcgs.
... are there any other functions you'd like to rename while I'm
changing pages to folios?
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec()
2021-06-30 19:18 ` Matthew Wilcox
@ 2021-06-30 21:21 ` Johannes Weiner
0 siblings, 0 replies; 59+ messages in thread
From: Johannes Weiner @ 2021-06-30 21:21 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 08:18:33PM +0100, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 05:00:31AM +0100, Matthew Wilcox (Oracle) wrote:
> > This is the folio equivalent of mem_cgroup_page_lruvec().
>
> I'm just going through and removing the wrappers.
>
> Why is this function called this? There's an odd mix of
>
> lock_page_memcg()
This was chosen to match lock_page().
> page_memcg()
And this to match page_mapping(), page_zone() etc.
> count_memcg_page_event()
count_vm_event()
> split_page_memcg()
split_page(), split_page_owner()
> mem_cgroup_charge()
> mem_cgroup_swapin_charge_page()
These are larger, heavier subsystem API calls that modify all kinds of
state, not just the page. Hence the namespacing.
With the smaller getter/setter type functions on pages we have
traditionally used <verb>_<object> rather than page_<verb>, simply
because the page is such a low-level object and many functions do
sequences of page manipulations. Namespacing would turn them into:
page_do_this(page);
page_set_that(page);
page_lock(page);
if (page_is_blah(page))
page_mark_foo(page);
page_unlock(page);
page_put(page);
which is hard on the reader because it obscures the salient part of
each line behind repetetive boiler plate.
> mem_cgroup_lruvec()
This is arguably not a namespace prefix, but rather an accessor
function to look up the memcg's lruvec.
> mem_cgroup_from_task()
This is a pattern to look up memcgs from various objects:
- mem_cgroup_from_css()
- mem_cgroup_from_counter()
- mem_cgroup_from_id()
- mem_cgroup_from_seq()
- mem_cgroup_from_obj()
and we used to have mem_cgroup_from_page() at some point...
> I'd really like to call this function folio_lruvec().
That would be a better name indeed.
However, pairing renames with conversion is worrisome because it means
having two unintuitively diverging names for the same operation in the
API during a time where everybody has to relearn the code base already.
Please either precede it with a rename to page_lruvec(), or keep the
existing naming pattern in the conversion.
> It happens to behave differently if the folio is part of a memcg,
> but conceptually, lruvecs aren't particularly tied to memcgs.
Conceptually, lruvecs are always tied to a memcg. On !CONFIG_MEMCG
kernels that just happens to be the root cgroup.
But because it's an accessor to a page attribute, dropping the
namespacing prefix is in line with things like page_memcg().
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 01/18] mm: Add folio_nid()
2021-06-30 4:00 ` [PATCH v3 01/18] mm: Add folio_nid() Matthew Wilcox (Oracle)
@ 2021-07-01 6:56 ` Christoph Hellwig
0 siblings, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 6:56 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree()
2021-06-30 4:00 ` [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree() Matthew Wilcox (Oracle)
2021-06-30 6:55 ` Michal Hocko
2021-06-30 14:18 ` Johannes Weiner
@ 2021-07-01 6:57 ` Christoph Hellwig
2 siblings, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 6:57 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:19AM +0100, Matthew Wilcox (Oracle) wrote:
> By using the node id in mem_cgroup_update_tree(), we can delete
> soft_limit_tree_from_page() and mem_cgroup_page_nodeinfo(). Saves 42
> bytes of kernel text on my config.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node()
2021-06-30 4:00 ` [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node() Matthew Wilcox (Oracle)
2021-06-30 6:56 ` Michal Hocko
2021-06-30 14:19 ` Johannes Weiner
@ 2021-07-01 7:09 ` Christoph Hellwig
2 siblings, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:09 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:20AM +0100, Matthew Wilcox (Oracle) wrote:
> Opencode this one-line function in its three callers.
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID
2021-06-30 4:00 ` [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID Matthew Wilcox (Oracle)
2021-06-30 6:58 ` Michal Hocko
@ 2021-07-01 7:09 ` Christoph Hellwig
1 sibling, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:09 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions
2021-06-30 4:00 ` [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions Matthew Wilcox (Oracle)
2021-06-30 6:53 ` kernel test robot
@ 2021-07-01 7:12 ` Christoph Hellwig
1 sibling, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:12 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:22AM +0100, Matthew Wilcox (Oracle) wrote:
> memcg information is only stored in the head page, so the memcg
> subsystem needs to assure that all accesses are to the head page.
> The first step is converting page_memcg() to folio_memcg().
>
> Retain page_memcg() as a wrapper around folio_memcg() and PageMemcgKmem()
> as a wrapper around folio_memcg_kmem() but convert __page_memcg() to
> __folio_memcg() and __page_objcg() to __folio_objcg().
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
> include/linux/memcontrol.h | 105 +++++++++++++++++++++----------------
> mm/memcontrol.c | 21 ++++----
> 2 files changed, 73 insertions(+), 53 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 6d66037be646..92689fb2dab4 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -372,6 +372,7 @@ enum page_memcg_data_flags {
> #define MEMCG_DATA_FLAGS_MASK (__NR_MEMCG_DATA_FLAGS - 1)
>
> static inline bool PageMemcgKmem(struct page *page);
> +static inline bool folio_memcg_kmem(struct folio *folio);
>
> /*
> * After the initialization objcg->memcg is always pointing at
> @@ -386,73 +387,78 @@ static inline struct mem_cgroup *obj_cgroup_memcg(struct obj_cgroup *objcg)
> }
>
> /*
> - * __page_memcg - get the memory cgroup associated with a non-kmem page
> - * @page: a pointer to the page struct
> + * __folio_memcg - Get the memory cgroup associated with a non-kmem folio
> + * @folio: Pointer to the folio.
> *
> - * Returns a pointer to the memory cgroup associated with the page,
> - * or NULL. This function assumes that the page is known to have a
> + * Returns a pointer to the memory cgroup associated with the folio,
> + * or NULL. This function assumes that the folio is known to have a
> * proper memory cgroup pointer. It's not safe to call this function
> - * against some type of pages, e.g. slab pages or ex-slab pages or
> - * kmem pages.
> + * against some type of folios, e.g. slab folios or ex-slab folios or
> + * kmem folios.
> */
> -static inline struct mem_cgroup *__page_memcg(struct page *page)
> +static inline struct mem_cgroup *__folio_memcg(struct folio *folio)
> {
> - unsigned long memcg_data = page->memcg_data;
> + unsigned long memcg_data = folio->memcg_data;
>
> - VM_BUG_ON_PAGE(PageSlab(page), page);
> - VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
> - VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_KMEM, page);
> + VM_BUG_ON_FOLIO(folio_slab(folio), folio);
> + VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
> + VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_KMEM, folio);
>
> return (struct mem_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
> }
>
> /*
> - * __page_objcg - get the object cgroup associated with a kmem page
> - * @page: a pointer to the page struct
> + * __folio_objcg - get the object cgroup associated with a kmem folio.
> + * @folio: Pointer to the folio.
> *
> - * Returns a pointer to the object cgroup associated with the page,
> - * or NULL. This function assumes that the page is known to have a
> + * Returns a pointer to the object cgroup associated with the folio,
> + * or NULL. This function assumes that the folio is known to have a
> * proper object cgroup pointer. It's not safe to call this function
> - * against some type of pages, e.g. slab pages or ex-slab pages or
> - * LRU pages.
> + * against some type of folios, e.g. slab folios or ex-slab folios or
> + * LRU folios.
> */
> -static inline struct obj_cgroup *__page_objcg(struct page *page)
> +static inline struct obj_cgroup *__folio_objcg(struct folio *folio)
> {
> - unsigned long memcg_data = page->memcg_data;
> + unsigned long memcg_data = folio->memcg_data;
>
> - VM_BUG_ON_PAGE(PageSlab(page), page);
> - VM_BUG_ON_PAGE(memcg_data & MEMCG_DATA_OBJCGS, page);
> - VM_BUG_ON_PAGE(!(memcg_data & MEMCG_DATA_KMEM), page);
> + VM_BUG_ON_FOLIO(folio_slab(folio), folio);
> + VM_BUG_ON_FOLIO(memcg_data & MEMCG_DATA_OBJCGS, folio);
> + VM_BUG_ON_FOLIO(!(memcg_data & MEMCG_DATA_KMEM), folio);
>
> return (struct obj_cgroup *)(memcg_data & ~MEMCG_DATA_FLAGS_MASK);
> }
>
> /*
> - * page_memcg - get the memory cgroup associated with a page
> - * @page: a pointer to the page struct
> + * folio_memcg - Get the memory cgroup associated with a folio.
> + * @folio: Pointer to the folio.
> *
> - * Returns a pointer to the memory cgroup associated with the page,
> - * or NULL. This function assumes that the page is known to have a
> + * Returns a pointer to the memory cgroup associated with the folio,
> + * or NULL. This function assumes that the folio is known to have a
> * proper memory cgroup pointer. It's not safe to call this function
> - * against some type of pages, e.g. slab pages or ex-slab pages.
> + * against some type of folios, e.g. slab folios or ex-slab folios.
> *
> - * For a non-kmem page any of the following ensures page and memcg binding
> + * For a non-kmem folio any of the following ensures folio and memcg binding
> * stability:
> *
> - * - the page lock
> + * - the folio lock
> * - LRU isolation
> * - lock_page_memcg()
> * - exclusive reference
> *
> - * For a kmem page a caller should hold an rcu read lock to protect memcg
> - * associated with a kmem page from being released.
> + * For a kmem folio a caller should hold an rcu read lock to protect memcg
> + * associated with a kmem folio from being released.
> */
> -static inline struct mem_cgroup *page_memcg(struct page *page)
> +static inline struct mem_cgroup *folio_memcg(struct folio *folio)
> {
> - if (PageMemcgKmem(page))
> - return obj_cgroup_memcg(__page_objcg(page));
> + if (folio_memcg_kmem(folio))
> + return obj_cgroup_memcg(__folio_objcg(folio));
> else
> + return __folio_memcg(folio);
Nit: it might be worth to drop the pointless else while you're at it.
therwise looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() to take a folio
2021-06-30 4:00 ` [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() " Matthew Wilcox (Oracle)
2021-06-30 7:17 ` kernel test robot
@ 2021-07-01 7:13 ` Christoph Hellwig
1 sibling, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:13 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio()
2021-06-30 4:00 ` [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio() Matthew Wilcox (Oracle)
@ 2021-07-01 7:15 ` Christoph Hellwig
0 siblings, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:15 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio
2021-06-30 4:00 ` [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio Matthew Wilcox (Oracle)
2021-06-30 8:46 ` kernel test robot
@ 2021-07-01 7:17 ` Christoph Hellwig
2021-07-07 12:09 ` Matthew Wilcox
1 sibling, 1 reply; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:17 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:26AM +0100, Matthew Wilcox (Oracle) wrote:
> Convert all the callers to call page_folio().
>
> Signed-off-by: Matthew Wilcox (Oracle) <willy@infradead.org>
> ---
> include/linux/memcontrol.h | 4 ++--
> mm/filemap.c | 2 +-
> mm/khugepaged.c | 4 ++--
> mm/memcontrol.c | 14 +++++++-------
> mm/memory-failure.c | 2 +-
> mm/memremap.c | 2 +-
> mm/page_alloc.c | 2 +-
> mm/swap.c | 2 +-
> 8 files changed, 16 insertions(+), 16 deletions(-)
>
> diff --git a/include/linux/memcontrol.h b/include/linux/memcontrol.h
> index 90d48b0e3191..d6386a2b9d7a 100644
> --- a/include/linux/memcontrol.h
> +++ b/include/linux/memcontrol.h
> @@ -710,7 +710,7 @@ int mem_cgroup_swapin_charge_page(struct page *page, struct mm_struct *mm,
> gfp_t gfp, swp_entry_t entry);
> void mem_cgroup_swapin_uncharge_swap(swp_entry_t entry);
>
> -void mem_cgroup_uncharge(struct page *page);
> +void mem_cgroup_uncharge(struct folio *);
why do you drop the parameter name?
Otherwise looks good:
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios
2021-06-30 4:00 ` [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios Matthew Wilcox (Oracle)
@ 2021-07-01 7:20 ` Christoph Hellwig
0 siblings, 0 replies; 59+ messages in thread
From: Christoph Hellwig @ 2021-07-01 7:20 UTC (permalink / raw)
To: Matthew Wilcox (Oracle)
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:27AM +0100, Matthew Wilcox (Oracle) wrote:
> Convert all callers of mem_cgroup_migrate() to call page_folio() first.
Looks good,
Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio
2021-07-01 7:17 ` Christoph Hellwig
@ 2021-07-07 12:09 ` Matthew Wilcox
0 siblings, 0 replies; 59+ messages in thread
From: Matthew Wilcox @ 2021-07-07 12:09 UTC (permalink / raw)
To: Christoph Hellwig
Cc: linux-mm, cgroups, Johannes Weiner, Michal Hocko, Vladimir Davydov
On Thu, Jul 01, 2021 at 08:17:48AM +0100, Christoph Hellwig wrote:
> On Wed, Jun 30, 2021 at 05:00:26AM +0100, Matthew Wilcox (Oracle) wrote:
> > -void mem_cgroup_uncharge(struct page *page);
> > +void mem_cgroup_uncharge(struct folio *);
>
> why do you drop the parameter name?
I usually do where it's 'struct foo *foo' or 'foo_t foo'. If the best
you can do is say "it's a page", well, yes, I knew that from the type.
But since you've complained, I've added the pointless 'folio' to it.
> Otherwise looks good:
>
> Reviewed-by: Christoph Hellwig <hch@lst.de>
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-06-30 8:32 ` Michal Hocko
@ 2021-07-07 15:10 ` Matthew Wilcox
2021-07-08 7:28 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2021-07-07 15:10 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed, Jun 30, 2021 at 10:32:02AM +0200, Michal Hocko wrote:
> On Wed 30-06-21 05:00:29, Matthew Wilcox wrote:
> > These are the folio equivalents of lock_page_memcg() and
> > unlock_page_memcg(). Reimplement them as wrappers.
>
> Is there any reason why you haven't followed the same approach as for
> the previous patches. I mean callers can call page_folio and then
> lock_page_memcg wrapper shouldn't be really needed.
At this point in the patch series there are ~20 places which call
lock_page_memcg(). I think it makes more sense to leave the wrapper
in place, and then we can remove the wrapper once all/most of these
places are converted to use folios. There are another 5 conversions
already in the patch series, eg here:
https://git.infradead.org/users/willy/pagecache.git/commitdiff/a41c942c8e4b41df30be128ef6998ff1849fa36a
> I do not really want to be annoying here but I have to say that I like
> the conversion by previous patches much better than this wrapper
> approach as mentioned during the previous review already. If you have
> some reasons to stick with this approach for this particular case then
> make it explicit in the changelog.
OK, I can point to the number of callers as a reason to keep the
wrappers in place. I intended to just do the conversion here, but
seeing the number of callers made me reconsider.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-06-30 12:45 ` Michal Hocko
@ 2021-07-07 15:25 ` Matthew Wilcox
2021-07-08 7:30 ` Michal Hocko
0 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2021-07-07 15:25 UTC (permalink / raw)
To: Michal Hocko; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed, Jun 30, 2021 at 02:45:33PM +0200, Michal Hocko wrote:
> On Wed 30-06-21 13:31:17, Matthew Wilcox wrote:
> > On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> > > On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > > > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > > > we have folios of all orders. Does it count only the number of pages
> > > > in folios >= HPAGE_PMD_SIZE? Or does it count the number of pages in
> > > > folios > PAGE_SIZE?
> > >
> > > At this stage we only have PMD based, right? I believe it would be
> > > simpler to stick with that at the moment and change that to a more
> > > generic way along with other places which need updating.
> > >
> > > Wrt. counters they do count pages so in this case this shouldn't be a
> > > problem. But we do have counters for pmd mappings and that might need
> > > some care.
> >
> > Looking at how these are reported:
> >
> > show_val_kb(m, "AnonHugePages: ",
> > global_node_page_state(NR_ANON_THPS));
> > show_val_kb(m, "ShmemHugePages: ",
> > global_node_page_state(NR_SHMEM_THPS));
> > show_val_kb(m, "ShmemPmdMapped: ",
> > global_node_page_state(NR_SHMEM_PMDMAPPED));
> > show_val_kb(m, "FileHugePages: ",
> > global_node_page_state(NR_FILE_THPS));
> > show_val_kb(m, "FilePmdMapped: ",
> > global_node_page_state(NR_FILE_PMDMAPPED));
> >
> > it specifically refers to 'HugePages', so I think we need to only
> > count folios with order >= PMD_ORDER.
>
> Why? The presented value is in kB. It gives us a cumulative number of
> transparent large pages. Sure breakdown to respective orders would be
> impossible in general but the same would be the case if order > PMD_ORDER.
>
> I am not really sure how useful that information is in practice but that
> is a different story.
The scenario I'm thinking about is a situation where we have gigabytes
of memory in the page cache in 16k-64k chunks and we'll see
FileHugePages: 5219348 kB
FilePmdMapped: 0 kB
which might cause the slightly-too-clever user to think there's a
problem.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-06-30 4:00 ` [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Matthew Wilcox (Oracle)
2021-06-30 8:32 ` Michal Hocko
@ 2021-07-07 17:08 ` Johannes Weiner
2021-07-07 19:28 ` Matthew Wilcox
1 sibling, 1 reply; 59+ messages in thread
From: Johannes Weiner @ 2021-07-07 17:08 UTC (permalink / raw)
To: Matthew Wilcox (Oracle); +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> +static void __memcg_unlock(struct mem_cgroup *memcg)
This is too generic a name. There are several locks in the memcg, and
this one only locks the page->memcg bindings in the group.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-07-07 17:08 ` Johannes Weiner
@ 2021-07-07 19:28 ` Matthew Wilcox
2021-07-07 20:41 ` Johannes Weiner
0 siblings, 1 reply; 59+ messages in thread
From: Matthew Wilcox @ 2021-07-07 19:28 UTC (permalink / raw)
To: Johannes Weiner; +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jul 07, 2021 at 01:08:51PM -0400, Johannes Weiner wrote:
> On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> > -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> > +static void __memcg_unlock(struct mem_cgroup *memcg)
>
> This is too generic a name. There are several locks in the memcg, and
> this one only locks the page->memcg bindings in the group.
Fair. __memcg_move_unlock looks like the right name to me?
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-07-07 19:28 ` Matthew Wilcox
@ 2021-07-07 20:41 ` Johannes Weiner
2021-07-09 19:37 ` Matthew Wilcox
0 siblings, 1 reply; 59+ messages in thread
From: Johannes Weiner @ 2021-07-07 20:41 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jul 07, 2021 at 08:28:39PM +0100, Matthew Wilcox wrote:
> On Wed, Jul 07, 2021 at 01:08:51PM -0400, Johannes Weiner wrote:
> > On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> > > -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> > > +static void __memcg_unlock(struct mem_cgroup *memcg)
> >
> > This is too generic a name. There are several locks in the memcg, and
> > this one only locks the page->memcg bindings in the group.
>
> Fair. __memcg_move_unlock looks like the right name to me?
Could you please elaborate what the problem with the current name is?
mem_cgroup_move_account() does this:
lock_page_memcg(page);
page->memcg = to;
__unlock_page_memcg(from);
It locks and unlocks the page->memcg binding which can be done coming
from the page or the memcg. The current names are symmetrical to
reflect that it's the same lock.
We could switch them both to move_lock, but as per the other email,
lock_page_memcg() was chosen to resemble lock_page(). Because from a
memcg POV they're interchangeable - the former is just a more narrowly
scoped version for contexts that don't hold the page lock. It used to
be called something else and we had several contexts taking redundant
locks on accident because this hierarchy wasn't clear.
I don't mind fixing poorly chosen or misleading naming schemes, but I
think we need better explanations to overcome the reasoning behind the
existing names, not just the assumption that there weren't any.
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-07-07 15:10 ` Matthew Wilcox
@ 2021-07-08 7:28 ` Michal Hocko
0 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-07-08 7:28 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 07-07-21 16:10:49, Matthew Wilcox wrote:
[...]
> > I do not really want to be annoying here but I have to say that I like
> > the conversion by previous patches much better than this wrapper
> > approach as mentioned during the previous review already. If you have
> > some reasons to stick with this approach for this particular case then
> > make it explicit in the changelog.
>
> OK, I can point to the number of callers as a reason to keep the
> wrappers in place. I intended to just do the conversion here, but
> seeing the number of callers made me reconsider.
OK, fair enough. My worry is that we will have this lingering for way
too long. People simply tend to copy code... Anyway, please add a
comment warning that the wrapper shouldn't be used in any new code at
least.
Thanks!
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio
2021-07-07 15:25 ` Matthew Wilcox
@ 2021-07-08 7:30 ` Michal Hocko
0 siblings, 0 replies; 59+ messages in thread
From: Michal Hocko @ 2021-07-08 7:30 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: linux-mm, cgroups, Johannes Weiner, Vladimir Davydov
On Wed 07-07-21 16:25:07, Matthew Wilcox wrote:
> On Wed, Jun 30, 2021 at 02:45:33PM +0200, Michal Hocko wrote:
> > On Wed 30-06-21 13:31:17, Matthew Wilcox wrote:
> > > On Wed, Jun 30, 2021 at 02:20:38PM +0200, Michal Hocko wrote:
> > > > On Wed 30-06-21 12:22:48, Matthew Wilcox wrote:
> > > > > We need to decide what 'NR_ANON_THPS' means in a folio-based world where
> > > > > we have folios of all orders. Does it count only the number of pages
> > > > > in folios >= HPAGE_PMD_SIZE? Or does it count the number of pages in
> > > > > folios > PAGE_SIZE?
> > > >
> > > > At this stage we only have PMD based, right? I believe it would be
> > > > simpler to stick with that at the moment and change that to a more
> > > > generic way along with other places which need updating.
> > > >
> > > > Wrt. counters they do count pages so in this case this shouldn't be a
> > > > problem. But we do have counters for pmd mappings and that might need
> > > > some care.
> > >
> > > Looking at how these are reported:
> > >
> > > show_val_kb(m, "AnonHugePages: ",
> > > global_node_page_state(NR_ANON_THPS));
> > > show_val_kb(m, "ShmemHugePages: ",
> > > global_node_page_state(NR_SHMEM_THPS));
> > > show_val_kb(m, "ShmemPmdMapped: ",
> > > global_node_page_state(NR_SHMEM_PMDMAPPED));
> > > show_val_kb(m, "FileHugePages: ",
> > > global_node_page_state(NR_FILE_THPS));
> > > show_val_kb(m, "FilePmdMapped: ",
> > > global_node_page_state(NR_FILE_PMDMAPPED));
> > >
> > > it specifically refers to 'HugePages', so I think we need to only
> > > count folios with order >= PMD_ORDER.
> >
> > Why? The presented value is in kB. It gives us a cumulative number of
> > transparent large pages. Sure breakdown to respective orders would be
> > impossible in general but the same would be the case if order > PMD_ORDER.
> >
> > I am not really sure how useful that information is in practice but that
> > is a different story.
>
> The scenario I'm thinking about is a situation where we have gigabytes
> of memory in the page cache in 16k-64k chunks and we'll see
> FileHugePages: 5219348 kB
> FilePmdMapped: 0 kB
>
> which might cause the slightly-too-clever user to think there's a
> problem.
Well, cases like this one shouldn't be really hard to explain though so
I wouldn't be worried about this.
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 59+ messages in thread
* Re: [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock()
2021-07-07 20:41 ` Johannes Weiner
@ 2021-07-09 19:37 ` Matthew Wilcox
0 siblings, 0 replies; 59+ messages in thread
From: Matthew Wilcox @ 2021-07-09 19:37 UTC (permalink / raw)
To: Johannes Weiner; +Cc: linux-mm, cgroups, Michal Hocko, Vladimir Davydov
On Wed, Jul 07, 2021 at 04:41:05PM -0400, Johannes Weiner wrote:
> On Wed, Jul 07, 2021 at 08:28:39PM +0100, Matthew Wilcox wrote:
> > On Wed, Jul 07, 2021 at 01:08:51PM -0400, Johannes Weiner wrote:
> > > On Wed, Jun 30, 2021 at 05:00:29AM +0100, Matthew Wilcox (Oracle) wrote:
> > > > -static void __unlock_page_memcg(struct mem_cgroup *memcg)
> > > > +static void __memcg_unlock(struct mem_cgroup *memcg)
> > >
> > > This is too generic a name. There are several locks in the memcg, and
> > > this one only locks the page->memcg bindings in the group.
> >
> > Fair. __memcg_move_unlock looks like the right name to me?
>
> Could you please elaborate what the problem with the current name is?
>
> mem_cgroup_move_account() does this:
>
> lock_page_memcg(page);
> page->memcg = to;
> __unlock_page_memcg(from);
>
> It locks and unlocks the page->memcg binding which can be done coming
> from the page or the memcg. The current names are symmetrical to
> reflect that it's the same lock.
OK, so in the prerequisite series to this patch, lock_page() becomes
folio_lock(). This series turns lock_page_memcg() into
folio_memcg_lock(). As a minimum, then, this needs to turn into
__folio_memcg_unlock().
> We could switch them both to move_lock, but as per the other email,
> lock_page_memcg() was chosen to resemble lock_page(). Because from a
> memcg POV they're interchangeable - the former is just a more narrowly
> scoped version for contexts that don't hold the page lock. It used to
> be called something else and we had several contexts taking redundant
> locks on accident because this hierarchy wasn't clear.
Unfortunately, it's still not clear. I've answered questions from
people who think that they have the page locked because they called
lock_page_memcg() ;-(
^ permalink raw reply [flat|nested] 59+ messages in thread
end of thread, other threads:[~2021-07-09 19:37 UTC | newest]
Thread overview: 59+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-06-30 4:00 [PATCH v3 00/18] Folio conversion of memcg Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 01/18] mm: Add folio_nid() Matthew Wilcox (Oracle)
2021-07-01 6:56 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 02/18] mm/memcg: Remove 'page' parameter to mem_cgroup_charge_statistics() Matthew Wilcox (Oracle)
2021-06-30 14:17 ` Johannes Weiner
2021-06-30 4:00 ` [PATCH v3 03/18] mm/memcg: Use the node id in mem_cgroup_update_tree() Matthew Wilcox (Oracle)
2021-06-30 6:55 ` Michal Hocko
2021-06-30 14:18 ` Johannes Weiner
2021-07-01 6:57 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 04/18] mm/memcg: Remove soft_limit_tree_node() Matthew Wilcox (Oracle)
2021-06-30 6:56 ` Michal Hocko
2021-06-30 14:19 ` Johannes Weiner
2021-07-01 7:09 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 05/18] mm/memcg: Convert memcg_check_events to take a node ID Matthew Wilcox (Oracle)
2021-06-30 6:58 ` Michal Hocko
2021-06-30 6:59 ` Michal Hocko
2021-07-01 7:09 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 06/18] mm/memcg: Add folio_memcg() and related functions Matthew Wilcox (Oracle)
2021-06-30 6:53 ` kernel test robot
2021-07-01 7:12 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 07/18] mm/memcg: Convert commit_charge() to take a folio Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 08/18] mm/memcg: Convert mem_cgroup_charge() " Matthew Wilcox (Oracle)
2021-06-30 7:17 ` kernel test robot
2021-07-01 7:13 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 09/18] mm/memcg: Convert uncharge_page() to uncharge_folio() Matthew Wilcox (Oracle)
2021-07-01 7:15 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 10/18] mm/memcg: Convert mem_cgroup_uncharge() to take a folio Matthew Wilcox (Oracle)
2021-06-30 8:46 ` kernel test robot
2021-07-01 7:17 ` Christoph Hellwig
2021-07-07 12:09 ` Matthew Wilcox
2021-06-30 4:00 ` [PATCH v3 11/18] mm/memcg: Convert mem_cgroup_migrate() to take folios Matthew Wilcox (Oracle)
2021-07-01 7:20 ` Christoph Hellwig
2021-06-30 4:00 ` [PATCH v3 12/18] mm/memcg: Convert mem_cgroup_track_foreign_dirty_slowpath() to folio Matthew Wilcox (Oracle)
2021-06-30 4:00 ` [PATCH v3 13/18] mm/memcg: Add folio_memcg_lock() and folio_memcg_unlock() Matthew Wilcox (Oracle)
2021-06-30 8:32 ` Michal Hocko
2021-07-07 15:10 ` Matthew Wilcox
2021-07-08 7:28 ` Michal Hocko
2021-07-07 17:08 ` Johannes Weiner
2021-07-07 19:28 ` Matthew Wilcox
2021-07-07 20:41 ` Johannes Weiner
2021-07-09 19:37 ` Matthew Wilcox
2021-06-30 4:00 ` [PATCH v3 14/18] mm/memcg: Convert mem_cgroup_move_account() to use a folio Matthew Wilcox (Oracle)
2021-06-30 8:30 ` Michal Hocko
2021-06-30 11:22 ` Matthew Wilcox
2021-06-30 12:20 ` Michal Hocko
2021-06-30 12:31 ` Matthew Wilcox
2021-06-30 12:45 ` Michal Hocko
2021-07-07 15:25 ` Matthew Wilcox
2021-07-08 7:30 ` Michal Hocko
2021-06-30 4:00 ` [PATCH v3 15/18] mm/memcg: Add mem_cgroup_folio_lruvec() Matthew Wilcox (Oracle)
2021-06-30 8:12 ` kernel test robot
2021-06-30 19:18 ` Matthew Wilcox
2021-06-30 21:21 ` Johannes Weiner
2021-06-30 4:00 ` [PATCH v3 16/18] mm/memcg: Add folio_lruvec_lock() and similar functions Matthew Wilcox (Oracle)
2021-06-30 8:36 ` Michal Hocko
2021-06-30 4:00 ` [PATCH v3 17/18] mm/memcg: Add folio_lruvec_relock_irq() and folio_lruvec_relock_irqsave() Matthew Wilcox (Oracle)
2021-06-30 8:39 ` Michal Hocko
2021-06-30 4:00 ` [PATCH v3 18/18] mm/workingset: Convert workingset_activation to take a folio Matthew Wilcox (Oracle)
2021-06-30 8:44 ` [PATCH v3 00/18] Folio conversion of memcg Michal Hocko
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).