* [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block()
@ 2023-06-12 14:34 Kefeng Wang
2023-06-12 14:34 ` [PATCH -next 2/2] mm: kill [add|del]_page_to_lru_list() Kefeng Wang
2023-06-12 14:41 ` [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Matthew Wilcox
0 siblings, 2 replies; 6+ messages in thread
From: Kefeng Wang @ 2023-06-12 14:34 UTC (permalink / raw)
To: akpm; +Cc: linux-mm, linux-kernel, willy, jgowans, yuzhao, Kefeng Wang
Directly use a folio instead of page_folio() when page successfully
isolated (hugepage and movable page) and after folio_get_nontail_page(),
which removes several calls to compound_head().
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
mm/compaction.c | 71 ++++++++++++++++++++++++++-----------------------
1 file changed, 38 insertions(+), 33 deletions(-)
diff --git a/mm/compaction.c b/mm/compaction.c
index 3398ef3a55fe..5d3f0aaa6785 100644
--- a/mm/compaction.c
+++ b/mm/compaction.c
@@ -831,6 +831,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
struct lruvec *lruvec;
unsigned long flags = 0;
struct lruvec *locked = NULL;
+ struct folio *folio = NULL;
struct page *page = NULL, *valid_page = NULL;
struct address_space *mapping;
unsigned long start_pfn = low_pfn;
@@ -927,7 +928,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
if (!valid_page && pageblock_aligned(low_pfn)) {
if (!isolation_suitable(cc, page)) {
low_pfn = end_pfn;
- page = NULL;
+ folio = NULL;
goto isolate_abort;
}
valid_page = page;
@@ -959,7 +960,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* Hugepage was successfully isolated and placed
* on the cc->migratepages list.
*/
- low_pfn += compound_nr(page) - 1;
+ folio = page_folio(page);
+ low_pfn += folio_nr_pages(folio) - 1;
goto isolate_success_no_list;
}
@@ -1027,8 +1029,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
locked = NULL;
}
- if (isolate_movable_page(page, mode))
+ if (isolate_movable_page(page, mode)) {
+ folio = page_folio(page);
goto isolate_success;
+ }
}
goto isolate_fail;
@@ -1039,7 +1043,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* sure the page is not being freed elsewhere -- the
* page release code relies on it.
*/
- if (unlikely(!get_page_unless_zero(page)))
+ folio = folio_get_nontail_page(page);
+ if (unlikely(!folio))
goto isolate_fail;
/*
@@ -1047,7 +1052,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* from long term pinning preventing it from migrating,
* so avoid taking lru_lock and isolating it unnecessarily.
*/
- mapping = page_mapping(page);
+ mapping = folio_mapping(folio);
if (!cc->alloc_contig && page_has_extra_refs(page, mapping))
goto isolate_fail_put;
@@ -1063,7 +1068,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
goto isolate_fail_put;
/* Compaction might skip unevictable pages but CMA takes them */
- if (!(mode & ISOLATE_UNEVICTABLE) && PageUnevictable(page))
+ if (!(mode & ISOLATE_UNEVICTABLE) && folio_test_unevictable(folio))
goto isolate_fail_put;
/*
@@ -1072,10 +1077,10 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* it will be able to migrate without blocking - clean pages
* for the most part. PageWriteback would require blocking.
*/
- if ((mode & ISOLATE_ASYNC_MIGRATE) && PageWriteback(page))
+ if ((mode & ISOLATE_ASYNC_MIGRATE) && folio_test_writeback(folio))
goto isolate_fail_put;
- if ((mode & ISOLATE_ASYNC_MIGRATE) && PageDirty(page)) {
+ if ((mode & ISOLATE_ASYNC_MIGRATE) && folio_test_dirty(folio)) {
bool migrate_dirty;
/*
@@ -1087,22 +1092,22 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* the page lock until after the page is removed
* from the page cache.
*/
- if (!trylock_page(page))
+ if (!folio_trylock(folio))
goto isolate_fail_put;
- mapping = page_mapping(page);
+ mapping = folio_mapping(folio);
migrate_dirty = !mapping ||
mapping->a_ops->migrate_folio;
- unlock_page(page);
+ folio_unlock(folio);
if (!migrate_dirty)
goto isolate_fail_put;
}
/* Try isolate the page */
- if (!TestClearPageLRU(page))
+ if (!folio_test_clear_lru(folio))
goto isolate_fail_put;
- lruvec = folio_lruvec(page_folio(page));
+ lruvec = folio_lruvec(folio);
/* If we already hold the lock, we can skip some rechecking */
if (lruvec != locked) {
@@ -1112,7 +1117,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
compact_lock_irqsave(&lruvec->lru_lock, &flags, cc);
locked = lruvec;
- lruvec_memcg_debug(lruvec, page_folio(page));
+ lruvec_memcg_debug(lruvec, folio);
/*
* Try get exclusive access under lock. If marked for
@@ -1132,30 +1137,30 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
* and it's on LRU. It can only be a THP so the order
* is safe to read and it's 0 for tail pages.
*/
- if (unlikely(PageCompound(page) && !cc->alloc_contig)) {
- low_pfn += compound_nr(page) - 1;
- nr_scanned += compound_nr(page) - 1;
- SetPageLRU(page);
+ if (unlikely(folio_test_large(folio) && !cc->alloc_contig)) {
+ low_pfn += folio_nr_pages(folio) - 1;
+ nr_scanned += folio_nr_pages(folio) - 1;
+ folio_set_lru(folio);
goto isolate_fail_put;
}
}
/* The whole page is taken off the LRU; skip the tail pages. */
- if (PageCompound(page))
- low_pfn += compound_nr(page) - 1;
+ if (folio_test_large(folio))
+ low_pfn += folio_nr_pages(folio) - 1;
/* Successfully isolated */
- del_page_from_lru_list(page, lruvec);
- mod_node_page_state(page_pgdat(page),
- NR_ISOLATED_ANON + page_is_file_lru(page),
- thp_nr_pages(page));
+ lruvec_del_folio(lruvec, folio);
+ mod_node_page_state(folio_pgdat(folio),
+ NR_ISOLATED_ANON + folio_is_file_lru(folio),
+ folio_nr_pages(folio));
isolate_success:
- list_add(&page->lru, &cc->migratepages);
+ list_add(&folio->lru, &cc->migratepages);
isolate_success_no_list:
- cc->nr_migratepages += compound_nr(page);
- nr_isolated += compound_nr(page);
- nr_scanned += compound_nr(page) - 1;
+ cc->nr_migratepages += folio_nr_pages(folio);
+ nr_isolated += folio_nr_pages(folio);
+ nr_scanned += folio_nr_pages(folio) - 1;
/*
* Avoid isolating too much unless this block is being
@@ -1177,7 +1182,7 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
unlock_page_lruvec_irqrestore(locked, flags);
locked = NULL;
}
- put_page(page);
+ folio_put(folio);
isolate_fail:
if (!skip_on_failure && ret != -ENOMEM)
@@ -1218,14 +1223,14 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
if (unlikely(low_pfn > end_pfn))
low_pfn = end_pfn;
- page = NULL;
+ folio = NULL;
isolate_abort:
if (locked)
unlock_page_lruvec_irqrestore(locked, flags);
- if (page) {
- SetPageLRU(page);
- put_page(page);
+ if (folio) {
+ folio_set_lru(folio);
+ folio_put(folio);
}
/*
--
2.35.3
^ permalink raw reply related [flat|nested] 6+ messages in thread
* [PATCH -next 2/2] mm: kill [add|del]_page_to_lru_list()
2023-06-12 14:34 [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Kefeng Wang
@ 2023-06-12 14:34 ` Kefeng Wang
2023-06-12 19:10 ` Yu Zhao
2023-06-12 14:41 ` [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Matthew Wilcox
1 sibling, 1 reply; 6+ messages in thread
From: Kefeng Wang @ 2023-06-12 14:34 UTC (permalink / raw)
To: akpm; +Cc: linux-mm, linux-kernel, willy, jgowans, yuzhao, Kefeng Wang
Now no one call [add|del]_page_to_lru_list(), let's drop unused
page interfaces.
Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
---
include/linux/mm_inline.h | 12 ------------
1 file changed, 12 deletions(-)
diff --git a/include/linux/mm_inline.h b/include/linux/mm_inline.h
index 0e1d239a882c..e9cdeb290841 100644
--- a/include/linux/mm_inline.h
+++ b/include/linux/mm_inline.h
@@ -323,12 +323,6 @@ void lruvec_add_folio(struct lruvec *lruvec, struct folio *folio)
list_add(&folio->lru, &lruvec->lists[lru]);
}
-static __always_inline void add_page_to_lru_list(struct page *page,
- struct lruvec *lruvec)
-{
- lruvec_add_folio(lruvec, page_folio(page));
-}
-
static __always_inline
void lruvec_add_folio_tail(struct lruvec *lruvec, struct folio *folio)
{
@@ -357,12 +351,6 @@ void lruvec_del_folio(struct lruvec *lruvec, struct folio *folio)
-folio_nr_pages(folio));
}
-static __always_inline void del_page_from_lru_list(struct page *page,
- struct lruvec *lruvec)
-{
- lruvec_del_folio(lruvec, page_folio(page));
-}
-
#ifdef CONFIG_ANON_VMA_NAME
/*
* mmap_lock should be read-locked when calling anon_vma_name(). Caller should
--
2.35.3
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block()
2023-06-12 14:34 [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Kefeng Wang
2023-06-12 14:34 ` [PATCH -next 2/2] mm: kill [add|del]_page_to_lru_list() Kefeng Wang
@ 2023-06-12 14:41 ` Matthew Wilcox
2023-06-13 1:53 ` Kefeng Wang
1 sibling, 1 reply; 6+ messages in thread
From: Matthew Wilcox @ 2023-06-12 14:41 UTC (permalink / raw)
To: Kefeng Wang; +Cc: akpm, linux-mm, linux-kernel, jgowans, yuzhao
On Mon, Jun 12, 2023 at 10:34:13PM +0800, Kefeng Wang wrote:
> @@ -959,7 +960,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> * Hugepage was successfully isolated and placed
> * on the cc->migratepages list.
> */
> - low_pfn += compound_nr(page) - 1;
> + folio = page_folio(page);
> + low_pfn += folio_nr_pages(folio) - 1;
> goto isolate_success_no_list;
Why is this safe? That is, how do we know that the folio can't be
dissolved under us at this point, then reallocated and hit the
VM_BUG_ON_PGFLAGS(PageTail(page), page) in folio_flags() when we
test folio_test_large()?
> @@ -1132,30 +1137,30 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
> * and it's on LRU. It can only be a THP so the order
> * is safe to read and it's 0 for tail pages.
> */
^^^ This comment needs to be updated too.
> - mod_node_page_state(page_pgdat(page),
> - NR_ISOLATED_ANON + page_is_file_lru(page),
> - thp_nr_pages(page));
> + lruvec_del_folio(lruvec, folio);
> + mod_node_page_state(folio_pgdat(folio),
> + NR_ISOLATED_ANON + folio_is_file_lru(folio),
> + folio_nr_pages(folio));
node_stat_mod_folio()
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH -next 2/2] mm: kill [add|del]_page_to_lru_list()
2023-06-12 14:34 ` [PATCH -next 2/2] mm: kill [add|del]_page_to_lru_list() Kefeng Wang
@ 2023-06-12 19:10 ` Yu Zhao
0 siblings, 0 replies; 6+ messages in thread
From: Yu Zhao @ 2023-06-12 19:10 UTC (permalink / raw)
To: Kefeng Wang; +Cc: akpm, linux-mm, linux-kernel, willy, jgowans
On Mon, Jun 12, 2023 at 8:19 AM Kefeng Wang <wangkefeng.wang@huawei.com> wrote:
>
> Now no one call [add|del]_page_to_lru_list(), let's drop unused
> page interfaces.
>
> Signed-off-by: Kefeng Wang <wangkefeng.wang@huawei.com>
Acked-by: Yu Zhao <yuzhao@google.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block()
2023-06-12 14:41 ` [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Matthew Wilcox
@ 2023-06-13 1:53 ` Kefeng Wang
2023-06-14 11:21 ` Kefeng Wang
0 siblings, 1 reply; 6+ messages in thread
From: Kefeng Wang @ 2023-06-13 1:53 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: akpm, linux-mm, linux-kernel, jgowans, yuzhao
On 2023/6/12 22:41, Matthew Wilcox wrote:
> On Mon, Jun 12, 2023 at 10:34:13PM +0800, Kefeng Wang wrote:
>> @@ -959,7 +960,8 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
>> * Hugepage was successfully isolated and placed
>> * on the cc->migratepages list.
>> */
>> - low_pfn += compound_nr(page) - 1;
>> + folio = page_folio(page);
>> + low_pfn += folio_nr_pages(folio) - 1;
>> goto isolate_success_no_list;
>
> Why is this safe? That is, how do we know that the folio can't be
> dissolved under us at this point, then reallocated and hit the
> VM_BUG_ON_PGFLAGS(PageTail(page), page) in folio_flags() when we
> test folio_test_large()?
This is successfully isolated path, after isolate_hugetlb(), the folio
reference is incremented, so I think the folio can't be dissolved here,
correct me if I am wrong.
>
>> @@ -1132,30 +1137,30 @@ isolate_migratepages_block(struct compact_control *cc, unsigned long low_pfn,
>> * and it's on LRU. It can only be a THP so the order
>> * is safe to read and it's 0 for tail pages.
>> */
>
> ^^^ This comment needs to be updated too.
will update
>
>> - mod_node_page_state(page_pgdat(page),
>> - NR_ISOLATED_ANON + page_is_file_lru(page),
>> - thp_nr_pages(page));
>> + lruvec_del_folio(lruvec, folio);
>> + mod_node_page_state(folio_pgdat(folio),
>> + NR_ISOLATED_ANON + folio_is_file_lru(folio),
>> + folio_nr_pages(folio));
>
> node_stat_mod_folio()
> ok
Thanks
>
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block()
2023-06-13 1:53 ` Kefeng Wang
@ 2023-06-14 11:21 ` Kefeng Wang
0 siblings, 0 replies; 6+ messages in thread
From: Kefeng Wang @ 2023-06-14 11:21 UTC (permalink / raw)
To: Matthew Wilcox; +Cc: akpm, linux-mm, linux-kernel, jgowans, yuzhao
On 2023/6/13 9:53, Kefeng Wang wrote:
>
>
> On 2023/6/12 22:41, Matthew Wilcox wrote:
>> On Mon, Jun 12, 2023 at 10:34:13PM +0800, Kefeng Wang wrote:
>>> @@ -959,7 +960,8 @@ isolate_migratepages_block(struct compact_control
>>> *cc, unsigned long low_pfn,
>>> * Hugepage was successfully isolated and placed
>>> * on the cc->migratepages list.
>>> */
>>> - low_pfn += compound_nr(page) - 1;
>>> + folio = page_folio(page);
>>> + low_pfn += folio_nr_pages(folio) - 1;
>>> goto isolate_success_no_list;
>>
>> Why is this safe? That is, how do we know that the folio can't be
>> dissolved under us at this point, then reallocated and hit the
>> VM_BUG_ON_PGFLAGS(PageTail(page), page) in folio_flags() when we
>> test folio_test_large()?
>
> This is successfully isolated path, after isolate_hugetlb(), the folio
> reference is incremented, so I think the folio can't be dissolved here,
> correct me if I am wrong.
Hi Matthew，is it enough for the above conversion from page to folio with
reference?
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2023-06-14 11:21 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-06-12 14:34 [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Kefeng Wang
2023-06-12 14:34 ` [PATCH -next 2/2] mm: kill [add|del]_page_to_lru_list() Kefeng Wang
2023-06-12 19:10 ` Yu Zhao
2023-06-12 14:41 ` [PATCH -next 1/2] mm: compaction: convert to use a folio in isolate_migratepages_block() Matthew Wilcox
2023-06-13 1:53 ` Kefeng Wang
2023-06-14 11:21 ` Kefeng Wang
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.