* [PATCH] mm: thp: update split_queue_len correctly
@ 2021-11-23 17:46 Shakeel Butt
2021-11-23 17:59 ` Shakeel Butt
2021-11-23 18:51 ` Yang Shi
0 siblings, 2 replies; 4+ messages in thread
From: Shakeel Butt @ 2021-11-23 17:46 UTC (permalink / raw)
To: David Hildenbrand, Kirill A . Shutemov, Yang Shi, Zi Yan, Matthew Wilcox
Cc: Andrew Morton, linux-mm, linux-kernel, Shakeel Butt
The deferred THPs are split on memory pressure through shrinker
callback and splitting of THP during reclaim can fail for several
reasons like unable to lock the THP, under writeback or unexpected
number of pins on the THP. Such pages are put back on the deferred split
list for consideration later. However kernel does not update the
deferred queue size on putting back the pages whose split was failed.
This patch fixes that.
Fixes: 364c1eebe453 ("mm: thp: extract split_queue_* into a struct")
Signed-off-by: Shakeel Butt <shakeelb@google.com>
---
mm/huge_memory.c | 4 +++-
1 file changed, 3 insertions(+), 1 deletion(-)
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e5483347291c..4fff9584815b 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2809,7 +2809,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
unsigned long flags;
LIST_HEAD(list), *pos, *next;
struct page *page;
- int split = 0;
+ unsigned long split = 0, num = 0;
#ifdef CONFIG_MEMCG
if (sc->memcg)
@@ -2823,6 +2823,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
page = compound_head(page);
if (get_page_unless_zero(page)) {
list_move(page_deferred_list(page), &list);
+ num++;
} else {
/* We lost race with put_compound_page() */
list_del_init(page_deferred_list(page));
@@ -2847,6 +2848,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
spin_lock_irqsave(&ds_queue->split_queue_lock, flags);
list_splice_tail(&list, &ds_queue->split_queue);
+ ds_queue->split_queue_len += (num - split);
spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags);
/*
--
2.34.0.rc2.393.gf8c9666880-goog
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] mm: thp: update split_queue_len correctly
2021-11-23 17:46 [PATCH] mm: thp: update split_queue_len correctly Shakeel Butt
@ 2021-11-23 17:59 ` Shakeel Butt
2021-11-23 18:51 ` Yang Shi
1 sibling, 0 replies; 4+ messages in thread
From: Shakeel Butt @ 2021-11-23 17:59 UTC (permalink / raw)
To: David Hildenbrand, Kirill A . Shutemov, Yang Shi, Zi Yan, Matthew Wilcox
Cc: Andrew Morton, linux-mm, linux-kernel
On Tue, Nov 23, 2021 at 9:47 AM Shakeel Butt <shakeelb@google.com> wrote:
>
> The deferred THPs are split on memory pressure through shrinker
> callback and splitting of THP during reclaim can fail for several
> reasons like unable to lock the THP, under writeback or unexpected
> number of pins on the THP. Such pages are put back on the deferred split
> list for consideration later. However kernel does not update the
> deferred queue size on putting back the pages whose split was failed.
> This patch fixes that.
I forgot to add the user visible impact.
"Without this patch the split_queue_len can underflow. Shrinker will
always get that there are some THPs to split even if there are not and
waste some cpu to scan the empty list."
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: [PATCH] mm: thp: update split_queue_len correctly
2021-11-23 17:46 [PATCH] mm: thp: update split_queue_len correctly Shakeel Butt
2021-11-23 17:59 ` Shakeel Butt
@ 2021-11-23 18:51 ` Yang Shi
2021-11-23 19:03 ` Shakeel Butt
1 sibling, 1 reply; 4+ messages in thread
From: Yang Shi @ 2021-11-23 18:51 UTC (permalink / raw)
To: Shakeel Butt
Cc: David Hildenbrand, Kirill A . Shutemov, Zi Yan, Matthew Wilcox,
Andrew Morton, Linux MM, Linux Kernel Mailing List
On Tue, Nov 23, 2021 at 9:47 AM Shakeel Butt <shakeelb@google.com> wrote:
>
> The deferred THPs are split on memory pressure through shrinker
> callback and splitting of THP during reclaim can fail for several
> reasons like unable to lock the THP, under writeback or unexpected
> number of pins on the THP. Such pages are put back on the deferred split
> list for consideration later. However kernel does not update the
> deferred queue size on putting back the pages whose split was failed.
> This patch fixes that.
>
> Fixes: 364c1eebe453 ("mm: thp: extract split_queue_* into a struct")
> Signed-off-by: Shakeel Butt <shakeelb@google.com>
> ---
> mm/huge_memory.c | 4 +++-
> 1 file changed, 3 insertions(+), 1 deletion(-)
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index e5483347291c..4fff9584815b 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2809,7 +2809,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> unsigned long flags;
> LIST_HEAD(list), *pos, *next;
> struct page *page;
> - int split = 0;
> + unsigned long split = 0, num = 0;
>
> #ifdef CONFIG_MEMCG
> if (sc->memcg)
> @@ -2823,6 +2823,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> page = compound_head(page);
> if (get_page_unless_zero(page)) {
> list_move(page_deferred_list(page), &list);
> + num++;
Thanks for catching this. But I don't think "num" is needed, isn't the
below code good enough?
diff --git a/mm/huge_memory.c b/mm/huge_memory.c
index e5483347291c..1fbd8299db0c 100644
--- a/mm/huge_memory.c
+++ b/mm/huge_memory.c
@@ -2847,6 +2847,7 @@ static unsigned long deferred_split_scan(struct
shrinker *shrink,
spin_lock_irqsave(&ds_queue->split_queue_lock, flags);
list_splice_tail(&list, &ds_queue->split_queue);
+ ds_queue->split_queue_len -= split;
spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags);
/*
> } else {
> /* We lost race with put_compound_page() */
> list_del_init(page_deferred_list(page));
> @@ -2847,6 +2848,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
>
> spin_lock_irqsave(&ds_queue->split_queue_lock, flags);
> list_splice_tail(&list, &ds_queue->split_queue);
> + ds_queue->split_queue_len += (num - split);
> spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags);
>
> /*
> --
> 2.34.0.rc2.393.gf8c9666880-goog
>
^ permalink raw reply related [flat|nested] 4+ messages in thread
* Re: [PATCH] mm: thp: update split_queue_len correctly
2021-11-23 18:51 ` Yang Shi
@ 2021-11-23 19:03 ` Shakeel Butt
0 siblings, 0 replies; 4+ messages in thread
From: Shakeel Butt @ 2021-11-23 19:03 UTC (permalink / raw)
To: Yang Shi
Cc: David Hildenbrand, Kirill A . Shutemov, Zi Yan, Matthew Wilcox,
Andrew Morton, Linux MM, Linux Kernel Mailing List
On Tue, Nov 23, 2021 at 10:51 AM Yang Shi <shy828301@gmail.com> wrote:
>
> On Tue, Nov 23, 2021 at 9:47 AM Shakeel Butt <shakeelb@google.com> wrote:
> >
> > The deferred THPs are split on memory pressure through shrinker
> > callback and splitting of THP during reclaim can fail for several
> > reasons like unable to lock the THP, under writeback or unexpected
> > number of pins on the THP. Such pages are put back on the deferred split
> > list for consideration later. However kernel does not update the
> > deferred queue size on putting back the pages whose split was failed.
> > This patch fixes that.
> >
> > Fixes: 364c1eebe453 ("mm: thp: extract split_queue_* into a struct")
> > Signed-off-by: Shakeel Butt <shakeelb@google.com>
> > ---
> > mm/huge_memory.c | 4 +++-
> > 1 file changed, 3 insertions(+), 1 deletion(-)
> >
> > diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> > index e5483347291c..4fff9584815b 100644
> > --- a/mm/huge_memory.c
> > +++ b/mm/huge_memory.c
> > @@ -2809,7 +2809,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> > unsigned long flags;
> > LIST_HEAD(list), *pos, *next;
> > struct page *page;
> > - int split = 0;
> > + unsigned long split = 0, num = 0;
> >
> > #ifdef CONFIG_MEMCG
> > if (sc->memcg)
> > @@ -2823,6 +2823,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> > page = compound_head(page);
> > if (get_page_unless_zero(page)) {
> > list_move(page_deferred_list(page), &list);
> > + num++;
>
> Thanks for catching this. But I don't think "num" is needed, isn't the
> below code good enough?
Yes you are right. I will send the next version. I will at least
change the type of split to unsigned long.
>
> diff --git a/mm/huge_memory.c b/mm/huge_memory.c
> index e5483347291c..1fbd8299db0c 100644
> --- a/mm/huge_memory.c
> +++ b/mm/huge_memory.c
> @@ -2847,6 +2847,7 @@ static unsigned long deferred_split_scan(struct
> shrinker *shrink,
>
> spin_lock_irqsave(&ds_queue->split_queue_lock, flags);
> list_splice_tail(&list, &ds_queue->split_queue);
> + ds_queue->split_queue_len -= split;
> spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags);
>
> /*
>
> > } else {
> > /* We lost race with put_compound_page() */
> > list_del_init(page_deferred_list(page));
> > @@ -2847,6 +2848,7 @@ static unsigned long deferred_split_scan(struct shrinker *shrink,
> >
> > spin_lock_irqsave(&ds_queue->split_queue_lock, flags);
> > list_splice_tail(&list, &ds_queue->split_queue);
> > + ds_queue->split_queue_len += (num - split);
> > spin_unlock_irqrestore(&ds_queue->split_queue_lock, flags);
> >
> > /*
> > --
> > 2.34.0.rc2.393.gf8c9666880-goog
> >
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2021-11-23 19:03 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2021-11-23 17:46 [PATCH] mm: thp: update split_queue_len correctly Shakeel Butt
2021-11-23 17:59 ` Shakeel Butt
2021-11-23 18:51 ` Yang Shi
2021-11-23 19:03 ` Shakeel Butt
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.