* [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
@ 2019-11-05 22:02 Yang Shi
2019-11-06 15:18 ` Michal Hocko
0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2019-11-05 22:02 UTC (permalink / raw)
To: hughd, akpm; +Cc: yang.shi, linux-mm, linux-kernel
The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
__GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should
write out to swap only in response to memory pressure, so
__GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
or in direct reclaim already.
In addition, XArray node allocations from PF_MEMALLOC contexts could
completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
reserves from being allocated.
Here just copy the gfp flags used by add_to_swap().
Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
---
mm/shmem.c | 3 ++-
1 file changed, 2 insertions(+), 1 deletion(-)
diff --git a/mm/shmem.c b/mm/shmem.c
index 220be9f..9691dec 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
if (list_empty(&info->swaplist))
list_add(&info->swaplist, &shmem_swaplist);
- if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+ if (add_to_swap_cache(page, swap,
+ __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
spin_lock_irq(&info->lock);
shmem_recalc_inode(inode);
info->swapped++;
--
1.8.3.1
^ permalink raw reply related [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
2019-11-05 22:02 [PATCH] mm: shmem: use proper gfp flags for shmem_writepage() Yang Shi
@ 2019-11-06 15:18 ` Michal Hocko
2019-11-06 18:23 ` Yang Shi
0 siblings, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2019-11-06 15:18 UTC (permalink / raw)
To: Yang Shi; +Cc: hughd, akpm, linux-mm, linux-kernel
On Wed 06-11-19 06:02:31, Yang Shi wrote:
> The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
> GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
> __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should
> write out to swap only in response to memory pressure, so
> __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
> or in direct reclaim already.
What kind of problem are you trying to fix here?
> In addition, XArray node allocations from PF_MEMALLOC contexts could
> completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
> reserves from being allocated.
I am not really familiar with XArray much, could you be more specific
please?
> Here just copy the gfp flags used by add_to_swap().
>
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
> ---
> mm/shmem.c | 3 ++-
> 1 file changed, 2 insertions(+), 1 deletion(-)
>
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 220be9f..9691dec 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
> if (list_empty(&info->swaplist))
> list_add(&info->swaplist, &shmem_swaplist);
>
> - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
> + if (add_to_swap_cache(page, swap,
> + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
> spin_lock_irq(&info->lock);
> shmem_recalc_inode(inode);
> info->swapped++;
> --
> 1.8.3.1
--
Michal Hocko
SUSE Labs
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
2019-11-06 15:18 ` Michal Hocko
@ 2019-11-06 18:23 ` Yang Shi
2019-11-06 18:59 ` Hugh Dickins
0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2019-11-06 18:23 UTC (permalink / raw)
To: Michal Hocko; +Cc: hughd, akpm, linux-mm, linux-kernel
On 11/6/19 7:18 AM, Michal Hocko wrote:
> On Wed 06-11-19 06:02:31, Yang Shi wrote:
>> The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
>> GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
>> __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should
>> write out to swap only in response to memory pressure, so
>> __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
>> or in direct reclaim already.
> What kind of problem are you trying to fix here?
I didn't run into any visible problem. I just happened to find this
inconsistency when I was looking into the other problem.
The add_to_swap() does:
int add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry,
__GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}
Actually, shmem_writepage() does almost the same thing and both of them
are called in reclaim context, so I didn't see why they should use
different gfp flag. And, GFP_ATOMIC is also different from the old
definition as I mentioned in the commit log.
>
>> In addition, XArray node allocations from PF_MEMALLOC contexts could
>> completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
>> reserves from being allocated.
> I am not really familiar with XArray much, could you be more specific
> please?
It comes from the comments of add_to_swap(), says:
/*
* XArray node allocations from PF_MEMALLOC contexts could
* completely exhaust the page allocator. __GFP_NOMEMALLOC
* stops emergency reserves from being allocated.
And, it looks the original comment came from pre-git time, TBH I'm not
quite sure about the specific problem which incurred this. I suspect it
may be because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.
>
>> Here just copy the gfp flags used by add_to_swap().
>>
>> Cc: Hugh Dickins <hughd@google.com>
>> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
>> ---
>> mm/shmem.c | 3 ++-
>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/shmem.c b/mm/shmem.c
>> index 220be9f..9691dec 100644
>> --- a/mm/shmem.c
>> +++ b/mm/shmem.c
>> @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
>> if (list_empty(&info->swaplist))
>> list_add(&info->swaplist, &shmem_swaplist);
>>
>> - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
>> + if (add_to_swap_cache(page, swap,
>> + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
>> spin_lock_irq(&info->lock);
>> shmem_recalc_inode(inode);
>> info->swapped++;
>> --
>> 1.8.3.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
2019-11-06 18:23 ` Yang Shi
@ 2019-11-06 18:59 ` Hugh Dickins
2019-11-07 20:19 ` Yang Shi
0 siblings, 1 reply; 6+ messages in thread
From: Hugh Dickins @ 2019-11-06 18:59 UTC (permalink / raw)
To: Yang Shi; +Cc: Michal Hocko, hughd, akpm, linux-mm, linux-kernel
[-- Attachment #1: Type: TEXT/PLAIN, Size: 3462 bytes --]
On Wed, 6 Nov 2019, Yang Shi wrote:
> On 11/6/19 7:18 AM, Michal Hocko wrote:
> > On Wed 06-11-19 06:02:31, Yang Shi wrote:
> > > The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
> > > GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
> > > __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should
> > > write out to swap only in response to memory pressure, so
> > > __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
> > > or in direct reclaim already.
> > What kind of problem are you trying to fix here?
>
> I didn't run into any visible problem. I just happened to find this
> inconsistency when I was looking into the other problem.
Yes, I don't think it fixes any actual problem: just a cleanup to
make the two calls look the same when they don't need to be different
(whereas the call from __read_swap_cache_async() rightly uses a
lower priority gfp).
If it does fix a problem, then you need to worry also about the
* TODO: this could cause a theoretical memory reclaim
* deadlock in the swap out path.
comment still against the call in add_to_swap(): but I think that
is equally theoretical, demanding no attention since 2.6.12.
>
> The add_to_swap() does:
>
> int add_to_swap(struct page *page)
> {
> ...
> err = add_to_swap_cache(page, entry,
> __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
> ...
> }
>
> Actually, shmem_writepage() does almost the same thing and both of them are
> called in reclaim context, so I didn't see why they should use different gfp
> flag. And, GFP_ATOMIC is also different from the old definition as I
> mentioned in the commit log.
>
> >
> > > In addition, XArray node allocations from PF_MEMALLOC contexts could
> > > completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
> > > reserves from being allocated.
> > I am not really familiar with XArray much, could you be more specific
> > please?
>
> It comes from the comments of add_to_swap(), says:
>
> /*
> * XArray node allocations from PF_MEMALLOC contexts could
> * completely exhaust the page allocator. __GFP_NOMEMALLOC
> * stops emergency reserves from being allocated.
>
> And, it looks the original comment came from pre-git time, TBH I'm not quite
> sure about the specific problem which incurred this. I suspect it may be
> because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.
>
> >
> > > Here just copy the gfp flags used by add_to_swap().
> > >
> > > Cc: Hugh Dickins <hughd@google.com>
> > > Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
> > > ---
> > > mm/shmem.c | 3 ++-
> > > 1 file changed, 2 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/mm/shmem.c b/mm/shmem.c
> > > index 220be9f..9691dec 100644
> > > --- a/mm/shmem.c
> > > +++ b/mm/shmem.c
> > > @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page,
> > > struct writeback_control *wbc)
> > > if (list_empty(&info->swaplist))
> > > list_add(&info->swaplist, &shmem_swaplist);
> > > - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
> > > + if (add_to_swap_cache(page, swap,
> > > + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
> > > spin_lock_irq(&info->lock);
> > > shmem_recalc_inode(inode);
> > > info->swapped++;
> > > --
> > > 1.8.3.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
2019-11-06 18:59 ` Hugh Dickins
@ 2019-11-07 20:19 ` Yang Shi
2019-11-07 22:59 ` Hugh Dickins
0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2019-11-07 20:19 UTC (permalink / raw)
To: Hugh Dickins; +Cc: Michal Hocko, akpm, linux-mm, linux-kernel
On 11/6/19 10:59 AM, Hugh Dickins wrote:
> On Wed, 6 Nov 2019, Yang Shi wrote:
>> On 11/6/19 7:18 AM, Michal Hocko wrote:
>>> On Wed 06-11-19 06:02:31, Yang Shi wrote:
>>>> The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
>>>> GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
>>>> __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM. However, shmem_writepage() should
>>>> write out to swap only in response to memory pressure, so
>>>> __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
>>>> or in direct reclaim already.
>>> What kind of problem are you trying to fix here?
>> I didn't run into any visible problem. I just happened to find this
>> inconsistency when I was looking into the other problem.
> Yes, I don't think it fixes any actual problem: just a cleanup to
> make the two calls look the same when they don't need to be different
> (whereas the call from __read_swap_cache_async() rightly uses a
> lower priority gfp).
I'm supposed it is because __read_swap_cache_async()is typically called
from page fault context which is less crucial than reclaim.
Shall I consider this as an ack but with commit log rephrased to reflect
the cleanup?
>
> If it does fix a problem, then you need to worry also about the
> * TODO: this could cause a theoretical memory reclaim
> * deadlock in the swap out path.
> comment still against the call in add_to_swap(): but I think that
> is equally theoretical, demanding no attention since 2.6.12.
>
>> The add_to_swap() does:
>>
>> int add_to_swap(struct page *page)
>> {
>> ...
>> err = add_to_swap_cache(page, entry,
>> __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
>> ...
>> }
>>
>> Actually, shmem_writepage() does almost the same thing and both of them are
>> called in reclaim context, so I didn't see why they should use different gfp
>> flag. And, GFP_ATOMIC is also different from the old definition as I
>> mentioned in the commit log.
>>
>>>> In addition, XArray node allocations from PF_MEMALLOC contexts could
>>>> completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
>>>> reserves from being allocated.
>>> I am not really familiar with XArray much, could you be more specific
>>> please?
>> It comes from the comments of add_to_swap(), says:
>>
>> /*
>> * XArray node allocations from PF_MEMALLOC contexts could
>> * completely exhaust the page allocator. __GFP_NOMEMALLOC
>> * stops emergency reserves from being allocated.
>>
>> And, it looks the original comment came from pre-git time, TBH I'm not quite
>> sure about the specific problem which incurred this. I suspect it may be
>> because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.
>>
>>>> Here just copy the gfp flags used by add_to_swap().
>>>>
>>>> Cc: Hugh Dickins <hughd@google.com>
>>>> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
>>>> ---
>>>> mm/shmem.c | 3 ++-
>>>> 1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>>> index 220be9f..9691dec 100644
>>>> --- a/mm/shmem.c
>>>> +++ b/mm/shmem.c
>>>> @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page,
>>>> struct writeback_control *wbc)
>>>> if (list_empty(&info->swaplist))
>>>> list_add(&info->swaplist, &shmem_swaplist);
>>>> - if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
>>>> + if (add_to_swap_cache(page, swap,
>>>> + __GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
>>>> spin_lock_irq(&info->lock);
>>>> shmem_recalc_inode(inode);
>>>> info->swapped++;
>>>> --
>>>> 1.8.3.1
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
2019-11-07 20:19 ` Yang Shi
@ 2019-11-07 22:59 ` Hugh Dickins
0 siblings, 0 replies; 6+ messages in thread
From: Hugh Dickins @ 2019-11-07 22:59 UTC (permalink / raw)
To: Yang Shi; +Cc: Hugh Dickins, Michal Hocko, akpm, linux-mm, linux-kernel
On Thu, 7 Nov 2019, Yang Shi wrote:
> On 11/6/19 10:59 AM, Hugh Dickins wrote:
> > Yes, I don't think it fixes any actual problem: just a cleanup to
> > make the two calls look the same when they don't need to be different
> > (whereas the call from __read_swap_cache_async() rightly uses a
> > lower priority gfp).
>
> I'm supposed it is because __read_swap_cache_async()is typically called from
> page fault context which is less crucial than reclaim.
Exactly.
>
> Shall I consider this as an ack but with commit log rephrased to reflect the
> cleanup?
Okay,
Acked-by: Hugh Dickins <hughd@google.com>
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2019-11-07 22:59 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-05 22:02 [PATCH] mm: shmem: use proper gfp flags for shmem_writepage() Yang Shi
2019-11-06 15:18 ` Michal Hocko
2019-11-06 18:23 ` Yang Shi
2019-11-06 18:59 ` Hugh Dickins
2019-11-07 20:19 ` Yang Shi
2019-11-07 22:59 ` Hugh Dickins
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).