Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
* [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
@ 2019-11-05 22:02 Yang Shi
  2019-11-06 15:18 ` Michal Hocko
  0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2019-11-05 22:02 UTC (permalink / raw)
  To: hughd, akpm; +Cc: yang.shi, linux-mm, linux-kernel

The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
__GFP_ATOMIC | __GFP_KSWAPD_RECLAIM.  However, shmem_writepage() should
write out to swap only in response to memory pressure, so
__GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
or in direct reclaim already.

In addition, XArray node allocations from PF_MEMALLOC contexts could
completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
reserves from being allocated.

Here just copy the gfp flags used by add_to_swap().

Cc: Hugh Dickins <hughd@google.com>
Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
---
 mm/shmem.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/mm/shmem.c b/mm/shmem.c
index 220be9f..9691dec 100644
--- a/mm/shmem.c
+++ b/mm/shmem.c
@@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
 	if (list_empty(&info->swaplist))
 		list_add(&info->swaplist, &shmem_swaplist);
 
-	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
+	if (add_to_swap_cache(page, swap,
+			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
 		spin_lock_irq(&info->lock);
 		shmem_recalc_inode(inode);
 		info->swapped++;
-- 
1.8.3.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
  2019-11-05 22:02 [PATCH] mm: shmem: use proper gfp flags for shmem_writepage() Yang Shi
@ 2019-11-06 15:18 ` Michal Hocko
  2019-11-06 18:23   ` Yang Shi
  0 siblings, 1 reply; 6+ messages in thread
From: Michal Hocko @ 2019-11-06 15:18 UTC (permalink / raw)
  To: Yang Shi; +Cc: hughd, akpm, linux-mm, linux-kernel

On Wed 06-11-19 06:02:31, Yang Shi wrote:
> The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
> GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
> __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM.  However, shmem_writepage() should
> write out to swap only in response to memory pressure, so
> __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
> or in direct reclaim already.

What kind of problem are you trying to fix here?

> In addition, XArray node allocations from PF_MEMALLOC contexts could
> completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
> reserves from being allocated.

I am not really familiar with XArray much, could you be more specific
please?

> Here just copy the gfp flags used by add_to_swap().
> 
> Cc: Hugh Dickins <hughd@google.com>
> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
> ---
>  mm/shmem.c | 3 ++-
>  1 file changed, 2 insertions(+), 1 deletion(-)
> 
> diff --git a/mm/shmem.c b/mm/shmem.c
> index 220be9f..9691dec 100644
> --- a/mm/shmem.c
> +++ b/mm/shmem.c
> @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
>  	if (list_empty(&info->swaplist))
>  		list_add(&info->swaplist, &shmem_swaplist);
>  
> -	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
> +	if (add_to_swap_cache(page, swap,
> +			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
>  		spin_lock_irq(&info->lock);
>  		shmem_recalc_inode(inode);
>  		info->swapped++;
> -- 
> 1.8.3.1

-- 
Michal Hocko
SUSE Labs


^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
  2019-11-06 15:18 ` Michal Hocko
@ 2019-11-06 18:23   ` Yang Shi
  2019-11-06 18:59     ` Hugh Dickins
  0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2019-11-06 18:23 UTC (permalink / raw)
  To: Michal Hocko; +Cc: hughd, akpm, linux-mm, linux-kernel



On 11/6/19 7:18 AM, Michal Hocko wrote:
> On Wed 06-11-19 06:02:31, Yang Shi wrote:
>> The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
>> GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
>> __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM.  However, shmem_writepage() should
>> write out to swap only in response to memory pressure, so
>> __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
>> or in direct reclaim already.
> What kind of problem are you trying to fix here?

I didn't run into any visible problem. I just happened to find this 
inconsistency when I was looking into the other problem.

The add_to_swap() does:

int add_to_swap(struct page *page)
{
...
err = add_to_swap_cache(page, entry,
                         __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
...
}

Actually, shmem_writepage() does almost the same thing and both of them 
are called in reclaim context, so I didn't see why they should use 
different gfp flag. And, GFP_ATOMIC is also different from the old 
definition as I mentioned in the commit log.

>
>> In addition, XArray node allocations from PF_MEMALLOC contexts could
>> completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
>> reserves from being allocated.
> I am not really familiar with XArray much, could you be more specific
> please?

It comes from the comments of add_to_swap(), says:

/*
          * XArray node allocations from PF_MEMALLOC contexts could
          * completely exhaust the page allocator. __GFP_NOMEMALLOC
          * stops emergency reserves from being allocated.

And, it looks the original comment came from pre-git time, TBH I'm not 
quite sure about the specific problem which incurred this. I suspect it 
may be because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.

>
>> Here just copy the gfp flags used by add_to_swap().
>>
>> Cc: Hugh Dickins <hughd@google.com>
>> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
>> ---
>>   mm/shmem.c | 3 ++-
>>   1 file changed, 2 insertions(+), 1 deletion(-)
>>
>> diff --git a/mm/shmem.c b/mm/shmem.c
>> index 220be9f..9691dec 100644
>> --- a/mm/shmem.c
>> +++ b/mm/shmem.c
>> @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page, struct writeback_control *wbc)
>>   	if (list_empty(&info->swaplist))
>>   		list_add(&info->swaplist, &shmem_swaplist);
>>   
>> -	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
>> +	if (add_to_swap_cache(page, swap,
>> +			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
>>   		spin_lock_irq(&info->lock);
>>   		shmem_recalc_inode(inode);
>>   		info->swapped++;
>> -- 
>> 1.8.3.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
  2019-11-06 18:23   ` Yang Shi
@ 2019-11-06 18:59     ` Hugh Dickins
  2019-11-07 20:19       ` Yang Shi
  0 siblings, 1 reply; 6+ messages in thread
From: Hugh Dickins @ 2019-11-06 18:59 UTC (permalink / raw)
  To: Yang Shi; +Cc: Michal Hocko, hughd, akpm, linux-mm, linux-kernel

[-- Attachment #1: Type: TEXT/PLAIN, Size: 3462 bytes --]

On Wed, 6 Nov 2019, Yang Shi wrote:
> On 11/6/19 7:18 AM, Michal Hocko wrote:
> > On Wed 06-11-19 06:02:31, Yang Shi wrote:
> > > The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
> > > GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
> > > __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM.  However, shmem_writepage() should
> > > write out to swap only in response to memory pressure, so
> > > __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
> > > or in direct reclaim already.
> > What kind of problem are you trying to fix here?
> 
> I didn't run into any visible problem. I just happened to find this
> inconsistency when I was looking into the other problem.

Yes, I don't think it fixes any actual problem: just a cleanup to
make the two calls look the same when they don't need to be different
(whereas the call from __read_swap_cache_async() rightly uses a 
lower priority gfp).

If it does fix a problem, then you need to worry also about the
	 * TODO: this could cause a theoretical memory reclaim
	 * deadlock in the swap out path.
comment still against the call in add_to_swap(): but I think that
is equally theoretical, demanding no attention since 2.6.12.

> 
> The add_to_swap() does:
> 
> int add_to_swap(struct page *page)
> {
> ...
> err = add_to_swap_cache(page, entry,
>                         __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
> ...
> }
> 
> Actually, shmem_writepage() does almost the same thing and both of them are
> called in reclaim context, so I didn't see why they should use different gfp
> flag. And, GFP_ATOMIC is also different from the old definition as I
> mentioned in the commit log.
> 
> > 
> > > In addition, XArray node allocations from PF_MEMALLOC contexts could
> > > completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
> > > reserves from being allocated.
> > I am not really familiar with XArray much, could you be more specific
> > please?
> 
> It comes from the comments of add_to_swap(), says:
> 
> /*
>          * XArray node allocations from PF_MEMALLOC contexts could
>          * completely exhaust the page allocator. __GFP_NOMEMALLOC
>          * stops emergency reserves from being allocated.
> 
> And, it looks the original comment came from pre-git time, TBH I'm not quite
> sure about the specific problem which incurred this. I suspect it may be
> because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.
> 
> > 
> > > Here just copy the gfp flags used by add_to_swap().
> > > 
> > > Cc: Hugh Dickins <hughd@google.com>
> > > Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
> > > ---
> > >   mm/shmem.c | 3 ++-
> > >   1 file changed, 2 insertions(+), 1 deletion(-)
> > > 
> > > diff --git a/mm/shmem.c b/mm/shmem.c
> > > index 220be9f..9691dec 100644
> > > --- a/mm/shmem.c
> > > +++ b/mm/shmem.c
> > > @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page,
> > > struct writeback_control *wbc)
> > >   	if (list_empty(&info->swaplist))
> > >   		list_add(&info->swaplist, &shmem_swaplist);
> > >   -	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
> > > +	if (add_to_swap_cache(page, swap,
> > > +			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
> > >   		spin_lock_irq(&info->lock);
> > >   		shmem_recalc_inode(inode);
> > >   		info->swapped++;
> > > -- 
> > > 1.8.3.1

^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
  2019-11-06 18:59     ` Hugh Dickins
@ 2019-11-07 20:19       ` Yang Shi
  2019-11-07 22:59         ` Hugh Dickins
  0 siblings, 1 reply; 6+ messages in thread
From: Yang Shi @ 2019-11-07 20:19 UTC (permalink / raw)
  To: Hugh Dickins; +Cc: Michal Hocko, akpm, linux-mm, linux-kernel



On 11/6/19 10:59 AM, Hugh Dickins wrote:
> On Wed, 6 Nov 2019, Yang Shi wrote:
>> On 11/6/19 7:18 AM, Michal Hocko wrote:
>>> On Wed 06-11-19 06:02:31, Yang Shi wrote:
>>>> The shmem_writepage() uses GFP_ATOMIC to allocate swap cache.
>>>> GFP_ATOMIC used to mean __GFP_HIGH, but now it means __GFP_HIGH |
>>>> __GFP_ATOMIC | __GFP_KSWAPD_RECLAIM.  However, shmem_writepage() should
>>>> write out to swap only in response to memory pressure, so
>>>> __GFP_KSWAPD_RECLAIM looks useless since the caller may be kswapd itself
>>>> or in direct reclaim already.
>>> What kind of problem are you trying to fix here?
>> I didn't run into any visible problem. I just happened to find this
>> inconsistency when I was looking into the other problem.
> Yes, I don't think it fixes any actual problem: just a cleanup to
> make the two calls look the same when they don't need to be different
> (whereas the call from __read_swap_cache_async() rightly uses a
> lower priority gfp).

I'm supposed it is because __read_swap_cache_async()is typically called 
from page fault context which is less crucial than reclaim.

Shall I consider this as an ack but with commit log rephrased to reflect 
the cleanup?

>
> If it does fix a problem, then you need to worry also about the
> 	 * TODO: this could cause a theoretical memory reclaim
> 	 * deadlock in the swap out path.
> comment still against the call in add_to_swap(): but I think that
> is equally theoretical, demanding no attention since 2.6.12.
>
>> The add_to_swap() does:
>>
>> int add_to_swap(struct page *page)
>> {
>> ...
>> err = add_to_swap_cache(page, entry,
>>                          __GFP_HIGH|__GFP_NOMEMALLOC|__GFP_NOWARN);
>> ...
>> }
>>
>> Actually, shmem_writepage() does almost the same thing and both of them are
>> called in reclaim context, so I didn't see why they should use different gfp
>> flag. And, GFP_ATOMIC is also different from the old definition as I
>> mentioned in the commit log.
>>
>>>> In addition, XArray node allocations from PF_MEMALLOC contexts could
>>>> completely exhaust the page allocator, __GFP_NOMEMALLOC stops emergency
>>>> reserves from being allocated.
>>> I am not really familiar with XArray much, could you be more specific
>>> please?
>> It comes from the comments of add_to_swap(), says:
>>
>> /*
>>           * XArray node allocations from PF_MEMALLOC contexts could
>>           * completely exhaust the page allocator. __GFP_NOMEMALLOC
>>           * stops emergency reserves from being allocated.
>>
>> And, it looks the original comment came from pre-git time, TBH I'm not quite
>> sure about the specific problem which incurred this. I suspect it may be
>> because PF_MEMALLOC context allows ALLOC_NO_WATERMARK.
>>
>>>> Here just copy the gfp flags used by add_to_swap().
>>>>
>>>> Cc: Hugh Dickins <hughd@google.com>
>>>> Signed-off-by: Yang Shi <yang.shi@linux.alibaba.com>
>>>> ---
>>>>    mm/shmem.c | 3 ++-
>>>>    1 file changed, 2 insertions(+), 1 deletion(-)
>>>>
>>>> diff --git a/mm/shmem.c b/mm/shmem.c
>>>> index 220be9f..9691dec 100644
>>>> --- a/mm/shmem.c
>>>> +++ b/mm/shmem.c
>>>> @@ -1369,7 +1369,8 @@ static int shmem_writepage(struct page *page,
>>>> struct writeback_control *wbc)
>>>>    	if (list_empty(&info->swaplist))
>>>>    		list_add(&info->swaplist, &shmem_swaplist);
>>>>    -	if (add_to_swap_cache(page, swap, GFP_ATOMIC) == 0) {
>>>> +	if (add_to_swap_cache(page, swap,
>>>> +			__GFP_HIGH | __GFP_NOMEMALLOC | __GFP_NOWARN) == 0) {
>>>>    		spin_lock_irq(&info->lock);
>>>>    		shmem_recalc_inode(inode);
>>>>    		info->swapped++;
>>>> -- 
>>>> 1.8.3.1



^ permalink raw reply	[flat|nested] 6+ messages in thread

* Re: [PATCH] mm: shmem: use proper gfp flags for shmem_writepage()
  2019-11-07 20:19       ` Yang Shi
@ 2019-11-07 22:59         ` Hugh Dickins
  0 siblings, 0 replies; 6+ messages in thread
From: Hugh Dickins @ 2019-11-07 22:59 UTC (permalink / raw)
  To: Yang Shi; +Cc: Hugh Dickins, Michal Hocko, akpm, linux-mm, linux-kernel

On Thu, 7 Nov 2019, Yang Shi wrote:
> On 11/6/19 10:59 AM, Hugh Dickins wrote:
> > Yes, I don't think it fixes any actual problem: just a cleanup to
> > make the two calls look the same when they don't need to be different
> > (whereas the call from __read_swap_cache_async() rightly uses a
> > lower priority gfp).
> 
> I'm supposed it is because __read_swap_cache_async()is typically called from
> page fault context which is less crucial than reclaim.

Exactly.

> 
> Shall I consider this as an ack but with commit log rephrased to reflect the
> cleanup?

Okay,
Acked-by: Hugh Dickins <hughd@google.com>


^ permalink raw reply	[flat|nested] 6+ messages in thread

end of thread, back to index

Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-11-05 22:02 [PATCH] mm: shmem: use proper gfp flags for shmem_writepage() Yang Shi
2019-11-06 15:18 ` Michal Hocko
2019-11-06 18:23   ` Yang Shi
2019-11-06 18:59     ` Hugh Dickins
2019-11-07 20:19       ` Yang Shi
2019-11-07 22:59         ` Hugh Dickins

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org
	public-inbox-index linux-mm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git