Should a swapped out page be deleted from swap cache?

Should a swapped out page be deleted from swap cache?

[Li Haifeng]

For explain my question, the two points should be displayed as below.

1.  If an anonymous page is swapped out, this page will be deleted
from swap cache and be put back into buddy system.

2. When a page is swapped out, the sharing count of swap slot must not
be zero. That is, page_swapcount(page) will not return zero.

Are both of them above right?

According the two points above, I was confused to the line 655 below.
When a page is swapped out, the return value of page_swapcount(page)
will not be zero. So, the page couldn't be deleted from swap cache.

 644  * If swap is getting full, or if there are no more mappings of this page,
 645  * then try_to_free_swap is called to free its swap space.
 646  */
 647 int try_to_free_swap(struct page *page)
 648 {
 649         VM_BUG_ON(!PageLocked(page));
 650
 651         if (!PageSwapCache(page))
 652                 return 0;
 653         if (PageWriteback(page))
 654                 return 0;
 655         if (page_swapcount(page))//Has referenced by other swap out page.
 656                 return 0;
 657
 658         /*
 659          * Once hibernation has begun to create its image of memory,
 660          * there's a danger that one of the calls to try_to_free_swap()
 661          * - most probably a call from __try_to_reclaim_swap() while
 662          * hibernation is allocating its own swap pages for the image,
 663          * but conceivably even a call from memory reclaim - will free
 664          * the swap from a page which has already been recorded in the
 665          * image as a clean swapcache page, and then reuse its swap for
 666          * another page of the image.  On waking from hibernation, the
 667          * original page might be freed under memory pressure, then
 668          * later read back in from swap, now with the wrong data.
 669          *
 670          * Hibration suspends storage while it is writing the image
 671          * to disk so check that here.
 672          */
 673         if (pm_suspended_storage())
 674                 return 0;
 675
 676         delete_from_swap_cache(page);
 677         SetPageDirty(page);
 678         return 1;
 679 }

Thanks.

Should a swapped out page be deleted from swap cache?

[Hugh Dickins]

On Mon, 18 Feb 2013, Li Haifeng wrote:

> For explain my question, the two points should be displayed as below.
> 
> 1.  If an anonymous page is swapped out, this page will be deleted
> from swap cache and be put back into buddy system.

Yes, unless the page is referenced again before it comes to be
deleted from swap cache.

> 
> 2. When a page is swapped out, the sharing count of swap slot must not
> be zero. That is, page_swapcount(page) will not return zero.

I would not say "must not": we just prefer not to waste time on swapping
a page out if its use count has already gone to 0.  And its use count
might go down to 0 an instant after swap_writepage() makes that check.

> 
> Are both of them above right?
> 
> According the two points above, I was confused to the line 655 below.
> When a page is swapped out, the return value of page_swapcount(page)
> will not be zero. So, the page couldn't be deleted from swap cache.

Yes, we cannot free the swap as long as its data might be needed again.

But a swap cache page may linger in memory for an indefinite time,
in between being queued for write out, and actually being freed from
the end of the lru by memory pressure.

At various points where we hold the page lock on a swap cache page,
it's worth checking whether it is still actually needed, or could
now be freed from swap cache, and the corresponding swap slot freed:
that's what try_to_free_swap() does.

Hugh

> 
>  644  * If swap is getting full, or if there are no more mappings of this page,
>  645  * then try_to_free_swap is called to free its swap space.
>  646  */
>  647 int try_to_free_swap(struct page *page)
>  648 {
>  649         VM_BUG_ON(!PageLocked(page));
>  650
>  651         if (!PageSwapCache(page))
>  652                 return 0;
>  653         if (PageWriteback(page))
>  654                 return 0;
>  655         if (page_swapcount(page))//Has referenced by other swap out page.
>  656                 return 0;
>  657
>  658         /*
>  659          * Once hibernation has begun to create its image of memory,
>  660          * there's a danger that one of the calls to try_to_free_swap()
>  661          * - most probably a call from __try_to_reclaim_swap() while
>  662          * hibernation is allocating its own swap pages for the image,
>  663          * but conceivably even a call from memory reclaim - will free
>  664          * the swap from a page which has already been recorded in the
>  665          * image as a clean swapcache page, and then reuse its swap for
>  666          * another page of the image.  On waking from hibernation, the
>  667          * original page might be freed under memory pressure, then
>  668          * later read back in from swap, now with the wrong data.
>  669          *
>  670          * Hibration suspends storage while it is writing the image
>  671          * to disk so check that here.
>  672          */
>  673         if (pm_suspended_storage())
>  674                 return 0;
>  675
>  676         delete_from_swap_cache(page);
>  677         SetPageDirty(page);
>  678         return 1;
>  679 }
> 
> Thanks.

Should a swapped out page be deleted from swap cache?

[Will Huck]

Hi Hugh,
On 02/19/2013 02:06 AM, Hugh Dickins wrote:

Another question:

Why kernel memory mapping use direct mapping instead of kmalloc/vmalloc 
which will setup mapping on demand?

> On Mon, 18 Feb 2013, Li Haifeng wrote:
>
>> For explain my question, the two points should be displayed as below.
>>
>> 1.  If an anonymous page is swapped out, this page will be deleted
>> from swap cache and be put back into buddy system.
> Yes, unless the page is referenced again before it comes to be
> deleted from swap cache.
>
>> 2. When a page is swapped out, the sharing count of swap slot must not
>> be zero. That is, page_swapcount(page) will not return zero.
> I would not say "must not": we just prefer not to waste time on swapping
> a page out if its use count has already gone to 0.  And its use count
> might go down to 0 an instant after swap_writepage() makes that check.
>
>> Are both of them above right?
>>
>> According the two points above, I was confused to the line 655 below.
>> When a page is swapped out, the return value of page_swapcount(page)
>> will not be zero. So, the page couldn't be deleted from swap cache.
> Yes, we cannot free the swap as long as its data might be needed again.
>
> But a swap cache page may linger in memory for an indefinite time,
> in between being queued for write out, and actually being freed from
> the end of the lru by memory pressure.
>
> At various points where we hold the page lock on a swap cache page,
> it's worth checking whether it is still actually needed, or could
> now be freed from swap cache, and the corresponding swap slot freed:
> that's what try_to_free_swap() does.
>
> Hugh
>
>>   644  * If swap is getting full, or if there are no more mappings of this page,
>>   645  * then try_to_free_swap is called to free its swap space.
>>   646  */
>>   647 int try_to_free_swap(struct page *page)
>>   648 {
>>   649         VM_BUG_ON(!PageLocked(page));
>>   650
>>   651         if (!PageSwapCache(page))
>>   652                 return 0;
>>   653         if (PageWriteback(page))
>>   654                 return 0;
>>   655         if (page_swapcount(page))//Has referenced by other swap out page.
>>   656                 return 0;
>>   657
>>   658         /*
>>   659          * Once hibernation has begun to create its image of memory,
>>   660          * there's a danger that one of the calls to try_to_free_swap()
>>   661          * - most probably a call from __try_to_reclaim_swap() while
>>   662          * hibernation is allocating its own swap pages for the image,
>>   663          * but conceivably even a call from memory reclaim - will free
>>   664          * the swap from a page which has already been recorded in the
>>   665          * image as a clean swapcache page, and then reuse its swap for
>>   666          * another page of the image.  On waking from hibernation, the
>>   667          * original page might be freed under memory pressure, then
>>   668          * later read back in from swap, now with the wrong data.
>>   669          *
>>   670          * Hibration suspends storage while it is writing the image
>>   671          * to disk so check that here.
>>   672          */
>>   673         if (pm_suspended_storage())
>>   674                 return 0;
>>   675
>>   676         delete_from_swap_cache(page);
>>   677         SetPageDirty(page);
>>   678         return 1;
>>   679 }
>>
>> Thanks.
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo at kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email at kvack.org </a>

Should a swapped out page be deleted from swap cache?

[Li Haifeng]

2013/2/19 Hugh Dickins <hughd@google.com>
>
> On Mon, 18 Feb 2013, Li Haifeng wrote:
>
> > For explain my question, the two points should be displayed as below.
> >
> > 1.  If an anonymous page is swapped out, this page will be deleted
> > from swap cache and be put back into buddy system.
>
> Yes, unless the page is referenced again before it comes to be
> deleted from swap cache.
>
> >
> > 2. When a page is swapped out, the sharing count of swap slot must not
> > be zero. That is, page_swapcount(page) will not return zero.
>
> I would not say "must not": we just prefer not to waste time on swapping
> a page out if its use count has already gone to 0.  And its use count
> might go down to 0 an instant after swap_writepage() makes that check.
>

Thanks for your reply and patience.

If a anonymous page is swapped out and  comes to be reclaimable,
shrink_page_list() will call __remove_mapping() to delete the page
swapped out from swap cache. Corresponding code lists as below.

 765 static unsigned long shrink_page_list(struct list_head *page_list,
 766                                       struct mem_cgroup_zone *mz,
 767                                       struct scan_control *sc,
 768                                       int priority,
 769                                       unsigned long *ret_nr_dirty,
 770                                       unsigned long *ret_nr_writeback)
 771 {
...
 971                 if (!mapping || !__remove_mapping(mapping, page))
 972                         goto keep_locked;
 973
 974                 /*
 975                  * At this point, we have no other references and there is
 976                  * no way to pick any more up (removed from LRU, removed
 977                  * from pagecache). Can use non-atomic bitops now (and
 978                  * we obviously don't have to worry about waking
up a process
 979                  * waiting on the page lock, because there are no
references.
 980                  */
 981                 __clear_page_locked(page);
 982 free_it:
 983                 nr_reclaimed++;
 984
 985                 /*
 986                  * Is there need to periodically free_page_list? It would
 987                  * appear not as the counts should be low
 988                  */
 989                 list_add(&page->lru, &free_pages);
 990                 continue;

Please correct me if my understanding is wrong.

Thanks.
> >
> > Are both of them above right?
> >
> > According the two points above, I was confused to the line 655 below.
> > When a page is swapped out, the return value of page_swapcount(page)
> > will not be zero. So, the page couldn't be deleted from swap cache.
>
> Yes, we cannot free the swap as long as its data might be needed again.
>
> But a swap cache page may linger in memory for an indefinite time,
> in between being queued for write out, and actually being freed from
> the end of the lru by memory pressure.
>
> At various points where we hold the page lock on a swap cache page,
> it's worth checking whether it is still actually needed, or could
> now be freed from swap cache, and the corresponding swap slot freed:
> that's what try_to_free_swap() does.

I do agree. Thanks again.
>
> Hugh
>
> >
> >  644  * If swap is getting full, or if there are no more mappings of
> > this page,
> >  645  * then try_to_free_swap is called to free its swap space.
> >  646  */
> >  647 int try_to_free_swap(struct page *page)
> >  648 {
> >  649         VM_BUG_ON(!PageLocked(page));
> >  650
> >  651         if (!PageSwapCache(page))
> >  652                 return 0;
> >  653         if (PageWriteback(page))
> >  654                 return 0;
> >  655         if (page_swapcount(page))//Has referenced by other swap out
> > page.
> >  656                 return 0;
> >  657
> >  658         /*
> >  659          * Once hibernation has begun to create its image of
> > memory,
> >  660          * there's a danger that one of the calls to
> > try_to_free_swap()
> >  661          * - most probably a call from __try_to_reclaim_swap()
> > while
> >  662          * hibernation is allocating its own swap pages for the
> > image,
> >  663          * but conceivably even a call from memory reclaim - will
> > free
> >  664          * the swap from a page which has already been recorded in
> > the
> >  665          * image as a clean swapcache page, and then reuse its swap
> > for
> >  666          * another page of the image.  On waking from hibernation,
> > the
> >  667          * original page might be freed under memory pressure, then
> >  668          * later read back in from swap, now with the wrong data.
> >  669          *
> >  670          * Hibration suspends storage while it is writing the image
> >  671          * to disk so check that here.
> >  672          */
> >  673         if (pm_suspended_storage())
> >  674                 return 0;
> >  675
> >  676         delete_from_swap_cache(page);
> >  677         SetPageDirty(page);
> >  678         return 1;
> >  679 }
> >
> > Thanks.

Should a swapped out page be deleted from swap cache?

[Will Huck]

On 02/19/2013 10:04 AM, Li Haifeng wrote:
> 2013/2/19 Hugh Dickins <hughd@google.com>
>> On Mon, 18 Feb 2013, Li Haifeng wrote:
>>
>>> For explain my question, the two points should be displayed as below.
>>>
>>> 1.  If an anonymous page is swapped out, this page will be deleted
>>> from swap cache and be put back into buddy system.
>> Yes, unless the page is referenced again before it comes to be
>> deleted from swap cache.
>>
>>> 2. When a page is swapped out, the sharing count of swap slot must not
>>> be zero. That is, page_swapcount(page) will not return zero.
>> I would not say "must not": we just prefer not to waste time on swapping
>> a page out if its use count has already gone to 0.  And its use count
>> might go down to 0 an instant after swap_writepage() makes that check.
>>
> Thanks for your reply and patience.
>
> If a anonymous page is swapped out and  comes to be reclaimable,
> shrink_page_list() will call __remove_mapping() to delete the page
> swapped out from swap cache. Corresponding code lists as below.

I'm not sure if
if (PageAnon(page) && !PageSwapCache(page)) {
  .................
}
will add the page to swap cache again.

>
>   765 static unsigned long shrink_page_list(struct list_head *page_list,
>   766                                       struct mem_cgroup_zone *mz,
>   767                                       struct scan_control *sc,
>   768                                       int priority,
>   769                                       unsigned long *ret_nr_dirty,
>   770                                       unsigned long *ret_nr_writeback)
>   771 {
> ...
>   971                 if (!mapping || !__remove_mapping(mapping, page))
>   972                         goto keep_locked;
>   973
>   974                 /*
>   975                  * At this point, we have no other references and there is
>   976                  * no way to pick any more up (removed from LRU, removed
>   977                  * from pagecache). Can use non-atomic bitops now (and
>   978                  * we obviously don't have to worry about waking
> up a process
>   979                  * waiting on the page lock, because there are no
> references.
>   980                  */
>   981                 __clear_page_locked(page);
>   982 free_it:
>   983                 nr_reclaimed++;
>   984
>   985                 /*
>   986                  * Is there need to periodically free_page_list? It would
>   987                  * appear not as the counts should be low
>   988                  */
>   989                 list_add(&page->lru, &free_pages);
>   990                 continue;
>
> Please correct me if my understanding is wrong.
>
> Thanks.
>>> Are both of them above right?
>>>
>>> According the two points above, I was confused to the line 655 below.
>>> When a page is swapped out, the return value of page_swapcount(page)
>>> will not be zero. So, the page couldn't be deleted from swap cache.
>> Yes, we cannot free the swap as long as its data might be needed again.
>>
>> But a swap cache page may linger in memory for an indefinite time,
>> in between being queued for write out, and actually being freed from
>> the end of the lru by memory pressure.
>>
>> At various points where we hold the page lock on a swap cache page,
>> it's worth checking whether it is still actually needed, or could
>> now be freed from swap cache, and the corresponding swap slot freed:
>> that's what try_to_free_swap() does.
> I do agree. Thanks again.
>> Hugh
>>
>>>   644  * If swap is getting full, or if there are no more mappings of
>>> this page,
>>>   645  * then try_to_free_swap is called to free its swap space.
>>>   646  */
>>>   647 int try_to_free_swap(struct page *page)
>>>   648 {
>>>   649         VM_BUG_ON(!PageLocked(page));
>>>   650
>>>   651         if (!PageSwapCache(page))
>>>   652                 return 0;
>>>   653         if (PageWriteback(page))
>>>   654                 return 0;
>>>   655         if (page_swapcount(page))//Has referenced by other swap out
>>> page.
>>>   656                 return 0;
>>>   657
>>>   658         /*
>>>   659          * Once hibernation has begun to create its image of
>>> memory,
>>>   660          * there's a danger that one of the calls to
>>> try_to_free_swap()
>>>   661          * - most probably a call from __try_to_reclaim_swap()
>>> while
>>>   662          * hibernation is allocating its own swap pages for the
>>> image,
>>>   663          * but conceivably even a call from memory reclaim - will
>>> free
>>>   664          * the swap from a page which has already been recorded in
>>> the
>>>   665          * image as a clean swapcache page, and then reuse its swap
>>> for
>>>   666          * another page of the image.  On waking from hibernation,
>>> the
>>>   667          * original page might be freed under memory pressure, then
>>>   668          * later read back in from swap, now with the wrong data.
>>>   669          *
>>>   670          * Hibration suspends storage while it is writing the image
>>>   671          * to disk so check that here.
>>>   672          */
>>>   673         if (pm_suspended_storage())
>>>   674                 return 0;
>>>   675
>>>   676         delete_from_swap_cache(page);
>>>   677         SetPageDirty(page);
>>>   678         return 1;
>>>   679 }
>>>
>>> Thanks.
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo at kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email at kvack.org </a>

Should a swapped out page be deleted from swap cache?

[Li Haifeng]

2013/2/19 Will Huck <will.huckk@gmail.com>:
> On 02/19/2013 10:04 AM, Li Haifeng wrote:
>>
>> 2013/2/19 Hugh Dickins <hughd@google.com>
>>>
>>> On Mon, 18 Feb 2013, Li Haifeng wrote:
>>>
>>>> For explain my question, the two points should be displayed as below.
>>>>
>>>> 1.  If an anonymous page is swapped out, this page will be deleted
>>>> from swap cache and be put back into buddy system.
>>>
>>> Yes, unless the page is referenced again before it comes to be
>>> deleted from swap cache.
>>>
>>>> 2. When a page is swapped out, the sharing count of swap slot must not
>>>> be zero. That is, page_swapcount(page) will not return zero.
>>>
>>> I would not say "must not": we just prefer not to waste time on swapping
>>> a page out if its use count has already gone to 0.  And its use count
>>> might go down to 0 an instant after swap_writepage() makes that check.
>>>
>> Thanks for your reply and patience.
>>
>> If a anonymous page is swapped out and  comes to be reclaimable,
>> shrink_page_list() will call __remove_mapping() to delete the page
>> swapped out from swap cache. Corresponding code lists as below.
>
>
> I'm not sure if
> if (PageAnon(page) && !PageSwapCache(page)) {
>  .................
> }
> will add the page to swap cache again.
>

Adding the page to swap cache is the first stage of memory reclaiming.

When an anonymous page will be reclaimed, it should be swapped out. If
it's not in the swap cache, it will insert into swap cache first and
set the bit of PG_swapcache on page->flags. Then, it will be swapped
out by try_to_unmap(). After it's swapped out, and no processes swap
in this page, the page will be deleted from swap cache and be put into
buddy system.

Please correct me if my understanding is wrong.

Regards,
Haifeng Li.

>>
>>   765 static unsigned long shrink_page_list(struct list_head *page_list,
>>   766                                       struct mem_cgroup_zone *mz,
>>   767                                       struct scan_control *sc,
>>   768                                       int priority,
>>   769                                       unsigned long *ret_nr_dirty,
>>   770                                       unsigned long
>> *ret_nr_writeback)
>>   771 {
>> ...
>>   971                 if (!mapping || !__remove_mapping(mapping, page))
>>   972                         goto keep_locked;
>>   973
>>   974                 /*
>>   975                  * At this point, we have no other references and
>> there is
>>   976                  * no way to pick any more up (removed from LRU,
>> removed
>>   977                  * from pagecache). Can use non-atomic bitops now
>> (and
>>   978                  * we obviously don't have to worry about waking
>> up a process
>>   979                  * waiting on the page lock, because there are no
>> references.
>>   980                  */
>>   981                 __clear_page_locked(page);
>>   982 free_it:
>>   983                 nr_reclaimed++;
>>   984
>>   985                 /*
>>   986                  * Is there need to periodically free_page_list? It
>> would
>>   987                  * appear not as the counts should be low
>>   988                  */
>>   989                 list_add(&page->lru, &free_pages);
>>   990                 continue;
>>
>> Please correct me if my understanding is wrong.
>>
>> Thanks.
>>>>
>>>> Are both of them above right?
>>>>
>>>> According the two points above, I was confused to the line 655 below.
>>>> When a page is swapped out, the return value of page_swapcount(page)
>>>> will not be zero. So, the page couldn't be deleted from swap cache.
>>>
>>> Yes, we cannot free the swap as long as its data might be needed again.
>>>
>>> But a swap cache page may linger in memory for an indefinite time,
>>> in between being queued for write out, and actually being freed from
>>> the end of the lru by memory pressure.
>>>
>>> At various points where we hold the page lock on a swap cache page,
>>> it's worth checking whether it is still actually needed, or could
>>> now be freed from swap cache, and the corresponding swap slot freed:
>>> that's what try_to_free_swap() does.
>>
>> I do agree. Thanks again.
>>>
>>> Hugh
>>>
>>>>   644  * If swap is getting full, or if there are no more mappings of
>>>> this page,
>>>>   645  * then try_to_free_swap is called to free its swap space.
>>>>   646  */
>>>>   647 int try_to_free_swap(struct page *page)
>>>>   648 {
>>>>   649         VM_BUG_ON(!PageLocked(page));
>>>>   650
>>>>   651         if (!PageSwapCache(page))
>>>>   652                 return 0;
>>>>   653         if (PageWriteback(page))
>>>>   654                 return 0;
>>>>   655         if (page_swapcount(page))//Has referenced by other swap
>>>> out
>>>> page.
>>>>   656                 return 0;
>>>>   657
>>>>   658         /*
>>>>   659          * Once hibernation has begun to create its image of
>>>> memory,
>>>>   660          * there's a danger that one of the calls to
>>>> try_to_free_swap()
>>>>   661          * - most probably a call from __try_to_reclaim_swap()
>>>> while
>>>>   662          * hibernation is allocating its own swap pages for the
>>>> image,
>>>>   663          * but conceivably even a call from memory reclaim - will
>>>> free
>>>>   664          * the swap from a page which has already been recorded in
>>>> the
>>>>   665          * image as a clean swapcache page, and then reuse its
>>>> swap
>>>> for
>>>>   666          * another page of the image.  On waking from hibernation,
>>>> the
>>>>   667          * original page might be freed under memory pressure,
>>>> then
>>>>   668          * later read back in from swap, now with the wrong data.
>>>>   669          *
>>>>   670          * Hibration suspends storage while it is writing the
>>>> image
>>>>   671          * to disk so check that here.
>>>>   672          */
>>>>   673         if (pm_suspended_storage())
>>>>   674                 return 0;
>>>>   675
>>>>   676         delete_from_swap_cache(page);
>>>>   677         SetPageDirty(page);
>>>>   678         return 1;
>>>>   679 }
>>>>
>>>> Thanks.
>>
>> --
>>
>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>> the body to majordomo at kvack.org.  For more info on Linux MM,
>> see: http://www.linux-mm.org/ .
>> Don't email: <a href=mailto:"dont@kvack.org"> email at kvack.org </a>
>
>

Should a swapped out page be deleted from swap cache?

[Ric Mason]

On 02/19/2013 02:06 AM, Hugh Dickins wrote:
> On Mon, 18 Feb 2013, Li Haifeng wrote:
>
>> For explain my question, the two points should be displayed as below.
>>
>> 1.  If an anonymous page is swapped out, this page will be deleted
>> from swap cache and be put back into buddy system.
> Yes, unless the page is referenced again before it comes to be
> deleted from swap cache.
>
>> 2. When a page is swapped out, the sharing count of swap slot must not
>> be zero. That is, page_swapcount(page) will not return zero.
> I would not say "must not": we just prefer not to waste time on swapping
> a page out if its use count has already gone to 0.  And its use count
> might go down to 0 an instant after swap_writepage() makes that check.
>
>> Are both of them above right?
>>
>> According the two points above, I was confused to the line 655 below.
>> When a page is swapped out, the return value of page_swapcount(page)
>> will not be zero. So, the page couldn't be deleted from swap cache.
> Yes, we cannot free the swap as long as its data might be needed again.
>
> But a swap cache page may linger in memory for an indefinite time,
> in between being queued for write out, and actually being freed from
> the end of the lru by memory pressure.
>
> At various points where we hold the page lock on a swap cache page,
> it's worth checking whether it is still actually needed, or could
> now be freed from swap cache, and the corresponding swap slot freed:
> that's what try_to_free_swap() does.

Hi Hugh,

There is a call of try_to_free_swap in function swap_writepage, if 
swap_writepage is call from shrink_page_list path, PageSwapCache(page) 
== trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, 
then will delete the page from swap cache and free swap slot, where I miss?

>
> Hugh
>
>>   644  * If swap is getting full, or if there are no more mappings of this page,
>>   645  * then try_to_free_swap is called to free its swap space.
>>   646  */
>>   647 int try_to_free_swap(struct page *page)
>>   648 {
>>   649         VM_BUG_ON(!PageLocked(page));
>>   650
>>   651         if (!PageSwapCache(page))
>>   652                 return 0;
>>   653         if (PageWriteback(page))
>>   654                 return 0;
>>   655         if (page_swapcount(page))//Has referenced by other swap out page.
>>   656                 return 0;
>>   657
>>   658         /*
>>   659          * Once hibernation has begun to create its image of memory,
>>   660          * there's a danger that one of the calls to try_to_free_swap()
>>   661          * - most probably a call from __try_to_reclaim_swap() while
>>   662          * hibernation is allocating its own swap pages for the image,
>>   663          * but conceivably even a call from memory reclaim - will free
>>   664          * the swap from a page which has already been recorded in the
>>   665          * image as a clean swapcache page, and then reuse its swap for
>>   666          * another page of the image.  On waking from hibernation, the
>>   667          * original page might be freed under memory pressure, then
>>   668          * later read back in from swap, now with the wrong data.
>>   669          *
>>   670          * Hibration suspends storage while it is writing the image
>>   671          * to disk so check that here.
>>   672          */
>>   673         if (pm_suspended_storage())
>>   674                 return 0;
>>   675
>>   676         delete_from_swap_cache(page);
>>   677         SetPageDirty(page);
>>   678         return 1;
>>   679 }
>>
>> Thanks.
> --
> To unsubscribe, send a message with 'unsubscribe linux-mm' in
> the body to majordomo at kvack.org.  For more info on Linux MM,
> see: http://www.linux-mm.org/ .
> Don't email: <a href=mailto:"dont@kvack.org"> email at kvack.org </a>

Should a swapped out page be deleted from swap cache?

[Sha Zhengju]

On Tue, Feb 19, 2013 at 2:53 PM, Li Haifeng <omycle@gmail.com> wrote:
> 2013/2/19 Will Huck <will.huckk@gmail.com>:
>> On 02/19/2013 10:04 AM, Li Haifeng wrote:
>>>
>>> 2013/2/19 Hugh Dickins <hughd@google.com>
>>>>
>>>> On Mon, 18 Feb 2013, Li Haifeng wrote:
>>>>
>>>>> For explain my question, the two points should be displayed as below.
>>>>>
>>>>> 1.  If an anonymous page is swapped out, this page will be deleted
>>>>> from swap cache and be put back into buddy system.
>>>>
>>>> Yes, unless the page is referenced again before it comes to be
>>>> deleted from swap cache.
>>>>
>>>>> 2. When a page is swapped out, the sharing count of swap slot must not
>>>>> be zero. That is, page_swapcount(page) will not return zero.
>>>>
>>>> I would not say "must not": we just prefer not to waste time on swapping
>>>> a page out if its use count has already gone to 0.  And its use count
>>>> might go down to 0 an instant after swap_writepage() makes that check.
>>>>
>>> Thanks for your reply and patience.
>>>
>>> If a anonymous page is swapped out and  comes to be reclaimable,
>>> shrink_page_list() will call __remove_mapping() to delete the page
>>> swapped out from swap cache. Corresponding code lists as below.
>>
>>
>> I'm not sure if
>> if (PageAnon(page) && !PageSwapCache(page)) {
>>  .................
>> }
>> will add the page to swap cache again.
>>
>
> Adding the page to swap cache is the first stage of memory reclaiming.
>
> When an anonymous page will be reclaimed, it should be swapped out. If
> it's not in the swap cache, it will insert into swap cache first and
> set the bit of PG_swapcache on page->flags. Then, it will be swapped
> out by try_to_unmap(). After it's swapped out, and no processes swap

Swapout(writing to swap disk) is not done by try_to_unmap() which only
tries to remove all page table mappings to a page. Before unmapping,
add_to_swap() will set the swap cache page dirty and it will be
written out by pageout()->swap_writepage().


Thanks,
Sha


> in this page, the page will be deleted from swap cache and be put into
> buddy system.
>
> Please correct me if my understanding is wrong.
>
> Regards,
> Haifeng Li.
>
>>>
>>>   765 static unsigned long shrink_page_list(struct list_head *page_list,
>>>   766                                       struct mem_cgroup_zone *mz,
>>>   767                                       struct scan_control *sc,
>>>   768                                       int priority,
>>>   769                                       unsigned long *ret_nr_dirty,
>>>   770                                       unsigned long
>>> *ret_nr_writeback)
>>>   771 {
>>> ...
>>>   971                 if (!mapping || !__remove_mapping(mapping, page))
>>>   972                         goto keep_locked;
>>>   973
>>>   974                 /*
>>>   975                  * At this point, we have no other references and
>>> there is
>>>   976                  * no way to pick any more up (removed from LRU,
>>> removed
>>>   977                  * from pagecache). Can use non-atomic bitops now
>>> (and
>>>   978                  * we obviously don't have to worry about waking
>>> up a process
>>>   979                  * waiting on the page lock, because there are no
>>> references.
>>>   980                  */
>>>   981                 __clear_page_locked(page);
>>>   982 free_it:
>>>   983                 nr_reclaimed++;
>>>   984
>>>   985                 /*
>>>   986                  * Is there need to periodically free_page_list? It
>>> would
>>>   987                  * appear not as the counts should be low
>>>   988                  */
>>>   989                 list_add(&page->lru, &free_pages);
>>>   990                 continue;
>>>
>>> Please correct me if my understanding is wrong.
>>>
>>> Thanks.
>>>>>
>>>>> Are both of them above right?
>>>>>
>>>>> According the two points above, I was confused to the line 655 below.
>>>>> When a page is swapped out, the return value of page_swapcount(page)
>>>>> will not be zero. So, the page couldn't be deleted from swap cache.
>>>>
>>>> Yes, we cannot free the swap as long as its data might be needed again.
>>>>
>>>> But a swap cache page may linger in memory for an indefinite time,
>>>> in between being queued for write out, and actually being freed from
>>>> the end of the lru by memory pressure.
>>>>
>>>> At various points where we hold the page lock on a swap cache page,
>>>> it's worth checking whether it is still actually needed, or could
>>>> now be freed from swap cache, and the corresponding swap slot freed:
>>>> that's what try_to_free_swap() does.
>>>
>>> I do agree. Thanks again.
>>>>
>>>> Hugh
>>>>
>>>>>   644  * If swap is getting full, or if there are no more mappings of
>>>>> this page,
>>>>>   645  * then try_to_free_swap is called to free its swap space.
>>>>>   646  */
>>>>>   647 int try_to_free_swap(struct page *page)
>>>>>   648 {
>>>>>   649         VM_BUG_ON(!PageLocked(page));
>>>>>   650
>>>>>   651         if (!PageSwapCache(page))
>>>>>   652                 return 0;
>>>>>   653         if (PageWriteback(page))
>>>>>   654                 return 0;
>>>>>   655         if (page_swapcount(page))//Has referenced by other swap
>>>>> out
>>>>> page.
>>>>>   656                 return 0;
>>>>>   657
>>>>>   658         /*
>>>>>   659          * Once hibernation has begun to create its image of
>>>>> memory,
>>>>>   660          * there's a danger that one of the calls to
>>>>> try_to_free_swap()
>>>>>   661          * - most probably a call from __try_to_reclaim_swap()
>>>>> while
>>>>>   662          * hibernation is allocating its own swap pages for the
>>>>> image,
>>>>>   663          * but conceivably even a call from memory reclaim - will
>>>>> free
>>>>>   664          * the swap from a page which has already been recorded in
>>>>> the
>>>>>   665          * image as a clean swapcache page, and then reuse its
>>>>> swap
>>>>> for
>>>>>   666          * another page of the image.  On waking from hibernation,
>>>>> the
>>>>>   667          * original page might be freed under memory pressure,
>>>>> then
>>>>>   668          * later read back in from swap, now with the wrong data.
>>>>>   669          *
>>>>>   670          * Hibration suspends storage while it is writing the
>>>>> image
>>>>>   671          * to disk so check that here.
>>>>>   672          */
>>>>>   673         if (pm_suspended_storage())
>>>>>   674                 return 0;
>>>>>   675
>>>>>   676         delete_from_swap_cache(page);
>>>>>   677         SetPageDirty(page);
>>>>>   678         return 1;
>>>>>   679 }
>>>>>
>>>>> Thanks.
>>>
>>> --
>>>
>>> To unsubscribe, send a message with 'unsubscribe linux-mm' in
>>> the body to majordomo at kvack.org.  For more info on Linux MM,
>>> see: http://www.linux-mm.org/ .
>>> Don't email: <a href=mailto:"dont@kvack.org"> email at kvack.org </a>
>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo at vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at  http://www.tux.org/lkml/

Should a swapped out page be deleted from swap cache?

[Hugh Dickins]

On Tue, 19 Feb 2013, Sha Zhengju wrote:
> On Tue, Feb 19, 2013 at 2:53 PM, Li Haifeng <omycle@gmail.com> wrote:
> > 2013/2/19 Will Huck <will.huckk@gmail.com>:
> >> On 02/19/2013 10:04 AM, Li Haifeng wrote:
> >>>
> >>> If a anonymous page is swapped out and  comes to be reclaimable,
> >>> shrink_page_list() will call __remove_mapping() to delete the page
> >>> swapped out from swap cache. Corresponding code lists as below.

Correct.

> >>
> >>
> >> I'm not sure if
> >> if (PageAnon(page) && !PageSwapCache(page)) {
> >>  .................
> >> }
> >> will add the page to swap cache again.

No, it's already in the swap cache.  Of course, the original pageframe
may be removed from swap cache, freed, data later read back from swap into
a new swap cache pageframe, that be mapped into user memory, removed from
swap cache and swap freed, then later arrive here in page reclaim at the
PageAnon(page) && !PageSwapCache(page) to be added to swap again.

> >>
> >
> > Adding the page to swap cache is the first stage of memory reclaiming.
> >
> > When an anonymous page will be reclaimed, it should be swapped out. If
> > it's not in the swap cache, it will insert into swap cache first and
> > set the bit of PG_swapcache on page->flags. Then, it will be swapped
> > out by try_to_unmap(). After it's swapped out, and no processes swap

Almost correct...

> 
> Swapout(writing to swap disk) is not done by try_to_unmap() which only
> tries to remove all page table mappings to a page. Before unmapping,
> add_to_swap() will set the swap cache page dirty and it will be
> written out by pageout()->swap_writepage().

... but yes, try_to_unmap() is not the one that writes out to swap.

Hugh

Should a swapped out page be deleted from swap cache?

[Hugh Dickins]

On Tue, 19 Feb 2013, Ric Mason wrote:
> 
> There is a call of try_to_free_swap in function swap_writepage, if
> swap_writepage is call from shrink_page_list path, PageSwapCache(page) ==
> trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, then will
> delete the page from swap cache and free swap slot, where I miss?

That's correct.  PageWriteback is sure to be false there.  page_swapcount
usually won't be 0 there, but sometimes it will be, and in that case we
do want to delete from swap cache and free the swap slot.

Hugh

Should a swapped out page be deleted from swap cache?

[Hugh Dickins]

On Tue, 19 Feb 2013, Will Huck wrote:
> 
> Another question:

I don't see the connection to deleting a swapped out page from swap cache.

> 
> Why kernel memory mapping use direct mapping instead of kmalloc/vmalloc which
> will setup mapping on demand?

I may misunderstand you, and "kernel memory mapping".

kmalloc does not set up a mapping, it uses the direct mapping already set up.

It would be circular if the basic page allocation primitives used kmalloc,
since kmalloc relies on the basic page allocation primitives.

vmalloc is less efficient than using the direct mapping (repeated setup
and teardown, no use of hugepages), but necessary when you want a larger
virtual array than you're likely to find from the buddy allocator.

Hugh

Should a swapped out page be deleted from swap cache?

[Ric Mason]

Hi Hugh,

On 02/20/2013 02:56 AM, Hugh Dickins wrote:
> On Tue, 19 Feb 2013, Ric Mason wrote:
>> There is a call of try_to_free_swap in function swap_writepage, if
>> swap_writepage is call from shrink_page_list path, PageSwapCache(page) ==
>> trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, then will
>> delete the page from swap cache and free swap slot, where I miss?
> That's correct.  PageWriteback is sure to be false there.  page_swapcount
> usually won't be 0 there, but sometimes it will be, and in that case we
> do want to delete from swap cache and free the swap slot.

1) If PageSwapCache(page)  == true, PageWriteback(page) == false, 
page_swapcount(page) == 0  in swap_writepage(shrink_page_list path), 
then will delete the page from swap cache and free swap slot, in 
function swap_writepage:

if (try_to_free_swap(page)) {
     unlock_page(page);
     goto out;
}
writeback will not execute, that's wrong. Where I miss?

2) In the function pageout, page will be set PG_Reclaim flag, since this 
flag is set, end_swap_bio_write->end_page_writeback:
if (TestClearPageReclaim(page))
      rotate_reclaimable_page(page);
it means that page will be add to the tail of lru list, page is clean 
anonymous page this time and will be reclaim to buddy system soon, correct?
If is correct, what is the meaning of rotate here?

>
> Hugh

Should a swapped out page be deleted from swap cache?

[Will Huck]

On 02/20/2013 03:06 AM, Hugh Dickins wrote:
> On Tue, 19 Feb 2013, Will Huck wrote:
>> Another question:
> I don't see the connection to deleting a swapped out page from swap cache.
>
>> Why kernel memory mapping use direct mapping instead of kmalloc/vmalloc which
>> will setup mapping on demand?
> I may misunderstand you, and "kernel memory mapping".
>
> kmalloc does not set up a mapping, it uses the direct mapping already set up.
>
> It would be circular if the basic page allocation primitives used kmalloc,
> since kmalloc relies on the basic page allocation primitives.
>
> vmalloc is less efficient than using the direct mapping (repeated setup
> and teardown, no use of hugepages), but necessary when you want a larger

Is there tlb flush in setup and teardown process? and they also expensive?

> virtual array than you're likely to find from the buddy allocator.
>
> Hugh

Should a swapped out page be deleted from swap cache?

[Li Haifeng]

2013/2/20 Ric Mason <ric.masonn@gmail.com>:
>
> Hi Hugh,
>
>
> On 02/20/2013 02:56 AM, Hugh Dickins wrote:
>>
>> On Tue, 19 Feb 2013, Ric Mason wrote:
>>>
>>> There is a call of try_to_free_swap in function swap_writepage, if
>>> swap_writepage is call from shrink_page_list path, PageSwapCache(page) ==
>>> trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, then
>>> will
>>> delete the page from swap cache and free swap slot, where I miss?
>>
>> That's correct.  PageWriteback is sure to be false there.  page_swapcount
>> usually won't be 0 there, but sometimes it will be, and in that case we
>> do want to delete from swap cache and free the swap slot.
>
>
> 1) If PageSwapCache(page)  == true, PageWriteback(page) == false,
> page_swapcount(page) == 0  in swap_writepage(shrink_page_list path), then
> will delete the page from swap cache and free swap slot, in function
> swap_writepage:
>
> if (try_to_free_swap(page)) {
>     unlock_page(page);
>     goto out;
> }
> writeback will not execute, that's wrong. Where I miss?

when the page is deleted from swap cache and corresponding swap slot
is free, the page is set dirty. The dirty page won't be reclaimed. It
is not wrong.

corresponding path lists as below.
when swap_writepage() is called by pageout() in shrink_page_list().
pageout() will return PAGE_SUCCESS. For PAGE_SUCCESS, when
PageDirty(page) is true, this reclaiming page will be keeped in the
inactive LRU list.
shrink_page_list()
{
...
 904                         switch (pageout(page, mapping, sc)) {
 905                         case PAGE_KEEP:
 906                                 nr_congested++;
 907                                 goto keep_locked;
 908                         case PAGE_ACTIVATE:
 909                                 goto activate_locked;
 910                         case PAGE_SUCCESS:
 911                                 if (PageWriteback(page))
 912                                         goto keep_lumpy;
 913                                 if (PageDirty(page))
 914                                         goto keep;
...}

>
> 2) In the function pageout, page will be set PG_Reclaim flag, since this
> flag is set, end_swap_bio_write->end_page_writeback:
> if (TestClearPageReclaim(page))
>      rotate_reclaimable_page(page);
> it means that page will be add to the tail of lru list, page is clean
> anonymous page this time and will be reclaim to buddy system soon, correct?
correct
> If is correct, what is the meaning of rotate here?

Rotating here is to add the page to the tail of inactive LRU list. So
this page will be reclaimed ASAP while reclaiming.

>
>>
>> Hugh
>
>

Should a swapped out page be deleted from swap cache?

[Ric Mason]

On 03/06/2013 01:34 PM, Li Haifeng wrote:
> 2013/2/20 Ric Mason <ric.masonn@gmail.com>:
>> Hi Hugh,
>>
>>
>> On 02/20/2013 02:56 AM, Hugh Dickins wrote:
>>> On Tue, 19 Feb 2013, Ric Mason wrote:
>>>> There is a call of try_to_free_swap in function swap_writepage, if
>>>> swap_writepage is call from shrink_page_list path, PageSwapCache(page) ==
>>>> trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, then
>>>> will
>>>> delete the page from swap cache and free swap slot, where I miss?
>>> That's correct.  PageWriteback is sure to be false there.  page_swapcount
>>> usually won't be 0 there, but sometimes it will be, and in that case we
>>> do want to delete from swap cache and free the swap slot.
>>
>> 1) If PageSwapCache(page)  == true, PageWriteback(page) == false,
>> page_swapcount(page) == 0  in swap_writepage(shrink_page_list path), then
>> will delete the page from swap cache and free swap slot, in function
>> swap_writepage:
>>
>> if (try_to_free_swap(page)) {
>>      unlock_page(page);
>>      goto out;
>> }
>> writeback will not execute, that's wrong. Where I miss?
> when the page is deleted from swap cache and corresponding swap slot
> is free, the page is set dirty. The dirty page won't be reclaimed. It
> is not wrong.

I don't think so. For dirty pages, there are two steps: 1)writeback 
2)reclaim. Since PageSwapCache(page) == true && PageWriteback(page) == 
false && page_swapcount(page) == 0 in swap_writeback(), 
try_to_free_swap() will return true and writeback will be skip. Then how 
can step one be executed?

>
> corresponding path lists as below.
> when swap_writepage() is called by pageout() in shrink_page_list().
> pageout() will return PAGE_SUCCESS. For PAGE_SUCCESS, when
> PageDirty(page) is true, this reclaiming page will be keeped in the
> inactive LRU list.
> shrink_page_list()
> {
> ...
>   904                         switch (pageout(page, mapping, sc)) {
>   905                         case PAGE_KEEP:
>   906                                 nr_congested++;
>   907                                 goto keep_locked;
>   908                         case PAGE_ACTIVATE:
>   909                                 goto activate_locked;
>   910                         case PAGE_SUCCESS:
>   911                                 if (PageWriteback(page))
>   912                                         goto keep_lumpy;
>   913                                 if (PageDirty(page))
>   914                                         goto keep;
> ...}
>
>> 2) In the function pageout, page will be set PG_Reclaim flag, since this
>> flag is set, end_swap_bio_write->end_page_writeback:
>> if (TestClearPageReclaim(page))
>>       rotate_reclaimable_page(page);
>> it means that page will be add to the tail of lru list, page is clean
>> anonymous page this time and will be reclaim to buddy system soon, correct?
> correct
>> If is correct, what is the meaning of rotate here?
> Rotating here is to add the page to the tail of inactive LRU list. So
> this page will be reclaimed ASAP while reclaiming.
>
>>> Hugh
>>

Should a swapped out page be deleted from swap cache?

[Ric Mason]

On 03/06/2013 07:04 PM, Ric Mason wrote:
> On 03/06/2013 01:34 PM, Li Haifeng wrote:
>> 2013/2/20 Ric Mason <ric.masonn@gmail.com>:
>>> Hi Hugh,
>>>
>>>
>>> On 02/20/2013 02:56 AM, Hugh Dickins wrote:
>>>> On Tue, 19 Feb 2013, Ric Mason wrote:
>>>>> There is a call of try_to_free_swap in function swap_writepage, if
>>>>> swap_writepage is call from shrink_page_list path, 
>>>>> PageSwapCache(page) ==
>>>>> trure, PageWriteback(page) maybe false, page_swapcount(page) == 0, 
>>>>> then
>>>>> will
>>>>> delete the page from swap cache and free swap slot, where I miss?
>>>> That's correct.  PageWriteback is sure to be false there. 
>>>> page_swapcount
>>>> usually won't be 0 there, but sometimes it will be, and in that 
>>>> case we
>>>> do want to delete from swap cache and free the swap slot.
>>>
>>> 1) If PageSwapCache(page)  == true, PageWriteback(page) == false,
>>> page_swapcount(page) == 0  in swap_writepage(shrink_page_list path), 
>>> then
>>> will delete the page from swap cache and free swap slot, in function
>>> swap_writepage:
>>>
>>> if (try_to_free_swap(page)) {
>>>      unlock_page(page);
>>>      goto out;
>>> }
>>> writeback will not execute, that's wrong. Where I miss?
>> when the page is deleted from swap cache and corresponding swap slot
>> is free, the page is set dirty. The dirty page won't be reclaimed. It
>> is not wrong.
>
> I don't think so. For dirty pages, there are two steps: 1)writeback 
> 2)reclaim. Since PageSwapCache(page) == true && PageWriteback(page) == 
> false && page_swapcount(page) == 0 in swap_writeback(), 
> try_to_free_swap() will return true and writeback will be skip. Then 
> how can step one be executed?

s/swap_writeback()/swap_writepage()

Btw, Hi Hugh, could you explain more to us? :-)

>
>>
>> corresponding path lists as below.
>> when swap_writepage() is called by pageout() in shrink_page_list().
>> pageout() will return PAGE_SUCCESS. For PAGE_SUCCESS, when
>> PageDirty(page) is true, this reclaiming page will be keeped in the
>> inactive LRU list.
>> shrink_page_list()
>> {
>> ...
>>   904                         switch (pageout(page, mapping, sc)) {
>>   905                         case PAGE_KEEP:
>>   906                                 nr_congested++;
>>   907                                 goto keep_locked;
>>   908                         case PAGE_ACTIVATE:
>>   909                                 goto activate_locked;
>>   910                         case PAGE_SUCCESS:
>>   911                                 if (PageWriteback(page))
>>   912                                         goto keep_lumpy;
>>   913                                 if (PageDirty(page))
>>   914                                         goto keep;
>> ...}
>>
>>> 2) In the function pageout, page will be set PG_Reclaim flag, since 
>>> this
>>> flag is set, end_swap_bio_write->end_page_writeback:
>>> if (TestClearPageReclaim(page))
>>>       rotate_reclaimable_page(page);
>>> it means that page will be add to the tail of lru list, page is clean
>>> anonymous page this time and will be reclaim to buddy system soon, 
>>> correct?
>> correct
>>> If is correct, what is the meaning of rotate here?
>> Rotating here is to add the page to the tail of inactive LRU list. So
>> this page will be reclaimed ASAP while reclaiming.
>>
>>>> Hugh
>>>
>