All of lore.kernel.org
 help / color / mirror / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	linux-api@vger.kernel.org, Hugh Dickins <hughd@google.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	zhangyanfei@cn.fujitsu.com, Rik van Riel <riel@redhat.com>,
	Mel Gorman <mgorman@suse.de>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Jason Evans <je@fb.com>, Daniel Micay <danielmicay@gmail.com>,
	"Kirill A. Shutemov" <kirill@shutemov.name>,
	yalin.wang2010@gmail.com, Shaohua Li <shli@kernel.org>
Subject: Re: [PATCH 7/8] mm: clear PG_dirty to mark page freeable
Date: Fri, 30 Oct 2015 13:55:04 +0100	[thread overview]
Message-ID: <20151030125504.GC23627@dhcp22.suse.cz> (raw)
In-Reply-To: <1446188504-28023-8-git-send-email-minchan@kernel.org>

On Fri 30-10-15 16:01:43, Minchan Kim wrote:
> Basically, MADV_FREE relies on dirty bit in page table entry to decide
> whether VM allows to discard the page or not.  IOW, if page table entry
> includes marked dirty bit, VM shouldn't discard the page.
> 
> However, as a example, if swap-in by read fault happens, page table entry
> doesn't have dirty bit so MADV_FREE could discard the page wrongly.
> 
> For avoiding the problem, MADV_FREE did more checks with PageDirty
> and PageSwapCache. It worked out because swapped-in page lives on
> swap cache and since it is evicted from the swap cache, the page has
> PG_dirty flag. So both page flags check effectively prevent
> wrong discarding by MADV_FREE.
> 
> However, a problem in above logic is that swapped-in page has
> PG_dirty still after they are removed from swap cache so VM cannot
> consider the page as freeable any more even if madvise_free is
> called in future.
> 
> Look at below example for detail.
> 
>     ptr = malloc();
>     memset(ptr);
>     ..
>     ..
>     .. heavy memory pressure so all of pages are swapped out
>     ..
>     ..
>     var = *ptr; -> a page swapped-in and could be removed from
>                    swapcache. Then, page table doesn't mark
>                    dirty bit and page descriptor includes PG_dirty
>     ..
>     ..
>     madvise_free(ptr); -> It doesn't clear PG_dirty of the page.
>     ..
>     ..
>     ..
>     .. heavy memory pressure again.
>     .. In this time, VM cannot discard the page because the page
>     .. has *PG_dirty*
> 
> To solve the problem, this patch clears PG_dirty if only the page is owned
> exclusively by current process when madvise is called because PG_dirty
> represents ptes's dirtiness in several processes so we could clear it only
> if we own it exclusively.
> 
> Acked-by: Hugh Dickins <hughd@google.com>
> Signed-off-by: Minchan Kim <minchan@kernel.org>

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  mm/madvise.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 9ee9df8c768d..fc24104d6b3a 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -303,11 +303,19 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
>  		if (!page)
>  			continue;
>  
> -		if (PageSwapCache(page)) {
> +		if (PageSwapCache(page) || PageDirty(page)) {
>  			if (!trylock_page(page))
>  				continue;
> +			/*
> +			 * If page is shared with others, we couldn't clear
> +			 * PG_dirty of the page.
> +			 */
> +			if (page_count(page) != 1 + !!PageSwapCache(page)) {
> +				unlock_page(page);
> +				continue;
> +			}
>  
> -			if (!try_to_free_swap(page)) {
> +			if (PageSwapCache(page) && !try_to_free_swap(page)) {
>  				unlock_page(page);
>  				continue;
>  			}
> -- 
> 1.9.1

-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Andrew Morton
	<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	Michael Kerrisk
	<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
	zhangyanfei-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org,
	Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>,
	KOSAKI Motohiro
	<kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org>,
	Jason Evans <je-b10kYP2dOMg@public.gmane.org>,
	Daniel Micay
	<danielmicay-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	"Kirill A. Shutemov"
	<kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org>,
	yalin.wang2010-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
	Shaohua Li <shli-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH 7/8] mm: clear PG_dirty to mark page freeable
Date: Fri, 30 Oct 2015 13:55:04 +0100	[thread overview]
Message-ID: <20151030125504.GC23627@dhcp22.suse.cz> (raw)
In-Reply-To: <1446188504-28023-8-git-send-email-minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

On Fri 30-10-15 16:01:43, Minchan Kim wrote:
> Basically, MADV_FREE relies on dirty bit in page table entry to decide
> whether VM allows to discard the page or not.  IOW, if page table entry
> includes marked dirty bit, VM shouldn't discard the page.
> 
> However, as a example, if swap-in by read fault happens, page table entry
> doesn't have dirty bit so MADV_FREE could discard the page wrongly.
> 
> For avoiding the problem, MADV_FREE did more checks with PageDirty
> and PageSwapCache. It worked out because swapped-in page lives on
> swap cache and since it is evicted from the swap cache, the page has
> PG_dirty flag. So both page flags check effectively prevent
> wrong discarding by MADV_FREE.
> 
> However, a problem in above logic is that swapped-in page has
> PG_dirty still after they are removed from swap cache so VM cannot
> consider the page as freeable any more even if madvise_free is
> called in future.
> 
> Look at below example for detail.
> 
>     ptr = malloc();
>     memset(ptr);
>     ..
>     ..
>     .. heavy memory pressure so all of pages are swapped out
>     ..
>     ..
>     var = *ptr; -> a page swapped-in and could be removed from
>                    swapcache. Then, page table doesn't mark
>                    dirty bit and page descriptor includes PG_dirty
>     ..
>     ..
>     madvise_free(ptr); -> It doesn't clear PG_dirty of the page.
>     ..
>     ..
>     ..
>     .. heavy memory pressure again.
>     .. In this time, VM cannot discard the page because the page
>     .. has *PG_dirty*
> 
> To solve the problem, this patch clears PG_dirty if only the page is owned
> exclusively by current process when madvise is called because PG_dirty
> represents ptes's dirtiness in several processes so we could clear it only
> if we own it exclusively.
> 
> Acked-by: Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
> Signed-off-by: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

Acked-by: Michal Hocko <mhocko-IBi9RG/b67k@public.gmane.org>

> ---
>  mm/madvise.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 9ee9df8c768d..fc24104d6b3a 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -303,11 +303,19 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
>  		if (!page)
>  			continue;
>  
> -		if (PageSwapCache(page)) {
> +		if (PageSwapCache(page) || PageDirty(page)) {
>  			if (!trylock_page(page))
>  				continue;
> +			/*
> +			 * If page is shared with others, we couldn't clear
> +			 * PG_dirty of the page.
> +			 */
> +			if (page_count(page) != 1 + !!PageSwapCache(page)) {
> +				unlock_page(page);
> +				continue;
> +			}
>  
> -			if (!try_to_free_swap(page)) {
> +			if (PageSwapCache(page) && !try_to_free_swap(page)) {
>  				unlock_page(page);
>  				continue;
>  			}
> -- 
> 1.9.1

-- 
Michal Hocko
SUSE Labs

WARNING: multiple messages have this Message-ID (diff)
From: Michal Hocko <mhocko@kernel.org>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	linux-api@vger.kernel.org, Hugh Dickins <hughd@google.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	zhangyanfei@cn.fujitsu.com, Rik van Riel <riel@redhat.com>,
	Mel Gorman <mgorman@suse.de>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Jason Evans <je@fb.com>, Daniel Micay <danielmicay@gmail.com>,
	"Kirill A. Shutemov" <kirill@shutemov.name>,
	yalin.wang2010@gmail.com, Shaohua Li <shli@kernel.org>
Subject: Re: [PATCH 7/8] mm: clear PG_dirty to mark page freeable
Date: Fri, 30 Oct 2015 13:55:04 +0100	[thread overview]
Message-ID: <20151030125504.GC23627@dhcp22.suse.cz> (raw)
In-Reply-To: <1446188504-28023-8-git-send-email-minchan@kernel.org>

On Fri 30-10-15 16:01:43, Minchan Kim wrote:
> Basically, MADV_FREE relies on dirty bit in page table entry to decide
> whether VM allows to discard the page or not.  IOW, if page table entry
> includes marked dirty bit, VM shouldn't discard the page.
> 
> However, as a example, if swap-in by read fault happens, page table entry
> doesn't have dirty bit so MADV_FREE could discard the page wrongly.
> 
> For avoiding the problem, MADV_FREE did more checks with PageDirty
> and PageSwapCache. It worked out because swapped-in page lives on
> swap cache and since it is evicted from the swap cache, the page has
> PG_dirty flag. So both page flags check effectively prevent
> wrong discarding by MADV_FREE.
> 
> However, a problem in above logic is that swapped-in page has
> PG_dirty still after they are removed from swap cache so VM cannot
> consider the page as freeable any more even if madvise_free is
> called in future.
> 
> Look at below example for detail.
> 
>     ptr = malloc();
>     memset(ptr);
>     ..
>     ..
>     .. heavy memory pressure so all of pages are swapped out
>     ..
>     ..
>     var = *ptr; -> a page swapped-in and could be removed from
>                    swapcache. Then, page table doesn't mark
>                    dirty bit and page descriptor includes PG_dirty
>     ..
>     ..
>     madvise_free(ptr); -> It doesn't clear PG_dirty of the page.
>     ..
>     ..
>     ..
>     .. heavy memory pressure again.
>     .. In this time, VM cannot discard the page because the page
>     .. has *PG_dirty*
> 
> To solve the problem, this patch clears PG_dirty if only the page is owned
> exclusively by current process when madvise is called because PG_dirty
> represents ptes's dirtiness in several processes so we could clear it only
> if we own it exclusively.
> 
> Acked-by: Hugh Dickins <hughd@google.com>
> Signed-off-by: Minchan Kim <minchan@kernel.org>

Acked-by: Michal Hocko <mhocko@suse.com>

> ---
>  mm/madvise.c | 12 ++++++++++--
>  1 file changed, 10 insertions(+), 2 deletions(-)
> 
> diff --git a/mm/madvise.c b/mm/madvise.c
> index 9ee9df8c768d..fc24104d6b3a 100644
> --- a/mm/madvise.c
> +++ b/mm/madvise.c
> @@ -303,11 +303,19 @@ static int madvise_free_pte_range(pmd_t *pmd, unsigned long addr,
>  		if (!page)
>  			continue;
>  
> -		if (PageSwapCache(page)) {
> +		if (PageSwapCache(page) || PageDirty(page)) {
>  			if (!trylock_page(page))
>  				continue;
> +			/*
> +			 * If page is shared with others, we couldn't clear
> +			 * PG_dirty of the page.
> +			 */
> +			if (page_count(page) != 1 + !!PageSwapCache(page)) {
> +				unlock_page(page);
> +				continue;
> +			}
>  
> -			if (!try_to_free_swap(page)) {
> +			if (PageSwapCache(page) && !try_to_free_swap(page)) {
>  				unlock_page(page);
>  				continue;
>  			}
> -- 
> 1.9.1

-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2015-10-30 12:55 UTC|newest]

Thread overview: 97+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-10-30  7:01 [PATCH 0/8] MADV_FREE support Minchan Kim
2015-10-30  7:01 ` Minchan Kim
2015-10-30  7:01 ` Minchan Kim
2015-10-30  7:01 ` [PATCH 1/8] mm: support madvise(MADV_FREE) Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 16:49   ` Shaohua Li
2015-10-30 16:49     ` Shaohua Li
2015-11-03  0:10     ` Minchan Kim
2015-11-03  0:10       ` Minchan Kim
2015-11-03  0:10       ` Minchan Kim
2015-10-30  7:01 ` [PATCH 2/8] mm: define MADV_FREE for some arches Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30  7:01 ` [PATCH 3/8] arch: uapi: asm: mman.h: Let MADV_FREE have same value for all architectures Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-11-02  0:08   ` Hugh Dickins
2015-11-02  0:08     ` Hugh Dickins
2015-11-02  0:08     ` Hugh Dickins
2015-11-02  0:08     ` Hugh Dickins
2015-11-03  2:32     ` Minchan Kim
2015-11-03  2:32       ` Minchan Kim
2015-11-03  2:32       ` Minchan Kim
2015-11-03  2:32       ` Minchan Kim
2015-11-03  2:36       ` Minchan Kim
2015-11-03  2:36         ` Minchan Kim
2015-11-03  2:36         ` Minchan Kim
2015-11-03  2:36         ` Minchan Kim
2015-11-03  3:36         ` David Miller
2015-11-03  3:36           ` David Miller
2015-11-03  3:36           ` David Miller
2015-11-03  4:31           ` Minchan Kim
2015-11-03  4:31             ` Minchan Kim
2015-11-03  4:31             ` Minchan Kim
2015-10-30  7:01 ` [PATCH 4/8] mm: free swp_entry in madvise_free Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 12:28   ` Michal Hocko
2015-10-30 12:28     ` Michal Hocko
2015-11-03  0:53     ` Minchan Kim
2015-11-03  0:53       ` Minchan Kim
2015-11-03  0:53       ` Minchan Kim
2015-10-30  7:01 ` [PATCH 5/8] mm: move lazily freed pages to inactive list Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 17:22   ` Shaohua Li
2015-10-30 17:22     ` Shaohua Li
2015-10-30 17:22     ` Shaohua Li
2015-11-03  0:52     ` Minchan Kim
2015-11-03  0:52       ` Minchan Kim
2015-11-03  0:52       ` Minchan Kim
2015-11-04  8:15       ` Michal Hocko
2015-11-04  8:15         ` Michal Hocko
2015-11-04 17:53       ` Shaohua Li
2015-11-04 17:53         ` Shaohua Li
2015-11-04 17:53         ` Shaohua Li
2015-11-04 18:20         ` Shaohua Li
2015-11-04 18:20           ` Shaohua Li
2015-11-05  1:11           ` Minchan Kim
2015-11-05  1:11             ` Minchan Kim
2015-11-05  1:03         ` Minchan Kim
2015-11-05  1:03           ` Minchan Kim
2015-11-05  1:03           ` Minchan Kim
2015-11-04 20:55   ` Johannes Weiner
2015-11-04 20:55     ` Johannes Weiner
2015-11-04 20:55     ` Johannes Weiner
2015-11-04 21:48     ` Daniel Micay
2015-11-04 21:48       ` Daniel Micay
2015-11-04 22:55       ` Johannes Weiner
2015-11-04 22:55         ` Johannes Weiner
2015-11-04 22:55         ` Johannes Weiner
2015-11-04 23:36         ` Daniel Micay
2015-11-04 23:49           ` Daniel Micay
2015-11-04 23:49             ` Daniel Micay
2015-10-30  7:01 ` [PATCH 6/8] mm: lru_deactivate_fn should clear PG_referenced Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 12:47   ` Michal Hocko
2015-10-30 12:47     ` Michal Hocko
2015-10-30 12:47     ` Michal Hocko
2015-11-03  1:10     ` Minchan Kim
2015-11-03  1:10       ` Minchan Kim
2015-11-04  8:22       ` Michal Hocko
2015-11-04  8:22         ` Michal Hocko
2015-11-04  8:22         ` Michal Hocko
2015-10-30  7:01 ` [PATCH 7/8] mm: clear PG_dirty to mark page freeable Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 12:55   ` Michal Hocko [this message]
2015-10-30 12:55     ` Michal Hocko
2015-10-30 12:55     ` Michal Hocko
2015-10-30  7:01 ` [PATCH 8/8] mm: mark stable page dirty in KSM Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-11-01  4:51 ` [PATCH 0/8] MADV_FREE support David Rientjes
2015-11-01  4:51   ` David Rientjes
2015-11-01  4:51   ` David Rientjes
2015-11-01  6:29   ` Daniel Micay
2015-11-03  2:23     ` Minchan Kim
2015-11-03  2:23       ` Minchan Kim
2015-11-03  2:23       ` Minchan Kim
2015-11-04 20:19     ` David Rientjes
2015-11-04 20:19       ` David Rientjes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20151030125504.GC23627@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=akpm@linux-foundation.org \
    --cc=danielmicay@gmail.com \
    --cc=hannes@cmpxchg.org \
    --cc=hughd@google.com \
    --cc=je@fb.com \
    --cc=kirill@shutemov.name \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=minchan@kernel.org \
    --cc=mtk.manpages@gmail.com \
    --cc=riel@redhat.com \
    --cc=shli@kernel.org \
    --cc=yalin.wang2010@gmail.com \
    --cc=zhangyanfei@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.