All of lore.kernel.org
 help / color / mirror / Atom feed
From: Shaohua Li <shli@kernel.org>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	linux-api@vger.kernel.org, Hugh Dickins <hughd@google.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	zhangyanfei@cn.fujitsu.com, Rik van Riel <riel@redhat.com>,
	Mel Gorman <mgorman@suse.de>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Jason Evans <je@fb.com>, Daniel Micay <danielmicay@gmail.com>,
	"Kirill A. Shutemov" <kirill@shutemov.name>,
	Michal Hocko <mhocko@suse.cz>,
	yalin.wang2010@gmail.com, "Wang,
	Yalin" <Yalin.Wang@sonymobile.com>
Subject: Re: [PATCH 5/8] mm: move lazily freed pages to inactive list
Date: Fri, 30 Oct 2015 10:22:12 -0700	[thread overview]
Message-ID: <20151030172212.GB44946@kernel.org> (raw)
In-Reply-To: <1446188504-28023-6-git-send-email-minchan@kernel.org>

On Fri, Oct 30, 2015 at 04:01:41PM +0900, Minchan Kim wrote:
> MADV_FREE is a hint that it's okay to discard pages if there is memory
> pressure and we use reclaimers(ie, kswapd and direct reclaim) to free them
> so there is no value keeping them in the active anonymous LRU so this
> patch moves them to inactive LRU list's head.
> 
> This means that MADV_FREE-ed pages which were living on the inactive list
> are reclaimed first because they are more likely to be cold rather than
> recently active pages.
> 
> An arguable issue for the approach would be whether we should put the page
> to the head or tail of the inactive list.  I chose head because the kernel
> cannot make sure it's really cold or warm for every MADV_FREE usecase but
> at least we know it's not *hot*, so landing of inactive head would be a
> comprimise for various usecases.
> 
> This fixes suboptimal behavior of MADV_FREE when pages living on the
> active list will sit there for a long time even under memory pressure
> while the inactive list is reclaimed heavily.  This basically breaks the
> whole purpose of using MADV_FREE to help the system to free memory which
> is might not be used.

My main concern is the policy how we should treat the FREE pages. Moving it to
inactive lru is definitionly a good start, I'm wondering if it's enough. The
MADV_FREE increases memory pressure and cause unnecessary reclaim because of
the lazy memory free. While MADV_FREE is intended to be a better replacement of
MADV_DONTNEED, MADV_DONTNEED doesn't have the memory pressure issue as it free
memory immediately. So I hope the MADV_FREE doesn't have impact on memory
pressure too. I'm thinking of adding an extra lru list and wartermark for this
to make sure FREE pages can be freed before system wide page reclaim. As you
said, this is arguable, but I hope we can discuss about this issue more.

Or do you want to push this first and address the policy issue later?

Thanks,
Shaohua

WARNING: multiple messages have this Message-ID (diff)
From: Shaohua Li <shli-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
To: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Andrew Morton
	<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
	linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
	Michael Kerrisk
	<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
	Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
	Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
	zhangyanfei-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org,
	Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
	Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>,
	KOSAKI Motohiro
	<kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org>,
	Jason Evans <je-b10kYP2dOMg@public.gmane.org>,
	Daniel Micay
	<danielmicay-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
	"Kirill A. Shutemov"
	<kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org>,
	Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>,
	yalin.wang2010-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org, "Wang,
	Yalin" <Yalin.Wang-/MT0OVThwyLZJqsBc5GL+g@public.gmane.org>
Subject: Re: [PATCH 5/8] mm: move lazily freed pages to inactive list
Date: Fri, 30 Oct 2015 10:22:12 -0700	[thread overview]
Message-ID: <20151030172212.GB44946@kernel.org> (raw)
In-Reply-To: <1446188504-28023-6-git-send-email-minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>

On Fri, Oct 30, 2015 at 04:01:41PM +0900, Minchan Kim wrote:
> MADV_FREE is a hint that it's okay to discard pages if there is memory
> pressure and we use reclaimers(ie, kswapd and direct reclaim) to free them
> so there is no value keeping them in the active anonymous LRU so this
> patch moves them to inactive LRU list's head.
> 
> This means that MADV_FREE-ed pages which were living on the inactive list
> are reclaimed first because they are more likely to be cold rather than
> recently active pages.
> 
> An arguable issue for the approach would be whether we should put the page
> to the head or tail of the inactive list.  I chose head because the kernel
> cannot make sure it's really cold or warm for every MADV_FREE usecase but
> at least we know it's not *hot*, so landing of inactive head would be a
> comprimise for various usecases.
> 
> This fixes suboptimal behavior of MADV_FREE when pages living on the
> active list will sit there for a long time even under memory pressure
> while the inactive list is reclaimed heavily.  This basically breaks the
> whole purpose of using MADV_FREE to help the system to free memory which
> is might not be used.

My main concern is the policy how we should treat the FREE pages. Moving it to
inactive lru is definitionly a good start, I'm wondering if it's enough. The
MADV_FREE increases memory pressure and cause unnecessary reclaim because of
the lazy memory free. While MADV_FREE is intended to be a better replacement of
MADV_DONTNEED, MADV_DONTNEED doesn't have the memory pressure issue as it free
memory immediately. So I hope the MADV_FREE doesn't have impact on memory
pressure too. I'm thinking of adding an extra lru list and wartermark for this
to make sure FREE pages can be freed before system wide page reclaim. As you
said, this is arguable, but I hope we can discuss about this issue more.

Or do you want to push this first and address the policy issue later?

Thanks,
Shaohua

WARNING: multiple messages have this Message-ID (diff)
From: Shaohua Li <shli@kernel.org>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
	linux-kernel@vger.kernel.org, linux-mm@kvack.org,
	Michael Kerrisk <mtk.manpages@gmail.com>,
	linux-api@vger.kernel.org, Hugh Dickins <hughd@google.com>,
	Johannes Weiner <hannes@cmpxchg.org>,
	zhangyanfei@cn.fujitsu.com, Rik van Riel <riel@redhat.com>,
	Mel Gorman <mgorman@suse.de>,
	KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
	Jason Evans <je@fb.com>, Daniel Micay <danielmicay@gmail.com>,
	"Kirill A. Shutemov" <kirill@shutemov.name>,
	Michal Hocko <mhocko@suse.cz>,
	yalin.wang2010@gmail.com, "Wang,
	Yalin" <Yalin.Wang@sonymobile.com>
Subject: Re: [PATCH 5/8] mm: move lazily freed pages to inactive list
Date: Fri, 30 Oct 2015 10:22:12 -0700	[thread overview]
Message-ID: <20151030172212.GB44946@kernel.org> (raw)
In-Reply-To: <1446188504-28023-6-git-send-email-minchan@kernel.org>

On Fri, Oct 30, 2015 at 04:01:41PM +0900, Minchan Kim wrote:
> MADV_FREE is a hint that it's okay to discard pages if there is memory
> pressure and we use reclaimers(ie, kswapd and direct reclaim) to free them
> so there is no value keeping them in the active anonymous LRU so this
> patch moves them to inactive LRU list's head.
> 
> This means that MADV_FREE-ed pages which were living on the inactive list
> are reclaimed first because they are more likely to be cold rather than
> recently active pages.
> 
> An arguable issue for the approach would be whether we should put the page
> to the head or tail of the inactive list.  I chose head because the kernel
> cannot make sure it's really cold or warm for every MADV_FREE usecase but
> at least we know it's not *hot*, so landing of inactive head would be a
> comprimise for various usecases.
> 
> This fixes suboptimal behavior of MADV_FREE when pages living on the
> active list will sit there for a long time even under memory pressure
> while the inactive list is reclaimed heavily.  This basically breaks the
> whole purpose of using MADV_FREE to help the system to free memory which
> is might not be used.

My main concern is the policy how we should treat the FREE pages. Moving it to
inactive lru is definitionly a good start, I'm wondering if it's enough. The
MADV_FREE increases memory pressure and cause unnecessary reclaim because of
the lazy memory free. While MADV_FREE is intended to be a better replacement of
MADV_DONTNEED, MADV_DONTNEED doesn't have the memory pressure issue as it free
memory immediately. So I hope the MADV_FREE doesn't have impact on memory
pressure too. I'm thinking of adding an extra lru list and wartermark for this
to make sure FREE pages can be freed before system wide page reclaim. As you
said, this is arguable, but I hope we can discuss about this issue more.

Or do you want to push this first and address the policy issue later?

Thanks,
Shaohua

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

  reply	other threads:[~2015-10-30 17:22 UTC|newest]

Thread overview: 97+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-10-30  7:01 [PATCH 0/8] MADV_FREE support Minchan Kim
2015-10-30  7:01 ` Minchan Kim
2015-10-30  7:01 ` Minchan Kim
2015-10-30  7:01 ` [PATCH 1/8] mm: support madvise(MADV_FREE) Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 16:49   ` Shaohua Li
2015-10-30 16:49     ` Shaohua Li
2015-11-03  0:10     ` Minchan Kim
2015-11-03  0:10       ` Minchan Kim
2015-11-03  0:10       ` Minchan Kim
2015-10-30  7:01 ` [PATCH 2/8] mm: define MADV_FREE for some arches Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30  7:01 ` [PATCH 3/8] arch: uapi: asm: mman.h: Let MADV_FREE have same value for all architectures Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-11-02  0:08   ` Hugh Dickins
2015-11-02  0:08     ` Hugh Dickins
2015-11-02  0:08     ` Hugh Dickins
2015-11-02  0:08     ` Hugh Dickins
2015-11-03  2:32     ` Minchan Kim
2015-11-03  2:32       ` Minchan Kim
2015-11-03  2:32       ` Minchan Kim
2015-11-03  2:32       ` Minchan Kim
2015-11-03  2:36       ` Minchan Kim
2015-11-03  2:36         ` Minchan Kim
2015-11-03  2:36         ` Minchan Kim
2015-11-03  2:36         ` Minchan Kim
2015-11-03  3:36         ` David Miller
2015-11-03  3:36           ` David Miller
2015-11-03  3:36           ` David Miller
2015-11-03  4:31           ` Minchan Kim
2015-11-03  4:31             ` Minchan Kim
2015-11-03  4:31             ` Minchan Kim
2015-10-30  7:01 ` [PATCH 4/8] mm: free swp_entry in madvise_free Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 12:28   ` Michal Hocko
2015-10-30 12:28     ` Michal Hocko
2015-11-03  0:53     ` Minchan Kim
2015-11-03  0:53       ` Minchan Kim
2015-11-03  0:53       ` Minchan Kim
2015-10-30  7:01 ` [PATCH 5/8] mm: move lazily freed pages to inactive list Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 17:22   ` Shaohua Li [this message]
2015-10-30 17:22     ` Shaohua Li
2015-10-30 17:22     ` Shaohua Li
2015-11-03  0:52     ` Minchan Kim
2015-11-03  0:52       ` Minchan Kim
2015-11-03  0:52       ` Minchan Kim
2015-11-04  8:15       ` Michal Hocko
2015-11-04  8:15         ` Michal Hocko
2015-11-04 17:53       ` Shaohua Li
2015-11-04 17:53         ` Shaohua Li
2015-11-04 17:53         ` Shaohua Li
2015-11-04 18:20         ` Shaohua Li
2015-11-04 18:20           ` Shaohua Li
2015-11-05  1:11           ` Minchan Kim
2015-11-05  1:11             ` Minchan Kim
2015-11-05  1:03         ` Minchan Kim
2015-11-05  1:03           ` Minchan Kim
2015-11-05  1:03           ` Minchan Kim
2015-11-04 20:55   ` Johannes Weiner
2015-11-04 20:55     ` Johannes Weiner
2015-11-04 20:55     ` Johannes Weiner
2015-11-04 21:48     ` Daniel Micay
2015-11-04 21:48       ` Daniel Micay
2015-11-04 22:55       ` Johannes Weiner
2015-11-04 22:55         ` Johannes Weiner
2015-11-04 22:55         ` Johannes Weiner
2015-11-04 23:36         ` Daniel Micay
2015-11-04 23:49           ` Daniel Micay
2015-11-04 23:49             ` Daniel Micay
2015-10-30  7:01 ` [PATCH 6/8] mm: lru_deactivate_fn should clear PG_referenced Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 12:47   ` Michal Hocko
2015-10-30 12:47     ` Michal Hocko
2015-10-30 12:47     ` Michal Hocko
2015-11-03  1:10     ` Minchan Kim
2015-11-03  1:10       ` Minchan Kim
2015-11-04  8:22       ` Michal Hocko
2015-11-04  8:22         ` Michal Hocko
2015-11-04  8:22         ` Michal Hocko
2015-10-30  7:01 ` [PATCH 7/8] mm: clear PG_dirty to mark page freeable Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-10-30 12:55   ` Michal Hocko
2015-10-30 12:55     ` Michal Hocko
2015-10-30 12:55     ` Michal Hocko
2015-10-30  7:01 ` [PATCH 8/8] mm: mark stable page dirty in KSM Minchan Kim
2015-10-30  7:01   ` Minchan Kim
2015-11-01  4:51 ` [PATCH 0/8] MADV_FREE support David Rientjes
2015-11-01  4:51   ` David Rientjes
2015-11-01  4:51   ` David Rientjes
2015-11-01  6:29   ` Daniel Micay
2015-11-03  2:23     ` Minchan Kim
2015-11-03  2:23       ` Minchan Kim
2015-11-03  2:23       ` Minchan Kim
2015-11-04 20:19     ` David Rientjes
2015-11-04 20:19       ` David Rientjes

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20151030172212.GB44946@kernel.org \
    --to=shli@kernel.org \
    --cc=Yalin.Wang@sonymobile.com \
    --cc=akpm@linux-foundation.org \
    --cc=danielmicay@gmail.com \
    --cc=hannes@cmpxchg.org \
    --cc=hughd@google.com \
    --cc=je@fb.com \
    --cc=kirill@shutemov.name \
    --cc=kosaki.motohiro@jp.fujitsu.com \
    --cc=linux-api@vger.kernel.org \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@suse.de \
    --cc=mhocko@suse.cz \
    --cc=minchan@kernel.org \
    --cc=mtk.manpages@gmail.com \
    --cc=riel@redhat.com \
    --cc=yalin.wang2010@gmail.com \
    --cc=zhangyanfei@cn.fujitsu.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.