From: Minchan Kim <minchan@kernel.org>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: linux-mm <linux-mm@kvack.org>,
LKML <linux-kernel@vger.kernel.org>,
linux-api@vger.kernel.org, Michal Hocko <mhocko@suse.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Tim Murray <timmurray@google.com>,
Joel Fernandes <joel@joelfernandes.org>,
Suren Baghdasaryan <surenb@google.com>,
Daniel Colascione <dancol@google.com>,
Shakeel Butt <shakeelb@google.com>,
Sonny Rao <sonnyrao@google.com>,
oleksandr@redhat.com, hdanton@sina.com, lizeb@google.com,
Dave Hansen <dave.hansen@intel.com>,
"Kirill A . Shutemov" <kirill.shutemov@linux.intel.com>
Subject: Re: [PATCH v3 0/5] Introduce MADV_COLD and MADV_PAGEOUT
Date: Mon, 1 Jul 2019 16:38:48 +0900 [thread overview]
Message-ID: <20190701073848.GB136163@google.com> (raw)
In-Reply-To: <20190627115405.255259-1-minchan@kernel.org>
Hi Folks,
Do you guys have comments? I think it would be long enough to be
pending. If there is no further comments, I want to ask to merge.
Thanks.
On Thu, Jun 27, 2019 at 08:54:00PM +0900, Minchan Kim wrote:
> This patch is part of previous series:
> https://lore.kernel.org/lkml/20190531064313.193437-1-minchan@kernel.org/T/#u
> Originally, it was created for external madvise hinting feature.
>
> https://lkml.org/lkml/2019/5/31/463
> Michal wanted to separte the discussion from external hinting interface
> so this patchset includes only first part of my entire patchset
>
> - introduce MADV_COLD and MADV_PAGEOUT hint to madvise.
>
> However, I keep entire description for others for easier understanding
> why this kinds of hint was born.
>
> Thanks.
>
> This patchset is against on next-20190530.
>
> Below is description of previous entire patchset.
> ================= &< =====================
>
> - Background
>
> The Android terminology used for forking a new process and starting an app
> from scratch is a cold start, while resuming an existing app is a hot start.
> While we continually try to improve the performance of cold starts, hot
> starts will always be significantly less power hungry as well as faster so
> we are trying to make hot start more likely than cold start.
>
> To increase hot start, Android userspace manages the order that apps should
> be killed in a process called ActivityManagerService. ActivityManagerService
> tracks every Android app or service that the user could be interacting with
> at any time and translates that into a ranked list for lmkd(low memory
> killer daemon). They are likely to be killed by lmkd if the system has to
> reclaim memory. In that sense they are similar to entries in any other cache.
> Those apps are kept alive for opportunistic performance improvements but
> those performance improvements will vary based on the memory requirements of
> individual workloads.
>
> - Problem
>
> Naturally, cached apps were dominant consumers of memory on the system.
> However, they were not significant consumers of swap even though they are
> good candidate for swap. Under investigation, swapping out only begins
> once the low zone watermark is hit and kswapd wakes up, but the overall
> allocation rate in the system might trip lmkd thresholds and cause a cached
> process to be killed(we measured performance swapping out vs. zapping the
> memory by killing a process. Unsurprisingly, zapping is 10x times faster
> even though we use zram which is much faster than real storage) so kill
> from lmkd will often satisfy the high zone watermark, resulting in very
> few pages actually being moved to swap.
>
> - Approach
>
> The approach we chose was to use a new interface to allow userspace to
> proactively reclaim entire processes by leveraging platform information.
> This allowed us to bypass the inaccuracy of the kernel’s LRUs for pages
> that are known to be cold from userspace and to avoid races with lmkd
> by reclaiming apps as soon as they entered the cached state. Additionally,
> it could provide many chances for platform to use much information to
> optimize memory efficiency.
>
> To achieve the goal, the patchset introduce two new options for madvise.
> One is MADV_COLD which will deactivate activated pages and the other is
> MADV_PAGEOUT which will reclaim private pages instantly. These new options
> complement MADV_DONTNEED and MADV_FREE by adding non-destructive ways to
> gain some free memory space. MADV_PAGEOUT is similar to MADV_DONTNEED in a way
> that it hints the kernel that memory region is not currently needed and
> should be reclaimed immediately; MADV_COLD is similar to MADV_FREE in a way
> that it hints the kernel that memory region is not currently needed and
> should be reclaimed when memory pressure rises.
>
> Minchan Kim (5):
> mm: introduce MADV_COLD
> mm: change PAGEREF_RECLAIM_CLEAN with PAGE_REFRECLAIM
> mm: account nr_isolated_xxx in [isolate|putback]_lru_page
> mm: introduce MADV_PAGEOUT
> mm: factor out pmd young/dirty bit handling and THP split
>
> include/linux/huge_mm.h | 3 -
> include/linux/swap.h | 2 +
> include/uapi/asm-generic/mman-common.h | 2 +
> mm/compaction.c | 2 -
> mm/gup.c | 7 +-
> mm/huge_memory.c | 74 -----
> mm/internal.h | 2 +-
> mm/khugepaged.c | 3 -
> mm/madvise.c | 438 ++++++++++++++++++++++++-
> mm/memory-failure.c | 3 -
> mm/memory_hotplug.c | 4 -
> mm/mempolicy.c | 6 +-
> mm/migrate.c | 37 +--
> mm/oom_kill.c | 2 +-
> mm/swap.c | 42 +++
> mm/vmscan.c | 86 ++++-
> 16 files changed, 566 insertions(+), 147 deletions(-)
>
> --
> 2.22.0.410.gd8fdbe21b5-goog
>
next prev parent reply other threads:[~2019-07-01 7:38 UTC|newest]
Thread overview: 27+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-06-27 11:54 [PATCH v3 0/5] Introduce MADV_COLD and MADV_PAGEOUT Minchan Kim
2019-06-27 11:54 ` [PATCH v3 1/5] mm: introduce MADV_COLD Minchan Kim
2019-06-27 13:13 ` Dave Hansen
2019-06-27 14:02 ` Michal Hocko
2019-06-27 14:36 ` Dave Hansen
2019-06-27 14:53 ` Michal Hocko
2019-06-27 23:56 ` Minchan Kim
2019-07-01 7:35 ` Minchan Kim
2019-07-09 9:19 ` Michal Hocko
2019-06-27 23:46 ` Minchan Kim
2019-06-27 11:54 ` [PATCH v3 2/5] mm: change PAGEREF_RECLAIM_CLEAN with PAGE_REFRECLAIM Minchan Kim
2019-06-27 11:54 ` [PATCH v3 3/5] mm: account nr_isolated_xxx in [isolate|putback]_lru_page Minchan Kim
2019-07-09 9:38 ` Michal Hocko
2019-06-27 11:54 ` [PATCH v3 4/5] mm: introduce MADV_PAGEOUT Minchan Kim
2019-07-09 9:55 ` Michal Hocko
2019-07-10 10:48 ` Minchan Kim
2019-07-10 11:16 ` Michal Hocko
2019-07-10 11:53 ` Minchan Kim
2019-07-10 19:47 ` Michal Hocko
2019-07-11 0:25 ` Minchan Kim
2019-06-27 11:54 ` [PATCH v3 5/5] mm: factor out pmd young/dirty bit handling and THP split Minchan Kim
2019-07-09 14:10 ` Michal Hocko
2019-07-10 10:56 ` Minchan Kim
2019-06-27 18:06 ` [PATCH v3 0/5] Introduce MADV_COLD and MADV_PAGEOUT Kirill A. Shutemov
2019-06-27 23:12 ` Minchan Kim
2019-07-01 7:38 ` Minchan Kim [this message]
2019-07-01 10:22 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20190701073848.GB136163@google.com \
--to=minchan@kernel.org \
--cc=akpm@linux-foundation.org \
--cc=dancol@google.com \
--cc=dave.hansen@intel.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=joel@joelfernandes.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=lizeb@google.com \
--cc=mhocko@suse.com \
--cc=oleksandr@redhat.com \
--cc=shakeelb@google.com \
--cc=sonnyrao@google.com \
--cc=surenb@google.com \
--cc=timmurray@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).