From: David Rientjes <rientjes@google.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Michael Kerrisk <mtk.manpages@gmail.com>,
linux-api@vger.kernel.org, Hugh Dickins <hughd@google.com>,
Johannes Weiner <hannes@cmpxchg.org>,
zhangyanfei@cn.fujitsu.com, Rik van Riel <riel@redhat.com>,
Mel Gorman <mgorman@suse.de>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Jason Evans <je@fb.com>, Daniel Micay <danielmicay@gmail.com>,
"Kirill A. Shutemov" <kirill@shutemov.name>,
Michal Hocko <mhocko@suse.cz>,
yalin.wang2010@gmail.com, Shaohua Li <shli@kernel.org>
Subject: Re: [PATCH 0/8] MADV_FREE support
Date: Sat, 31 Oct 2015 21:51:15 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.10.1510312142560.10406@chino.kir.corp.google.com> (raw)
In-Reply-To: <1446188504-28023-1-git-send-email-minchan@kernel.org>
On Fri, 30 Oct 2015, Minchan Kim wrote:
> MADV_FREE is on linux-next so long time. The reason was two, I think.
>
> 1. MADV_FREE code on reclaim path was really mess.
>
> 2. Andrew really want to see voice of userland people who want to use
> the syscall.
>
> A few month ago, Daniel Micay(jemalloc active contributor) requested me
> to make progress upstreaming but I was busy at that time so it took
> so long time for me to revist the code and finally, I clean it up the
> mess recently so it solves the #2 issue.
>
> As well, Daniel and Jason(jemalloc maintainer) requested it to Andrew
> again recently and they said it would be great to have even though
> it has swap dependency now so Andrew decided he will do that for v4.4.
>
First, thanks very much for refreshing the patchset and reposting after a
series of changes have been periodically added to -mm, it makes it much
easier.
For tcmalloc, we can do some things in the allocator itself to increase
the amount of memory backed by thp. Specifically, we can prefer to
release Spans to pageblocks that are already not backed by thp so there is
no additional split on each scavenge. This is somewhat easy if all memory
is organized into hugepage-aligned pageblocks in the allocator itself.
Second, we can prefer to release Spans of longer length on each scavenge
so we can delay scavenging for as long as possible in a hope we can find
more pages to coalesce. Third, we can discount refaulted released memory
from the scavenging period.
That significantly improves the amount of memory backed by thp for
tcmalloc. The problem, however, is that tcmalloc uses MADV_DONTNEED to
release memory to the system and MADV_FREE wouldn't help at all in a
swapless environment.
To combat that, I've proposed a new MADV bit that simply caches the
ranges freed by the allocator per vma and places them on both a per-vma
and per-memcg list. During reclaim, this list is iterated and ptes are
freed after thp split period to the normal directed reclaim. Without
memory pressure, this backs 100% of the heap with thp with a relatively
lightweight kernel change (the majority is vma manipulation on split) and
a couple line change to tcmalloc. When pulling memory from the returned
freelists, the memory that we have MADV_DONTNEED'd, we need to use another
MADV bit to remove it from this cache, so there is a second madvise(2)
syscall involved but the freeing call is much less expensive since there
is no pagetable walk without memory pressure or synchronous thp split.
I've been looking at MADV_FREE to see if there is common ground that could
be shared, but perhaps it's just easier to ask what your proposed strategy
is so that tcmalloc users, especially those in swapless environments,
would benefit from any of your work?
WARNING: multiple messages have this Message-ID (diff)
From: David Rientjes <rientjes-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>
To: Minchan Kim <minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Cc: Andrew Morton
<akpm-de/tnXTf+JLsfHDXvbKv3WD2FQJk+8+b@public.gmane.org>,
linux-kernel-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
linux-mm-Bw31MaZKKs3YtjvyW6yDsg@public.gmane.org,
Michael Kerrisk
<mtk.manpages-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
linux-api-u79uwXL29TY76Z2rM5mHXA@public.gmane.org,
Hugh Dickins <hughd-hpIqsD4AKlfQT0dZR+AlfA@public.gmane.org>,
Johannes Weiner <hannes-druUgvl0LCNAfugRpC6u6w@public.gmane.org>,
zhangyanfei-BthXqXjhjHXQFUHtdCDX3A@public.gmane.org,
Rik van Riel <riel-H+wXaHxf7aLQT0dZR+AlfA@public.gmane.org>,
Mel Gorman <mgorman-l3A5Bk7waGM@public.gmane.org>,
KOSAKI Motohiro
<kosaki.motohiro-+CUm20s59erQFUHtdCDX3A@public.gmane.org>,
Jason Evans <je-b10kYP2dOMg@public.gmane.org>,
Daniel Micay
<danielmicay-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org>,
"Kirill A. Shutemov"
<kirill-oKw7cIdHH8eLwutG50LtGA@public.gmane.org>,
Michal Hocko <mhocko-AlSwsSmVLrQ@public.gmane.org>,
yalin.wang2010-Re5JQEeQqe8AvxtiuMwx3w@public.gmane.org,
Shaohua Li <shli-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
Subject: Re: [PATCH 0/8] MADV_FREE support
Date: Sat, 31 Oct 2015 21:51:15 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.10.1510312142560.10406@chino.kir.corp.google.com> (raw)
In-Reply-To: <1446188504-28023-1-git-send-email-minchan-DgEjT+Ai2ygdnm+yROfE0A@public.gmane.org>
On Fri, 30 Oct 2015, Minchan Kim wrote:
> MADV_FREE is on linux-next so long time. The reason was two, I think.
>
> 1. MADV_FREE code on reclaim path was really mess.
>
> 2. Andrew really want to see voice of userland people who want to use
> the syscall.
>
> A few month ago, Daniel Micay(jemalloc active contributor) requested me
> to make progress upstreaming but I was busy at that time so it took
> so long time for me to revist the code and finally, I clean it up the
> mess recently so it solves the #2 issue.
>
> As well, Daniel and Jason(jemalloc maintainer) requested it to Andrew
> again recently and they said it would be great to have even though
> it has swap dependency now so Andrew decided he will do that for v4.4.
>
First, thanks very much for refreshing the patchset and reposting after a
series of changes have been periodically added to -mm, it makes it much
easier.
For tcmalloc, we can do some things in the allocator itself to increase
the amount of memory backed by thp. Specifically, we can prefer to
release Spans to pageblocks that are already not backed by thp so there is
no additional split on each scavenge. This is somewhat easy if all memory
is organized into hugepage-aligned pageblocks in the allocator itself.
Second, we can prefer to release Spans of longer length on each scavenge
so we can delay scavenging for as long as possible in a hope we can find
more pages to coalesce. Third, we can discount refaulted released memory
from the scavenging period.
That significantly improves the amount of memory backed by thp for
tcmalloc. The problem, however, is that tcmalloc uses MADV_DONTNEED to
release memory to the system and MADV_FREE wouldn't help at all in a
swapless environment.
To combat that, I've proposed a new MADV bit that simply caches the
ranges freed by the allocator per vma and places them on both a per-vma
and per-memcg list. During reclaim, this list is iterated and ptes are
freed after thp split period to the normal directed reclaim. Without
memory pressure, this backs 100% of the heap with thp with a relatively
lightweight kernel change (the majority is vma manipulation on split) and
a couple line change to tcmalloc. When pulling memory from the returned
freelists, the memory that we have MADV_DONTNEED'd, we need to use another
MADV bit to remove it from this cache, so there is a second madvise(2)
syscall involved but the freeing call is much less expensive since there
is no pagetable walk without memory pressure or synchronous thp split.
I've been looking at MADV_FREE to see if there is common ground that could
be shared, but perhaps it's just easier to ask what your proposed strategy
is so that tcmalloc users, especially those in swapless environments,
would benefit from any of your work?
WARNING: multiple messages have this Message-ID (diff)
From: David Rientjes <rientjes@google.com>
To: Minchan Kim <minchan@kernel.org>
Cc: Andrew Morton <akpm@linux-foundation.org>,
linux-kernel@vger.kernel.org, linux-mm@kvack.org,
Michael Kerrisk <mtk.manpages@gmail.com>,
linux-api@vger.kernel.org, Hugh Dickins <hughd@google.com>,
Johannes Weiner <hannes@cmpxchg.org>,
zhangyanfei@cn.fujitsu.com, Rik van Riel <riel@redhat.com>,
Mel Gorman <mgorman@suse.de>,
KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>,
Jason Evans <je@fb.com>, Daniel Micay <danielmicay@gmail.com>,
"Kirill A. Shutemov" <kirill@shutemov.name>,
Michal Hocko <mhocko@suse.cz>,
yalin.wang2010@gmail.com, Shaohua Li <shli@kernel.org>
Subject: Re: [PATCH 0/8] MADV_FREE support
Date: Sat, 31 Oct 2015 21:51:15 -0700 (PDT) [thread overview]
Message-ID: <alpine.DEB.2.10.1510312142560.10406@chino.kir.corp.google.com> (raw)
In-Reply-To: <1446188504-28023-1-git-send-email-minchan@kernel.org>
On Fri, 30 Oct 2015, Minchan Kim wrote:
> MADV_FREE is on linux-next so long time. The reason was two, I think.
>
> 1. MADV_FREE code on reclaim path was really mess.
>
> 2. Andrew really want to see voice of userland people who want to use
> the syscall.
>
> A few month ago, Daniel Micay(jemalloc active contributor) requested me
> to make progress upstreaming but I was busy at that time so it took
> so long time for me to revist the code and finally, I clean it up the
> mess recently so it solves the #2 issue.
>
> As well, Daniel and Jason(jemalloc maintainer) requested it to Andrew
> again recently and they said it would be great to have even though
> it has swap dependency now so Andrew decided he will do that for v4.4.
>
First, thanks very much for refreshing the patchset and reposting after a
series of changes have been periodically added to -mm, it makes it much
easier.
For tcmalloc, we can do some things in the allocator itself to increase
the amount of memory backed by thp. Specifically, we can prefer to
release Spans to pageblocks that are already not backed by thp so there is
no additional split on each scavenge. This is somewhat easy if all memory
is organized into hugepage-aligned pageblocks in the allocator itself.
Second, we can prefer to release Spans of longer length on each scavenge
so we can delay scavenging for as long as possible in a hope we can find
more pages to coalesce. Third, we can discount refaulted released memory
from the scavenging period.
That significantly improves the amount of memory backed by thp for
tcmalloc. The problem, however, is that tcmalloc uses MADV_DONTNEED to
release memory to the system and MADV_FREE wouldn't help at all in a
swapless environment.
To combat that, I've proposed a new MADV bit that simply caches the
ranges freed by the allocator per vma and places them on both a per-vma
and per-memcg list. During reclaim, this list is iterated and ptes are
freed after thp split period to the normal directed reclaim. Without
memory pressure, this backs 100% of the heap with thp with a relatively
lightweight kernel change (the majority is vma manipulation on split) and
a couple line change to tcmalloc. When pulling memory from the returned
freelists, the memory that we have MADV_DONTNEED'd, we need to use another
MADV bit to remove it from this cache, so there is a second madvise(2)
syscall involved but the freeing call is much less expensive since there
is no pagetable walk without memory pressure or synchronous thp split.
I've been looking at MADV_FREE to see if there is common ground that could
be shared, but perhaps it's just easier to ask what your proposed strategy
is so that tcmalloc users, especially those in swapless environments,
would benefit from any of your work?
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2015-11-01 4:51 UTC|newest]
Thread overview: 97+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-10-30 7:01 [PATCH 0/8] MADV_FREE support Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 7:01 ` [PATCH 1/8] mm: support madvise(MADV_FREE) Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 16:49 ` Shaohua Li
2015-10-30 16:49 ` Shaohua Li
2015-11-03 0:10 ` Minchan Kim
2015-11-03 0:10 ` Minchan Kim
2015-11-03 0:10 ` Minchan Kim
2015-10-30 7:01 ` [PATCH 2/8] mm: define MADV_FREE for some arches Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 7:01 ` [PATCH 3/8] arch: uapi: asm: mman.h: Let MADV_FREE have same value for all architectures Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-11-02 0:08 ` Hugh Dickins
2015-11-02 0:08 ` Hugh Dickins
2015-11-02 0:08 ` Hugh Dickins
2015-11-02 0:08 ` Hugh Dickins
2015-11-03 2:32 ` Minchan Kim
2015-11-03 2:32 ` Minchan Kim
2015-11-03 2:32 ` Minchan Kim
2015-11-03 2:32 ` Minchan Kim
2015-11-03 2:36 ` Minchan Kim
2015-11-03 2:36 ` Minchan Kim
2015-11-03 2:36 ` Minchan Kim
2015-11-03 2:36 ` Minchan Kim
2015-11-03 3:36 ` David Miller
2015-11-03 3:36 ` David Miller
2015-11-03 3:36 ` David Miller
2015-11-03 4:31 ` Minchan Kim
2015-11-03 4:31 ` Minchan Kim
2015-11-03 4:31 ` Minchan Kim
2015-10-30 7:01 ` [PATCH 4/8] mm: free swp_entry in madvise_free Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 12:28 ` Michal Hocko
2015-10-30 12:28 ` Michal Hocko
2015-11-03 0:53 ` Minchan Kim
2015-11-03 0:53 ` Minchan Kim
2015-11-03 0:53 ` Minchan Kim
2015-10-30 7:01 ` [PATCH 5/8] mm: move lazily freed pages to inactive list Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 17:22 ` Shaohua Li
2015-10-30 17:22 ` Shaohua Li
2015-10-30 17:22 ` Shaohua Li
2015-11-03 0:52 ` Minchan Kim
2015-11-03 0:52 ` Minchan Kim
2015-11-03 0:52 ` Minchan Kim
2015-11-04 8:15 ` Michal Hocko
2015-11-04 8:15 ` Michal Hocko
2015-11-04 17:53 ` Shaohua Li
2015-11-04 17:53 ` Shaohua Li
2015-11-04 17:53 ` Shaohua Li
2015-11-04 18:20 ` Shaohua Li
2015-11-04 18:20 ` Shaohua Li
2015-11-05 1:11 ` Minchan Kim
2015-11-05 1:11 ` Minchan Kim
2015-11-05 1:03 ` Minchan Kim
2015-11-05 1:03 ` Minchan Kim
2015-11-05 1:03 ` Minchan Kim
2015-11-04 20:55 ` Johannes Weiner
2015-11-04 20:55 ` Johannes Weiner
2015-11-04 20:55 ` Johannes Weiner
2015-11-04 21:48 ` Daniel Micay
2015-11-04 21:48 ` Daniel Micay
2015-11-04 22:55 ` Johannes Weiner
2015-11-04 22:55 ` Johannes Weiner
2015-11-04 22:55 ` Johannes Weiner
2015-11-04 23:36 ` Daniel Micay
2015-11-04 23:49 ` Daniel Micay
2015-11-04 23:49 ` Daniel Micay
2015-10-30 7:01 ` [PATCH 6/8] mm: lru_deactivate_fn should clear PG_referenced Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 12:47 ` Michal Hocko
2015-10-30 12:47 ` Michal Hocko
2015-10-30 12:47 ` Michal Hocko
2015-11-03 1:10 ` Minchan Kim
2015-11-03 1:10 ` Minchan Kim
2015-11-04 8:22 ` Michal Hocko
2015-11-04 8:22 ` Michal Hocko
2015-11-04 8:22 ` Michal Hocko
2015-10-30 7:01 ` [PATCH 7/8] mm: clear PG_dirty to mark page freeable Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-10-30 12:55 ` Michal Hocko
2015-10-30 12:55 ` Michal Hocko
2015-10-30 12:55 ` Michal Hocko
2015-10-30 7:01 ` [PATCH 8/8] mm: mark stable page dirty in KSM Minchan Kim
2015-10-30 7:01 ` Minchan Kim
2015-11-01 4:51 ` David Rientjes [this message]
2015-11-01 4:51 ` [PATCH 0/8] MADV_FREE support David Rientjes
2015-11-01 4:51 ` David Rientjes
2015-11-01 6:29 ` Daniel Micay
2015-11-03 2:23 ` Minchan Kim
2015-11-03 2:23 ` Minchan Kim
2015-11-03 2:23 ` Minchan Kim
2015-11-04 20:19 ` David Rientjes
2015-11-04 20:19 ` David Rientjes
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=alpine.DEB.2.10.1510312142560.10406@chino.kir.corp.google.com \
--to=rientjes@google.com \
--cc=akpm@linux-foundation.org \
--cc=danielmicay@gmail.com \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=je@fb.com \
--cc=kirill@shutemov.name \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-api@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.cz \
--cc=minchan@kernel.org \
--cc=mtk.manpages@gmail.com \
--cc=riel@redhat.com \
--cc=shli@kernel.org \
--cc=yalin.wang2010@gmail.com \
--cc=zhangyanfei@cn.fujitsu.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.