From: Yu Zhao <yuzhao@google.com>
To: Matthew Wilcox <willy@infradead.org>
Cc: linux-mm@kvack.org, Alex Shi <alex.shi@linux.alibaba.com>,
Andrew Morton <akpm@linux-foundation.org>,
Dave Hansen <dave.hansen@linux.intel.com>,
Hillf Danton <hdanton@sina.com>,
Johannes Weiner <hannes@cmpxchg.org>,
Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Mel Gorman <mgorman@suse.de>, Michal Hocko <mhocko@suse.com>,
Roman Gushchin <guro@fb.com>, Vlastimil Babka <vbabka@suse.cz>,
Wei Yang <richard.weiyang@linux.alibaba.com>,
Yang Shi <shy828301@gmail.com>, Ying Huang <ying.huang@intel.com>,
linux-kernel@vger.kernel.org, page-reclaim@google.com
Subject: Re: [PATCH v1 11/14] mm: multigenerational lru: page activation
Date: Tue, 16 Mar 2021 15:29:20 -0600 [thread overview]
Message-ID: <YFEjMCeVv9MARvp3@google.com> (raw)
In-Reply-To: <20210316163437.GB3420@casper.infradead.org>
On Tue, Mar 16, 2021 at 04:34:37PM +0000, Matthew Wilcox wrote:
> On Sat, Mar 13, 2021 at 12:57:44AM -0700, Yu Zhao wrote:
> > In the page fault path, we want to add pages to the per-zone lists
> > index by max_seq as they cannot be evicted without going through
> > the aging first. For anon pages, we rename
> > lru_cache_add_inactive_or_unevictable() to lru_cache_add_page_vma()
> > and add a new parameter, which is set to true in the page fault path,
> > to indicate whether they should be added to the per-zone lists index
> > by max_seq. For page/swap cache, since we cannot differentiate the
> > page fault path from the read ahead path at the time we call
> > lru_cache_add() in add_to_page_cache_lru() and
> > __read_swap_cache_async(), we have to add a new function
> > lru_gen_activate_page(), which is essentially activate_page(), to move
> > pages to the per-zone lists indexed by max_seq at a later time.
> > Hopefully we would find pages we want to activate in lru_pvecs.lru_add
> > and simply set PageActive() on them without having to actually move
> > them.
> >
> > In the reclaim path, pages mapped around a referenced PTE may also
> > have been referenced due to spatial locality. We add a new function
> > lru_gen_scan_around() to scan the vicinity of such a PTE.
> >
> > In addition, we add a new function page_is_active() to tell whether a
> > page is active. We cannot use PageActive() because it is only set on
> > active pages while they are not on multigenerational lru. It is
> > cleared while pages are on multigenerational lru, in order to spare
> > the aging the trouble of clearing it when an active generation becomes
> > inactive. Internally, page_is_active() compares the generation number
> > of a page with max_seq and max_seq-1, which are active generations and
> > protected from the eviction. Other generations, which may or may not
> > exist, are inactive.
>
> If we go with this multi-LRU approach, it feels like PageActive and
> PageInactive should go away as tests. We should have a LRU field in
> the page flags with some special values:
>
> - Not managed through LRU list
> - Not currently on any LRU list
> - Unevictable
> - Active list 1
> - Active list 2
> - ...
> - Active list 5
>
> Now you don't need any extra bits in the page flags. Or if you want to
> have 13 lists instead of 5, you can use just one extra bit. I'm not
> quite sure whether it makes sense to have that many lists, so I need
> to try to understand that better.
Yes, and this would be a lot cleaner. PG_{lru,unevictable,active,
referenced,reclaim,workingset,young,idle} could all go away. Look how
many bits we've added just for page reclaim. Sigh...
> I'd like to echo the comments from others that it'd be nice to split apart
> the multigenerational part of this and the physical scanning part of this.
> It's possible they don't make performance sense without each other,
> but from a review point of view, they seem entirely separate things.
Thanks for noticing. I do plan to see if the page table scanning part
could be better refactored. (I cut some corners by squashing it while
rebasing to latest kernel.)
next prev parent reply other threads:[~2021-03-16 21:29 UTC|newest]
Thread overview: 59+ messages / expand[flat|nested] mbox.gz Atom feed top
2021-03-13 7:57 [PATCH v1 00/14] Multigenerational LRU Yu Zhao
2021-03-13 7:57 ` [PATCH v1 01/14] include/linux/memcontrol.h: do not warn in page_memcg_rcu() if !CONFIG_MEMCG Yu Zhao
2021-03-13 15:09 ` Matthew Wilcox
2021-03-14 7:45 ` Yu Zhao
2021-03-13 7:57 ` [PATCH v1 02/14] include/linux/nodemask.h: define next_memory_node() if !CONFIG_NUMA Yu Zhao
2021-03-13 7:57 ` [PATCH v1 03/14] include/linux/huge_mm.h: define is_huge_zero_pmd() if !CONFIG_TRANSPARENT_HUGEPAGE Yu Zhao
2021-03-13 7:57 ` [PATCH v1 04/14] include/linux/cgroup.h: export cgroup_mutex Yu Zhao
2021-03-13 7:57 ` [PATCH v1 05/14] mm/swap.c: export activate_page() Yu Zhao
2021-03-13 7:57 ` [PATCH v1 06/14] mm, x86: support the access bit on non-leaf PMD entries Yu Zhao
2021-03-14 22:12 ` Zi Yan
2021-03-14 22:51 ` Matthew Wilcox
2021-03-15 0:03 ` Yu Zhao
2021-03-15 0:27 ` Zi Yan
2021-03-15 1:04 ` Yu Zhao
2021-03-14 23:22 ` Dave Hansen
2021-03-15 3:16 ` Yu Zhao
2021-03-13 7:57 ` [PATCH v1 07/14] mm/pagewalk.c: add pud_entry_post() for post-order traversals Yu Zhao
2021-03-13 7:57 ` [PATCH v1 08/14] mm/vmscan.c: refactor shrink_node() Yu Zhao
2021-03-13 7:57 ` [PATCH v1 09/14] mm: multigenerational lru: mm_struct list Yu Zhao
2021-03-15 19:40 ` Rik van Riel
2021-03-16 2:07 ` Huang, Ying
2021-03-16 3:57 ` Yu Zhao
2021-03-16 6:44 ` Huang, Ying
2021-03-16 7:56 ` Yu Zhao
2021-03-17 3:37 ` Huang, Ying
2021-03-17 10:46 ` Yu Zhao
2021-03-22 3:13 ` Huang, Ying
2021-03-22 8:08 ` Yu Zhao
2021-03-24 6:58 ` Huang, Ying
2021-04-10 18:48 ` Yu Zhao
2021-04-13 3:06 ` Huang, Ying
2021-03-13 7:57 ` [PATCH v1 10/14] mm: multigenerational lru: core Yu Zhao
2021-03-15 2:02 ` Andi Kleen
2021-03-15 3:37 ` Yu Zhao
2021-03-13 7:57 ` [PATCH v1 11/14] mm: multigenerational lru: page activation Yu Zhao
2021-03-16 16:34 ` Matthew Wilcox
2021-03-16 21:29 ` Yu Zhao [this message]
2021-03-13 7:57 ` [PATCH v1 12/14] mm: multigenerational lru: user space interface Yu Zhao
2021-03-13 12:23 ` kernel test robot
2021-03-13 7:57 ` [PATCH v1 13/14] mm: multigenerational lru: Kconfig Yu Zhao
2021-03-13 12:53 ` kernel test robot
2021-03-13 13:36 ` kernel test robot
2021-03-13 7:57 ` [PATCH v1 14/14] mm: multigenerational lru: documentation Yu Zhao
2021-03-19 9:31 ` Alex Shi
2021-03-22 6:09 ` Yu Zhao
2021-03-14 22:48 ` [PATCH v1 00/14] Multigenerational LRU Zi Yan
2021-03-15 0:52 ` Yu Zhao
2021-03-15 1:13 ` Hillf Danton
2021-03-15 6:49 ` Yu Zhao
2021-03-15 18:00 ` Dave Hansen
2021-03-16 2:24 ` Yu Zhao
2021-03-16 14:50 ` Dave Hansen
2021-03-16 20:30 ` Yu Zhao
2021-03-16 21:14 ` Dave Hansen
2021-04-10 9:21 ` Yu Zhao
2021-04-13 3:02 ` Huang, Ying
2021-04-13 23:00 ` Yu Zhao
2021-03-15 18:38 ` Yang Shi
2021-03-16 3:38 ` Yu Zhao
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=YFEjMCeVv9MARvp3@google.com \
--to=yuzhao@google.com \
--cc=akpm@linux-foundation.org \
--cc=alex.shi@linux.alibaba.com \
--cc=dave.hansen@linux.intel.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=hdanton@sina.com \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=mhocko@suse.com \
--cc=page-reclaim@google.com \
--cc=richard.weiyang@linux.alibaba.com \
--cc=shy828301@gmail.com \
--cc=vbabka@suse.cz \
--cc=willy@infradead.org \
--cc=ying.huang@intel.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).