[RFC][PATCH 00/12] mm: tweak page cache migration

[RFC][PATCH 00/12] mm: tweak page cache migration

[Dave Hansen]

First of all, I think this little slice of code is a bit
under-documented.  Perhaps this will help clarify things.

I'm pretty confident the page_count() check in the first
patch is right, which is why I removed it outright.  The
xas_load() check is a bit murkier, so I just left a
warning in for it.

Cc: Nicholas Piggin <npiggin@gmail.com>
Cc: Andrew Morton <akpm@linux-foundation.org>
Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
Cc: Yang Shi <yang.shi@linux.alibaba.com>
Cc: linux-mm@kvack.org
Cc: linux-kernel@vger.kernel.org

Re: [RFC][PATCH 00/12] mm: tweak page cache migration

[Dave Hansen]

Ugh, sorry about that.  I fat-fingered the wrong cover letter!

This should have been

Subject: [v4] Migrate Pages in lieu of discard

--

Changes since (automigrate-20200818):
 * Fall back to normal reclaim when demotion fails

The full series is also available here:

	https://github.com/hansendc/linux/tree/automigrate-20200818

I really just want folks to look at:

	[RFC][PATCH 08/12] mm/migrate: demote pages during reclaim

I've reworked that so that it can both use the high-level migration
API, and fall back to normal reclaim if migration fails.  I think
that gives us the best of both worlds.

I'm posting the series in case folks want to run the whole thing.

--

We're starting to see systems with more and more kinds of memory such
as Intel's implementation of persistent memory.

Let's say you have a system with some DRAM and some persistent memory.
Today, once DRAM fills up, reclaim will start and some of the DRAM
contents will be thrown out.  Allocations will, at some point, start
falling over to the slower persistent memory.

That has two nasty properties.  First, the newer allocations can end
up in the slower persistent memory.  Second, reclaimed data in DRAM
are just discarded even if there are gobs of space in persistent
memory that could be used.

This set implements a solution to these problems.  At the end of the
reclaim process in shrink_page_list() just before the last page
refcount is dropped, the page is migrated to persistent memory instead
of being dropped.

While I've talked about a DRAM/PMEM pairing, this approach would
function in any environment where memory tiers exist.

This is not perfect.  It "strands" pages in slower memory and never
brings them back to fast DRAM.  Other things need to be built to
promote hot pages back to DRAM.

== Open Issues ==

 * For cpusets and memory policies that restrict allocations
   to PMEM, is it OK to demote to PMEM?  Do we need a cgroup-
   level API to opt-in or opt-out of these migrations?

Cc: Yang Shi <yang.shi@linux.alibaba.com>
Cc: David Rientjes <rientjes@google.com>
Cc: Huang Ying <ying.huang@intel.com>
Cc: Dan Williams <dan.j.williams@intel.com>

--

Changes since (https://lwn.net/Articles/824830/):
 * Use higher-level migrate_pages() API approach from Yang Shi's
   earlier patches.
 * made sure to actually check node_reclaim_mode's new bit
 * disabled migration entirely before introducing RECLAIM_MIGRATE
 * Replace GFP_NOWAIT with explicit __GFP_KSWAPD_RECLAIM and
   comment why we want that.
 * Comment on effects of that keep multiple source nodes from
   sharing target nodes

Re: [RFC][PATCH 00/12] mm: tweak page cache migration

[Michal Hocko]

Am I the only one missing patch 1-5? lore.k.o doesn't seem to link them
under this message id either.

On Tue 06-10-20 13:51:03, Dave Hansen wrote:
> First of all, I think this little slice of code is a bit
> under-documented.  Perhaps this will help clarify things.
> 
> I'm pretty confident the page_count() check in the first
> patch is right, which is why I removed it outright.  The
> xas_load() check is a bit murkier, so I just left a
> warning in for it.
> 
> Cc: Nicholas Piggin <npiggin@gmail.com>
> Cc: Andrew Morton <akpm@linux-foundation.org>
> Cc: Matthew Wilcox (Oracle) <willy@infradead.org>
> Cc: Yang Shi <yang.shi@linux.alibaba.com>
> Cc: linux-mm@kvack.org
> Cc: linux-kernel@vger.kernel.org

-- 
Michal Hocko
SUSE Labs

Re: [RFC][PATCH 00/12] mm: tweak page cache migration

[David Hildenbrand]

On 07.10.20 11:52, Michal Hocko wrote:
> Am I the only one missing patch 1-5? lore.k.o doesn't seem to link them
> under this message id either.

I received no patches via linux-mm, only the cover letter and Dave's
reply. (maybe some are still in flight ...)

-- 
Thanks,

David / dhildenb

Re: [RFC][PATCH 00/12] mm: tweak page cache migration

[Yang Shi]

On Wed, Oct 7, 2020 at 2:55 AM David Hildenbrand <david@redhat.com> wrote:
>
> On 07.10.20 11:52, Michal Hocko wrote:
> > Am I the only one missing patch 1-5? lore.k.o doesn't seem to link them
> > under this message id either.
>
> I received no patches via linux-mm, only the cover letter and Dave's
> reply. (maybe some are still in flight ...)

Yes, exactly the same to me, but anyway I saw the patches via linux-kernel.

And, it seems the github series doesn't reflect the changes made by this series.

>
> --
> Thanks,
>
> David / dhildenb
>
>

Re: [RFC][PATCH 00/12] mm: tweak page cache migration

[Dave Hansen]

On 10/7/20 8:52 AM, Yang Shi wrote:
> On Wed, Oct 7, 2020 at 2:55 AM David Hildenbrand <david@redhat.com> wrote:
>> On 07.10.20 11:52, Michal Hocko wrote:
>>> Am I the only one missing patch 1-5? lore.k.o doesn't seem to link them
>>> under this message id either.
>> I received no patches via linux-mm, only the cover letter and Dave's
>> reply. (maybe some are still in flight ...)
> Yes, exactly the same to me, but anyway I saw the patches via linux-kernel.
> 
> And, it seems the github series doesn't reflect the changes made by this series.

Sorry about that.  I'll try to resend the series.

There have been some Intel->list troubles as of late, but I think I'm
probably to blame for this one.