Re: [Ksummit-discuss] [TECH TOPIC] Memory thrashing, was Re: Self nomination

From: Dave Hansen <dave@sr71.net>
To: James Bottomley <James.Bottomley@HansenPartnership.com>,
	Johannes Weiner <hannes@cmpxchg.org>
Cc: "Kleen, Andi" <andi.kleen@intel.com>,
	ksummit-discuss@lists.linuxfoundation.org
Subject: Re: [Ksummit-discuss] [TECH TOPIC] Memory thrashing, was Re: Self nomination
Date: Mon, 1 Aug 2016 12:51:44 -0700	[thread overview]
Message-ID: <579FA850.3020703@sr71.net> (raw)
In-Reply-To: <1470069183.18751.35.camel@HansenPartnership.com>

On 08/01/2016 09:33 AM, James Bottomley wrote:
>> All the LRU scanning is expensive and doesn't scale particularly
>> well, and there are some situations where we should be willing to
>> give up some of the precision of the current LRU in order to increase
>> the throughput of reclaim in general.
> 
> Would some type of hinting mechanism work (say via madvise)? 
>  MADV_DONTNEED may be good enough, but we could really do with
> MADV_SWAP_OUT_NOW to indicate objects we really don't want.  I suppose
> I can lose all my credibility by saying this would be the JVM: it knows
> roughly the expected lifetime and access patterns and is well qualified
> to mark objects as infrequently enough accessed to reside on swap.

I don't think MADV_DONTNEED is a good fit because it is destructive.  It
does seem like we are missing a true companion to MADV_WILLNEED which
would give memory a push in the direction of being swapped out.

But I don't think it's too crazy to expect apps to participate.  They
certainly have the potential to know more about their data than the
kernel does, and things like GPUs are already pretty actively optimizing
by moving memory around.

> I suppose another question is do we still want all of this to be page
> based?  We moved to extents in filesystems a while ago, wouldn't some
> extent based LRU mechanism be cheaper ... unfortunately it means
> something has to try to come up with an idea of what an extent means (I
> suspect it would be a bunch of virtually contiguous pages which have
> the same expected LRU properties, but I'm thinking from the application
> centric viewpoint).

One part of this (certainly not the _only_ one) is expanding where
transparent huge pages can be used.  That's one extent definition that's
relatively easy to agree on.

Past that, there are lots of things we can try (including something like
you've suggested), but I don't think anybody knows what will work yet.
There is no shortage of ideas.