Re: [RFC PATCH] mm: madvise: Ignore repeated MADV_DONTNEED hints

[Andrew Morton <akpm@linux-foundation.org>]

On Mon, 2 Feb 2015 16:55:25 +0000 Mel Gorman <mgorman@suse.de> wrote:

> glibc malloc changed behaviour in glibc 2.10 to have per-thread arenas
> instead of creating new areans if the existing ones were contended.
> The decision appears to have been made so the allocator scales better but the
> downside is that madvise(MADV_DONTNEED) is now called for these per-thread
> areans during free. This tears down pages that would have previously
> remained. There is nothing wrong with this decision from a functional point
> of view but any threaded application that frequently allocates/frees the
> same-sized region is going to incur the full teardown and refault costs.

MADV_DONTNEED has been there for many years.  How could this problem
not have been noticed during glibc 2.10 development/testing?  Is there
some more recent kernel change which is triggering this?

> This patch identifies when a thread is frequently calling MADV_DONTNEED
> on the same region of memory and starts ignoring the hint.

That's pretty nasty-looking :(

And presumably there are all sorts of behaviours which will still
trigger the problem but which will avoid the start/end equality test in
ignore_madvise_hint()?

Really, this is a glibc problem and only a glibc problem. 
MADV_DONTNEED is unavoidably expensive and glibc is calling
MADV_DONTNEED for a region which it *does* need.  Is there something
preventing this from being addressed within glibc?