Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: Michal Hocko <mhocko@kernel.org>
To: lsf-pc@lists.linux-foundation.org
Cc: linux-mm@kvack.org, Vlastimil Babka <vbabka@suse.cz>,
	Mel Gorman <mgorman@suse.de>,
	David Rientjes <rientjes@google.com>
Subject: [LSF/MM TOPIC] wmark based pro-active compaction
Date: Fri, 30 Dec 2016 14:14:12 +0100
Message-ID: <20161230131412.GI13301@dhcp22.suse.cz> (raw)

Hi,
I didn't originally want to send this proposal because Vlastimil is
planning to do some work in this area so I've expected him to send
something similar. But the recent discussion about the THP defrag
options pushed me to send out my thoughts.

So what is the problem? The demand for high order pages is growing and
that seems to be the general trend. The problem is that while they can
bring performance benefit they can get be really expensive to allocate
especially when we enter the direct compaction. So we really want to
prevent from expensive path and defer as much as possible to the
background. A huge step forward was kcompactd introduced by Vlastimil.
We are still not there yet though, because it might be already quite
late when we wakeup_kcompactd(). The memory might be already fragmented
when we hit there. Moreover we do not have any way to actually tell
which orders we do care about.

Therefore I believe we need a watermark based pro-active compaction
which would keep the background compaction busy as long as we have
less pages of the configured order. kcompactd should wake up
periodically, I think, and check for the status so that we can catch
the fragmentation before we get low on memory.
The interface could look something like:
/proc/sys/vm/compact_wmark
time_period order count

There are many details that would have to be solved of course - e.g. do
not burn cycles pointlessly when we know that no further progress can be
made etc... but in principle the idea show work.
-- 
Michal Hocko
SUSE Labs

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

             reply index

Thread overview: 8+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-12-30 13:14 Michal Hocko [this message]
2016-12-30 14:06 ` Mel Gorman
2017-01-05  9:53   ` Vlastimil Babka
2017-01-05 10:27     ` Michal Hocko
2017-01-06  8:57       ` Vlastimil Babka
2017-01-13  7:03     ` Joonsoo Kim
2017-01-19 14:18       ` Vlastimil Babka
2017-03-08 14:56 ` Vlastimil Babka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20161230131412.GI13301@dhcp22.suse.cz \
    --to=mhocko@kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=lsf-pc@lists.linux-foundation.org \
    --cc=mgorman@suse.de \
    --cc=rientjes@google.com \
    --cc=vbabka@suse.cz \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org
	public-inbox-index linux-mm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git