Linux-mm Archive on lore.kernel.org
 help / color / Atom feed
From: Vlastimil Babka <vbabka@suse.cz>
To: Andrew Morton <akpm@linux-foundation.org>
Cc: Michal Hocko <mhocko@kernel.org>,
	Mel Gorman <mgorman@techsingularity.net>,
	Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	David Rientjes <rientjes@google.com>,
	Rik van Riel <riel@redhat.com>,
	linux-mm@kvack.org, linux-kernel@vger.kernel.org,
	Vlastimil Babka <vbabka@suse.cz>
Subject: [PATCH v6 00/11] make direct compaction more deterministic
Date: Wed, 10 Aug 2016 11:12:15 +0200
Message-ID: <20160810091226.6709-1-vbabka@suse.cz> (raw)

Changes since v3
* The first part with cleanups (v4, v5) went separately to 4.8-rc1
* Rebased to 4.8-rc1
* Patch 1 - don't touch cached pfns in whole-zone compaction (Joonsoo)
* New patches 2 and 3 in response to Joonsoo pointing out missing adustments
  to watermark checks in patch 7 - turns out we can remove those watermark
  checks altogether.
* Patch 6 made less aggressive to avoid premature OOM (Joonsoo)

This is mostly a followup to Michal's oom detection rework, which highlighted
the need for direct compaction to provide better feedback in reclaim/compaction
loop, so that it can reliably recognize when compaction cannot make further
progress, and allocation should invoke OOM killer or fail. We've discussed
this at LSF/MM [1] where I proposed expanding the async/sync migration mode
used in compaction to more general "priorities". This patchset adds one new
priority that just overrides all the heuristics and makes compaction fully
scan all zones. I don't currently think that we need more fine-grained
priorities, but we'll see. Other than that there's some smaller fixes and
cleanups, mainly related to the THP-specific hacks.

I've tested this with stress-highalloc in GFP_KERNEL order-4 and
THP-like order-9 scenarios. There's some improvement for compaction stats
for the order-4, which is likely due to the better watermarks handling.
In the previous version I reported mostly noise wrt compaction stats, and
decreased direct reclaim - now the reclaim is without difference. I believe
this is due to the less aggressive compaction priority increase in patch 6.

"before" is a mmotm tree prior to 4.7 release plus the first part of the
series that was sent and merged separately

                                    before        after
order-4:

Compaction stalls                    27216       30759
Compaction success                   19598       25475
Compaction failures                   7617        5283
Page migrate success                370510      464919
Page migrate failure                 25712       27987
Compaction pages isolated           849601     1041581
Compaction migrate scanned       143146541   101084990
Compaction free scanned          208355124   144863510
Compaction cost                       1403        1210

order-9:

Compaction stalls                     7311        7401
Compaction success                    1634        1683
Compaction failures                   5677        5718
Page migrate success                194657      183988
Page migrate failure                  4753        4170
Compaction pages isolated           498790      456130
Compaction migrate scanned          565371      524174
Compaction free scanned            4230296     4250744
Compaction cost                        215         203

[1] https://lwn.net/Articles/684611/

Vlastimil Babka (11):
  mm, compaction: make whole_zone flag ignore cached scanner positions
  mm, compaction: cleanup unused functions
  mm, compaction: rename COMPACT_PARTIAL to COMPACT_SUCCESS
  mm, compaction: don't recheck watermarks after COMPACT_SUCCESS
  mm, compaction: add the ultimate direct compaction priority
  mm, compaction: more reliably increase direct compaction priority
  mm, compaction: use correct watermark when checking compaction success
  mm, compaction: create compact_gap wrapper
  mm, compaction: use proper alloc_flags in __compaction_suitable()
  mm, compaction: require only min watermarks for non-costly orders
  mm, vmscan: make compaction_ready() more accurate and readable

 include/linux/compaction.h        |  32 +++++---
 include/trace/events/compaction.h |   2 +-
 mm/compaction.c                   | 154 +++++++++++++++++---------------------
 mm/internal.h                     |   2 +-
 mm/page_alloc.c                   |  22 +++---
 mm/vmscan.c                       |  49 ++++++------
 6 files changed, 128 insertions(+), 133 deletions(-)

-- 
2.9.2

--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org.  For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>

             reply index

Thread overview: 32+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2016-08-10  9:12 Vlastimil Babka [this message]
2016-08-10  9:12 ` [PATCH v6 01/11] mm, compaction: make whole_zone flag ignore cached scanner positions Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 02/11] mm, compaction: cleanup unused functions Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 03/11] mm, compaction: rename COMPACT_PARTIAL to COMPACT_SUCCESS Vlastimil Babka
2016-08-18  9:01   ` Michal Hocko
2016-08-10  9:12 ` [PATCH v6 04/11] mm, compaction: don't recheck watermarks after COMPACT_SUCCESS Vlastimil Babka
2016-08-16  6:12   ` Joonsoo Kim
2016-08-16  6:11     ` Vlastimil Babka
2016-08-18 11:59     ` Vlastimil Babka
2016-08-18  9:03   ` Michal Hocko
2016-08-10  9:12 ` [PATCH v6 05/11] mm, compaction: add the ultimate direct compaction priority Vlastimil Babka
2016-08-16  5:58   ` Joonsoo Kim
2016-08-18 12:23     ` Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 06/11] mm, compaction: more reliably increase " Vlastimil Babka
2016-08-16  6:07   ` Joonsoo Kim
2016-08-16  6:31     ` Vlastimil Babka
2016-08-18  9:10   ` Michal Hocko
2016-08-18  9:44     ` Vlastimil Babka
2016-08-18  9:48       ` Michal Hocko
2016-08-10  9:12 ` [PATCH v6 07/11] mm, compaction: use correct watermark when checking compaction success Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 08/11] mm, compaction: create compact_gap wrapper Vlastimil Babka
2016-08-16  6:15   ` Joonsoo Kim
2016-08-16  6:15     ` Vlastimil Babka
2016-08-16  6:41       ` Joonsoo Kim
2016-08-18 12:13         ` Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 09/11] mm, compaction: use proper alloc_flags in __compaction_suitable() Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 10/11] mm, compaction: require only min watermarks for non-costly orders Vlastimil Babka
2016-08-16  6:16   ` Joonsoo Kim
2016-08-16  6:36     ` Vlastimil Babka
2016-08-16  6:46       ` Joonsoo Kim
2016-08-18 12:20         ` Vlastimil Babka
2016-08-10  9:12 ` [PATCH v6 11/11] mm, vmscan: make compaction_ready() more accurate and readable Vlastimil Babka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20160810091226.6709-1-vbabka@suse.cz \
    --to=vbabka@suse.cz \
    --cc=akpm@linux-foundation.org \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=linux-mm@kvack.org \
    --cc=mgorman@techsingularity.net \
    --cc=mhocko@kernel.org \
    --cc=riel@redhat.com \
    --cc=rientjes@google.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link

Linux-mm Archive on lore.kernel.org

Archives are clonable:
	git clone --mirror https://lore.kernel.org/linux-mm/0 linux-mm/git/0.git

	# If you have public-inbox 1.1+ installed, you may
	# initialize and index your mirror using the following commands:
	public-inbox-init -V2 linux-mm linux-mm/ https://lore.kernel.org/linux-mm \
		linux-mm@kvack.org
	public-inbox-index linux-mm

Example config snippet for mirrors

Newsgroup available over NNTP:
	nntp://nntp.lore.kernel.org/org.kvack.linux-mm


AGPL code for this site: git clone https://public-inbox.org/public-inbox.git