From: Mel Gorman <mgorman@techsingularity.net>
To: Debabrata Banerjee <dbavatar@gmail.com>
Cc: Roman Gushchin <guro@fb.com>,
linux-mm@kvack.org, Andrew Morton <akpm@linux-foundation.org>,
Johannes Weiner <hannes@cmpxchg.org>,
Michal Hocko <mhocko@suse.com>,
Vladimir Davydov <vdavydov.dev@gmail.com>,
Rik van Riel <riel@redhat.com>,
kernel-team@fb.com,
"linux-kernel@vger.kernel.org" <linux-kernel@vger.kernel.org>
Subject: Re: [PATCH] mm: make allocation counters per-order
Date: Thu, 6 Jul 2017 17:43:12 +0100 [thread overview]
Message-ID: <20170706164312.4nnjsrpzv5vbtbkm@techsingularity.net> (raw)
In-Reply-To: <CAATkVEzuFq5UWasE87Eo_F4aQxkuYWqSGJh5bBnieC=686NyqA@mail.gmail.com>
On Thu, Jul 06, 2017 at 12:12:47PM -0400, Debabrata Banerjee wrote:
> On Thu, Jul 6, 2017 at 11:51 AM, Mel Gorman <mgorman@techsingularity.net> wrote:
> >
> > These counters do not actually help you solve that particular problem.
> > Knowing how many allocations happened since the system booted doesn't tell
> > you much about how many failed or why they failed. You don't even know
> > what frequency they occured at unless you monitor it constantly so you're
> > back to square one whether this information is available from proc or not.
> > There even is a tracepoint that can be used to track information related
> > to events that degrade fragmentation (trace_mm_page_alloc_extfrag) although
> > the primary thing it tells you is that "the probability that an allocation
> > will fail due to fragmentation in the future is potentially higher".
>
> I agree these counters don't have enough information, but there a
> start to a first order approximation of the current state of memory.
That incurs a universal cost on the off-chance of debugging and ultimately
the debugging is only useful in combination with developing kernel patches
in which case it could be behind a kconfig option.
> buddyinfo and pagetypeinfo basically show no information now, because
They can be used to calculate a fragmentation index at a given point in
time. Admittedly, building a bigger picture requires a full scan of memory
(and that's what was required when fragmentation avoidance was first
being implemented).
> they only involve the small amount of free memory under the watermark
> and all our machines are in this state. As second order approximation,
> it would be nice to be able to get answers like: "There are
> reclaimable high order allocations of at least this order" and "None
> of this order allocation can become available due to unmovable and
> unreclaimable allocations"
Which this patch doesn't provide as what you are looking for requires
a full scan of memory to determine. I've done it in the past using a
severe abuse of systemtap to load a module that scans all of memory with
a variation of PAGE_OWNER to identify stack traces of pages that "don't
belonw" within a pageblock.
Even *with* that information, your options for tuning an unmodified kernel
are basically limited to increasing min_free_kbytes, altering THP's level
of aggression when compacting or brute forcing with either drop_caches,
compact_node or both. All other options after that require kernel patches
-- altering annotations, altering fallback mechanisms, altering compaction,
improving support for pages that can be migrated etc.
--
Mel Gorman
SUSE Labs
--
To unsubscribe, send a message with 'unsubscribe linux-mm' in
the body to majordomo@kvack.org. For more info on Linux MM,
see: http://www.linux-mm.org/ .
Don't email: <a href=mailto:"dont@kvack.org"> email@kvack.org </a>
next prev parent reply other threads:[~2017-07-06 16:43 UTC|newest]
Thread overview: 18+ messages / expand[flat|nested] mbox.gz Atom feed top
2017-07-06 13:04 [PATCH] mm: make allocation counters per-order Roman Gushchin
2017-07-06 13:19 ` Mel Gorman
2017-07-06 14:46 ` Roman Gushchin
2017-07-06 15:47 ` Mel Gorman
2017-07-06 16:43 ` Roman Gushchin
2017-07-06 17:16 ` Mel Gorman
2017-07-06 18:00 ` Debabrata Banerjee
2017-07-06 20:02 ` Mel Gorman
2017-07-16 13:27 ` Roman Gushchin
2017-07-16 13:29 ` Roman Gushchin
2017-07-16 13:55 ` [v2] " kbuild test robot
2017-07-06 14:54 ` [PATCH] " Debabrata Banerjee
2017-07-06 15:51 ` Mel Gorman
2017-07-06 16:12 ` Debabrata Banerjee
2017-07-06 16:43 ` Mel Gorman [this message]
2017-07-06 15:02 ` Christoph Lameter
2017-07-07 1:54 ` kbuild test robot
2017-07-07 6:06 ` kbuild test robot
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20170706164312.4nnjsrpzv5vbtbkm@techsingularity.net \
--to=mgorman@techsingularity.net \
--cc=akpm@linux-foundation.org \
--cc=dbavatar@gmail.com \
--cc=guro@fb.com \
--cc=hannes@cmpxchg.org \
--cc=kernel-team@fb.com \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mhocko@suse.com \
--cc=riel@redhat.com \
--cc=vdavydov.dev@gmail.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).