From: Andrew Morton <akpm@linux-foundation.org>
To: Linus Torvalds <torvalds@linux-foundation.org>
Cc: Joonsoo Kim <iamjoonsoo.kim@lge.com>,
Al Viro <viro@zeniv.linux.org.uk>,
Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: Re: get_vmalloc_info() and /proc/meminfo insanely expensive
Date: Wed, 12 Aug 2015 21:00:27 -0700 [thread overview]
Message-ID: <20150812210027.88dfcf90.akpm@linux-foundation.org> (raw)
In-Reply-To: <CA+55aFzUDiGRY_CZz0hON9AxkD1Fw_B-YSLiHH8tt_3rkKUrzw@mail.gmail.com>
On Wed, 12 Aug 2015 20:29:34 -0700 Linus Torvalds <torvalds@linux-foundation.org> wrote:
> I just did some profiling of a simple "make test" in the git repo, and
> was surprised by the top kernel offender: get_vmalloc_info() showed up
> at roughly 4% cpu use.
>
> It turns out that bash ends up reading /proc/meminfo on every single
> activation, and "make test" is basically just running a huge
> collection of shell scripts. You can verify by just doing
>
> strace -o trace sh -c "echo"
>
> to see what bash does on your system. I suspect it's actually glibc,
> because a quick google finds the function "get_phys_pages()" that just
> looks at the "MemTotal" line (or possibly get_avphys_pageslooks at the
> MemFree" line).
And bash surely isn't interested in vmalloc stats. Putting all these
things in the same file wasn't the smartest thing we've ever done.
> Ok, so bash is insane for caring so deeply that it does this
> regardless of anything else. But what else is new - user space does
> odd things. It's like a truism.
>
> My gut feel for this is that we should just rate-limit this and cache
> the vmalloc information for a fraction of a second or something. Maybe
> we could expose total memory sizes in some more efficient format, but
> it's not like existing binaries will magically de-crapify themselves,
> so just speeding up meminfo sounds like a good thing.
>
> Maybe we could even cache the whole seqfile buffer - Al? How painful
> would something like that be? Although from the profiles, it's really
> just the vmalloc info gathering that shows up as actually wasting CPU
> cycles..
>
Do your /proc/meminfo vmalloc numbers actually change during that build?
Mine don't. Perhaps we can cache the most recent vmalloc_info and
invalidate that cache whenever someone does a vmalloc/vfree/etc.
next prev parent reply other threads:[~2015-08-13 4:00 UTC|newest]
Thread overview: 12+ messages / expand[flat|nested] mbox.gz Atom feed top
2015-08-13 3:29 get_vmalloc_info() and /proc/meminfo insanely expensive Linus Torvalds
2015-08-13 4:00 ` Andrew Morton [this message]
2015-08-13 5:52 ` Linus Torvalds
2015-08-13 7:42 ` Rasmus Villemoes
2015-08-13 16:50 ` Linus Torvalds
2015-08-13 18:32 ` Linus Torvalds
2015-08-13 18:52 ` Linus Torvalds
2015-08-13 19:50 ` David Rientjes
2015-08-13 20:04 ` Linus Torvalds
2015-08-13 21:15 ` Tony Luck
2015-08-13 22:56 ` Linus Torvalds
2015-08-13 20:26 ` Andrew Morton
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20150812210027.88dfcf90.akpm@linux-foundation.org \
--to=akpm@linux-foundation.org \
--cc=iamjoonsoo.kim@lge.com \
--cc=linux-kernel@vger.kernel.org \
--cc=torvalds@linux-foundation.org \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).