linux-kernel.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
From: Linus Torvalds <torvalds@linux-foundation.org>
To: Joonsoo Kim <iamjoonsoo.kim@lge.com>,
	Andrew Morton <akpm@linux-foundation.org>,
	Al Viro <viro@zeniv.linux.org.uk>
Cc: Linux Kernel Mailing List <linux-kernel@vger.kernel.org>
Subject: get_vmalloc_info() and /proc/meminfo insanely expensive
Date: Wed, 12 Aug 2015 20:29:34 -0700	[thread overview]
Message-ID: <CA+55aFzUDiGRY_CZz0hON9AxkD1Fw_B-YSLiHH8tt_3rkKUrzw@mail.gmail.com> (raw)

I just did some profiling of a simple "make test" in the git repo, and
was surprised by the top kernel offender: get_vmalloc_info() showed up
at roughly 4% cpu use.

It turns out that bash ends up reading /proc/meminfo on every single
activation, and "make test" is basically just running a huge
collection of shell scripts. You can verify by just doing

    strace -o trace sh -c "echo"

to see what bash does on your system. I suspect it's actually glibc,
because a quick google finds the function "get_phys_pages()" that just
looks at the "MemTotal" line (or possibly get_avphys_pageslooks at the
MemFree" line).

Ok, so bash is insane for caring so deeply that it does this
regardless of anything else. But what else is new - user space does
odd things. It's like a truism.

My gut feel for this is that we should just rate-limit this and cache
the vmalloc information for a fraction of a second or something. Maybe
we could expose total memory sizes in some more efficient format, but
it's not like existing binaries will magically de-crapify themselves,
so just speeding up meminfo sounds like a good thing.

Maybe we could even cache the whole seqfile buffer - Al? How painful
would something like that be? Although from the profiles, it's really
just the vmalloc info gathering that shows up as actually wasting CPU
cycles..

                    Linus

             reply	other threads:[~2015-08-13  3:29 UTC|newest]

Thread overview: 12+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2015-08-13  3:29 Linus Torvalds [this message]
2015-08-13  4:00 ` get_vmalloc_info() and /proc/meminfo insanely expensive Andrew Morton
2015-08-13  5:52   ` Linus Torvalds
2015-08-13  7:42     ` Rasmus Villemoes
2015-08-13 16:50       ` Linus Torvalds
2015-08-13 18:32     ` Linus Torvalds
2015-08-13 18:52       ` Linus Torvalds
2015-08-13 19:50         ` David Rientjes
2015-08-13 20:04           ` Linus Torvalds
2015-08-13 21:15       ` Tony Luck
2015-08-13 22:56         ` Linus Torvalds
2015-08-13 20:26     ` Andrew Morton

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=CA+55aFzUDiGRY_CZz0hON9AxkD1Fw_B-YSLiHH8tt_3rkKUrzw@mail.gmail.com \
    --to=torvalds@linux-foundation.org \
    --cc=akpm@linux-foundation.org \
    --cc=iamjoonsoo.kim@lge.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=viro@zeniv.linux.org.uk \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).