From: Michal Hocko <mhocko@kernel.org>
To: Robert Foss <robert.foss@collabora.com>
Cc: sonnyrao@chromium.org, corbet@lwn.net, akpm@linux-foundation.org,
vbabka@suse.cz, koct9i@gmail.com, hughd@google.com,
n-horiguchi@ah.jp.nec.com, minchan@kernel.org,
john.stultz@linaro.org, ross.zwisler@linux.intel.com,
jmarchan@redhat.com, hannes@cmpxchg.org, keescook@chromium.org,
viro@zeniv.linux.org.uk, gorcunov@openvz.org,
plaguedbypenguins@gmail.com, rientjes@google.com,
eric.engestrom@imgtec.com, jdanis@google.com, calvinowens@fb.com,
adobriyan@gmail.com, jann@thejh.net,
kirill.shutemov@linux.intel.com, ldufour@linux.vnet.ibm.com,
linux-doc@vger.kernel.org, linux-kernel@vger.kernel.org,
Ben Zhang <benzh@chromium.org>, Bryan Freed <bfreed@chromium.org>,
Filipe Brandenburger <filbranden@chromium.org>,
Mateusz Guzik <mguzik@redhat.com>
Subject: Re: [PACTH v2 0/3] Implement /proc/<pid>/totmaps
Date: Wed, 17 Aug 2016 10:22:00 +0200 [thread overview]
Message-ID: <20160817082200.GA10547@dhcp22.suse.cz> (raw)
In-Reply-To: <336532d0-57f2-a430-d195-13c13f70e25a@collabora.com>
On Tue 16-08-16 12:46:51, Robert Foss wrote:
[...]
> $ /usr/bin/time -v -p zsh -c "repeat 25 { awk '/^Rss/{rss+=\$2}
> /^Pss/{pss+=\$2} END {printf \"rss:%d pss:%d\n\", rss, pss}\'
> /proc/5025/smaps }"
> [...]
> Command being timed: "zsh -c repeat 25 { awk '/^Rss/{rss+=$2}
> /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}\' /proc/5025/smaps
> }"
> User time (seconds): 0.37
> System time (seconds): 0.45
> Percent of CPU this job got: 92%
> Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.89
This is really unexpected. Where is the user time spent? Anyway, rather
than measuring some random processes I've tried to measure something
resembling the worst case. So I've created a simple program to mmap as
much as possible:
#include <sys/mman.h>
#include <sys/types.h>
#include <unistd.h>
#include <stdio.h>
int main()
{
while (mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_ANON|MAP_SHARED|MAP_POPULATE, -1, 0) != MAP_FAILED)
;
printf("pid:%d\n", getpid());
pause();
return 0;
}
so depending on /proc/sys/vm/max_map_count you will get the maximum
possible mmaps. I am using a default so 65k mappings. Then I have
retried your 25x file parsing:
$ cat s.sh
#!/bin/sh
pid=$1
for i in $(seq 25)
do
awk '/^Rss/{rss+=$2} /^Pss/{pss+=$2} END {printf "rss:%d pss:%d\n", rss, pss}' /proc/$pid/smaps
done
But I am getting different results from you:
$ awk '/^[0-9a-f]/{print}' /proc/14808/smaps | wc -l
65532
[...]
Command being timed: "sh s.sh 14808"
User time (seconds): 0.00
System time (seconds): 20.10
Percent of CPU this job got: 99%
Elapsed (wall clock) time (h:mm:ss or m:ss): 0:20.20
The results are stable when I try multiple times, in fact there
shouldn't be any reason for them not to be. Then I went on to increase
max_map_count to 250k and that behaves consistently:
$ awk '/^[0-9a-f]/{print}' /proc/16093/smaps | wc -l
250002
[...]
Command being timed: "sh s.sh 16093"
User time (seconds): 0.00
System time (seconds): 77.93
Percent of CPU this job got: 98%
Elapsed (wall clock) time (h:mm:ss or m:ss): 1:19.09
So with a reasonable user space the parsing is really not all that time
consuming wrt. smaps handling. That being said I am still very skeptical
about a dedicated proc file which accomplishes what userspace can done
in a trivial way.
--
Michal Hocko
SUSE Labs
next prev parent reply other threads:[~2016-08-17 8:22 UTC|newest]
Thread overview: 49+ messages / expand[flat|nested] mbox.gz Atom feed top
2016-08-12 22:04 [PACTH v2 0/3] Implement /proc/<pid>/totmaps robert.foss
2016-08-12 22:04 ` [PACTH v2 1/3] mm, proc: " robert.foss
2016-08-13 14:39 ` Jann Horn
2016-08-15 13:57 ` Robert Foss
2016-08-15 20:14 ` Robert Foss
2016-08-12 22:04 ` [PACTH v2 2/3] Documentation/filesystems: Fixed typo robert.foss
2016-08-12 22:04 ` [PACTH v2 3/3] Documentation/filesystems: Added /proc/PID/totmaps documentation robert.foss
2016-08-14 9:04 ` [PACTH v2 0/3] Implement /proc/<pid>/totmaps Michal Hocko
2016-08-15 13:00 ` Robert Foss
2016-08-15 13:42 ` Michal Hocko
2016-08-15 16:25 ` Robert Foss
2016-08-16 7:12 ` Michal Hocko
2016-08-16 16:46 ` Robert Foss
2016-08-17 8:22 ` Michal Hocko [this message]
2016-08-17 9:31 ` Jann Horn
2016-08-17 13:03 ` Michal Hocko
2016-08-17 16:48 ` Robert Foss
2016-08-17 18:57 ` Sonny Rao
2016-08-18 7:44 ` Michal Hocko
2016-08-18 17:47 ` Sonny Rao
2016-08-18 18:01 ` Michal Hocko
2016-08-18 21:05 ` Robert Foss
2016-08-19 6:27 ` Sonny Rao
2016-08-19 2:26 ` Minchan Kim
2016-08-19 6:47 ` Sonny Rao
2016-08-19 8:05 ` Michal Hocko
2016-08-19 18:20 ` Sonny Rao
2016-08-22 0:07 ` Minchan Kim
2016-08-22 7:40 ` Michal Hocko
2016-08-22 14:12 ` Minchan Kim
2016-08-22 14:37 ` Robert Foss
2016-08-22 16:45 ` Michal Hocko
2016-08-22 17:29 ` Michal Hocko
2016-08-22 17:47 ` Michal Hocko
2016-08-23 8:26 ` Michal Hocko
2016-08-23 14:33 ` utime accounting regression since 4.6 (was: Re: [PACTH v2 0/3] Implement /proc/<pid>/totmaps) Michal Hocko
2016-08-23 21:46 ` Rik van Riel
2016-08-24 16:56 ` Michal Hocko
2016-09-30 9:49 ` Michal Hocko
2016-09-30 12:35 ` Rik van Riel
2016-08-19 6:43 ` [PACTH v2 0/3] Implement /proc/<pid>/totmaps Sonny Rao
2016-08-19 7:59 ` Michal Hocko
2016-08-19 17:57 ` Sonny Rao
2016-08-22 7:54 ` Michal Hocko
2016-08-22 22:44 ` Sonny Rao
2016-08-24 10:14 ` Marcin Jabrzyk
2016-08-30 8:20 ` Michal Hocko
2016-08-29 14:37 ` Michal Hocko
2016-08-30 8:15 ` Michal Hocko
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=20160817082200.GA10547@dhcp22.suse.cz \
--to=mhocko@kernel.org \
--cc=adobriyan@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=benzh@chromium.org \
--cc=bfreed@chromium.org \
--cc=calvinowens@fb.com \
--cc=corbet@lwn.net \
--cc=eric.engestrom@imgtec.com \
--cc=filbranden@chromium.org \
--cc=gorcunov@openvz.org \
--cc=hannes@cmpxchg.org \
--cc=hughd@google.com \
--cc=jann@thejh.net \
--cc=jdanis@google.com \
--cc=jmarchan@redhat.com \
--cc=john.stultz@linaro.org \
--cc=keescook@chromium.org \
--cc=kirill.shutemov@linux.intel.com \
--cc=koct9i@gmail.com \
--cc=ldufour@linux.vnet.ibm.com \
--cc=linux-doc@vger.kernel.org \
--cc=linux-kernel@vger.kernel.org \
--cc=mguzik@redhat.com \
--cc=minchan@kernel.org \
--cc=n-horiguchi@ah.jp.nec.com \
--cc=plaguedbypenguins@gmail.com \
--cc=rientjes@google.com \
--cc=robert.foss@collabora.com \
--cc=ross.zwisler@linux.intel.com \
--cc=sonnyrao@chromium.org \
--cc=vbabka@suse.cz \
--cc=viro@zeniv.linux.org.uk \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).