All of lore.kernel.org
 help / color / mirror / Atom feed
From: Simon Kirby <sim@hostway.ca>
To: linux-kernel@vger.kernel.org,
	Shawn Bohrer <shawn.bohrer@gmail.com>,
	Eric Dumazet <eric.dumazet@gmail.com>
Subject: sys_epoll_wait high CPU load in 2.6.37
Date: Tue, 25 Jan 2011 16:09:32 -0800	[thread overview]
Message-ID: <20110126000932.GA23089@hostway.ca> (raw)

Hello!

Since upgrading 2.6.36 -> 2.6.37, dovecot's "anvil" process seems to end
up taking a lot more time in "top", and "perf top" shows output like this
(system-wide):

  samples  pcnt function                      DSO
  _______ _____ _____________________________ __________________________

  2405.00 68.8% sys_epoll_wait                [kernel.kallsyms]
    33.00  0.9% mail_cache_lookup_iter_next   libdovecot-storage.so.0.0.0
    30.00  0.9% _raw_spin_lock                [kernel.kallsyms]
    ...etc...

It only wakes up 5-10 times per second or so (on this box), and does
stuff like this:

epoll_wait(12, {{EPOLLIN, {u32=19417616, u64=19417616}}}, 25, 2147483647) = 1
read(29, "PENALTY-GET\t192.168.31.10\n"..., 738) = 26
write(29, "0 0\n"..., 4)                = 4
epoll_wait(12, {{EPOLLIN, {u32=19395632, u64=19395632}}}, 25, 2147483647) = 1
read(18, "LOOKUP\tpop3/192.168.31.10/tshield"..., 668) = 58
write(18, "0\n"..., 2)                  = 2
epoll_wait(12, {{EPOLLIN, {u32=19373072, u64=19373072}}}, 25, 2147483647) = 1
read(7, "CONNECT\t3490\tpop3/192.168.31.10/t"..., 254) = 64
epoll_wait(12, {{EPOLLIN, {u32=19373072, u64=19373072}}}, 25, 2147483647) = 1
read(7, "DISCONNECT\t3482\tpop3/192.168.31.1"..., 190) = 62

Anything obvious here?  anvil talks over UNIX sockets to the rest of
dovecot, and uses epoll_wait.  So, suspect commits might be:

95aac7b1cd224f568fb83937044cd303ff11b029
5456f09aaf88731e16dbcea7522cb330b6846415
or other bits from
 git log v2.6.36..v2.6.37 net/unix/af_unix.c fs/eventpoll.c

I suspect it has something to do with that "infinite value" check removal
in that first commit.  It doesn't show up easily on a test box, but I can
try reverting 95aac7b1cd in production if it's not obvious.

Simon-

             reply	other threads:[~2011-01-26  0:09 UTC|newest]

Thread overview: 14+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2011-01-26  0:09 Simon Kirby [this message]
2011-01-26  7:18 ` sys_epoll_wait high CPU load in 2.6.37 Eric Dumazet
2011-01-26 11:16   ` Eric Dumazet
2011-01-26 15:31     ` Davide Libenzi
2011-01-26 15:43       ` Eric Dumazet
2011-01-26 15:52         ` Davide Libenzi
2011-01-26 15:59           ` Eric Dumazet
2011-01-26 16:13             ` Eric Dumazet
2011-01-26 17:20               ` Shawn Bohrer
2011-01-26 17:51                 ` Eric Dumazet
2011-01-26 18:06                   ` Davide Libenzi
2011-01-26 18:18                     ` Eric Dumazet
2011-01-26 18:45                       ` Eric Dumazet
2011-01-28  0:17                         ` Davide Libenzi

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20110126000932.GA23089@hostway.ca \
    --to=sim@hostway.ca \
    --cc=eric.dumazet@gmail.com \
    --cc=linux-kernel@vger.kernel.org \
    --cc=shawn.bohrer@gmail.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.