From: Zdenek Kabelac <zkabelac@redhat.com>
To: Mel Gorman <mgorman@suse.de>
Cc: Seth Jennings <sjenning@linux.vnet.ibm.com>,
Jiri Slaby <jslaby@suse.cz>,
Valdis.Kletnieks@vt.edu, Jiri Slaby <jirislaby@gmail.com>,
linux-mm@kvack.org, LKML <linux-kernel@vger.kernel.org>,
Andrew Morton <akpm@linux-foundation.org>,
Rik van Riel <riel@redhat.com>,
Robert Jennings <rcj@linux.vnet.ibm.com>
Subject: Re: kswapd0: excessive CPU usage
Date: Sun, 11 Nov 2012 10:13:14 +0100 [thread overview]
Message-ID: <509F6C2A.9060502@redhat.com> (raw)
In-Reply-To: <20121109090635.GG8218@suse.de>
Dne 9.11.2012 10:06, Mel Gorman napsal(a):
> On Fri, Nov 09, 2012 at 09:07:45AM +0100, Zdenek Kabelac wrote:
>>> fe2c2a106663130a5ab45cb0e3414b52df2fff0c is the first bad commit
>>> commit fe2c2a106663130a5ab45cb0e3414b52df2fff0c
>>> Author: Rik van Riel <riel@redhat.com>
>>> Date: Wed Mar 21 16:33:51 2012 -0700
>>>
>>> vmscan: reclaim at order 0 when compaction is enabled
>>> ...
>>>
>>> This is plausible since the issue seems to be in the kswapd + compaction
>>> realm. I've yet to figure out exactly what about this commit results in
>>> kswapd spinning.
>>>
>>> I would be interested if someone can confirm this finding.
>>>
>>> --
>>> Seth
>>>
>>
>>
>> On my system 3.7-rc4 the problem seems to be effectively solved by
>> revert patch: https://lkml.org/lkml/2012/11/5/308
>>
>
> Ok, while there is still a question on whether it's enough I think it's
> sensible to at least start with the obvious one.
>
Hmm, so it's just took longer to hit the problem and observe kswapd0
spinning on my CPU again - it's not as endless like before - but still it
easily eats minutes - it helps to turn off Firefox or TB (memory hungry
apps) so kswapd0 stops soon - and restart those apps again.
(And I still have like >1GB of cached memory)
kswapd0 R running task 0 30 2 0x00000000
ffff8801331efae8 0000000000000082 0000000000000018 0000000000000246
ffff880135b9a340 ffff8801331effd8 ffff8801331effd8 ffff8801331effd8
ffff880055dfa340 ffff880135b9a340 00000000331efad8 ffff8801331ee000
Call Trace:
[<ffffffff81555bf2>] preempt_schedule+0x42/0x60
[<ffffffff81557a95>] _raw_spin_unlock+0x55/0x60
[<ffffffff81192971>] put_super+0x31/0x40
[<ffffffff81192a42>] drop_super+0x22/0x30
[<ffffffff81193b89>] prune_super+0x149/0x1b0
[<ffffffff81141e2a>] shrink_slab+0xba/0x510
[<ffffffff81185b4a>] ? mem_cgroup_iter+0x17a/0x2e0
[<ffffffff81185a9a>] ? mem_cgroup_iter+0xca/0x2e0
[<ffffffff81145099>] balance_pgdat+0x629/0x7f0
[<ffffffff811453d4>] kswapd+0x174/0x620
[<ffffffff8106fd20>] ? __init_waitqueue_head+0x60/0x60
[<ffffffff81145260>] ? balance_pgdat+0x7f0/0x7f0
[<ffffffff8106f50b>] kthread+0xdb/0xe0
[<ffffffff8106f430>] ? kthread_create_on_node+0x140/0x140
[<ffffffff8155fa1c>] ret_from_fork+0x7c/0xb0
[<ffffffff8106f430>] ? kthread_create_on_node+0x140/0x140
runnable tasks:
task PID tree-key switches prio exec-runtime
sum-exec sum-sleep
----------------------------------------------------------------------------------------------------------
kswapd0 30 8689943.729790 36266 120 8689943.729790
201495.640629 56609485.489414 /
kworker/0:1 14790 8689937.729790 16969 120 8689937.729790
374.385996 150405.181652 /
R bash 14855 821.749268 50 120 821.749268
24.027535 5252.291128 /autogroup-304
Mem-Info:
DMA per-cpu:
CPU 0: hi: 0, btch: 1 usd: 0
CPU 1: hi: 0, btch: 1 usd: 0
DMA32 per-cpu:
CPU 0: hi: 186, btch: 31 usd: 146
CPU 1: hi: 186, btch: 31 usd: 135
Normal per-cpu:
CPU 0: hi: 186, btch: 31 usd: 131
CPU 1: hi: 186, btch: 31 usd: 132
active_anon:726521 inactive_anon:26442 isolated_anon:0
active_file:77765 inactive_file:76890 isolated_file:0
unevictable:12 dirty:4 writeback:0 unstable:0
free:40261 slab_reclaimable:12414 slab_unreclaimable:9694
mapped:26382 shmem:162712 pagetables:6618 bounce:0
free_cma:0
DMA free:15676kB min:272kB low:340kB high:408kB active_anon:208kB
inactive_anon:0kB active_file:0kB inactive_file:0kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:15900kB mlocked:0kB dirty:0kB
writeback:0kB mapped:0kB shmem:208kB slab_reclaimable:8kB
slab_unreclaimable:8kB kernel_stack:0kB pagetables:0kB unstable:0kB bounce:0kB
free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? yes
lowmem_reserve[]: 0 2951 3836 3836
DMA32 free:126072kB min:51776kB low:64720kB high:77664kB active_anon:2175104kB
inactive_anon:98976kB active_file:296252kB inactive_file:297648kB
unevictable:48kB isolated(anon):0kB isolated(file):0kB present:3021960kB
mlocked:48kB dirty:12kB writeback:0kB mapped:77664kB shmem:620388kB
slab_reclaimable:19128kB slab_unreclaimable:6292kB kernel_stack:624kB
pagetables:8900kB unstable:0kB bounce:0kB free_cma:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 885 885
Normal free:19296kB min:15532kB low:19412kB high:23296kB active_anon:730772kB
inactive_anon:6792kB active_file:14808kB inactive_file:9912kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:906664kB mlocked:0kB dirty:4kB
writeback:0kB mapped:27864kB shmem:30252kB slab_reclaimable:30520kB
slab_unreclaimable:32476kB kernel_stack:2496kB pagetables:17572kB unstable:0kB
bounce:0kB free_cma:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0 0 0
DMA: 1*4kB 1*8kB 3*16kB 2*32kB 3*64kB 2*128kB 3*256kB 2*512kB 3*1024kB
3*2048kB 1*4096kB = 15676kB
DMA32: 730*4kB 328*8kB 223*16kB 123*32kB 182*64kB 96*128kB 172*256kB 56*512kB
12*1024kB 1*2048kB 1*4096kB = 128120kB
Normal: 600*4kB 384*8kB 164*16kB 122*32kB 40*64kB 7*128kB 1*256kB 1*512kB
1*1024kB 1*2048kB 0*4096kB = 19296kB
317367 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap = 0kB
Total swap = 0kB
1032176 pages RAM
42789 pages reserved
642501 pages shared
869271 pages non-shared
next prev parent reply other threads:[~2012-11-11 9:13 UTC|newest]
Thread overview: 52+ messages / expand[flat|nested] mbox.gz Atom feed top
2012-10-11 8:52 kswapd0: wxcessive CPU usage Jiri Slaby
2012-10-11 13:44 ` Valdis.Kletnieks
2012-10-11 15:34 ` Jiri Slaby
2012-10-11 17:56 ` Valdis.Kletnieks
2012-10-11 17:59 ` Jiri Slaby
2012-10-11 18:19 ` Valdis.Kletnieks
2012-10-11 22:08 ` kswapd0: excessive " Jiri Slaby
2012-10-12 12:37 ` Jiri Slaby
2012-10-12 13:57 ` Mel Gorman
2012-10-15 9:54 ` Jiri Slaby
2012-10-15 11:09 ` Mel Gorman
2012-10-29 10:52 ` Thorsten Leemhuis
2012-10-30 19:18 ` Mel Gorman
2012-10-31 11:25 ` Thorsten Leemhuis
2012-10-31 15:04 ` Mel Gorman
2012-11-04 16:36 ` Rik van Riel
2012-11-02 10:44 ` Zdenek Kabelac
2012-11-02 10:53 ` Jiri Slaby
2012-11-02 19:45 ` Jiri Slaby
2012-11-04 11:26 ` Zdenek Kabelac
2012-11-05 14:24 ` [PATCH] Revert "mm: vmscan: scale number of pages reclaimed by reclaim/compaction based on failures" Mel Gorman
2012-11-06 10:15 ` Johannes Hirte
2012-11-09 8:36 ` Mel Gorman
2012-11-14 21:43 ` Johannes Hirte
2012-11-09 9:12 ` Mel Gorman
2012-11-09 4:22 ` kswapd0: excessive CPU usage Seth Jennings
2012-11-09 8:07 ` Zdenek Kabelac
2012-11-09 9:06 ` Mel Gorman
2012-11-11 9:13 ` Zdenek Kabelac [this message]
2012-11-12 11:37 ` [PATCH] Revert "mm: remove __GFP_NO_KSWAPD" Mel Gorman
2012-11-16 19:14 ` Josh Boyer
2012-11-16 19:51 ` Andrew Morton
2012-11-20 1:43 ` Valdis.Kletnieks
2012-11-16 20:06 ` Mel Gorman
2012-11-20 15:38 ` Josh Boyer
2012-11-20 16:13 ` Bruno Wolff III
2012-11-20 17:43 ` Thorsten Leemhuis
2012-11-23 15:20 ` Thorsten Leemhuis
2012-11-27 11:12 ` Mel Gorman
2012-11-21 15:08 ` Mel Gorman
2012-11-20 9:18 ` Glauber Costa
2012-11-20 20:18 ` Andrew Morton
2012-11-21 8:30 ` Glauber Costa
2012-11-12 12:19 ` kswapd0: excessive CPU usage Mel Gorman
2012-11-12 13:13 ` Zdenek Kabelac
2012-11-12 13:31 ` Mel Gorman
2012-11-12 14:50 ` Zdenek Kabelac
2012-11-18 19:00 ` Zdenek Kabelac
2012-11-18 19:07 ` Jiri Slaby
2012-11-09 8:40 ` Mel Gorman
2012-10-11 22:14 ` kswapd0: wxcessive " Andrew Morton
2012-10-11 22:26 ` Jiri Slaby
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=509F6C2A.9060502@redhat.com \
--to=zkabelac@redhat.com \
--cc=Valdis.Kletnieks@vt.edu \
--cc=akpm@linux-foundation.org \
--cc=jirislaby@gmail.com \
--cc=jslaby@suse.cz \
--cc=linux-kernel@vger.kernel.org \
--cc=linux-mm@kvack.org \
--cc=mgorman@suse.de \
--cc=rcj@linux.vnet.ibm.com \
--cc=riel@redhat.com \
--cc=sjenning@linux.vnet.ibm.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).