From: "Roman Jarosz" <kedgedev@gmail.com>
To: "David Rientjes" <rientjes@google.com>
Cc: linux-kernel@vger.kernel.org,
"KOSAKI Motohiro" <kosaki.motohiro@jp.fujitsu.com>,
"Andrew Morton" <akpm@linux-foundation.org>
Subject: Re: OOM-Killer kills too much with 2.6.32.2
Date: Mon, 25 Jan 2010 23:12:38 +0100 [thread overview]
Message-ID: <op.u63zzcz6asvm2a@kedge> (raw)
In-Reply-To: <alpine.DEB.2.00.1001221623480.12712@chino.kir.corp.google.com>
On Sat, 23 Jan 2010 01:40:17 +0100, David Rientjes <rientjes@google.com>
wrote:
> On Thu, 14 Jan 2010, Roman Jarosz wrote:
>
>> Hi,
>>
>> since kernel 2.6.32.2 (also tried 2.6.32.3) I get a lot of oom-killer
>> kills
>> when I do hard disk intensive tasks (mainly in VirtualBox which is
>> running
>> Windows XP) and IMHO it kills processes even if I have a lot of free
>> memory.
>>
>> Is this a known bug? I have self compiled kernel so I can try patches.
>>
>> Regards
>> Roman Jarosz
>>
>> PS. Please CC me.
>>
>> Jan 7 12:39:27 kedge kernel: X invoked oom-killer: gfp_mask=0x0,
>> order=0,
>> oom_adj=0
>
> The gfp_mask of 0x0 and order of 0 indicate these are triggered by
> pagefaults that end up returning VM_FAULT_OOM. Prior to 2.6.29, the
> current task (X in all cases from your log) would have been SIGKILLed; we
> now call the oom killer instead of kill a memory hogging task so that the
> fault is retried with more success. If you've upgraded from a 2.6.29 or
> later kernel and are only now experiencing these errors, it may indicate
> a
> regression in VM that we need to investigate (and, if so, you may want to
> try merging f50de2d38 from 2.6.33 to see if it helps keep more
> ZONE_NORMAL
> memory available so that such drastic measures aren't necessary).
I only have logs since 2.6.31-rc7 and there is one oom-kill in 2.6.32-rc7
and then the kills in 2.6.32.2.
Nov 25 13:49:04 kedge kernel: X invoked oom-killer: gfp_mask=0x0, order=0,
oom_adj=0
Nov 25 13:49:04 kedge kernel: Pid: 1883, comm: X Not tainted 2.6.32-rc7 #1
Nov 25 13:49:04 kedge kernel: Call Trace:
Nov 25 13:49:04 kedge kernel: [<ffffffff8107a66d>] ? 0xffffffff8107a66d
Nov 25 13:49:04 kedge kernel: [<ffffffff8107a953>] ? 0xffffffff8107a953
Nov 25 13:49:04 kedge kernel: [<ffffffff8107aac7>] ? 0xffffffff8107aac7
Nov 25 13:49:04 kedge kernel: [<ffffffff8155453f>] ? 0xffffffff8155453f
Nov 25 13:49:04 kedge kernel: Mem-Info:
Nov 25 13:49:04 kedge kernel: DMA per-cpu:
Nov 25 13:49:04 kedge kernel: CPU 0: hi: 0, btch: 1 usd: 0
Nov 25 13:49:04 kedge kernel: CPU 1: hi: 0, btch: 1 usd: 0
Nov 25 13:49:04 kedge kernel: DMA32 per-cpu:
Nov 25 13:49:04 kedge kernel: CPU 0: hi: 186, btch: 31 usd: 178
Nov 25 13:49:04 kedge kernel: CPU 1: hi: 186, btch: 31 usd: 174
Nov 25 13:49:04 kedge kernel: Normal per-cpu:
Nov 25 13:49:04 kedge kernel: CPU 0: hi: 186, btch: 31 usd: 153
Nov 25 13:49:04 kedge kernel: CPU 1: hi: 186, btch: 31 usd: 81
Nov 25 13:49:04 kedge kernel: active_anon:135012 inactive_anon:77065
isolated_anon:0
Nov 25 13:49:04 kedge kernel: active_file:332368 inactive_file:411112
isolated_file:32
Nov 25 13:49:04 kedge kernel: unevictable:0 dirty:139692 writeback:2395
unstable:0
Nov 25 13:49:04 kedge kernel: free:6869 slab_reclaimable:15618
slab_unreclaimable:7026
Nov 25 13:49:04 kedge kernel: mapped:31999 shmem:26786 pagetables:4497
bounce:0
Nov 25 13:49:04 kedge kernel: DMA free:15776kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:1
60kB unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15344kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB slab_rec
laimable:0kB slab_unreclaimable:8kB kernel_stack:0kB pagetables:0kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0 all_unreclaima
ble? no
Nov 25 13:49:04 kedge kernel: lowmem_reserve[]: 0 2990 3937 3937
Nov 25 13:49:04 kedge kernel: DMA32 free:9856kB min:6084kB low:7604kB
high:9124kB active_anon:327516kB inactive_anon:87488kB active_file:112
2804kB inactive_file:1374360kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3062688kB mlocked:0kB dirty:440312kB writeback:
5556kB mapped:24652kB shmem:57952kB slab_reclaimable:42492kB
slab_unreclaimable:10528kB kernel_stack:472kB pagetables:3464kB
unstable:0kB bo
unce:0kB writeback_tmp:0kB pages_scanned:384 all_unreclaimable? no
Nov 25 13:49:04 kedge kernel: lowmem_reserve[]: 0 0 946 946
Nov 25 13:49:04 kedge kernel: Normal free:1844kB min:1924kB low:2404kB
high:2884kB active_anon:212532kB inactive_anon:220772kB active_file:2
06668kB inactive_file:269928kB unevictable:0kB isolated(anon):0kB
isolated(file):128kB present:969600kB mlocked:0kB dirty:118456kB writeback
:4024kB mapped:103344kB shmem:49192kB slab_reclaimable:19980kB
slab_unreclaimable:17568kB kernel_stack:1608kB pagetables:14524kB
unstable:0k
B bounce:0kB writeback_tmp:0kB pages_scanned:448 all_unreclaimable? no
Nov 25 13:49:04 kedge kernel: lowmem_reserve[]: 0 0 0 0
Nov 25 13:49:04 kedge kernel: DMA: 0*4kB 2*8kB 1*16kB 2*32kB 1*64kB
2*128kB 2*256kB 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15776kB
Nov 25 13:49:04 kedge kernel: DMA32: 300*4kB 4*8kB 29*16kB 1*32kB 1*64kB
1*128kB 1*256kB 1*512kB 1*1024kB 1*2048kB 1*4096kB = 9856kB
Nov 25 13:49:04 kedge kernel: Normal: 461*4kB 0*8kB 0*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1844kB
Nov 25 13:49:04 kedge kernel: 754208 total pagecache pages
Nov 25 13:49:04 kedge kernel: 64 pages in swap cache
Nov 25 13:49:04 kedge kernel: Swap cache stats: add 357, delete 293, find
2/8
Nov 25 13:49:04 kedge kernel: Free swap = 2006888kB
Nov 25 13:49:04 kedge kernel: Total swap = 2008116kB
Nov 25 13:49:04 kedge kernel: 1032192 pages RAM
Nov 25 13:49:04 kedge kernel: 36869 pages reserved
Nov 25 13:49:04 kedge kernel: 377334 pages shared
Nov 25 13:49:04 kedge kernel: 705565 pages non-shared
Nov 25 13:49:04 kedge kernel: Out of memory: kill process 6816
(services.exe) score 1652318 or a child
Nov 25 13:49:04 kedge kernel: Killed process 6818 (winedevice.exe)
>> Jan 7 12:39:27 kedge kernel: Pid: 1954, comm: X Not tainted 2.6.32.2 #1
>> Jan 7 12:39:27 kedge kernel: Call Trace:
>> Jan 7 12:39:27 kedge kernel: [<ffffffff8107b17d>] ? 0xffffffff8107b17d
>> Jan 7 12:39:27 kedge kernel: [<ffffffff8107b463>] ? 0xffffffff8107b463
>> Jan 7 12:39:27 kedge kernel: [<ffffffff8107b5d7>] ? 0xffffffff8107b5d7
>> Jan 7 12:39:27 kedge kernel: [<ffffffff815581df>] ? 0xffffffff815581df
>
> Can you find out what these symbols are?
Can I somehow get the symbols without recompiling kernel?
I have the source tree with *.o files and other stuff which was created
during compilation.
I've also got the kill in 2.6.33-rc5, what do I have to do to get the
symbols?
Is "Load all symbols for debugging/ksymoops (KALLSYMS)" and
"Compile the kernel with frame pointers (FRAME_POINTER)" enough?
Jan 25 15:51:45 kedge kernel: X invoked oom-killer: gfp_mask=0x0, order=0,
oom_adj=0
Jan 25 15:51:45 kedge kernel: Pid: 1904, comm: X Not tainted 2.6.33-rc5 #1
Jan 25 15:51:45 kedge kernel: Call Trace:
Jan 25 15:51:45 kedge kernel: [<ffffffff8107e1a4>] 0xffffffff8107e1a4
Jan 25 15:51:45 kedge kernel: [<ffffffff8107e306>] 0xffffffff8107e306
Jan 25 15:51:45 kedge kernel: [<ffffffff8107e4f0>] 0xffffffff8107e4f0
Jan 25 15:51:45 kedge kernel: [<ffffffff8107e678>] 0xffffffff8107e678
Jan 25 15:51:45 kedge kernel: [<ffffffff81021e60>] 0xffffffff81021e60
Jan 25 15:51:45 kedge kernel: [<ffffffff810221a8>] 0xffffffff810221a8
Jan 25 15:51:45 kedge kernel: [<ffffffff8156f95f>] 0xffffffff8156f95f
Jan 25 15:51:45 kedge kernel: Mem-Info:
Jan 25 15:51:45 kedge kernel: DMA per-cpu:
Jan 25 15:51:45 kedge kernel: CPU 0: hi: 0, btch: 1 usd: 0
Jan 25 15:51:45 kedge kernel: CPU 1: hi: 0, btch: 1 usd: 0
Jan 25 15:51:45 kedge kernel: DMA32 per-cpu:
Jan 25 15:51:45 kedge kernel: CPU 0: hi: 186, btch: 31 usd: 152
Jan 25 15:51:45 kedge kernel: CPU 1: hi: 186, btch: 31 usd: 171
Jan 25 15:51:45 kedge kernel: Normal per-cpu:
Jan 25 15:51:45 kedge kernel: CPU 0: hi: 186, btch: 31 usd: 44
Jan 25 15:51:45 kedge kernel: CPU 1: hi: 186, btch: 31 usd: 167
Jan 25 15:51:45 kedge kernel: active_anon:303490 inactive_anon:130718
isolated_anon:32
Jan 25 15:51:45 kedge kernel: active_file:308170 inactive_file:183887
isolated_file:0
Jan 25 15:51:45 kedge kernel: unevictable:0 dirty:172639 writeback:8741
unstable:0
Jan 25 15:51:45 kedge kernel: free:6871 slab_reclaimable:26390
slab_unreclaimable:11635
Jan 25 15:51:45 kedge kernel: mapped:38879 shmem:29781 pagetables:5198
bounce:0
Jan 25 15:51:45 kedge kernel: DMA free:15764kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:4kB inactive_file:156kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15332kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:4kB slab_unreclaimable:16kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:0
all_unreclaimable? no
Jan 25 15:51:45 kedge kernel: lowmem_reserve[]: 0 2990 3937 3937
Jan 25 15:51:45 kedge kernel: DMA32 free:9832kB min:6084kB low:7604kB
high:9124kB active_anon:970168kB inactive_anon:242576kB
active_file:1015796kB inactive_file:619228kB unevictable:0kB
isolated(anon):128kB isolated(file):0kB present:3062688kB mlocked:0kB
dirty:584004kB writeback:31268kB mapped:35252kB shmem:43448kB
slab_reclaimable:85656kB slab_unreclaimable:25596kB kernel_stack:264kB
pagetables:4724kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:238 all_unreclaimable? no
Jan 25 15:51:45 kedge kernel: lowmem_reserve[]: 0 0 946 946
Jan 25 15:51:45 kedge kernel: Normal free:1888kB min:1924kB low:2404kB
high:2884kB active_anon:243792kB inactive_anon:280296kB
active_file:216880kB inactive_file:116164kB unevictable:0kB
isolated(anon):0kB isolated(file):0kB present:969600kB mlocked:0kB
dirty:106552kB writeback:3696kB mapped:120264kB shmem:75676kB
slab_reclaimable:19900kB slab_unreclaimable:20928kB kernel_stack:1920kB
pagetables:16068kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:33 all_unreclaimable? no
Jan 25 15:51:45 kedge kernel: lowmem_reserve[]: 0 0 0 0
Jan 25 15:51:45 kedge kernel: DMA: 1*4kB 2*8kB 2*16kB 3*32kB 2*64kB
3*128kB 3*256kB 2*512kB 3*1024kB 3*2048kB 1*4096kB = 15764kB
Jan 25 15:51:45 kedge kernel: DMA32: 1108*4kB 1*8kB 2*16kB 0*32kB 0*64kB
2*128kB 2*256kB 1*512kB 0*1024kB 0*2048kB 1*4096kB = 9848kB
Jan 25 15:51:45 kedge kernel: Normal: 472*4kB 0*8kB 0*16kB 0*32kB 0*64kB
0*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1888kB
Jan 25 15:51:45 kedge kernel: 522113 total pagecache pages
Jan 25 15:51:45 kedge kernel: 274 pages in swap cache
Jan 25 15:51:45 kedge kernel: Swap cache stats: add 5340, delete 5066,
find 0/0
Jan 25 15:51:45 kedge kernel: Free swap = 1986756kB
Jan 25 15:51:45 kedge kernel: Total swap = 2008116kB
Jan 25 15:51:45 kedge kernel: 1032192 pages RAM
Jan 25 15:51:45 kedge kernel: 49813 pages reserved
Jan 25 15:51:45 kedge kernel: 821582 pages shared
Jan 25 15:51:45 kedge kernel: 260542 pages non-shared
Jan 25 15:51:45 kedge kernel: Out of memory: kill process 1746 (hald)
score 10058 or a child
Jan 25 15:51:45 kedge kernel: Killed process 1747 (hald-runner)
vsz:17860kB, anon-rss:24kB, file-rss:1044kB
next prev parent reply other threads:[~2010-01-25 22:10 UTC|newest]
Thread overview: 57+ messages / expand[flat|nested] mbox.gz Atom feed top
2010-01-14 13:15 OOM-Killer kills too much with 2.6.32.2 Roman Jarosz
2010-01-23 0:40 ` David Rientjes
2010-01-25 22:12 ` Roman Jarosz [this message]
2010-01-25 1:48 ` KOSAKI Motohiro
2010-01-25 20:47 ` Roman Jarosz
2010-01-26 5:19 ` KOSAKI Motohiro
2010-01-26 7:51 ` A Rojas
2010-01-26 9:06 ` Roman Jarosz
2010-01-26 11:07 ` KOSAKI Motohiro
2010-01-26 12:33 ` Chris Wilson
2010-01-26 13:03 ` KOSAKI Motohiro
2010-01-26 13:18 ` Chris Wilson
2010-01-26 13:59 ` Michael Reinelt
2010-01-26 14:07 ` Michael Reinelt
2010-01-27 0:50 ` KOSAKI Motohiro
2010-01-27 9:56 ` Pekka Enberg
2010-01-27 10:55 ` Linus Torvalds
2010-01-27 11:12 ` Pekka Enberg
2010-01-27 11:14 ` [PATCH] drm/i915: Selectively enable self-reclaim Chris Wilson
2010-01-27 11:20 ` Pekka Enberg
2010-01-27 11:30 ` Michael Reinelt
2010-01-28 3:15 ` Michael Reinelt
2010-01-28 18:21 ` Roman Jarosz
2010-01-27 11:50 ` KOSAKI Motohiro
2010-01-27 12:16 ` Linus Torvalds
2010-01-27 12:28 ` Linus Torvalds
2010-01-27 15:25 ` Chris Wilson
2010-01-27 16:09 ` Linus Torvalds
2010-01-27 17:14 ` Chris Wilson
2010-01-27 17:19 ` Linus Torvalds
2010-01-27 21:03 ` Roman Jarosz
2010-06-30 6:54 ` [Intel-gfx] " Dave Airlie
2010-06-30 7:05 ` Chris Wilson
2010-06-30 7:05 ` Chris Wilson
2010-06-30 23:07 ` [Intel-gfx] " Linus Torvalds
2010-07-01 1:24 ` Linus Torvalds
2010-07-01 1:55 ` KOSAKI Motohiro
2010-07-01 10:15 ` Dave Airlie
2010-07-01 11:19 ` Chris Wilson
2010-07-01 11:19 ` Chris Wilson
2010-07-01 22:34 ` M. Vefa Bicakci
2010-07-01 23:59 ` Linus Torvalds
2010-07-01 23:59 ` Linus Torvalds
2010-07-02 0:06 ` [Intel-gfx] " Dave Airlie
2010-07-02 0:49 ` Dave Airlie
2010-07-02 1:28 ` Linus Torvalds
2010-07-02 1:28 ` Linus Torvalds
2010-07-17 18:58 ` [Intel-gfx] " M. Vefa Bicakci
2010-07-17 19:15 ` Linus Torvalds
2010-07-18 14:27 ` M. Vefa Bicakci
2010-07-18 14:27 ` M. Vefa Bicakci
2010-07-18 16:59 ` Linus Torvalds
2010-01-28 6:37 ` Willy Tarreau
2010-01-26 13:41 ` OOM-Killer kills too much with 2.6.32.2 Roman Jarosz
2010-01-27 0:14 ` KOSAKI Motohiro
2010-01-27 9:53 ` Roman Jarosz
2010-01-26 13:57 ` Pekka Enberg
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=op.u63zzcz6asvm2a@kedge \
--to=kedgedev@gmail.com \
--cc=akpm@linux-foundation.org \
--cc=kosaki.motohiro@jp.fujitsu.com \
--cc=linux-kernel@vger.kernel.org \
--cc=rientjes@google.com \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.