cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin Mattock]

Hello,
I'm not sure how to handle this,
while compiling firefox-3.6b1.source
I get this with the default compiling options,
as well as custom:


[  532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
[  532.942330] Pid: 16002, comm: cc1plus Tainted: P
2.6.32-rc5-00083-g04ea458 #2
[  532.942333] Call Trace:
[  532.942342]  [<ffffffff810bce11>] T.417+0x7c/0x245
[  532.942347]  [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
[  532.942352]  [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
[  532.942357]  [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
[  532.942363]  [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
[  532.942369]  [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
[  532.942373]  [<ffffffff813ddb25>] page_fault+0x25/0x30
[  532.942376] Mem-Info:
[  532.942378] DMA per-cpu:
[  532.942380] CPU    0: hi:    0, btch:   1 usd:   0
[  532.942383] CPU    1: hi:    0, btch:   1 usd:   0
[  532.942385] DMA32 per-cpu:
[  532.942388] CPU    0: hi:  186, btch:  31 usd:  94
[  532.942391] CPU    1: hi:  186, btch:  31 usd:  23
[  532.942393] Normal per-cpu:
[  532.942395] CPU    0: hi:  186, btch:  31 usd: 150
[  532.942398] CPU    1: hi:  186, btch:  31 usd: 155
[  532.942404] active_anon:707575 inactive_anon:264673 isolated_anon:0
[  532.942406]  active_file:58 inactive_file:33 isolated_file:0
[  532.942407]  unevictable:0 dirty:0 writeback:0 unstable:0 buffer:71
[  532.942408]  free:6998 slab_reclaimable:2697 slab_unreclaimable:16267
[  532.942409]  mapped:136 shmem:64 pagetables:2761 bounce:0
[  532.942418] DMA free:15944kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[  532.942425] lowmem_reserve[]: 0 2976 3986 3986
[  532.942436] DMA32 free:10020kB min:6020kB low:7524kB high:9028kB
active_anon:2360492kB inactive_anon:590196kB active_file:84kB
inactive_file:68kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
writeback:0kB mapped:136kB shmem:80kB slab_reclaimable:0kB
slab_unreclaimable:88kB kernel_stack:0kB pagetables:6100kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:448
all_unreclaimable? yes
[  532.942443] lowmem_reserve[]: 0 0 1010 1010
[  532.942454] Normal free:2028kB min:2040kB low:2548kB high:3060kB
active_anon:469808kB inactive_anon:468496kB active_file:148kB
inactive_file:64kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
writeback:0kB mapped:408kB shmem:176kB slab_reclaimable:10788kB
slab_unreclaimable:64972kB kernel_stack:800kB pagetables:4944kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:413
all_unreclaimable? yes
[  532.942461] lowmem_reserve[]: 0 0 0 0
[  532.942465] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
[  532.942478] DMA32: 7*4kB 7*8kB 5*16kB 4*32kB 2*64kB 1*128kB 1*256kB
4*512kB 3*1024kB 0*2048kB 1*4096kB = 10020kB
[  532.942490] Normal: 507*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2028kB
[  532.942501] 7974 total pagecache pages
[  532.942503] 7778 pages in swap cache
[  532.942506] Swap cache stats: add 112290, delete 104512, find 3595/4051
[  532.942508] Free swap  = 0kB
[  532.942510] Total swap = 431632kB
[  532.957941] 1048576 pages RAM
[  532.957943] 40518 pages reserved
[  532.957945] 312 pages shared
[  532.957947] 1000176 pages non-shared
[  532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
[  532.957955] Killed process 16002 (cc1plus)

I just compiled the latest gcc snapshot a few days
ago.


-- 
Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Jiri Slaby]

On 11/02/2009 10:29 PM, Justin Mattock wrote:
> [  532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
> [  532.942330] Pid: 16002, comm: cc1plus Tainted: P
> 2.6.32-rc5-00083-g04ea458 #2
> [  532.942333] Call Trace:
> [  532.942342]  [<ffffffff810bce11>] T.417+0x7c/0x245
> [  532.942347]  [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
> [  532.942352]  [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
> [  532.942357]  [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
> [  532.942363]  [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
> [  532.942369]  [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
> [  532.942373]  [<ffffffff813ddb25>] page_fault+0x25/0x30
...
> [  532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
> [  532.957955] Killed process 16002 (cc1plus)
> 
> I just compiled the latest gcc snapshot a few days
> ago.

How many jobs did you run in parallel? Was there anything else memory
consuming running on that machine? Do you run the same jobs count every
time you compile such big c++ projects?

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin Mattock]

On Mon, Nov 2, 2009 at 1:49 PM, Jiri Slaby <jirislaby@gmail.com> wrote:
> On 11/02/2009 10:29 PM, Justin Mattock wrote:
>> [  532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
>> [  532.942330] Pid: 16002, comm: cc1plus Tainted: P
>> 2.6.32-rc5-00083-g04ea458 #2
>> [  532.942333] Call Trace:
>> [  532.942342]  [<ffffffff810bce11>] T.417+0x7c/0x245
>> [  532.942347]  [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
>> [  532.942352]  [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
>> [  532.942357]  [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
>> [  532.942363]  [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
>> [  532.942369]  [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
>> [  532.942373]  [<ffffffff813ddb25>] page_fault+0x25/0x30
> ...
>> [  532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
>> [  532.957955] Killed process 16002 (cc1plus)
>>
>> I just compiled the latest gcc snapshot a few days
>> ago.
>
> How many jobs did you run in parallel? Was there anything else memory
> consuming running on that machine? Do you run the same jobs count every
> time you compile such big c++ projects?
>

This would be the only job running.

as for other types of jobs under ps aux there's only
udev,dbus,acpid,sshd,wicd running.

Now with this oom-killer I'm
hitting this on an imac9,1 with
gcc (GCC) 4.5.0 20091029 (experimental)
2.6.32-rc5-00083-g04ea458

On my other machine macbook pro ati chipset
(same system pure64)firefox seems to be going a lot farther
than on the imac(doing this in the past I think this takes a good
45min to compile).

the macbook has:
gcc (GCC) Cross-LFS 4.4.1.20090722) 4.4.1
2.6.32-rc4-00039-ga3f6f2e



-- 
Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Jiri Slaby]

On 11/02/2009 11:02 PM, Justin Mattock wrote:
> Now with this oom-killer I'm
> hitting this on an imac9,1 with
> gcc (GCC) 4.5.0 20091029 (experimental)

So there is probably a leak in the gcc chain. Does this happen with a
stable gcc version?

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Jiri Slaby wrote:
> On 11/02/2009 11:02 PM, Justin Mattock wrote:
>    
>> Now with this oom-killer I'm
>> hitting this on an imac9,1 with
>> gcc (GCC) 4.5.0 20091029 (experimental)
>>      
>
> So there is probably a leak in the gcc chain. Does this happen with a
> stable gcc version?
>
>    
well right now I'(on the macbook), seems to be going for a good ten 
minutes before
I hit an error with a header file(nss)

I'll see if I can get this thing to compile all the way through.
(knock on wood) then go from there.

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Jiri Slaby wrote:
> On 11/02/2009 11:02 PM, Justin Mattock wrote:
>    
>> Now with this oom-killer I'm
>> hitting this on an imac9,1 with
>> gcc (GCC) 4.5.0 20091029 (experimental)
>>      
>
> So there is probably a leak in the gcc chain. Does this happen with a
> stable gcc version?
>
>    
o.k. I think it's something with the latest gcc(snapshot)
right now she's been compiling firefox for 45min
(then crapped out because I compiled nss without sqlite)
without no omm-killer.

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[KOSAKI Motohiro]

> Hello,
> I'm not sure how to handle this,
> while compiling firefox-3.6b1.source
> I get this with the default compiling options,
> as well as custom:
> 
> 
> [  532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
> [  532.942330] Pid: 16002, comm: cc1plus Tainted: P
> 2.6.32-rc5-00083-g04ea458 #2
> [  532.942333] Call Trace:
> [  532.942342]  [<ffffffff810bce11>] T.417+0x7c/0x245
> [  532.942347]  [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
> [  532.942352]  [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
> [  532.942357]  [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
> [  532.942363]  [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
> [  532.942369]  [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
> [  532.942373]  [<ffffffff813ddb25>] page_fault+0x25/0x30
> [  532.942376] Mem-Info:
> [  532.942378] DMA per-cpu:
> [  532.942380] CPU    0: hi:    0, btch:   1 usd:   0
> [  532.942383] CPU    1: hi:    0, btch:   1 usd:   0
> [  532.942385] DMA32 per-cpu:
> [  532.942388] CPU    0: hi:  186, btch:  31 usd:  94
> [  532.942391] CPU    1: hi:  186, btch:  31 usd:  23
> [  532.942393] Normal per-cpu:
> [  532.942395] CPU    0: hi:  186, btch:  31 usd: 150
> [  532.942398] CPU    1: hi:  186, btch:  31 usd: 155
> [  532.942404] active_anon:707575 inactive_anon:264673 isolated_anon:0
> [  532.942406]  active_file:58 inactive_file:33 isolated_file:0

file cache (active_file + inactive_file) was very little. It indicate anyone waste too much memory.
I doubt you use buggy compiler.



> [  532.942407]  unevictable:0 dirty:0 writeback:0 unstable:0 buffer:71
> [  532.942408]  free:6998 slab_reclaimable:2697 slab_unreclaimable:16267
> [  532.942409]  mapped:136 shmem:64 pagetables:2761 bounce:0
> [  532.942418] DMA free:15944kB min:28kB low:32kB high:40kB
> active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
> unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
> mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
> slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
> pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
> pages_scanned:0 all_unreclaimable? yes
> [  532.942425] lowmem_reserve[]: 0 2976 3986 3986
> [  532.942436] DMA32 free:10020kB min:6020kB low:7524kB high:9028kB
> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
> inactive_file:68kB unevictable:0kB isolated(anon):0kB
> isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
> writeback:0kB mapped:136kB shmem:80kB slab_reclaimable:0kB
> slab_unreclaimable:88kB kernel_stack:0kB pagetables:6100kB
> unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:448
> all_unreclaimable? yes
> [  532.942443] lowmem_reserve[]: 0 0 1010 1010
> [  532.942454] Normal free:2028kB min:2040kB low:2548kB high:3060kB
> active_anon:469808kB inactive_anon:468496kB active_file:148kB
> inactive_file:64kB unevictable:0kB isolated(anon):0kB
> isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
> writeback:0kB mapped:408kB shmem:176kB slab_reclaimable:10788kB
> slab_unreclaimable:64972kB kernel_stack:800kB pagetables:4944kB
> unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:413
> all_unreclaimable? yes
> [  532.942461] lowmem_reserve[]: 0 0 0 0
> [  532.942465] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
> 1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
> [  532.942478] DMA32: 7*4kB 7*8kB 5*16kB 4*32kB 2*64kB 1*128kB 1*256kB
> 4*512kB 3*1024kB 0*2048kB 1*4096kB = 10020kB
> [  532.942490] Normal: 507*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
> 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 2028kB
> [  532.942501] 7974 total pagecache pages
> [  532.942503] 7778 pages in swap cache
> [  532.942506] Swap cache stats: add 112290, delete 104512, find 3595/4051
> [  532.942508] Free swap  = 0kB
> [  532.942510] Total swap = 431632kB
> [  532.957941] 1048576 pages RAM
> [  532.957943] 40518 pages reserved
> [  532.957945] 312 pages shared
> [  532.957947] 1000176 pages non-shared
> [  532.957951] Out of memory: kill process 16001 (c++) score 543727 or a child
> [  532.957955] Killed process 16002 (cc1plus)
> 
> I just compiled the latest gcc snapshot a few days
> ago.

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

KOSAKI Motohiro wrote:
>> Hello,
>> I'm not sure how to handle this,
>> while compiling firefox-3.6b1.source
>> I get this with the default compiling options,
>> as well as custom:
>>
>>
>> [  532.942324] cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0
>> [  532.942330] Pid: 16002, comm: cc1plus Tainted: P
>> 2.6.32-rc5-00083-g04ea458 #2
>> [  532.942333] Call Trace:
>> [  532.942342]  [<ffffffff810bce11>] T.417+0x7c/0x245
>> [  532.942347]  [<ffffffff810bd11c>] __out_of_memory+0x142/0x159
>> [  532.942352]  [<ffffffff810bd1a1>] out_of_memory+0x6e/0x9d
>> [  532.942357]  [<ffffffff810c0086>] __alloc_pages_nodemask+0x47e/0x5cc
>> [  532.942363]  [<ffffffff810d2115>] handle_mm_fault+0x25d/0x68e
>> [  532.942369]  [<ffffffff813dfc1d>] do_page_fault+0x2bb/0x2d3
>> [  532.942373]  [<ffffffff813ddb25>] page_fault+0x25/0x30
>> [  532.942376] Mem-Info:
>> [  532.942378] DMA per-cpu:
>> [  532.942380] CPU    0: hi:    0, btch:   1 usd:   0
>> [  532.942383] CPU    1: hi:    0, btch:   1 usd:   0
>> [  532.942385] DMA32 per-cpu:
>> [  532.942388] CPU    0: hi:  186, btch:  31 usd:  94
>> [  532.942391] CPU    1: hi:  186, btch:  31 usd:  23
>> [  532.942393] Normal per-cpu:
>> [  532.942395] CPU    0: hi:  186, btch:  31 usd: 150
>> [  532.942398] CPU    1: hi:  186, btch:  31 usd: 155
>> [  532.942404] active_anon:707575 inactive_anon:264673 isolated_anon:0
>> [  532.942406]  active_file:58 inactive_file:33 isolated_file:0
>>      
>
> file cache (active_file + inactive_file) was very little. It indicate anyone waste too much memory.
> I doubt you use buggy compiler.
>
>
>
>    
hmm... then this is something with firefox then..
In that case I'll continue to build my system with
this compiler.
Although a bit concerned building everything
for the system with a compiler that shows some
memory issue,but if you say its not the compiler,
then I'll carry on with what I'm doing.
(and use an older compiler for firefox).

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Andrew Morton]

On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock <justinmattock@gmail.com> wrote:

> Hello,
> I'm not sure how to handle this,
> while compiling firefox-3.6b1.source
> I get this with the default compiling options,
> as well as custom:
> 
> ...
>
> active_anon:2360492kB inactive_anon:590196kB active_file:84kB

2.8GB of anonymous memory

> [  532.942508] Free swap  = 0kB
> [  532.942510] Total swap = 431632kB

430MB of swap, all used up.

That's a genuine OOM.  Something (presumably cc1plus) has consumed
waaaay too much memory, quite possibly leaked it.

It would help if the oom-killer were to print some information about
the oom-killed process's memory footprint.

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Andrew Morton wrote:
> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock<justinmattock@gmail.com>  wrote:
>
>    
>> Hello,
>> I'm not sure how to handle this,
>> while compiling firefox-3.6b1.source
>> I get this with the default compiling options,
>> as well as custom:
>>
>> ...
>>
>> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
>>      
>
> 2.8GB of anonymous memory
>
>    
figured it would be good enough
(I think I have 4gig's total)
>> [  532.942508] Free swap  = 0kB
>> [  532.942510] Total swap = 431632kB
>>      
>
> 430MB of swap, all used up.
>
>    
yep, narrow down to the smallest amount.
> That's a genuine OOM.  Something (presumably cc1plus) has consumed
> waaaay too much memory, quite possibly leaked it.
>
> It would help if the oom-killer were to print some information about
> the oom-killed process's memory footprint.
>
>
>    
I still have everything setup(if you need me to add a debug patch
let me know)
as for compiling: libc compiled fine, kernel fine,
and every package on the clfs list up to boot up the fresh system.
(was figuring out how to compiling/install firefox before
  I threw the old system away).

stable gcc(4.4*) on the macbook(same os/kernel) compiled fine
firefox, xulrunner, and in the process thunderbird...

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Jiri Slaby]

On 11/04/2009 07:44 AM, Justin P. Mattock wrote:
> as for compiling: libc compiled fine, kernel fine,
> and every package on the clfs list up to boot up the fresh system.

It might be pretty c++ only, I think.

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[KOSAKI Motohiro]

> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock <justinmattock@gmail.com> wrote:
> 
> > Hello,
> > I'm not sure how to handle this,
> > while compiling firefox-3.6b1.source
> > I get this with the default compiling options,
> > as well as custom:
> > 
> > ...
> >
> > active_anon:2360492kB inactive_anon:590196kB active_file:84kB
> 
> 2.8GB of anonymous memory
> 
> > [  532.942508] Free swap  = 0kB
> > [  532.942510] Total swap = 431632kB
> 
> 430MB of swap, all used up.
> 
> That's a genuine OOM.  Something (presumably cc1plus) has consumed
> waaaay too much memory, quite possibly leaked it.
> 
> It would help if the oom-killer were to print some information about
> the oom-killed process's memory footprint.
> 


How about this?

========
Subject: [PATCH] oom: show vsz and rss information of the killed process

In typical oom anylysis scenario, we frequently want to know the killed
process has memory leak or not at first step.
This patch add vsz and rss information to oom log for helping its
analysis. It save much times of debugging guys.

example:
===================================================================
rsyslogd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
Pid: 1308, comm: rsyslogd Not tainted 2.6.32-rc6 #24
Call Trace:
[<ffffffff8132e35b>] ?_spin_unlock+0x2b/0x40
[<ffffffff810f186e>] oom_kill_process+0xbe/0x2b0

(snip)

492283 pages non-shared
Out of memory: kill process 2341 (memhog) score 527276 or a child
Killed process 2341 (memhog) vsz:1054552kB, anon-rss:970588kB, file-rss:4kB
===========================================================================
                             ^
                             |
                            here

Signed-off-by: KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com>
---
 mm/oom_kill.c |   15 ++++++++++++---
 1 files changed, 12 insertions(+), 3 deletions(-)

diff --git a/mm/oom_kill.c b/mm/oom_kill.c
index ea2147d..498e6f6 100644
--- a/mm/oom_kill.c
+++ b/mm/oom_kill.c
@@ -337,6 +337,8 @@ static void dump_tasks(const struct mem_cgroup *mem)
 	} while_each_thread(g, p);
 }
 
+#define K(x) ((x) << (PAGE_SHIFT-10))
+
 /*
  * Send SIGKILL to the selected  process irrespective of  CAP_SYS_RAW_IO
  * flag though it's unlikely that  we select a process with CAP_SYS_RAW_IO
@@ -356,9 +358,16 @@ static void __oom_kill_task(struct task_struct *p, int verbose)
 		return;
 	}
 
-	if (verbose)
-		printk(KERN_ERR "Killed process %d (%s)\n",
-				task_pid_nr(p), p->comm);
+	if (verbose) {
+		task_lock(p);
+		printk(KERN_ERR "Killed process %d (%s) "
+		       "vsz:%lukB, anon-rss:%lukB, file-rss:%lukB\n",
+		       task_pid_nr(p), p->comm,
+		       K(p->mm->total_vm),
+		       K(get_mm_counter(p->mm, anon_rss)),
+		       K(get_mm_counter(p->mm, file_rss)));
+		task_unlock(p);
+	}
 
 	/*
 	 * We give our sacrificial lamb high priority and access to
-- 
1.6.2.5

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Dave Korn]

Andrew Morton wrote:
> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock <justinmattock@gmail.com> wrote:
> 
>> Hello,
>> I'm not sure how to handle this,
>> while compiling firefox-3.6b1.source
>> I get this with the default compiling options,
>> as well as custom:
>>
>> ...
>>
>> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
> 
> 2.8GB of anonymous memory
> 
>> [  532.942508] Free swap  = 0kB
>> [  532.942510] Total swap = 431632kB
> 
> 430MB of swap, all used up.
> 
> That's a genuine OOM.  Something (presumably cc1plus) has consumed
> waaaay too much memory, quite possibly leaked it.
> 
> It would help if the oom-killer were to print some information about
> the oom-killed process's memory footprint.

  I would think that the quickest way to proceed would be to re-run the
failing compile command under gdb at the command-line and see what it's doing
when the oom killer signals it, wouldn't it?  Or turn up the swap until it
doesn't get killed and see what info can be gleaned from the cc1(plus?)
-fmem-report output.

    cheers,
      DaveK

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Dave Korn wrote:
> Andrew Morton wrote:
>    
>> On Mon, 2 Nov 2009 13:29:29 -0800 Justin Mattock<justinmattock@gmail.com>  wrote:
>>
>>      
>>> Hello,
>>> I'm not sure how to handle this,
>>> while compiling firefox-3.6b1.source
>>> I get this with the default compiling options,
>>> as well as custom:
>>>
>>> ...
>>>
>>> active_anon:2360492kB inactive_anon:590196kB active_file:84kB
>>>        
>> 2.8GB of anonymous memory
>>
>>      
>>> [  532.942508] Free swap  = 0kB
>>> [  532.942510] Total swap = 431632kB
>>>        
>> 430MB of swap, all used up.
>>
>> That's a genuine OOM.  Something (presumably cc1plus) has consumed
>> waaaay too much memory, quite possibly leaked it.
>>
>> It would help if the oom-killer were to print some information about
>> the oom-killed process's memory footprint.
>>      
>
>    I would think that the quickest way to proceed would be to re-run the
> failing compile command under gdb at the command-line and see what it's doing
> when the oom killer signals it, wouldn't it?  Or turn up the swap until it
> doesn't get killed and see what info can be gleaned from the cc1(plus?)
> -fmem-report output.
>
>      cheers,
>        DaveK
>
>
>    
I can try, only issue I have is I don't
use a distro, so building anything requires me
to hand compile it(hopefully not difficult for gdb).

So give me some time on this and I'll see if I can get this up
and running, and add that patch to kernel then go from there.

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Andrew Morton]

On Wed,  4 Nov 2009 18:32:16 +0900 (JST) KOSAKI Motohiro <kosaki.motohiro@jp.fujitsu.com> wrote:

> > It would help if the oom-killer were to print some information about
> > the oom-killed process's memory footprint.
> > 
> 
> 
> How about this?

looks good, thanks.

> ========
> Subject: [PATCH] oom: show vsz and rss information of the killed process
> 
> In typical oom anylysis scenario, we frequently want to know the killed
> process has memory leak or not at first step.
> This patch add vsz and rss information to oom log for helping its
> analysis. It save much times of debugging guys.
> 
> example:
> ===================================================================
> rsyslogd invoked oom-killer: gfp_mask=0x201da, order=0, oom_adj=0
> Pid: 1308, comm: rsyslogd Not tainted 2.6.32-rc6 #24
> Call Trace:
> [<ffffffff8132e35b>] ?_spin_unlock+0x2b/0x40
> [<ffffffff810f186e>] oom_kill_process+0xbe/0x2b0
> 
> (snip)
> 
> 492283 pages non-shared
> Out of memory: kill process 2341 (memhog) score 527276 or a child
> Killed process 2341 (memhog) vsz:1054552kB, anon-rss:970588kB, file-rss:4kB
> ===========================================================================
>                              ^
>                              |
>                             here
> ...
>
> +	if (verbose) {
> +		task_lock(p);

We need to be careful with which locks we take on the oom-killer path,
because it can be called by code which already holds locks.  But I
expect task_lock() will be OK.

> +		printk(KERN_ERR "Killed process %d (%s) "
> +		       "vsz:%lukB, anon-rss:%lukB, file-rss:%lukB\n",
> +		       task_pid_nr(p), p->comm,
> +		       K(p->mm->total_vm),
> +		       K(get_mm_counter(p->mm, anon_rss)),
> +		       K(get_mm_counter(p->mm, file_rss)));
> +		task_unlock(p);
> +	}

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Dave Korn]

Justin P. Mattock wrote:

> I can try, only issue I have is I don't
> use a distro, so building anything requires me
> to hand compile it

  Oh, ouch!

> (hopefully not difficult for gdb).

  Indeed, hopefully not.

> So give me some time on this and I'll see if I can get this up
> and running, and add that patch to kernel then go from there.

  The one thing you can still try straight away for minimal effort is the
-fmem-report option, but it's also the least informative...

    cheers,
      DaveK

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin Mattock]

On Wed, Nov 4, 2009 at 7:45 AM, Dave Korn
<dave.korn.cygwin@googlemail.com> wrote:
> Justin P. Mattock wrote:
>
>> I can try, only issue I have is I don't
>> use a distro, so building anything requires me
>> to hand compile it
>
>  Oh, ouch!
>

I know.. I'm a horror for optimization

>> (hopefully not difficult for gdb).
>
>  Indeed, hopefully not.
>

you never know, some packages big/small turn into brain surgery
just to get going.(I'll try after I do some morning exercises)

>> So give me some time on this and I'll see if I can get this up
>> and running, and add that patch to kernel then go from there.
>
>  The one thing you can still try straight away for minimal effort is the
> -fmem-report option, but it's also the least informative...
>
>    cheers,
>      DaveK
>
>

O.k. here is the info from dmesg(with the patch added)
and what -fmem-report:


[  205.931940] kjournald starting.  Commit interval 5 seconds
[  205.931957] EXT3-fs warning: maximal mount count reached, running
e2fsck is recommended
[  205.935509] EXT3 FS on sdb1, internal journal
[  205.935513] EXT3-fs: mounted filesystem with writeback data mode.
[  205.956396] SELinux: initialized (dev sdb1, type ext3), uses xattr
[  434.205304] __ratelimit: 75 callbacks suppressed
[  434.205308] wicd-monitor invoked oom-killer: gfp_mask=0x201da,
order=0, oom_adj=0
[  434.205313] Pid: 1563, comm: wicd-monitor Tainted: P
2.6.32-rc5-00081-g964fe08-dirty #36
[  434.205316] Call Trace:
[  434.205325]  [<ffffffff810bc1af>] oom_kill_process+0x7c/0x243
[  434.205330]  [<ffffffff810bc6e0>] __out_of_memory+0x146/0x15d
[  434.205335]  [<ffffffff810bc909>] out_of_memory+0x6e/0x9d
[  434.205339]  [<ffffffff810bf7c0>] __alloc_pages_nodemask+0x498/0x5ce
[  434.205345]  [<ffffffff810c10e8>] __do_page_cache_readahead+0xa0/0x1a1
[  434.205350]  [<ffffffff810c1436>] ra_submit+0x1c/0x20
[  434.205353]  [<ffffffff810ba620>] filemap_fault+0x1a6/0x346
[  434.205359]  [<ffffffff810cf388>] __do_fault+0x4f/0x3d9
[  434.205363]  [<ffffffff810eec2e>] ? do_sync_read+0xe3/0x120
[  434.205369]  [<ffffffff811a2571>] ? file_has_perm+0x90/0x9e
[  434.205373]  [<ffffffff810d1cf7>] handle_mm_fault+0x3ab/0x6a7
[  434.205379]  [<ffffffff813d44a3>] do_page_fault+0x2bb/0x2d3
[  434.205383]  [<ffffffff813d23a5>] page_fault+0x25/0x30
[  434.205386] Mem-Info:
[  434.205388] DMA per-cpu:
[  434.205391] CPU    0: hi:    0, btch:   1 usd:   0
[  434.205394] CPU    1: hi:    0, btch:   1 usd:   0
[  434.205396] DMA32 per-cpu:
[  434.205399] CPU    0: hi:  186, btch:  31 usd: 125
[  434.205401] CPU    1: hi:  186, btch:  31 usd: 105
[  434.205404] Normal per-cpu:
[  434.205406] CPU    0: hi:  186, btch:  31 usd: 172
[  434.205409] CPU    1: hi:  186, btch:  31 usd: 154
[  434.205416] active_anon:708764 inactive_anon:266208 isolated_anon:0
[  434.205417]  active_file:71 inactive_file:11 isolated_file:0
[  434.205419]  unevictable:0 dirty:0 writeback:0 unstable:0 buffer:74
[  434.205420]  free:6961 slab_reclaimable:2782 slab_unreclaimable:16224
[  434.205421]  mapped:65 shmem:35 pagetables:2861 bounce:0
[  434.205430] DMA free:15944kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[  434.205438] lowmem_reserve[]: 0 2976 3986 3986
[  434.205449] DMA32 free:9976kB min:6020kB low:7524kB high:9028kB
active_anon:2360156kB inactive_anon:589924kB active_file:60kB
inactive_file:44kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
writeback:0kB mapped:88kB shmem:4kB slab_reclaimable:148kB
slab_unreclaimable:316kB kernel_stack:40kB pagetables:5952kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:225
all_unreclaimable? yes
[  434.205457] lowmem_reserve[]: 0 0 1010 1010
[  434.205468] Normal free:1924kB min:2040kB low:2548kB high:3060kB
active_anon:474900kB inactive_anon:474908kB active_file:224kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
writeback:0kB mapped:172kB shmem:136kB slab_reclaimable:10980kB
slab_unreclaimable:64572kB kernel_stack:824kB pagetables:5492kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:677
all_unreclaimable? yes
[  434.205476] lowmem_reserve[]: 0 0 0 0
[  434.205481] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
[  434.205493] DMA32: 2*4kB 14*8kB 16*16kB 10*32kB 3*64kB 1*128kB
1*256kB 1*512kB 2*1024kB 1*2048kB 1*4096kB = 9976kB
[  434.205505] Normal: 481*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1924kB
[  434.205516] 8029 total pagecache pages
[  434.205519] 7893 pages in swap cache
[  434.205521] Swap cache stats: add 112490, delete 104597, find 5058/5479
[  434.205524] Free swap  = 0kB
[  434.205526] Total swap = 431632kB
[  434.220125] 1048576 pages RAM
[  434.220127] 40493 pages reserved
[  434.220129] 170 pages shared
[  434.220131] 1000179 pages non-shared
[  434.220135] Out of memory: kill process 7925 (c++) score 539395 or a child
[  434.220141] Killed process 7926 (cc1plus) vsz:4280180kB,
anon-rss:3831924kB, file-rss:4kB
[  434.259045] cc1plus: page allocation failure. order:0, mode:0x280da
[  434.259051] Pid: 7926, comm: cc1plus Tainted: P
2.6.32-rc5-00081-g964fe08-dirty #36
[  434.259054] Call Trace:
[  434.259063]  [<ffffffff810bf874>] __alloc_pages_nodemask+0x54c/0x5ce
[  434.259070]  [<ffffffff810d1bc3>] handle_mm_fault+0x277/0x6a7
[  434.259076]  [<ffffffff813d44a3>] do_page_fault+0x2bb/0x2d3
[  434.259080]  [<ffffffff813d23a5>] page_fault+0x25/0x30
[  434.259083] Mem-Info:
[  434.259085] DMA per-cpu:
[  434.259088] CPU    0: hi:    0, btch:   1 usd:   0
[  434.259090] CPU    1: hi:    0, btch:   1 usd:   0
[  434.259092] DMA32 per-cpu:
[  434.259095] CPU    0: hi:  186, btch:  31 usd: 125
[  434.259098] CPU    1: hi:  186, btch:  31 usd: 105
[  434.259100] Normal per-cpu:
[  434.259103] CPU    0: hi:  186, btch:  31 usd: 172
[  434.259106] CPU    1: hi:  186, btch:  31 usd: 154
[  434.259113] active_anon:708764 inactive_anon:266208 isolated_anon:0
[  434.259115]  active_file:71 inactive_file:11 isolated_file:0
[  434.259116]  unevictable:0 dirty:0 writeback:0 unstable:0 buffer:74
[  434.259117]  free:6961 slab_reclaimable:2782 slab_unreclaimable:16224
[  434.259119]  mapped:65 shmem:35 pagetables:2861 bounce:0
[  434.259128] DMA free:15944kB min:28kB low:32kB high:40kB
active_anon:0kB inactive_anon:0kB active_file:0kB inactive_file:0kB
unevictable:0kB isolated(anon):0kB isolated(file):0kB present:15360kB
mlocked:0kB dirty:0kB writeback:0kB mapped:0kB shmem:0kB
slab_reclaimable:0kB slab_unreclaimable:8kB kernel_stack:0kB
pagetables:0kB unstable:0kB bounce:0kB writeback_tmp:0kB
pages_scanned:0 all_unreclaimable? yes
[  434.259135] lowmem_reserve[]: 0 2976 3986 3986
[  434.259147] DMA32 free:9976kB min:6020kB low:7524kB high:9028kB
active_anon:2360156kB inactive_anon:589924kB active_file:60kB
inactive_file:44kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:3047792kB mlocked:0kB dirty:0kB
writeback:0kB mapped:88kB shmem:4kB slab_reclaimable:148kB
slab_unreclaimable:316kB kernel_stack:40kB pagetables:5952kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:225
all_unreclaimable? yes
[  434.259154] lowmem_reserve[]: 0 0 1010 1010
[  434.259166] Normal free:1924kB min:2040kB low:2548kB high:3060kB
active_anon:474900kB inactive_anon:474908kB active_file:224kB
inactive_file:0kB unevictable:0kB isolated(anon):0kB
isolated(file):0kB present:1034240kB mlocked:0kB dirty:0kB
writeback:0kB mapped:172kB shmem:136kB slab_reclaimable:10980kB
slab_unreclaimable:64572kB kernel_stack:824kB pagetables:5492kB
unstable:0kB bounce:0kB writeback_tmp:0kB pages_scanned:677
all_unreclaimable? yes
[  434.259173] lowmem_reserve[]: 0 0 0 0
[  434.259178] DMA: 2*4kB 2*8kB 3*16kB 2*32kB 3*64kB 2*128kB 2*256kB
1*512kB 2*1024kB 2*2048kB 2*4096kB = 15944kB
[  434.259190] DMA32: 2*4kB 14*8kB 16*16kB 10*32kB 3*64kB 1*128kB
1*256kB 1*512kB 2*1024kB 1*2048kB 1*4096kB = 9976kB
[  434.259202] Normal: 481*4kB 0*8kB 0*16kB 0*32kB 0*64kB 0*128kB
0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 1924kB
[  434.259214] 8029 total pagecache pages
[  434.259216] 7893 pages in swap cache
[  434.259219] Swap cache stats: add 112490, delete 104597, find 5058/5480
[  434.259221] Free swap  = 0kB
[  434.259223] Total swap = 431632kB
[  434.273830] 1048576 pages RAM
[  434.273832] 40493 pages reserved
[  434.273834] 170 pages shared
[  434.273836] 1000179 pages non-shared




/********** and -fmem-report **************/




Memory still allocated at the end of the compilation process
Size   Allocated        Used    Overhead
8             56k         43k       1680
16           196k         81k       4312
32           188k         40k       3384
64           336k        232k       5376
128          132k        128k       1848
512           28k         13k        392
1024          20k       9216         280
2048          16k       8192         224
4096         348k        348k       4872
8192          56k         56k        392
16384         16k         16k         56
32768         64k         64k        112
24           256k         53k       4608
40           256k        199k       4096
48           800k        579k         12k
56            92k       6608        1472
72            72k       5040        1008
80          8192        1440         112
88          8192         704         112
96           940k        580k         12k
112           80k         39k       1120
120           16k        840         224
192          192k        157k       2688
136          600k        580k       8400
160          200k        172k       2800
176          976k        795k         13k
152           84k         33k       1176
104          120k         28k       1680
256         1040k       1019k         14k
144         4096         144          56
Total       7200k       5293k        104k

String pool
entries         9928
identifiers     6178 (62.23%)
slots           16384
deleted         3726
bytes           86k (17592186044415M overhead)
table size      128k
coll/search     0.3177
ins/search      0.1518
avg. entry      8.92 bytes (+/- 9.51)
longest entry   112

??? tree nodes created

(No per-node statistics)
Type hash: size 4093, 2550 elements, 1.102267 collisions
DECL_DEBUG_EXPR  hash: size 1021, 0 elements, 0.015692 collisions
DECL_VALUE_EXPR  hash: size 1021, 0 elements, 0.000000 collisions
no search statistics
No gimple statistics

Alias oracle query stats:
  refs_may_alias_p: 11 disambiguations, 29 queries
  ref_maybe_used_by_call_p: 0 disambiguations, 62 queries
  call_may_clobber_ref_p: 0 disambiguations, 0 queries

PTA query stats:
  pt_solution_includes: 7 disambiguations, 166 queries
  pt_solutions_intersect: 0 disambiguations, 507 queries
jsxml.cpp
c++ -o jsxml.o -c  -DOSTYPE=\"Linux2.6\" -DOSARCH=Linux
-DEXPORT_JS_API  -DJS_USE_SAFE_ARENA
-I/home/name/LFS/firefox/mozilla-1.9.2/js/src -I.
-I./../../dist/include -I./../../dist/include/nsprpub
-I/usr/include/nspr
-I/home/name/LFS/firefox/mozilla-1.9.2/js/src    -fPIC   -fno-rtti
-fno-exceptions -Wall -Wpointer-arith -Woverloaded-virtual
-Wsynth -Wno-ctor-dtor-privacy -Wno-non-virtual-dtor -Wcast-align
-Wno-invalid-offsetof -Wno-variadic-macros -Wno-long-long
-pedantic -fno-strict-aliasing -pthread -pipe  -DNDEBUG -DTRIMMED -m64
-mtune=core2 -march=core2 -O2 -pipe -fmem-report
-fomit-frame-pointer   -DMOZILLA_CLIENT -include ./js-confdefs.h
-Wp,-MD,.deps/jsxml.pp
/home/name/LFS/firefox/mozilla-1.9.2/js/src/jsxml.cpp
{standard input}: Assembler messages:
{standard input}:271839: Warning: end of file not at end of a line;
newline inserted
{standard input}:271896: Error: suffix or operands invalid for `movq'
{standard input}:271896: Error: open CFI at the end of file; missing
.cfi_endproc directive
c++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[4]: *** [jsxml.o] Error 1
make[4]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu/js/src'
make[3]: *** [libs_tier_js] Error 2
make[3]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[2]: *** [tier_js] Error 2
make[2]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[1]: *** [default] Error 2
make[1]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make: *** [build] Error 2




-- 
Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Dave Korn]

Justin Mattock wrote:

> O.k. here is the info from dmesg(with the patch added)
> and what -fmem-report:

  I don't know how to read the oom dmesg, but as to the -fmem-report:

> Memory still allocated at the end of the compilation process
> Size   Allocated        Used    Overhead
> Total       7200k       5293k        104k

... what that's telling us is that there isn't a substantial leak in GCC, as
there's only 7 meg left unreclaimed by GC at the end.  I think we'll have to
wait and see what the debugger tells us; either GCC really is using that much
memory in processing the file, or there's some kind of system or kernel bug
you're running into that is causing a leak in the VMM rather than the application.

> String pool

> bytes           86k (17592186044415M overhead)

  0xFFFFFFFFFFF00000, lol, wut?  It's possible that indicates some sort of
memory corruption going on.  Maybe valgrind can help, do you have that?

    cheers,
      DaveK

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Dave Korn wrote:
> Justin Mattock wrote:
>
>    
>> O.k. here is the info from dmesg(with the patch added)
>> and what -fmem-report:
>>      
>
>    I don't know how to read the oom dmesg, but as to the -fmem-report:
>
>    
>> Memory still allocated at the end of the compilation process
>> Size   Allocated        Used    Overhead
>> Total       7200k       5293k        104k
>>      
>
> ... what that's telling us is that there isn't a substantial leak in GCC, as
> there's only 7 meg left unreclaimed by GC at the end.  I think we'll have to
> wait and see what the debugger tells us; either GCC really is using that much
> memory in processing the file, or there's some kind of system or kernel bug
> you're running into that is causing a leak in the VMM rather than the application.
>
>    
just finished compiling and installing gdb/valgrind
>> String pool
>>      
>
>    
>> bytes           86k (17592186044415M overhead)
>>      
>
>    0xFFFFFFFFFFF00000, lol, wut?  It's possible that indicates some sort of
> memory corruption going on.  Maybe valgrind can help, do you have that?
>
>      cheers,
>        DaveK
>
>
>    
Not sure how to use these.(need to read)
Any quick commands I can do to get the info
to you?

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin Mattock]

here's what I did:
 valgrind --tool=memcheck --leak-check=full -v make -f client.mk build


e2 -march=core2 -O2 -pipe -fomit-frame-pointer   -DMOZILLA_CLIENT
-include ./js-confdefs.h -Wp,-MD,.deps/jsxml.pp
/home/justin/LFS/firefox/mozilla-1.9.2/js/src/jsxml.cpp
{standard input}: Assembler messages:
{standard input}:271839: Warning: end of file not at end of a line;
newline inserted
{standard input}:271896: Error: suffix or operands invalid for `movq'
{standard input}:271896: Error: open CFI at the end of file; missing
.cfi_endproc directive
c++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
make[4]: *** [jsxml.o] Error 1
make[4]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu/js/src'
make[3]: *** [libs_tier_js] Error 2
make[3]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[2]: *** [tier_js] Error 2
make[2]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make[1]: *** [default] Error 2
make[1]: Leaving directory
`/home/name/LFS/firefox/mozilla-1.9.2/obj-x86_64-unknown-linux-gnu'
make: *** [build] Error 2
==4072==
==4072== HEAP SUMMARY:
==4072==     in use at exit: 201,183 bytes in 4,237 blocks
==4072==   total heap usage: 28,879 allocs, 24,642 frees, 2,947,434
bytes allocated
==4072==
==4072== Searching for pointers to 4,237 not-freed blocks
==4072== Checked 268,808 bytes
==4072==
==4072== 6 bytes in 1 blocks are possibly lost in loss record 41 of 295
==4072==    at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==4072==    by 0x50ABEE1: strdup (in /lib/libc-2.10.90.so)
==4072==    by 0x4118D8: xstrdup (in /usr/bin/make)
==4072==    by 0x41BAA4: define_variable_in_set (in /usr/bin/make)
==4072==    by 0x4160B4: eval (in /usr/bin/make)
==4072==    by 0x416766: eval_makefile (in /usr/bin/make)
==4072==    by 0x416A3F: read_all_makefiles (in /usr/bin/make)
==4072==    by 0x410853: main (in /usr/bin/make)
==4072==
==4072== 14 bytes in 1 blocks are possibly lost in loss record 69 of 295
==4072==    at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==4072==    by 0x411977: xmalloc (in /usr/bin/make)
==4072==    by 0x411AA8: savestring (in /usr/bin/make)
==4072==    by 0x41BB7A: define_variable_in_set (in /usr/bin/make)
==4072==    by 0x410832: main (in /usr/bin/make)
==4072==
==4072== LEAK SUMMARY:
==4072==    definitely lost: 0 bytes in 0 blocks
==4072==    indirectly lost: 0 bytes in 0 blocks
==4072==      possibly lost: 20 bytes in 2 blocks
==4072==    still reachable: 201,163 bytes in 4,235 blocks
==4072==         suppressed: 0 bytes in 0 blocks
==4072== Reachable blocks (those to which a pointer was found) are not shown.
==4072== To see them, rerun with: --leak-check=full --show-reachable=yes
==4072==
==4072== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 5 from 5)
--4072--
--4072-- used_suppression:      2 dl-hack3-cond-1
--4072-- used_suppression:      3 glibc-2.5.x-on-SUSE-10.2-(PPC)-2a
==4072==
==4072== ERROR SUMMARY: 2 errors from 2 contexts (suppressed: 5 from 5)

I'll try out gdb, and more of valgrind.

-- 
Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[KOSAKI Motohiro]

> > +	if (verbose) {
> > +		task_lock(p);
> 
> We need to be careful with which locks we take on the oom-killer path,
> because it can be called by code which already holds locks.  But I
> expect task_lock() will be OK.

Sure.
task_lock() is already used various oom path. I think this is ok.

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Dave Korn]

Justin Mattock wrote:
> here's what I did:
>  valgrind --tool=memcheck --leak-check=full -v make -f client.mk build

> ==4072== LEAK SUMMARY:

> I'll try out gdb, and more of valgrind.

  Yep, that doesn't tell us a lot in its default modes.  I'm not a valgrind
expert but it looks from the docs like you want to try the Massif tool: it
looks really thorough.

http://valgrind.org/docs/manual/ms-manual.html

    cheers,
      DaveK

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin Mattock]

On Wed, Nov 4, 2009 at 4:36 PM, Dave Korn
<dave.korn.cygwin@googlemail.com> wrote:
> Justin Mattock wrote:
>> here's what I did:
>>  valgrind --tool=memcheck --leak-check=full -v make -f client.mk build
>
>> ==4072== LEAK SUMMARY:
>
>> I'll try out gdb, and more of valgrind.
>
>  Yep, that doesn't tell us a lot in its default modes.  I'm not a valgrind
> expert but it looks from the docs like you want to try the Massif tool: it
> looks really thorough.
>
> http://valgrind.org/docs/manual/ms-manual.html
>
>    cheers,
>      DaveK
>
>
>

o.k. round2 I can try gdb(now that I have somewhat of an idea of
how to use one of these tools).

in the meanwhile hopefully this is useful(if not just clip, and I'll try
to gather something useful)


using this for valgrind and the command(below in log)
valgrind -v --tool=memcheck --leak-check=yes --num-callers=10
--leak-check=full --show-reachable=yes --gen-suppressions=yes
(log of valgrind is a bit long, so here's just a few of what
suppresions printed out:)

==1830== Memcheck, a memory error detector
==1830== Copyright (C) 2002-2009, and GNU GPL'd, by Julian Seward et al.
==1830== Using Valgrind-3.5.0 and LibVEX; rerun with -h for copyright info
==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
-DEXPORT_JS_API -DJS_USE_SAFE_ARENA
-I/home/justin/LFS/firefox/mozilla-1.9.2/js/src -I.
-I./../../dist/include -I./../../dist/include/nsprpub
-I/usr/include/nspr -I/home/justin/LFS/firefox/mozilla-1.9.2/js/src
-fPIC -fno-rtti -fno-exceptions -Wall -Wpointer-arith
-Woverloaded-virtual -Wsynth -Wno-ctor-dtor-privacy
-Wno-non-virtual-dtor -Wcast-align -Wno-invalid-offsetof
-Wno-variadic-macros -Wno-long-long -pedantic -fno-strict-aliasing
-pthread -pipe -DNDEBUG -DTRIMMED -m64 -mtune=core2 -march=core2 -O2
-pipe -fomit-frame-pointer -DMOZILLA_CLIENT -include ./js-confdefs.h
-Wp,-MD,.deps/jsxml.pp
/home/justin/LFS/firefox/mozilla-1.9.2/js/src/jsxml.cpp
==1830==
--1830-- Valgrind options:
--1830--    -v
--1830--    --tool=memcheck
--1830--    --leak-check=yes
--1830--    --num-callers=10
--1830--    --leak-check=full
--1830--    --show-reachable=yes
--1830--    --gen-suppressions=yes
--1830-- Contents of /proc/version:
--1830--   Linux version 2.6.32-rc5-00083-g04ea458 (justin@Linux-0)
(gcc version 4.4.1 (GCC for Cross-LFS 4.4.1.20090722) ) #2 SMP Sat Oct
24 21:38:54 PDT 2009
--1830-- Arch and hwcaps: AMD64, amd64-sse3-cx16
--1830-- Page sizes: currently 4096, max supported 4096
--1830-- Valgrind library directory: /usr/lib/valgrind
--1830-- Reading syms from /usr/bin/c++ (0x400000)
--1830-- Reading syms from /lib/ld-2.10.90.so (0x4000000)
--1830-- Reading syms from /usr/lib/valgrind/memcheck-amd64-linux (0x38000000)
--1830--    object doesn't have a dynamic symbol table
--1830-- Reading suppressions file: /usr/lib/valgrind/default.supp
--1830-- REDIR: 0x4016110 (strlen) redirected to 0x38040607
(vgPlain_amd64_linux_REDIR_FOR_strlen)
--1830-- Reading syms from
/usr/lib/valgrind/vgpreload_core-amd64-linux.so (0x4a20000)
--1830-- Reading syms from
/usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so (0x4c21000)
==1830== WARNING: new redirection conflicts with existing -- ignoring it
--1830--     new: 0x04016110 (strlen              ) R-> 0x04c24fd0 strlen
--1830-- REDIR: 0x4015f30 (index) redirected to 0x4c24d50 (index)
--1830-- REDIR: 0x40160e0 (strcmp) redirected to 0x4c252e0 (strcmp)
--1830-- Reading syms from /lib/libc-2.10.90.so (0x4e28000)
--1830-- REDIR: 0x4ea4c40 (rindex) redirected to 0x4c24bb0 (rindex)
--1830-- REDIR: 0x4e9e010 (malloc) redirected to 0x4c24808 (malloc)
--1830-- REDIR: 0x4ea3330 (strncmp) redirected to 0x4c25210 (strncmp)
--1830-- REDIR: 0x4ea7670 (memcpy) redirected to 0x4c253e0 (memcpy)
--1830-- REDIR: 0x4e9df30 (free) redirected to 0x4c23b44 (free)
--1830-- REDIR: 0x4ea17d0 (strcmp) redirected to 0x4c25280 (strcmp)
--1830-- REDIR: 0x4ea3170 (strlen) redirected to 0x4c24f90 (strlen)
--1830-- REDIR: 0x4ea1750 (index) redirected to 0x4c24c50 (index)
--1830-- REDIR: 0x4ea9da0 (strchrnul) redirected to 0x4c25ff0 (strchrnul)
--1830-- REDIR: 0x4ea6d30 (mempcpy) redirected to 0x4c260e0 (mempcpy)
--1830-- REDIR: 0x4ea5bc0 (memchr) redirected to 0x4c253a0 (memchr)
--1830-- REDIR: 0x4e9f060 (realloc) redirected to 0x4c248b9 (realloc)
--1830-- REDIR: 0x4ea31c0 (strnlen) redirected to 0x4c24f60 (strnlen)
--1830-- REDIR: 0x4ea7360 (stpcpy) redirected to 0x4c25c10 (stpcpy)
--1830-- REDIR: 0x4ea2c20 (strcpy) redirected to 0x4c24ff0 (strcpy)
--1830-- REDIR: 0x4ea9d50 (rawmemchr) redirected to 0x4c26020 (rawmemchr)
--1830-- REDIR: 0x4e9d5d0 (calloc) redirected to 0x4c2322c (calloc)
--1830-- REDIR: 0x4ea4b80 (strncpy) redirected to 0x4c250c0 (strncpy)
--1830-- REDIR: 0x4ea1590 (strcat) redirected to 0x4c24d90 (strcat)
--1830-- REDIR: 0x4e5bf40 (putenv) redirected to 0x4c263d0 (putenv)
{standard input}: Assembler messages:
{standard input}:271839: Warning: end of file not at end of a line;
newline inserted
{standard input}:271896: Error: suffix or operands invalid for `movq'
{standard input}:271896: Error: open CFI at the end of file; missing
.cfi_endproc directive
c++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See <http://gcc.gnu.org/bugs.html> for instructions.
==1830==
==1830== HEAP SUMMARY:
==1830==     in use at exit: 25,444 bytes in 75 blocks
==1830==   total heap usage: 212 allocs, 137 frees, 36,791 bytes allocated
==1830==
==1830== Searching for pointers to 75 not-freed blocks
==1830== Checked 74,760 bytes
==1830==
==1830== 1 bytes in 1 blocks are still reachable in loss record 1 of 59
==1830==    at 0x4C232DA: calloc (vg_replace_malloc.c:418)
==1830==    by 0x41BE49: xcalloc (xmalloc.c:162)
==1830==    by 0x40F5D6: main (gcc.c:7270)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   fun:calloc
   fun:xcalloc
   fun:main
}
==1830== 4 bytes in 1 blocks are still reachable in loss record 2 of 59
==1830==    at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==1830==    by 0x41BDF7: xmalloc (xmalloc.c:147)
==1830==    by 0x402255: save_string (gcc.c:7660)
==1830==    by 0x4034ED: add_preprocessor_option (gcc.c:3447)
==1830==    by 0x408A06: process_command (gcc.c:3943)
==1830==    by 0x40DE6F: main (gcc.c:6871)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   fun:malloc
   fun:xmalloc
   fun:save_string
   fun:add_preprocessor_option
   fun:process_command
   fun:main
}
==1830== 6 bytes in 1 blocks are still reachable in loss record 3 of 59
==1830==    at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==1830==    by 0x41BDF7: xmalloc (xmalloc.c:147)
==1830==    by 0x41BF11: xstrdup (xstrdup.c:34)
==1830==    by 0x411F75: update_path (prefix.c:273)
==1830==    by 0x403631: add_prefix (gcc.c:2871)
==1830==    by 0x40E1CF: main (gcc.c:7043)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   fun:malloc
   fun:xmalloc
   fun:xstrdup
   fun:update_path
   fun:add_prefix
   fun:main
}
==1830== 7 bytes in 1 blocks are still reachable in loss record 4 of 59
==1830==    at 0x4C2488A: malloc (vg_replace_malloc.c:195)
==1830==    by 0x41BDF7: xmalloc (xmalloc.c:147)
==1830==    by 0x40E64E: main (gcc.c:8236)
==1830==
==1830==
==1830== ---- Print suppression ? --- [Return/N/n/Y/y/C/c] ---- y
{
   <insert_a_suppression_name_here>
   Memcheck:Leak
   fun:malloc
   fun:xmalloc
   fun:main
}


all I have to do is go into the directory
obj-x86_64-unknown-linux-gnu/js/src/
of mozillas source tree and use valgrind c++ ***
(I'll see what gdb does)

-- 
Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Dave Korn]

Justin Mattock wrote:


> ==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux

  Ah, you're running it on the "c++" utility and it's reporting the stats for
that, but how it works is that "c++" (and "gcc", "g++", et al) is just a
driver, that parses the command line arguments and shells out to the actual
compiler ("cc1plus"), assembler and linker to get them to do all the work.

  If you add "-v --save-temps" to the c++ invocation, it'll show you the
separate command lines it executes for the subprograms; the first invocation
will be of cc1plus, using the -E flag to generate the preprocessed source into
a .ii file, it's the second invocation you want, the one which uses the
"-fpreprocessed" flag and names the .ii file as input, which is the one that
actually then compiles the pre-processed source into assembly.  For fuller
explanation, see the GCC wiki:

http://gcc.gnu.org/wiki/DebuggingGCC

    cheers,
      DaveK

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Dave Korn wrote:
> Justin Mattock wrote:
>
>
>    
>> ==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
>>      
>
>    Ah, you're running it on the "c++" utility and it's reporting the stats for
> that, but how it works is that "c++" (and "gcc", "g++", et al) is just a
> driver, that parses the command line arguments and shells out to the actual
> compiler ("cc1plus"), assembler and linker to get them to do all the work.
>
>    If you add "-v --save-temps" to the c++ invocation, it'll show you the
> separate command lines it executes for the subprograms; the first invocation
> will be of cc1plus, using the -E flag to generate the preprocessed source into
> a .ii file, it's the second invocation you want, the one which uses the
> "-fpreprocessed" flag and names the .ii file as input, which is the one that
> actually then compiles the pre-processed source into assembly.  For fuller
> explanation, see the GCC wiki:
>
> http://gcc.gnu.org/wiki/DebuggingGCC
>
>      cheers,
>        DaveK
>
>
>
>    
o.k. after some reading and some experimenting,
I finally got firefox to compile all the way without a
memory leak..
here's what I found:
using the default .mozconfig I see during compiling there
is a -O3 in the CFLAGS(mozilla default optimization)
When using my optimization I use -O2
by mistake I accidentally forgot to add the -O2 while
tweaking my .mozconfig for a right CFLAGS to use for
debugging, in doing so firefox compiled all the way.

So using a custom CFLAGS in mozconfig and leaving out
-O* works..

Any ideas what might be happening here?

Justin P. Mattock

Re: cc1plus invoked oom-killer: gfp_mask=0x280da, order=0, oom_adj=0

[Justin P. Mattock]

Dave Korn wrote:
> Justin Mattock wrote:
>
>
>    
>> ==1830== Command: c++ -o jsxml.o -c -DOSTYPE="Linux2.6" -DOSARCH=Linux
>>      
>
>    Ah, you're running it on the "c++" utility and it's reporting the stats for
> that, but how it works is that "c++" (and "gcc", "g++", et al) is just a
> driver, that parses the command line arguments and shells out to the actual
> compiler ("cc1plus"), assembler and linker to get them to do all the work.
>
>    If you add "-v --save-temps" to the c++ invocation, it'll show you the
> separate command lines it executes for the subprograms; the first invocation
> will be of cc1plus, using the -E flag to generate the preprocessed source into
> a .ii file, it's the second invocation you want, the one which uses the
> "-fpreprocessed" flag and names the .ii file as input, which is the one that
> actually then compiles the pre-processed source into assembly.  For fuller
> explanation, see the GCC wiki:
>
> http://gcc.gnu.org/wiki/DebuggingGCC
>
>      cheers,
>        DaveK
>
>
>
>    
I didn't think at the time, but when compiling
4.5.*(snapshot) I added the pure64 patch to
gcc, everything seemed to go fine, but maybe
there needs to be more to it.

would this might cause an issue(memory leak or something)
like what I was receiving even though there's a symlink
to lib64?

BTW: I just cleared the deck and started fresh,
compiled gcc+pure64 patch(snapshot) into a single directory,
and then looked at ldd /usr/bin/gcc*
I see it pointing to /lib64(looking on my other system with
4.4.1 it points to /lib)

Justin P. Mattock