* swapper: page allocation failure.
@ 2006-05-21 8:10 Haar János
2006-05-21 8:16 ` Chris Wedgwood
2006-05-21 12:03 ` Andrew Morton
0 siblings, 2 replies; 28+ messages in thread
From: Haar János @ 2006-05-21 8:10 UTC (permalink / raw)
To: linux-kernel
Hello, list,
I seriously gets this, and dont know why.
This server have 2GB ram, and ~1.1G always free!
Anybody have an idea?
Thanks,
Janos
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel: <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f88315de>
e1000_clean_rx_irq+0x4fd/0x50a [e1000]
May 21 09:05:35 st-0003 kernel: <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel: <c01369cf> __do_IRQ+0x7c/0xcd <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel: <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel: <c01016c0> cpu_idle+0x5a/0x6f <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages: 1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap: 0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel: <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel: <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel: <c01369cf> __do_IRQ+0x7c/0xcd <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel: <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel: <c01016c0> cpu_idle+0x5a/0x6f <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages: 1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap: 0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel: <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel: <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel: <c01369cf> __do_IRQ+0x7c/0xcd <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel: <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel: <c01016c0> cpu_idle+0x5a/0x6f <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages: 1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap: 0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel: <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel: <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel: <c01369cf> __do_IRQ+0x7c/0xcd <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel: <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel: <c01016c0> cpu_idle+0x5a/0x6f <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages: 1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap: 0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel: <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel: <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel: <c01369cf> __do_IRQ+0x7c/0xcd <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel: <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel: <c01016c0> cpu_idle+0x5a/0x6f <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages: 1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap: 0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
mode:0x20
May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
<c014af1d> cache_alloc_refill+0x2a6/0x45c
May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
__alloc_skb+0x49/0xf5
May 21 09:05:35 st-0003 kernel: <f88336b1>
e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f8831570>
e1000_clean_rx_irq+0x48f/0x50a [e1000]
May 21 09:05:35 st-0003 kernel: <f88342d2> e1000_intr+0x96/0xf4 [e1000]
<c0136927> handle_IRQ_event+0x20/0x4c
May 21 09:05:35 st-0003 kernel: <c01369cf> __do_IRQ+0x7c/0xcd <c0104a65>
do_IRQ+0x44/0x53
May 21 09:05:35 st-0003 kernel: <c01030ba> common_interrupt+0x1a/0x20
<c01016f5> mwait_idle+0x20/0x34
May 21 09:05:35 st-0003 kernel: <c01016c0> cpu_idle+0x5a/0x6f <c05966c9>
start_kernel+0x317/0x31d
May 21 09:05:35 st-0003 kernel: Mem-info:
May 21 09:05:35 st-0003 kernel: DMA per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 0, batch 1 used:0
May 21 09:05:35 st-0003 kernel: DMA32 per-cpu: empty
May 21 09:05:35 st-0003 kernel: Normal per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:30
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:47
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:51
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:55
May 21 09:05:35 st-0003 kernel: HighMem per-cpu:
May 21 09:05:35 st-0003 kernel: cpu 0 hot: high 186, batch 31 used:98
May 21 09:05:35 st-0003 kernel: cpu 0 cold: high 62, batch 15 used:10
May 21 09:05:35 st-0003 kernel: cpu 1 hot: high 186, batch 31 used:183
May 21 09:05:35 st-0003 kernel: cpu 1 cold: high 62, batch 15 used:6
May 21 09:05:35 st-0003 kernel: Free pages: 1165384kB (1160424kB
HighMem)
May 21 09:05:35 st-0003 kernel: Active:3002 inactive:212594 dirty:35905
writeback:1 unstable:0 free:291346 slab:6782 mapped:929 pagetables:57
May 21 09:05:35 st-0003 kernel: DMA free:3548kB min:68kB low:84kB high:100kB
active:0kB inactive:5808kB present:16384kB pages_scanned:0
all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: DMA32 free:0kB min:0kB low:0kB high:0kB
active:0kB inactive:0kB present:0kB pages_scanned:0 all_unreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 880 2031
May 21 09:05:35 st-0003 kernel: Normal free:1412kB min:3756kB low:4692kB
high:5632kB active:8kB inactive:838708kB present:901120kB pages_scanned:0
all_unrecla
imable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 9215
May 21 09:05:35 st-0003 kernel: HighMem free:1160424kB min:512kB low:1740kB
high:2972kB active:12000kB inactive:5860kB present:1179584kB pages_scanned:0
all_u
nreclaimable? no
May 21 09:05:35 st-0003 kernel: lowmem_reserve[]: 0 0 0 0
May 21 09:05:35 st-0003 kernel: DMA: 1*4kB 1*8kB 1*16kB 0*32kB 1*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 0*4096kB = 3548kB
May 21 09:05:35 st-0003 kernel: DMA32: empty
May 21 09:05:35 st-0003 kernel: Normal: 1*4kB 0*8kB 0*16kB 0*32kB 0*64kB
1*128kB 1*256kB 0*512kB 1*1024kB 0*2048kB 0*4096kB = 1412kB
May 21 09:05:35 st-0003 kernel: HighMem: 142*4kB 34*8kB 18*16kB 8*32kB
6*64kB 2*128kB 1*256kB 0*512kB 1*1024kB 1*2048kB 282*4096kB = 1160424kB
May 21 09:05:35 st-0003 kernel: Free swap: 0kB
May 21 09:05:35 st-0003 kernel: 524272 pages of RAM
May 21 09:05:35 st-0003 kernel: 294896 pages of HIGHMEM
May 21 09:05:35 st-0003 kernel: 6010 reserved pages
May 21 09:05:35 st-0003 kernel: 213818 pages shared
May 21 09:05:35 st-0003 kernel: 0 pages swap cached
May 21 09:05:35 st-0003 kernel: 35905 pages dirty
May 21 09:05:35 st-0003 kernel: 1 pages writeback
May 21 09:05:35 st-0003 kernel: 929 pages mapped
May 21 09:05:35 st-0003 kernel: 6782 pages slab
May 21 09:05:35 st-0003 kernel: 57 pages pagetables
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 8:10 swapper: page allocation failure Haar János
@ 2006-05-21 8:16 ` Chris Wedgwood
2006-05-21 8:37 ` Haar János
2006-05-21 12:03 ` Andrew Morton
1 sibling, 1 reply; 28+ messages in thread
From: Chris Wedgwood @ 2006-05-21 8:16 UTC (permalink / raw)
To: Haar J?nos; +Cc: linux-kernel
On Sun, May 21, 2006 at 10:10:13AM +0200, Haar J?nos wrote:
> This server have 2GB ram, and ~1.1G always free!
> Anybody have an idea?
you ran out of lowmem?
what kernel is this and how do you trigger it?
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 8:16 ` Chris Wedgwood
@ 2006-05-21 8:37 ` Haar János
2006-05-21 8:47 ` Chris Wedgwood
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-21 8:37 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: linux-kernel
----- Original Message -----
From: "Chris Wedgwood" <cw@f00f.org>
To: "Haar J?nos" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 10:16 AM
Subject: Re: swapper: page allocation failure.
> On Sun, May 21, 2006 at 10:10:13AM +0200, Haar J?nos wrote:
>
> > This server have 2GB ram, and ~1.1G always free!
> > Anybody have an idea?
>
> you ran out of lowmem?
>
> what kernel is this and how do you trigger it?
I have 4 disk nodes that acts as nbd-server.
The only processes:
PID TTY STAT TIME COMMAND
1 ? S 0:12 init [3]
2 ? SW 0:00 [migration/0]
3 ? SWN 0:19 [ksoftirqd/0]
4 ? SW 0:00 [migration/1]
5 ? SWN 0:03 [ksoftirqd/1]
6 ? SW< 0:00 [events/0]
7 ? SW< 0:00 [events/1]
8 ? SW< 0:00 [khelper]
9 ? SW< 0:00 [kthread]
12 ? SW< 0:07 \_ [kblockd/0]
13 ? SW< 0:02 \_ [kblockd/1]
14 ? SW< 0:00 \_ [kacpid]
101 ? SW< 0:00 \_ [khubd]
103 ? SW< 0:00 \_ [kseriod]
247 ? SW 12:09 \_ [pdflush]
248 ? SW 10:04 \_ [pdflush]
250 ? SW< 0:00 \_ [aio/0]
251 ? SW< 0:00 \_ [aio/1]
252 ? SW< 0:00 \_ [xfslogd/0]
253 ? SW< 0:00 \_ [xfslogd/1]
254 ? SW< 0:00 \_ [xfsdatad/0]
255 ? SW< 0:00 \_ [xfsdatad/1]
921 ? SW< 0:00 \_ [ata/0]
922 ? SW< 0:00 \_ [ata/1]
926 ? SW< 0:00 \_ [scsi_eh_0]
927 ? SW< 0:00 \_ [scsi_eh_1]
935 ? SW< 0:00 \_ [scsi_eh_2]
936 ? SW< 0:00 \_ [scsi_eh_3]
937 ? SW< 0:00 \_ [scsi_eh_4]
938 ? SW< 0:00 \_ [scsi_eh_5]
951 ? SW< 0:00 \_ [scsi_eh_6]
952 ? SW< 0:00 \_ [scsi_eh_7]
1045 ? SW< 0:00 \_ [exec-osm/0]
1047 ? SW< 0:00 \_ [exec-osm/1]
1088 ? SW< 0:00 \_ [kcryptd/0]
1089 ? SW< 0:00 \_ [kcryptd/1]
1090 ? SW< 0:00 \_ [kmpathd/0]
1091 ? SW< 0:00 \_ [kmpathd/1]
1092 ? SW< 0:00 \_ [kmirrord]
1093 ? SW< 0:14 \_ [kedac]
1111 ? SW< 219:01 \_ [md0_raid5]
1112 ? SW< 0:23 \_ [rpciod/0]
1113 ? SW< 0:00 \_ [rpciod/1]
249 ? SW 51:30 [kswapd0]
2532 ? S 0:05 syslogd -m 0
2536 ? S 0:00 klogd -x
2615 ? S 0:00 /usr/sbin/sshd
21484 ? S 0:00 \_ /usr/sbin/sshd
21486 pts/2 S 0:00 \_ -bash
22444 pts/2 R 0:00 \_ ps fax
2629 ? S 0:03 crond
2638 ? S 0:55 /bin/bwbar --text-file
/mnt/DY_SYSTEM/sysinfo/st4-eth0_o.php --png-file
/mnt/DY_SYSTEM/sysinfo/st4-eth0_o.png eth0 100 -x 300
2639 ? S 0:54 /bin/bwbar --text-file
/mnt/DY_SYSTEM/sysinfo/st4-eth0_i.php -i --png-file
/mnt/DY_SYSTEM/sysinfo/st4-eth0_i.png eth0 100 -x 300
2674 tty2 S 0:00 /sbin/mingetty tty2
2675 tty3 S 0:00 /sbin/mingetty tty3
2676 tty4 S 0:00 /sbin/mingetty tty4
2677 tty5 S 0:00 /sbin/mingetty tty5
2678 tty6 S 0:00 /sbin/mingetty tty6
2925 tty1 S 0:00 /sbin/mingetty tty1 --noclear
5167 ? SW 0:00 [lockd]
21290 ? S< 0:00 /usr/local/bin/nbd-server 1230 /dev/md0
21780 ? S< 0:38 \_ /usr/local/bin/nbd-server 1230 /dev/md0
The kernel : 2.6.17-rc3-git1, but if i have right, the 2.6.16.1 is the same
if i try to swich back. :-)
2 of 4 nodes frequently reboots without any error message.
I have turned on some debug option, and catch this message.
At this point i watch the free -l -s 1, and i can see, the nbd-server (or
kernel) didn't use the highmem for buffer cache, but why?
total used free shared buffers cached
Mem: 2073048 893360 1179688 0 829092 19820
Low: 893464 868352 25112 0 0 0
High: 1179584 25008 1154576 0 0 0
-/+ buffers/cache: 44448 2028600
Swap: 0 0 0
(min_free_kbytes = 16000 now.)
Cheers,
Janos
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 8:37 ` Haar János
@ 2006-05-21 8:47 ` Chris Wedgwood
2006-05-21 9:03 ` Haar János
2006-05-21 9:23 ` swapper: page allocation failure Haar János
0 siblings, 2 replies; 28+ messages in thread
From: Chris Wedgwood @ 2006-05-21 8:47 UTC (permalink / raw)
To: Haar J?nos; +Cc: linux-kernel
On Sun, May 21, 2006 at 10:37:00AM +0200, Haar J?nos wrote:
> total used free shared buffers cached
> Mem: 2073048 893360 1179688 0 829092 19820
> Low: 893464 868352 25112 0 0 0
> High: 1179584 25008 1154576 0 0 0
> -/+ buffers/cache: 44448 2028600
> Swap: 0 0 0
looks bad for lowmem
what does /proc/meminfo say?
what about the output from "slabtop -sc" ?
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 8:47 ` Chris Wedgwood
@ 2006-05-21 9:03 ` Haar János
2006-05-21 9:10 ` Chris Wedgwood
2006-05-21 9:23 ` swapper: page allocation failure Haar János
1 sibling, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-21 9:03 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: linux-kernel
----- Original Message -----
From: "Chris Wedgwood" <cw@f00f.org>
To: "Haar J?nos" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 10:47 AM
Subject: Re: swapper: page allocation failure.
> On Sun, May 21, 2006 at 10:37:00AM +0200, Haar J?nos wrote:
>
> > total used free shared buffers
cached
> > Mem: 2073048 893360 1179688 0 829092
19820
> > Low: 893464 868352 25112 0 0
0
> > High: 1179584 25008 1154576 0 0
0
> > -/+ buffers/cache: 44448 2028600
> > Swap: 0 0 0
>
> looks bad for lowmem
>
> what does /proc/meminfo say?
[root@st-0001 vm]# cat /proc/meminfo
MemTotal: 2073048 kB
MemFree: 1179376 kB
Buffers: 829764 kB
Cached: 19896 kB
SwapCached: 0 kB
Active: 15604 kB
Inactive: 837636 kB
HighTotal: 1179584 kB
HighFree: 1154736 kB
LowTotal: 893464 kB
LowFree: 24640 kB
SwapTotal: 0 kB
SwapFree: 0 kB
Dirty: 21352 kB
Writeback: 0 kB
Mapped: 7000 kB
Slab: 22612 kB
CommitLimit: 1036524 kB
Committed_AS: 10968 kB
PageTables: 284 kB
VmallocTotal: 114680 kB
VmallocUsed: 308 kB
VmallocChunk: 114036 kB
>
> what about the output from "slabtop -sc" ?
Not installed.
Wich package or where can i find the source? (i use redhat 9.0)
Cheers,
Janos
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 9:03 ` Haar János
@ 2006-05-21 9:10 ` Chris Wedgwood
2006-05-21 9:31 ` Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Chris Wedgwood @ 2006-05-21 9:10 UTC (permalink / raw)
To: Haar J?nos; +Cc: linux-kernel
On Sun, May 21, 2006 at 11:03:33AM +0200, Haar J?nos wrote:
> MemTotal: 2073048 kB
> MemFree: 1179376 kB
fine
> Buffers: 829764 kB
ok
> Cached: 19896 kB
> SwapCached: 0 kB
> Active: 15604 kB
> Inactive: 837636 kB
hrm
> HighTotal: 1179584 kB
> HighFree: 1154736 kB
krm
> LowTotal: 893464 kB
> LowFree: 24640 kB
bad
> SwapTotal: 0 kB
> SwapFree: 0 kB
ok
> Dirty: 21352 kB
ok
> Writeback: 0 kB
> Mapped: 7000 kB
> Slab: 22612 kB
ok
you have very little low
> Not installed.
urgh
> Wich package or where can i find the source? (i use redhat 9.0)
google i guess, i have very little idea how to drive RH to be honest
anyhow, it's not the slab
something is eating/using/leaking all your lowmemory
what kernel version is this?
how long has the machine been up?
do you see it get worse over time?
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 8:47 ` Chris Wedgwood
2006-05-21 9:03 ` Haar János
@ 2006-05-21 9:23 ` Haar János
1 sibling, 0 replies; 28+ messages in thread
From: Haar János @ 2006-05-21 9:23 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: linux-kernel
----- Original Message -----
From: "Chris Wedgwood" <cw@f00f.org>
To: "Haar J?nos" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 10:47 AM
Subject: Re: swapper: page allocation failure.
> On Sun, May 21, 2006 at 10:37:00AM +0200, Haar J?nos wrote:
>
> > total used free shared buffers
cached
> > Mem: 2073048 893360 1179688 0 829092
19820
> > Low: 893464 868352 25112 0 0
0
> > High: 1179584 25008 1154576 0 0
0
> > -/+ buffers/cache: 44448 2028600
> > Swap: 0 0 0
>
> looks bad for lowmem
>
> what does /proc/meminfo say?
>
> what about the output from "slabtop -sc" ?
Here comes:
Active / Total Objects (% used) : 243896 / 258956 (94.2%)
Active / Total Slabs (% used) : 6243 / 6261 (99.7%)
Active / Total Caches (% used) : 76 / 151 (50.3%)
Active / Total Size (% used) : 20769.86K / 23348.68K (89.0%)
Minimum / Average / Maximum Object : 0.01K / 0.09K / 128.00K
OBJS ACTIVE USE OBJ SIZE SLABS OBJ/SLAB CACHE SIZE NAME
208872 206768 98% 0.05K 2901 72 11604K buffer_head
4960 4051 81% 0.68K 992 5 3968K nfs_inode_cache
9996 6478 64% 0.27K 714 14 2856K radix_tree_node
566 530 93% 2.00K 283 2 1132K size-2048
4680 4014 85% 0.12K 156 30 624K dentry_cache
1430 1409 98% 0.39K 143 10 572K inode_cache
65 65 100% 8.00K 65 1 520K size-8192
258 256 99% 1.32K 86 3 344K raid5/md0
5460 5435 99% 0.04K 65 84 260K sysfs_dir_cache
945 882 93% 0.25K 63 15 252K size-256
456 438 96% 0.50K 57 8 228K size-512
2891 2514 86% 0.06K 49 59 196K size-64
675 474 70% 0.25K 45 15 180K skbuff_head_cache
176 165 93% 1.00K 44 4 176K size-1024
1320 744 56% 0.12K 44 30 176K bio
42 37 88% 4.00K 42 1 168K size-4096
1170 1073 91% 0.12K 39 30 156K size-128
351 316 90% 0.40K 39 9 156K proc_inode_cache
4068 3298 81% 0.03K 36 113 144K size-32
660 160 24% 0.19K 33 20 132K filp
84 64 76% 1.30K 28 3 112K task_struct
1232 441 35% 0.09K 28 44 112K vm_area_struct
69 63 91% 1.31K 23 3 92K sighand_cache
20 20 100% 4.00K 20 1 80K pgd
396 244 61% 0.17K 18 22 72K blkdev_requests
1 1 100% 64.00K 1 1 64K size-65536
160 153 95% 0.38K 16 10 64K skbuff_fclone_cache
32 32 100% 2.00K 16 2 64K sgpool-128
15 3 20% 4.00K 15 1 60K names_cache
1196 1138 95% 0.04K 13 92 52K acpi_operand
2639 640 24% 0.02K 13 203 52K biovec-1
12 11 91% 3.00K 6 2 48K biovec-(256)
180 162 90% 0.25K 12 15 48K sgpool-16
96 82 85% 0.50K 12 8 48K sgpool-32
110 64 58% 0.38K 11 10 44K signal_cache
40 37 92% 0.97K 10 4 40K blkdev_queue
32 32 100% 1.00K 8 4 32K sgpool-64
80 80 100% 0.38K 8 10 32K scsi_cmd_cache
413 263 63% 0.06K 7 59 28K as_arq
180 150 83% 0.12K 6 30 24K kmem_cache
54 20 37% 0.44K 6 9 24K mm_struct
15 15 100% 1.50K 3 5 24K biovec-128
180 180 100% 0.12K 6 30 24K sgpool-8
42 36 85% 0.50K 6 7 24K nfs_write_data
505 64 12% 0.04K 5 101 20K pid
50 21 42% 0.38K 5 10 20K files_cache
145 112 77% 0.13K 5 29 20K idr_layer_cache
35 26 74% 0.56K 5 7 20K bdev_cache
35 23 65% 0.50K 5 7 20K sock_inode_cache
25 23 92% 0.75K 5 5 20K biovec-64
35 32 91% 0.50K 5 7 20K nfs_read_data
Cheers,
Janos
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 9:10 ` Chris Wedgwood
@ 2006-05-21 9:31 ` Haar János
2006-05-21 10:24 ` Chris Wedgwood
2006-05-21 10:26 ` Chris Wedgwood
0 siblings, 2 replies; 28+ messages in thread
From: Haar János @ 2006-05-21 9:31 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: linux-kernel
----- Original Message -----
From: "Chris Wedgwood" <cw@f00f.org>
To: "Haar J?nos" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 11:10 AM
Subject: Re: swapper: page allocation failure.
> On Sun, May 21, 2006 at 11:03:33AM +0200, Haar J?nos wrote:
>
> > MemTotal: 2073048 kB
> > MemFree: 1179376 kB
>
> fine
>
> > Buffers: 829764 kB
>
> ok
>
> > Cached: 19896 kB
> > SwapCached: 0 kB
> > Active: 15604 kB
>
> > Inactive: 837636 kB
>
> hrm
>
> > HighTotal: 1179584 kB
> > HighFree: 1154736 kB
>
> krm
>
> > LowTotal: 893464 kB
> > LowFree: 24640 kB
>
> bad
>
> > SwapTotal: 0 kB
> > SwapFree: 0 kB
>
> ok
>
> > Dirty: 21352 kB
>
> ok
>
> > Writeback: 0 kB
> > Mapped: 7000 kB
> > Slab: 22612 kB
>
> ok
>
>
>
>
> you have very little low
>
>
> > Not installed.
>
> urgh
>
> > Wich package or where can i find the source? (i use redhat 9.0)
>
> google i guess, i have very little idea how to drive RH to be honest
>
> anyhow, it's not the slab
I found it already, thanks.
>
>
> something is eating/using/leaking all your lowmemory
>
>
> what kernel version is this?
[root@st-0001 /]# uname -a
Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686 i686
i386 GNU/Linux
> how long has the machine been up?
[root@st-0001 /]# uptime
11:22:10 up 2:52, 1 user, load average: 0.35, 0.42, 0.43
> do you see it get worse over time?
No.
This is a simple disk node.
It serves the md0 array, and uses mem for buffering-caching.
If it reboots, fill the memory on the first couple of minutes, and stay on
full, but this is relatively good.
But 2 question is remaining:
1. why don't use highmem for caching?
2. why can not allocate enough lowmem from shared-buffer for the e1000
driver if it needs some memory?
Cheers,
Janos
>
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 9:31 ` Haar János
@ 2006-05-21 10:24 ` Chris Wedgwood
2006-05-21 10:42 ` Haar János
2006-05-21 10:26 ` Chris Wedgwood
1 sibling, 1 reply; 28+ messages in thread
From: Chris Wedgwood @ 2006-05-21 10:24 UTC (permalink / raw)
To: Haar J?nos; +Cc: linux-kernel
On Sun, May 21, 2006 at 11:31:12AM +0200, Haar J?nos wrote:
> 1. why don't use highmem for caching?
> 2. why can not allocate enough lowmem from shared-buffer for the e1000
> driver if it needs some memory?
highmem can't be used as freely as lowmem, it has additional
complexity and a slight over head that makes it hard or impossible to
use in many places
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 9:31 ` Haar János
2006-05-21 10:24 ` Chris Wedgwood
@ 2006-05-21 10:26 ` Chris Wedgwood
2006-05-21 10:45 ` Haar János
2006-05-21 12:01 ` Nick Piggin
1 sibling, 2 replies; 28+ messages in thread
From: Chris Wedgwood @ 2006-05-21 10:26 UTC (permalink / raw)
To: Haar J?nos; +Cc: linux-kernel
On Sun, May 21, 2006 at 11:31:12AM +0200, Haar J?nos wrote:
> [root@st-0001 /]# uname -a
> Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686 i686 i386 GNU/Linux
did earlier kernels work OK?
> This is a simple disk node.
> It serves the md0 array, and uses mem for buffering-caching.
odd, i looks like you've leaked alot of lowmem but i can't think why
i've got major (induced) brain-fog right now so i'll have to think
about it tomorrow sorry
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 10:24 ` Chris Wedgwood
@ 2006-05-21 10:42 ` Haar János
0 siblings, 0 replies; 28+ messages in thread
From: Haar János @ 2006-05-21 10:42 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: linux-kernel
----- Original Message -----
From: "Chris Wedgwood" <cw@f00f.org>
To: "Haar J?nos" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 12:24 PM
Subject: Re: swapper: page allocation failure.
> On Sun, May 21, 2006 at 11:31:12AM +0200, Haar J?nos wrote:
>
> > 1. why don't use highmem for caching?
> > 2. why can not allocate enough lowmem from shared-buffer for the e1000
> > driver if it needs some memory?
>
> highmem can't be used as freely as lowmem, it has additional
> complexity and a slight over head that makes it hard or impossible to
> use in many places
OK, i understand this, but buffer-cache is an optional thing, and looks like
really simple (for me), and i cannot understand why cannot use exactly for
buffering the highmem...
On my concentrator the nbd-client uses ~ 3.5GB of ram for PAGE-cache.
The buffer cache why can not use the highmem?
Whats the difference?
Cheers,
Janos
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 10:26 ` Chris Wedgwood
@ 2006-05-21 10:45 ` Haar János
2006-05-21 12:01 ` Nick Piggin
1 sibling, 0 replies; 28+ messages in thread
From: Haar János @ 2006-05-21 10:45 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: linux-kernel
----- Original Message -----
From: "Chris Wedgwood" <cw@f00f.org>
To: "Haar J?nos" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 12:26 PM
Subject: Re: swapper: page allocation failure.
> On Sun, May 21, 2006 at 11:31:12AM +0200, Haar J?nos wrote:
>
> > [root@st-0001 /]# uname -a
> > Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686
i686 i386 GNU/Linux
>
> did earlier kernels work OK?
No, the my reboot problem starts about january, but my tracking progress is
slow because this is the first error message.
(On january i have upgrade the nodes, and few things are changed.)
>
> > This is a simple disk node.
> > It serves the md0 array, and uses mem for buffering-caching.
>
> odd, i looks like you've leaked alot of lowmem but i can't think why
>
> i've got major (induced) brain-fog right now so i'll have to think
> about it tomorrow sorry
OK, thanks.
Cheers,
Janos
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 10:26 ` Chris Wedgwood
2006-05-21 10:45 ` Haar János
@ 2006-05-21 12:01 ` Nick Piggin
2006-05-21 13:58 ` Haar János
1 sibling, 1 reply; 28+ messages in thread
From: Nick Piggin @ 2006-05-21 12:01 UTC (permalink / raw)
To: Chris Wedgwood; +Cc: Haar J?nos, linux-kernel
Chris Wedgwood wrote:
> On Sun, May 21, 2006 at 11:31:12AM +0200, Haar J?nos wrote:
>
>
>>[root@st-0001 /]# uname -a
>>Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686 i686 i386 GNU/Linux
>
>
> did earlier kernels work OK?
>
>
>>This is a simple disk node.
>>It serves the md0 array, and uses mem for buffering-caching.
>
>
> odd, i looks like you've leaked alot of lowmem but i can't think why
>
> i've got major (induced) brain-fog right now so i'll have to think
> about it tomorrow sorry
The buffers are buffercache rather than the usual pagecache; due to
nbd I guess. Buffercache cannot be satisfied by highmem.
This would be a relatively uncommon setup, which explains why it
isn't working 100%. I don't know of any reason why reclaim speed
should be worse for buffercache, however one notable thing will be
that zone_normal's lowmem reserve that is untouchable by pagecache
will be eaten by buffercache...
Anyway, increasing /proc/sys/vm/min_free_kbytes should help. Janos,
perhaps you could try doubling it and see how you go?
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 8:10 swapper: page allocation failure Haar János
2006-05-21 8:16 ` Chris Wedgwood
@ 2006-05-21 12:03 ` Andrew Morton
2006-05-21 14:01 ` Haar János
1 sibling, 1 reply; 28+ messages in thread
From: Andrew Morton @ 2006-05-21 12:03 UTC (permalink / raw)
To: Haar János; +Cc: linux-kernel
Haar János <djani22@netcenter.hu> wrote:
>
> I seriously gets this, and dont know why.
> This server have 2GB ram, and ~1.1G always free!
> Anybody have an idea?
>
> Thanks,
> Janos
>
> May 21 09:05:35 st-0003 kernel: swapper: page allocation failure. order:0,
> mode:0x20
> May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
> <c014af1d> cache_alloc_refill+0x2a6/0x45c
> May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61 <c03d5bfc>
> __alloc_skb+0x49/0xf5
> May 21 09:05:35 st-0003 kernel: <f88336b1>
> e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f88315de>
e1000 gobbled up all your lowmem memory from interrupt context.
Increasing /proc/sys/vm/min_free_kbytes will help.
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 12:01 ` Nick Piggin
@ 2006-05-21 13:58 ` Haar János
2006-05-22 0:09 ` Nick Piggin
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-21 13:58 UTC (permalink / raw)
To: Nick Piggin; +Cc: Chris Wedgwood, linux-kernel
----- Original Message -----
From: "Nick Piggin" <nickpiggin@yahoo.com.au>
To: "Chris Wedgwood" <cw@f00f.org>
Cc: "Haar J?nos" <djani22@netcenter.hu>; <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 2:01 PM
Subject: Re: swapper: page allocation failure.
> Chris Wedgwood wrote:
> > On Sun, May 21, 2006 at 11:31:12AM +0200, Haar J?nos wrote:
> >
> >
> >>[root@st-0001 /]# uname -a
> >>Linux st-0001 2.6.17-rc3-git1 #2 SMP Sun May 21 01:12:22 CEST 2006 i686
i686 i386 GNU/Linux
> >
> >
> > did earlier kernels work OK?
> >
> >
> >>This is a simple disk node.
> >>It serves the md0 array, and uses mem for buffering-caching.
> >
> >
> > odd, i looks like you've leaked alot of lowmem but i can't think why
> >
> > i've got major (induced) brain-fog right now so i'll have to think
> > about it tomorrow sorry
>
> The buffers are buffercache rather than the usual pagecache; due to
> nbd I guess. Buffercache cannot be satisfied by highmem.
>
> This would be a relatively uncommon setup, which explains why it
> isn't working 100%. I don't know of any reason why reclaim speed
> should be worse for buffercache, however one notable thing will be
> that zone_normal's lowmem reserve that is untouchable by pagecache
> will be eaten by buffercache...
>
> Anyway, increasing /proc/sys/vm/min_free_kbytes should help. Janos,
> perhaps you could try doubling it and see how you go?
I did it allready, and it looks like solves the problem.
Yesterday i have more than 6 random reboots, and after i set from 3800 to
16000 the min free limit, i have none at this point. :-)
15:51:45 up 7:21, 1 user, load average: 0.85, 0.79, 0.67
Anyway, i interested about cache/buffer mechanism, because i have some
performance problems too, and i can see, these systems wastes the half of
memory instead of speeds up the operation.
Thanks,
Janos
>
> --
> SUSE Labs, Novell Inc.
> Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 12:03 ` Andrew Morton
@ 2006-05-21 14:01 ` Haar János
0 siblings, 0 replies; 28+ messages in thread
From: Haar János @ 2006-05-21 14:01 UTC (permalink / raw)
To: Andrew Morton; +Cc: linux-kernel
----- Original Message -----
From: "Andrew Morton" <akpm@osdl.org>
To: "Haar János" <djani22@netcenter.hu>
Cc: <linux-kernel@vger.kernel.org>
Sent: Sunday, May 21, 2006 2:03 PM
Subject: Re: swapper: page allocation failure.
> Haar János <djani22@netcenter.hu> wrote:
> >
> > I seriously gets this, and dont know why.
> > This server have 2GB ram, and ~1.1G always free!
> > Anybody have an idea?
> >
> > Thanks,
> > Janos
> >
> > May 21 09:05:35 st-0003 kernel: swapper: page allocation failure.
order:0,
> > mode:0x20
> > May 21 09:05:35 st-0003 kernel: <c013c6ac> __alloc_pages+0x274/0x286
> > <c014af1d> cache_alloc_refill+0x2a6/0x45c
> > May 21 09:05:35 st-0003 kernel: <c014b12b> __kmalloc+0x58/0x61
<c03d5bfc>
> > __alloc_skb+0x49/0xf5
> > May 21 09:05:35 st-0003 kernel: <f88336b1>
> > e1000_alloc_rx_buffers+0x5c/0x2e5 [e1000] <f88315de>
>
> e1000 gobbled up all your lowmem memory from interrupt context.
>
> Increasing /proc/sys/vm/min_free_kbytes will help.
Yes, i know. ;-)
Thanks,
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-21 13:58 ` Haar János
@ 2006-05-22 0:09 ` Nick Piggin
2006-05-22 7:41 ` Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Nick Piggin @ 2006-05-22 0:09 UTC (permalink / raw)
To: Haar János; +Cc: Chris Wedgwood, linux-kernel
Haar János wrote:
> I did it allready, and it looks like solves the problem.
> Yesterday i have more than 6 random reboots, and after i set from 3800 to
> 16000 the min free limit, i have none at this point. :-)
>
> 15:51:45 up 7:21, 1 user, load average: 0.85, 0.79, 0.67
Oh that's good. It's sad that you had random reboots though :(
>
> Anyway, i interested about cache/buffer mechanism, because i have some
> performance problems too, and i can see, these systems wastes the half of
> memory instead of speeds up the operation.
Yeah, as I said, block device's pagecache (aka buffercache) can't
use highmem. If nbd can export regular files as block devices, or
you use loop devices from regular files, that might help (or slow
things down :P).
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-22 0:09 ` Nick Piggin
@ 2006-05-22 7:41 ` Haar János
2006-05-22 11:17 ` Con Kolivas
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-22 7:41 UTC (permalink / raw)
To: Nick Piggin; +Cc: cw, linux-kernel
----- Original Message -----
From: "Nick Piggin" <nickpiggin@yahoo.com.au>
To: "Haar János" <djani22@netcenter.hu>
Cc: "Chris Wedgwood" <cw@f00f.org>; <linux-kernel@vger.kernel.org>
Sent: Monday, May 22, 2006 2:09 AM
Subject: Re: swapper: page allocation failure.
> Haar János wrote:
>
> > I did it allready, and it looks like solves the problem.
> > Yesterday i have more than 6 random reboots, and after i set from 3800
to
> > 16000 the min free limit, i have none at this point. :-)
> >
> > 15:51:45 up 7:21, 1 user, load average: 0.85, 0.79, 0.67
>
> Oh that's good. It's sad that you had random reboots though :(
09:12:44 up 1 day, 43 min, 1 user, load average: 0.48, 0.44, 0.42
It really fixes this issue. :-)
Thanks to you all!
>
> >
> > Anyway, i interested about cache/buffer mechanism, because i have some
> > performance problems too, and i can see, these systems wastes the half
of
> > memory instead of speeds up the operation.
>
> Yeah, as I said, block device's pagecache (aka buffercache) can't
> use highmem. If nbd can export regular files as block devices, or
> you use loop devices from regular files, that might help (or slow
> things down :P).
Hmm.
That sounds bad.
I think, if highmem is unreachable some times that makes lowmem more
valuable!
The kernel needs to keep (reserve) it free as much as possible.
The buffer-cache is an unimportant thing next to keeping lowmem free, but it
is blocks the performance and wastes the systems resources!
It is possible any workaround?
The NBD is finally fixed, and production ready.
The big systems should use it, because it is easy.
Additionally the big system usually needs the maximum of performance.....
(+ the good cache/buffer extends the hardware lifetime.)
I think it is important to be changed.
(e.g. i use 4 nodes, 4x 3.3TB to use it as one big blockdev.)
Cheers,
Janos
>
> --
> SUSE Labs, Novell Inc.
> Send instant messages to your online friends http://au.messenger.yahoo.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-22 7:41 ` Haar János
@ 2006-05-22 11:17 ` Con Kolivas
2006-05-22 15:08 ` Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Con Kolivas @ 2006-05-22 11:17 UTC (permalink / raw)
To: linux-kernel; +Cc: Haar János, Nick Piggin, cw
On Monday 22 May 2006 17:41, Haar János wrote:
> ----- Original Message -----
> From: "Nick Piggin" <nickpiggin@yahoo.com.au>
> > Yeah, as I said, block device's pagecache (aka buffercache) can't
> > use highmem. If nbd can export regular files as block devices, or
> > you use loop devices from regular files, that might help (or slow
> > things down :P).
>
> Hmm.
> That sounds bad.
> I think, if highmem is unreachable some times that makes lowmem more
> valuable!
> The kernel needs to keep (reserve) it free as much as possible.
> The buffer-cache is an unimportant thing next to keeping lowmem free, but
> it is blocks the performance and wastes the systems resources!
>
> It is possible any workaround?
Try with one of the alternative vmsplit options that gives you more lowmem?
That might break certain applications though.
--
-ck
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-22 11:17 ` Con Kolivas
@ 2006-05-22 15:08 ` Haar János
2006-05-22 15:12 ` Con Kolivas
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-22 15:08 UTC (permalink / raw)
To: Con Kolivas; +Cc: nickpiggin, cw, linux-kernel
----- Original Message -----
From: "Con Kolivas" <kernel@kolivas.org>
To: <linux-kernel@vger.kernel.org>
Cc: "Haar János" <djani22@netcenter.hu>; "Nick Piggin"
<nickpiggin@yahoo.com.au>; <cw@f00f.org>
Sent: Monday, May 22, 2006 1:17 PM
Subject: Re: swapper: page allocation failure.
> On Monday 22 May 2006 17:41, Haar János wrote:
> > ----- Original Message -----
> > From: "Nick Piggin" <nickpiggin@yahoo.com.au>
> > > Yeah, as I said, block device's pagecache (aka buffercache) can't
> > > use highmem. If nbd can export regular files as block devices, or
> > > you use loop devices from regular files, that might help (or slow
> > > things down :P).
> >
> > Hmm.
> > That sounds bad.
> > I think, if highmem is unreachable some times that makes lowmem more
> > valuable!
> > The kernel needs to keep (reserve) it free as much as possible.
> > The buffer-cache is an unimportant thing next to keeping lowmem free,
but
> > it is blocks the performance and wastes the systems resources!
> >
> > It is possible any workaround?
>
> Try with one of the alternative vmsplit options that gives you more
lowmem?
> That might break certain applications though.
total used free shared buffers cached
Mem: 4049724 4021196 28528 0 16384 3217288
Low: 4049724 4021196 28528
High: 0 0 0
-/+ buffers/cache: 787524 3262200
Swap: 0 0 0
This is an 64 bit machine, the "concentrator".
It looks like use all, the 4G ram as "lowmem".
If i replace the cpu on my nodes to 64bit capable ones, i can use all the
memory as buffer-cache? :-)
Cheers,
Janos
>
> --
> -ck
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure.
2006-05-22 15:08 ` Haar János
@ 2006-05-22 15:12 ` Con Kolivas
2006-05-22 21:06 ` swapper: page allocation failure. - random reboot problem Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Con Kolivas @ 2006-05-22 15:12 UTC (permalink / raw)
To: Haar János; +Cc: nickpiggin, cw, linux-kernel
On Tuesday 23 May 2006 01:08, Haar János wrote:
> ----- Original Message -----
> From: "Con Kolivas" <kernel@kolivas.org>
> > Try with one of the alternative vmsplit options that gives you more
>
> lowmem?
>
> > That might break certain applications though.
>
> total used free shared buffers cached
> Mem: 4049724 4021196 28528 0 16384 3217288
> Low: 4049724 4021196 28528
> High: 0 0 0
> -/+ buffers/cache: 787524 3262200
> Swap: 0 0 0
>
> This is an 64 bit machine, the "concentrator".
>
> It looks like use all, the 4G ram as "lowmem".
> If i replace the cpu on my nodes to 64bit capable ones, i can use all the
> memory as buffer-cache? :-)
Heh yes indeed. It's only if you're stuck on 32bit for whatever reason that
you'd need a different vmsplit. There is no need for highmem when 64bit
allows bazillions of bytes of lowmem :)
--
-ck
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-22 15:12 ` Con Kolivas
@ 2006-05-22 21:06 ` Haar János
2006-05-23 9:17 ` Nick Piggin
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-22 21:06 UTC (permalink / raw)
To: Con Kolivas; +Cc: nickpiggin, cw, linux-kernel
----- Original Message -----
From: "Con Kolivas" <kernel@kolivas.org>
To: "Haar János" <djani22@netcenter.hu>
Cc: <nickpiggin@yahoo.com.au>; <cw@f00f.org>; <linux-kernel@vger.kernel.org>
Sent: Monday, May 22, 2006 5:12 PM
Subject: Re: swapper: page allocation failure.
> On Tuesday 23 May 2006 01:08, Haar János wrote:
> > ----- Original Message -----
> > From: "Con Kolivas" <kernel@kolivas.org>
> > > Try with one of the alternative vmsplit options that gives you more
> >
> > lowmem?
> >
> > > That might break certain applications though.
> >
> > total used free shared buffers
cached
> > Mem: 4049724 4021196 28528 0 16384
3217288
> > Low: 4049724 4021196 28528
> > High: 0 0 0
> > -/+ buffers/cache: 787524 3262200
> > Swap: 0 0 0
> >
> > This is an 64 bit machine, the "concentrator".
> >
> > It looks like use all, the 4G ram as "lowmem".
> > If i replace the cpu on my nodes to 64bit capable ones, i can use all
the
> > memory as buffer-cache? :-)
>
> Heh yes indeed. It's only if you're stuck on 32bit for whatever reason
that
> you'd need a different vmsplit. There is no need for highmem when 64bit
> allows bazillions of bytes of lowmem :)
OK, it is enough, to switch to 64bit, thanks!
But i have a little problem.
My node #3 reboots again.
At this point i have run out of ideas. :-(
This is checked already:
- the complete hardware, except the 12 hdd. (smart reports, no errors at
all, 4x ide + 8xSATA all 300GB.)
- the SMP race. (checked with non-smp kernel)
- APIC/ACPI (tested with non... kernel)
- the e1000 driver (tested with realtek gige adapter)
- the complete filesystem, OS (NFS-ROOT, and copy between nodes.)
- the memory allocation proble, (checked with debug-kernel, and rised
min_free_kbytes)
The systems only service is nbd. (nbd-server serving md0, raid4 array)
Anybody have an idea?
Please let me know!
Thanks,
Janos
>
> --
> -ck
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-22 21:06 ` swapper: page allocation failure. - random reboot problem Haar János
@ 2006-05-23 9:17 ` Nick Piggin
2006-05-23 10:16 ` Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Nick Piggin @ 2006-05-23 9:17 UTC (permalink / raw)
To: Haar János; +Cc: Con Kolivas, cw, linux-kernel
Haar János wrote:
> OK, it is enough, to switch to 64bit, thanks!
>
> But i have a little problem.
> My node #3 reboots again.
>
> At this point i have run out of ideas. :-(
>
> This is checked already:
>
> - the complete hardware, except the 12 hdd. (smart reports, no errors at
> all, 4x ide + 8xSATA all 300GB.)
> - the SMP race. (checked with non-smp kernel)
> - APIC/ACPI (tested with non... kernel)
> - the e1000 driver (tested with realtek gige adapter)
> - the complete filesystem, OS (NFS-ROOT, and copy between nodes.)
> - the memory allocation proble, (checked with debug-kernel, and rised
> min_free_kbytes)
>
> The systems only service is nbd. (nbd-server serving md0, raid4 array)
>
> Anybody have an idea?
Bad hardware. Run memtest overnight. Can your power supply deal with
that many drives? etc.
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-23 9:17 ` Nick Piggin
@ 2006-05-23 10:16 ` Haar János
2006-05-23 10:24 ` Nick Piggin
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-23 10:16 UTC (permalink / raw)
To: Nick Piggin; +Cc: kernel, cw, linux-kernel
----- Original Message -----
From: "Nick Piggin" <nickpiggin@yahoo.com.au>
To: "Haar János" <djani22@netcenter.hu>
Cc: "Con Kolivas" <kernel@kolivas.org>; <cw@f00f.org>;
<linux-kernel@vger.kernel.org>
Sent: Tuesday, May 23, 2006 11:17 AM
Subject: Re: swapper: page allocation failure. - random reboot problem
> Haar János wrote:
>
> > OK, it is enough, to switch to 64bit, thanks!
> >
> > But i have a little problem.
> > My node #3 reboots again.
> >
> > At this point i have run out of ideas. :-(
> >
> > This is checked already:
> >
> > - the complete hardware, except the 12 hdd. (smart reports, no errors at
> > all, 4x ide + 8xSATA all 300GB.)
> > - the SMP race. (checked with non-smp kernel)
> > - APIC/ACPI (tested with non... kernel)
> > - the e1000 driver (tested with realtek gige adapter)
> > - the complete filesystem, OS (NFS-ROOT, and copy between nodes.)
> > - the memory allocation proble, (checked with debug-kernel, and rised
> > min_free_kbytes)
> >
> > The systems only service is nbd. (nbd-server serving md0, raid4 array)
> >
> > Anybody have an idea?
>
> Bad hardware. Run memtest overnight. Can your power supply deal with
> that many drives? etc.
Sorry, i have allready did these things.
The motherboard + CPU + RAM successed the overnight memtest, but anyway i
have replaced this group with another pre-tested ones, but no change!
(Additionally, i replaced the NIC [e1000 to e1000, and e1000 to realtek],
the sata cards [promise to promise], the sata and ide cables, mb, cpu, ram,
ps, and the power cable too.
Only the 12hdd is the same, but smart reports no errors at all!)
The power supply is the 3rd. and the problem is the same.
This is really a software bug, but i dont know exactly where.
Cheers,
Janos
>
> --
> SUSE Labs, Novell Inc.
> Send instant messages to your online friends http://au.messenger.yahoo.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-23 10:16 ` Haar János
@ 2006-05-23 10:24 ` Nick Piggin
2006-05-23 10:31 ` Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Nick Piggin @ 2006-05-23 10:24 UTC (permalink / raw)
To: Haar János; +Cc: kernel, cw, linux-kernel
Haar János wrote:
> Sorry, i have allready did these things.
>
> The motherboard + CPU + RAM successed the overnight memtest, but anyway i
> have replaced this group with another pre-tested ones, but no change!
> (Additionally, i replaced the NIC [e1000 to e1000, and e1000 to realtek],
> the sata cards [promise to promise], the sata and ide cables, mb, cpu, ram,
> ps, and the power cable too.
> Only the 12hdd is the same, but smart reports no errors at all!)
> The power supply is the 3rd. and the problem is the same.
But is the power supply rated enough to support all the drives?
I have seen random reboots where the power supply wasn't good
enough.
>
> This is really a software bug, but i dont know exactly where.
Could be. memtest doesn't guarantee anything though...
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-23 10:24 ` Nick Piggin
@ 2006-05-23 10:31 ` Haar János
2006-05-23 10:45 ` Nick Piggin
0 siblings, 1 reply; 28+ messages in thread
From: Haar János @ 2006-05-23 10:31 UTC (permalink / raw)
To: Nick Piggin; +Cc: kernel, cw, linux-kernel
----- Original Message -----
From: "Nick Piggin" <nickpiggin@yahoo.com.au>
To: "Haar János" <djani22@netcenter.hu>
Cc: <kernel@kolivas.org>; <cw@f00f.org>; <linux-kernel@vger.kernel.org>
Sent: Tuesday, May 23, 2006 12:24 PM
Subject: Re: swapper: page allocation failure. - random reboot problem
> Haar János wrote:
>
> > Sorry, i have allready did these things.
> >
> > The motherboard + CPU + RAM successed the overnight memtest, but anyway
i
> > have replaced this group with another pre-tested ones, but no change!
> > (Additionally, i replaced the NIC [e1000 to e1000, and e1000 to
realtek],
> > the sata cards [promise to promise], the sata and ide cables, mb, cpu,
ram,
> > ps, and the power cable too.
> > Only the 12hdd is the same, but smart reports no errors at all!)
> > The power supply is the 3rd. and the problem is the same.
>
> But is the power supply rated enough to support all the drives?
> I have seen random reboots where the power supply wasn't good
> enough.
This is the 3rd modell, currently 550W.
The system is P4 3G.
The another 2 stable node uses only 460W, and all hardware is equal.
But i tried to swap ps between the stable and unstabe nodes, but nothing is
changed....
Cheers,
Janos
>
> >
> > This is really a software bug, but i dont know exactly where.
>
> Could be. memtest doesn't guarantee anything though...
>
> --
> SUSE Labs, Novell Inc.
> Send instant messages to your online friends http://au.messenger.yahoo.com
> -
> To unsubscribe from this list: send the line "unsubscribe linux-kernel" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
> Please read the FAQ at http://www.tux.org/lkml/
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-23 10:31 ` Haar János
@ 2006-05-23 10:45 ` Nick Piggin
2006-05-23 11:01 ` Haar János
0 siblings, 1 reply; 28+ messages in thread
From: Nick Piggin @ 2006-05-23 10:45 UTC (permalink / raw)
To: Haar János; +Cc: kernel, cw, linux-kernel
Haar János wrote:
> ----- Original Message -----
>>But is the power supply rated enough to support all the drives?
>>I have seen random reboots where the power supply wasn't good
>>enough.
>
>
> This is the 3rd modell, currently 550W.
> The system is P4 3G.
>
> The another 2 stable node uses only 460W, and all hardware is equal.
> But i tried to swap ps between the stable and unstabe nodes, but nothing is
> changed....
Not sure then, sorry.
If it is a software problem, then if you can narrow it down further
(eg. kernel 2.6.15 worked, 2.6.16 did not), or find a reproducable
test case for it, then some more progress might be made.
--
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
* Re: swapper: page allocation failure. - random reboot problem
2006-05-23 10:45 ` Nick Piggin
@ 2006-05-23 11:01 ` Haar János
0 siblings, 0 replies; 28+ messages in thread
From: Haar János @ 2006-05-23 11:01 UTC (permalink / raw)
To: Nick Piggin; +Cc: kernel, cw, linux-kernel
----- Original Message -----
From: "Nick Piggin" <nickpiggin@yahoo.com.au>
To: "Haar János" <djani22@netcenter.hu>
Cc: <kernel@kolivas.org>; <cw@f00f.org>; <linux-kernel@vger.kernel.org>
Sent: Tuesday, May 23, 2006 12:45 PM
Subject: Re: swapper: page allocation failure. - random reboot problem
> Haar János wrote:
> > ----- Original Message -----
>
> >>But is the power supply rated enough to support all the drives?
> >>I have seen random reboots where the power supply wasn't good
> >>enough.
> >
> >
> > This is the 3rd modell, currently 550W.
> > The system is P4 3G.
> >
> > The another 2 stable node uses only 460W, and all hardware is equal.
> > But i tried to swap ps between the stable and unstabe nodes, but nothing
is
> > changed....
>
> Not sure then, sorry.
OK, what do you recommend? :-)
>
> If it is a software problem, then if you can narrow it down further
> (eg. kernel 2.6.15 worked, 2.6.16 did not), or find a reproducable
> test case for it, then some more progress might be made.
Yes, you have right!
I try it allready, but i cannot step back enough, because my sata card works
only on 2.6.16+
Before i use the promise sata cards, and the sata 300G hdds, i use 2.6.15
kernel, and all my 4 nodes was really stable.
But this is not enough to exactly find the problem. :-(
Anyway, Hetbert Xu helps me a lot to track down the problem, but we only can
close out some thing.
If i have right, the problem is about libata, promise driver,
sata-error-handling or similar, but not so sure.
If there is no way to software debug, i will swap all the 12 hdd between 2
disk nodes, and this can show i have right, or not.
Cheers,
Janos
>
> --
> SUSE Labs, Novell Inc.
> Send instant messages to your online friends http://au.messenger.yahoo.com
^ permalink raw reply [flat|nested] 28+ messages in thread
end of thread, other threads:[~2006-05-23 11:01 UTC | newest]
Thread overview: 28+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2006-05-21 8:10 swapper: page allocation failure Haar János
2006-05-21 8:16 ` Chris Wedgwood
2006-05-21 8:37 ` Haar János
2006-05-21 8:47 ` Chris Wedgwood
2006-05-21 9:03 ` Haar János
2006-05-21 9:10 ` Chris Wedgwood
2006-05-21 9:31 ` Haar János
2006-05-21 10:24 ` Chris Wedgwood
2006-05-21 10:42 ` Haar János
2006-05-21 10:26 ` Chris Wedgwood
2006-05-21 10:45 ` Haar János
2006-05-21 12:01 ` Nick Piggin
2006-05-21 13:58 ` Haar János
2006-05-22 0:09 ` Nick Piggin
2006-05-22 7:41 ` Haar János
2006-05-22 11:17 ` Con Kolivas
2006-05-22 15:08 ` Haar János
2006-05-22 15:12 ` Con Kolivas
2006-05-22 21:06 ` swapper: page allocation failure. - random reboot problem Haar János
2006-05-23 9:17 ` Nick Piggin
2006-05-23 10:16 ` Haar János
2006-05-23 10:24 ` Nick Piggin
2006-05-23 10:31 ` Haar János
2006-05-23 10:45 ` Nick Piggin
2006-05-23 11:01 ` Haar János
2006-05-21 9:23 ` swapper: page allocation failure Haar János
2006-05-21 12:03 ` Andrew Morton
2006-05-21 14:01 ` Haar János
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).