* Kernel Panic on OOM with 10 HTB rules
@ 2010-09-08 10:52 Anand Raj Manickam
2010-09-08 12:42 ` Eric Dumazet
0 siblings, 1 reply; 8+ messages in thread
From: Anand Raj Manickam @ 2010-09-08 10:52 UTC (permalink / raw)
To: netdev, netfilter-devel, shemminger
Hi,
I m struck with
Kernel Panic - Out of Memory Issue when traffic is more than 100k
pps and have around 10 tc rules.
There is NO kernel panic with 1 tc rule .
I using a 2.6.34.6 ( latest stable kernel) with SMP enabled
The rules are simple -
10 iptable rules :
iptables -t mangle -I FORWARD -s x.x.x.x -j CLASSIFY --set-class 0xz:0xy
10 tc rules :
tc class add dev imq0 parent 1:z classid 1:y htb rate 2kbit ceil 32768kbps
I m using Intel Xeon Quad core x5570, 12GB RAM , with 8 e1000e , irq
affined ( 1 pair per core)
There is NO issue / CRASH with multiple iptable rules marking to the
same tc rule/HTB class , i can sucessfully pass more than 200k pps /
800Mbps traffic .
But when each iptable rule is marking to seprate tc rule/HTB class and
when the traffic exceeds 100K pps / 20000 connections per second , we
hit this Out of Memory - Kernel panic condition .
Any help is appreciated . Do let me know if any more further
information is need.
Thanks,
Anand
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-08 10:52 Kernel Panic on OOM with 10 HTB rules Anand Raj Manickam
@ 2010-09-08 12:42 ` Eric Dumazet
2010-09-08 14:09 ` Anand Raj Manickam
2010-09-16 9:35 ` Patrick McHardy
0 siblings, 2 replies; 8+ messages in thread
From: Eric Dumazet @ 2010-09-08 12:42 UTC (permalink / raw)
To: Anand Raj Manickam; +Cc: netdev, netfilter-devel, shemminger
Le mercredi 08 septembre 2010 à 16:22 +0530, Anand Raj Manickam a
écrit :
> Hi,
>
> I m struck with
> Kernel Panic - Out of Memory Issue when traffic is more than 100k
> pps and have around 10 tc rules.
>
> There is NO kernel panic with 1 tc rule .
>
> I using a 2.6.34.6 ( latest stable kernel) with SMP enabled
>
> The rules are simple -
> 10 iptable rules :
> iptables -t mangle -I FORWARD -s x.x.x.x -j CLASSIFY --set-class 0xz:0xy
>
> 10 tc rules :
> tc class add dev imq0 parent 1:z classid 1:y htb rate 2kbit ceil 32768kbps
>
> I m using Intel Xeon Quad core x5570, 12GB RAM , with 8 e1000e , irq
> affined ( 1 pair per core)
>
> There is NO issue / CRASH with multiple iptable rules marking to the
> same tc rule/HTB class , i can sucessfully pass more than 200k pps /
> 800Mbps traffic .
>
> But when each iptable rule is marking to seprate tc rule/HTB class and
> when the traffic exceeds 100K pps / 20000 connections per second , we
> hit this Out of Memory - Kernel panic condition .
> Any help is appreciated . Do let me know if any more further
> information is need.
Hi Anand
Could you give :
1) A complete stack trace / panic report
2) cat /proc/interrupts
3) ifconfig -a
4) cat /proc/sys/vm/min_free_kbytes
5) cat /proc/meminfo
for each eth*
ethtool -g eth*
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-08 12:42 ` Eric Dumazet
@ 2010-09-08 14:09 ` Anand Raj Manickam
2010-09-08 14:45 ` Eric Dumazet
2010-09-16 9:35 ` Patrick McHardy
1 sibling, 1 reply; 8+ messages in thread
From: Anand Raj Manickam @ 2010-09-08 14:09 UTC (permalink / raw)
To: Eric Dumazet; +Cc: netdev, netfilter-devel, shemminger
Please find the info below ... its a long list though
On Wed, Sep 8, 2010 at 6:12 PM, Eric Dumazet <eric.dumazet@gmail.com> wrote:
> Le mercredi 08 septembre 2010 à 16:22 +0530, Anand Raj Manickam a
> écrit :
>> Hi,
>>
>> I m struck with
>> Kernel Panic - Out of Memory Issue when traffic is more than 100k
>> pps and have around 10 tc rules.
>>
>> There is NO kernel panic with 1 tc rule .
>>
>> I using a 2.6.34.6 ( latest stable kernel) with SMP enabled
>>
>> The rules are simple -
>> 10 iptable rules :
>> iptables -t mangle -I FORWARD -s x.x.x.x -j CLASSIFY --set-class 0xz:0xy
>>
>> 10 tc rules :
>> tc class add dev imq0 parent 1:z classid 1:y htb rate 2kbit ceil 32768kbps
>>
>> I m using Intel Xeon Quad core x5570, 12GB RAM , with 8 e1000e , irq
>> affined ( 1 pair per core)
>>
>> There is NO issue / CRASH with multiple iptable rules marking to the
>> same tc rule/HTB class , i can sucessfully pass more than 200k pps /
>> 800Mbps traffic .
>>
>> But when each iptable rule is marking to seprate tc rule/HTB class and
>> when the traffic exceeds 100K pps / 20000 connections per second , we
>> hit this Out of Memory - Kernel panic condition .
>> Any help is appreciated . Do let me know if any more further
>> information is need.
>
> Hi Anand
>
> Could you give :
>
> 1) A complete stack trace / panic report
Sorry there was no stack trace :-(
e5570#Out of memory: kill process 10240 (apache2) score 58700 or a child
Killed process 10240 (apache2) vsz:234800kB, anon-rss:1576kB, file-rss:1212kB
Out of memory: kill process 10241 (apache2) score 58700 or a child
Killed process 10241 (apache2) vsz:234800kB, anon-rss:1576kB, file-rss:1232kB
Out of memory: kill process 16120 (java) score 19383 or a child
Killed process 16120 (java) vsz:1240516kB, anon-rss:161428kB, file-rss:7516kB
Out of memory: kill process 2185 (mysqld) score 16377 or a child
Killed process 2185 (mysqld) vsz:131020kB, anon-rss:15392kB, file-rss:5144kB
Out of memory: kill process 10238 (apache2) score 3630 or a child
Killed process 21304 (tcstat_total.cg) vsz:2848kB, anon-rss:192kB,
file-rss:1016kB
Out of memory: kill process 10238 (apache2) score 3273 or a child
Killed process 10238 (apache2) vsz:13092kB, anon-rss:1408kB, file-rss:624kB
Out of memory: kill process 21315 (tcstat_total.cg) score 712 or a child
Killed process 21315 (tcstat_total.cg) vsz:2848kB, anon-rss:192kB,
file-rss:196kB
Out of memory: kill process 1983 (rsyslogd) score 449 or a child
Killed process 1983 (rsyslogd) vsz:28744kB, anon-rss:784kB, file-rss:912kB
Out of memory: kill process 10236 (apache2) score 208 or a child
Killed process 10236 (apache2) vsz:13320kB, anon-rss:1412kB, file-rss:1332kB
Out of memory: kill process 4051 (login) score 77 or a child
Killed process 4831 (bash) vsz:4556kB, anon-rss:396kB, file-rss:1336kB
Out of memory: kill process 2728 (cron) score 58 or a child
Killed process 2728 (cron) vsz:3716kB, anon-rss:156kB, file-rss:628kB
Out of memory: kill process 2068 (mysqld_safe) score 45 or a child
Killed process 2068 (mysqld_safe) vsz:2904kB, anon-rss:248kB, file-rss:1084kB
Out of memory: kill process 4051 (login) score 41 or a child
Killed process 4051 (login) vsz:2680kB, anon-rss:252kB, file-rss:948kB
Out of memory: kill process 2313 (inetd) score 31 or a child
Killed process 2313 (inetd) vsz:2004kB, anon-rss:92kB, file-rss:496kB
Out of memory: kill process 4045 (getty) score 28 or a child
Killed process 4045 (getty) vsz:1816kB, anon-rss:68kB, file-rss:400kB
Out of memory: kill process 4046 (getty) score 28 or a child
Killed process 4046 (getty) vsz:1816kB, anon-rss:72kB, file-rss:400kB
Out of memory: kill process 4047 (getty) score 28 or a child
Killed process 4047 (getty) vsz:1816kB, anon-rss:72kB, file-rss:400kB
Out of memory: kill process 4048 (getty) score 28 or a child
Killed process 4048 (getty) vsz:1816kB, anon-rss:72kB, file-rss:400kB
Out of memory: kill process 4049 (getty) score 28 or a child
Killed process 4049 (getty) vsz:1816kB, anon-rss:68kB, file-rss:400kB
Out of memory: kill process 4050 (getty) score 28 or a child
Killed process 4050 (getty) vsz:1816kB, anon-rss:68kB, file-rss:400kB
Kernel panic - not syncing: Out of memory and no killable processes...
> 2) cat /proc/interrupts
CPU0 CPU1 CPU2 CPU3
0: 250 0 0 0 IO-APIC-edge timer
1: 0 0 0 2 IO-APIC-edge i8042
2: 0 0 0 0 XT-PIC-XT cascade
4: 0 0 0 226 IO-APIC-edge serial
8: 0 0 0 2 IO-APIC-edge rtc0
12: 0 0 0 3 IO-APIC-edge i8042
19: 0 10673 0 0 IO-APIC-fasteoi ata_piix
33: 0 0 0 0 PCI-MSI-edge pciehp
34: 0 0 0 0 PCI-MSI-edge pciehp
35: 0 0 0 1 PCI-MSI-edge pciehp
36: 1 0 0 0 PCI-MSI-edge pciehp
37: 1 0 0 0 PCI-MSI-edge pciehp
38: 0 1 0 0 PCI-MSI-edge pciehp
39: 94 0 0 0 PCI-MSI-edge eth0
40: 88 6 0 0 PCI-MSI-edge eth1
41: 0 88 6 0 PCI-MSI-edge eth2
42: 0 88 6 0 PCI-MSI-edge eth3
43: 0 0 92 0 PCI-MSI-edge eth4-rx-0
44: 0 0 0 0 PCI-MSI-edge eth4-tx-0
45: 3 0 0 0 PCI-MSI-edge eth4
46: 0 4 88 0 PCI-MSI-edge eth5-rx-0
47: 0 0 0 0 PCI-MSI-edge eth5-tx-0
48: 0 0 3 0 PCI-MSI-edge eth5
49: 0 0 4 88 PCI-MSI-edge eth6-rx-0
50: 0 0 0 0 PCI-MSI-edge eth6-tx-0
51: 0 0 0 3 PCI-MSI-edge eth6
52: 4 0 0 88 PCI-MSI-edge eth7-rx-0
53: 0 0 0 0 PCI-MSI-edge eth7-tx-0
54: 0 3 0 0 PCI-MSI-edge eth7
NMI: 0 0 0 0 Non-maskable interrupts
LOC: 196962 205933 209322 208530 Local timer interrupts
SPU: 0 0 0 0 Spurious interrupts
PMI: 0 0 0 0 Performance
monitoring interrupts
PND: 0 0 0 0 Performance pending work
RES: 4866 761 416 467 Rescheduling interrupts
CAL: 39 102 107 109 Function call interrupts
TLB: 6019 5481 4678 6759 TLB shootdowns
TRM: 0 0 0 0 Thermal event interrupts
THR: 0 0 0 0 Threshold APIC interrupts
MCE: 0 0 0 0 Machine check exceptions
MCP: 2 2 2 2 Machine check polls
ERR: 3
MIS: 0
> 3) ifconfig -a
br0 Link encap:Ethernet HWaddr 00:90:0b:14:ac:38
inet addr:192.168.164.222 Bcast:192.168.167.255 Mask:255.255.252.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:50 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:2300 (2.2 KiB) TX bytes:0 (0.0 B)
eth0 Link encap:Ethernet HWaddr 00:90:0b:14:ac:38
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:10119 errors:0 dropped:0 overruns:0 frame:0
TX packets:20075 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6654384 (6.3 MiB) TX bytes:1427890 (1.3 MiB)
Memory:fb580000-fb5a0000
eth1 Link encap:Ethernet HWaddr 00:90:0b:14:ac:39
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:20039 errors:0 dropped:0 overruns:0 frame:0
TX packets:10155 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1495356 (1.4 MiB) TX bytes:6646438 (6.3 MiB)
Memory:fb5e0000-fb600000
eth2 Link encap:Ethernet HWaddr 00:90:0b:14:ac:3a
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:19190 errors:0 dropped:0 overruns:0 frame:0
TX packets:9727 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1432004 (1.3 MiB) TX bytes:6365028 (6.0 MiB)
Memory:fb680000-fb6a0000
eth3 Link encap:Ethernet HWaddr 00:90:0b:14:ac:3b
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:9689 errors:0 dropped:0 overruns:0 frame:0
TX packets:19228 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6372426 (6.0 MiB) TX bytes:1367600 (1.3 MiB)
Memory:fb6e0000-fb700000
eth4 Link encap:Ethernet HWaddr 00:90:0b:15:79:18
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:9252 errors:0 dropped:0 overruns:0 frame:0
TX packets:18354 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6085260 (5.8 MiB) TX bytes:1305576 (1.2 MiB)
Memory:fb8e0000-fb900000
eth5 Link encap:Ethernet HWaddr 00:90:0b:15:79:19
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:18316 errors:0 dropped:0 overruns:0 frame:0
TX packets:9290 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1366920 (1.3 MiB) TX bytes:6078300 (5.7 MiB)
Memory:fb9e0000-fba00000
eth6 Link encap:Ethernet HWaddr 00:90:0b:15:79:1a
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:18709 errors:0 dropped:0 overruns:0 frame:0
TX packets:9484 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:1396126 (1.3 MiB) TX bytes:6204656 (5.9 MiB)
Memory:fbae0000-fbb00000
eth7 Link encap:Ethernet HWaddr 00:90:0b:15:79:1b
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:9446 errors:0 dropped:0 overruns:0 frame:0
TX packets:18746 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:6211812 (5.9 MiB) TX bytes:1333280 (1.2 MiB)
Memory:fbbe0000-fbc00000
imq0 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
UP RUNNING NOARP MTU:16000 Metric:1
RX packets:129112 errors:0 dropped:0 overruns:0 frame:0
TX packets:129114 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:31060964 (29.6 MiB) TX bytes:31062207 (29.6 MiB)
imq1 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
imq2 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
imq3 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
imq4 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
imq5 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
imq6 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
imq7 Link encap:UNSPEC HWaddr
00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
NOARP MTU:16000 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:11000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:14454 errors:0 dropped:0 overruns:0 frame:0
TX packets:14454 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:1962561 (1.8 MiB) TX bytes:1962561 (1.8 MiB)
> 4) cat /proc/sys/vm/min_free_kbytes
3806
> 5) cat /proc/meminfo
MemTotal: 12462064 kB
MemFree: 12265628 kB
Buffers: 29672 kB
Cached: 64800 kB
SwapCached: 0 kB
Active: 87748 kB
Inactive: 75756 kB
Active(anon): 69040 kB
Inactive(anon): 288 kB
Active(file): 18708 kB
Inactive(file): 75468 kB
Unevictable: 0 kB
Mlocked: 0 kB
HighTotal: 11660872 kB
HighFree: 11515792 kB
LowTotal: 801192 kB
LowFree: 749836 kB
SwapTotal: 27226684 kB
SwapFree: 27226684 kB
Dirty: 120 kB
Writeback: 0 kB
AnonPages: 69032 kB
Mapped: 14508 kB
Shmem: 296 kB
Slab: 15728 kB
SReclaimable: 6044 kB
SUnreclaim: 9684 kB
KernelStack: 1376 kB
PageTables: 996 kB
NFS_Unstable: 0 kB
Bounce: 0 kB
WritebackTmp: 0 kB
CommitLimit: 33457716 kB
Committed_AS: 848336 kB
VmallocTotal: 122880 kB
VmallocUsed: 9468 kB
VmallocChunk: 106708 kB
DirectMap4k: 6136 kB
DirectMap2M: 907264 kB
> for each eth*
> ethtool -g eth*
e5570#ethtool -g eth0 - eth7 ( have the same config)
Ring parameters for eth0:
Pre-set maximums:
RX: 4096
RX Mini: 0
RX Jumbo: 0
TX: 4096
Current hardware settings:
RX: 256
RX Mini: 0
RX Jumbo: 0
TX: 256
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-08 14:09 ` Anand Raj Manickam
@ 2010-09-08 14:45 ` Eric Dumazet
2010-09-08 15:02 ` Krzysztof Olędzki
2010-09-08 17:25 ` Jarek Poplawski
0 siblings, 2 replies; 8+ messages in thread
From: Eric Dumazet @ 2010-09-08 14:45 UTC (permalink / raw)
To: Anand Raj Manickam; +Cc: netdev, netfilter-devel, shemminger
Le mercredi 08 septembre 2010 à 19:39 +0530, Anand Raj Manickam a
écrit :
> Please find the info below ... its a long list though
> Sorry there was no stack trace :-(
>
>
> e5570#Out of memory: kill process 10240 (apache2) score 58700 or a child
> Killed process 10240 (apache2) vsz:234800kB, anon-rss:1576kB, file-rss:1212kB
> Out of memory: kill process 10241 (apache2) score 58700 or a child
> Killed process 10241 (apache2) vsz:234800kB, anon-rss:1576kB, file-rss:1232kB
> Out of memory: kill process 16120 (java) score 19383 or a child
> Killed process 16120 (java) vsz:1240516kB, anon-rss:161428kB, file-rss:7516kB
> Out of memory: kill process 2185 (mysqld) score 16377 or a child
> Killed process 2185 (mysqld) vsz:131020kB, anon-rss:15392kB, file-rss:5144kB
> Out of memory: kill process 10238 (apache2) score 3630 or a child
> Killed process 21304 (tcstat_total.cg) vsz:2848kB, anon-rss:192kB,
> file-rss:1016kB
> Out of memory: kill process 10238 (apache2) score 3273 or a child
> Killed process 10238 (apache2) vsz:13092kB, anon-rss:1408kB, file-rss:624kB
> Out of memory: kill process 21315 (tcstat_total.cg) score 712 or a child
> Killed process 21315 (tcstat_total.cg) vsz:2848kB, anon-rss:192kB,
> file-rss:196kB
> Out of memory: kill process 1983 (rsyslogd) score 449 or a child
> Killed process 1983 (rsyslogd) vsz:28744kB, anon-rss:784kB, file-rss:912kB
> Out of memory: kill process 10236 (apache2) score 208 or a child
> Killed process 10236 (apache2) vsz:13320kB, anon-rss:1412kB, file-rss:1332kB
> Out of memory: kill process 4051 (login) score 77 or a child
> Killed process 4831 (bash) vsz:4556kB, anon-rss:396kB, file-rss:1336kB
> Out of memory: kill process 2728 (cron) score 58 or a child
> Killed process 2728 (cron) vsz:3716kB, anon-rss:156kB, file-rss:628kB
> Out of memory: kill process 2068 (mysqld_safe) score 45 or a child
> Killed process 2068 (mysqld_safe) vsz:2904kB, anon-rss:248kB, file-rss:1084kB
> Out of memory: kill process 4051 (login) score 41 or a child
> Killed process 4051 (login) vsz:2680kB, anon-rss:252kB, file-rss:948kB
> Out of memory: kill process 2313 (inetd) score 31 or a child
> Killed process 2313 (inetd) vsz:2004kB, anon-rss:92kB, file-rss:496kB
> Out of memory: kill process 4045 (getty) score 28 or a child
> Killed process 4045 (getty) vsz:1816kB, anon-rss:68kB, file-rss:400kB
> Out of memory: kill process 4046 (getty) score 28 or a child
> Killed process 4046 (getty) vsz:1816kB, anon-rss:72kB, file-rss:400kB
> Out of memory: kill process 4047 (getty) score 28 or a child
> Killed process 4047 (getty) vsz:1816kB, anon-rss:72kB, file-rss:400kB
> Out of memory: kill process 4048 (getty) score 28 or a child
> Killed process 4048 (getty) vsz:1816kB, anon-rss:72kB, file-rss:400kB
> Out of memory: kill process 4049 (getty) score 28 or a child
> Killed process 4049 (getty) vsz:1816kB, anon-rss:68kB, file-rss:400kB
> Out of memory: kill process 4050 (getty) score 28 or a child
> Killed process 4050 (getty) vsz:1816kB, anon-rss:68kB, file-rss:400kB
> Kernel panic - not syncing: Out of memory and no killable processes...
>
>
> > 2) cat /proc/interrupts
>
> CPU0 CPU1 CPU2 CPU3
> 0: 250 0 0 0 IO-APIC-edge timer
> 1: 0 0 0 2 IO-APIC-edge i8042
> 2: 0 0 0 0 XT-PIC-XT cascade
> 4: 0 0 0 226 IO-APIC-edge serial
> 8: 0 0 0 2 IO-APIC-edge rtc0
> 12: 0 0 0 3 IO-APIC-edge i8042
> 19: 0 10673 0 0 IO-APIC-fasteoi ata_piix
> 33: 0 0 0 0 PCI-MSI-edge pciehp
> 34: 0 0 0 0 PCI-MSI-edge pciehp
> 35: 0 0 0 1 PCI-MSI-edge pciehp
> 36: 1 0 0 0 PCI-MSI-edge pciehp
> 37: 1 0 0 0 PCI-MSI-edge pciehp
> 38: 0 1 0 0 PCI-MSI-edge pciehp
> 39: 94 0 0 0 PCI-MSI-edge eth0
> 40: 88 6 0 0 PCI-MSI-edge eth1
> 41: 0 88 6 0 PCI-MSI-edge eth2
> 42: 0 88 6 0 PCI-MSI-edge eth3
> 43: 0 0 92 0 PCI-MSI-edge eth4-rx-0
> 44: 0 0 0 0 PCI-MSI-edge eth4-tx-0
> 45: 3 0 0 0 PCI-MSI-edge eth4
> 46: 0 4 88 0 PCI-MSI-edge eth5-rx-0
> 47: 0 0 0 0 PCI-MSI-edge eth5-tx-0
> 48: 0 0 3 0 PCI-MSI-edge eth5
> 49: 0 0 4 88 PCI-MSI-edge eth6-rx-0
> 50: 0 0 0 0 PCI-MSI-edge eth6-tx-0
> 51: 0 0 0 3 PCI-MSI-edge eth6
> 52: 4 0 0 88 PCI-MSI-edge eth7-rx-0
> 53: 0 0 0 0 PCI-MSI-edge eth7-tx-0
> 54: 0 3 0 0 PCI-MSI-edge eth7
> NMI: 0 0 0 0 Non-maskable interrupts
> LOC: 196962 205933 209322 208530 Local timer interrupts
> SPU: 0 0 0 0 Spurious interrupts
> PMI: 0 0 0 0 Performance
> monitoring interrupts
> PND: 0 0 0 0 Performance pending work
> RES: 4866 761 416 467 Rescheduling interrupts
> CAL: 39 102 107 109 Function call interrupts
> TLB: 6019 5481 4678 6759 TLB shootdowns
> TRM: 0 0 0 0 Thermal event interrupts
> THR: 0 0 0 0 Threshold APIC interrupts
> MCE: 0 0 0 0 Machine check exceptions
> MCP: 2 2 2 2 Machine check polls
> ERR: 3
> MIS: 0
>
>
>
> > 3) ifconfig -a
> br0 Link encap:Ethernet HWaddr 00:90:0b:14:ac:38
> inet addr:192.168.164.222 Bcast:192.168.167.255 Mask:255.255.252.0
> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
> RX packets:50 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:2300 (2.2 KiB) TX bytes:0 (0.0 B)
>
> eth0 Link encap:Ethernet HWaddr 00:90:0b:14:ac:38
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:10119 errors:0 dropped:0 overruns:0 frame:0
> TX packets:20075 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:6654384 (6.3 MiB) TX bytes:1427890 (1.3 MiB)
> Memory:fb580000-fb5a0000
>
> eth1 Link encap:Ethernet HWaddr 00:90:0b:14:ac:39
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:20039 errors:0 dropped:0 overruns:0 frame:0
> TX packets:10155 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1495356 (1.4 MiB) TX bytes:6646438 (6.3 MiB)
> Memory:fb5e0000-fb600000
>
> eth2 Link encap:Ethernet HWaddr 00:90:0b:14:ac:3a
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:19190 errors:0 dropped:0 overruns:0 frame:0
> TX packets:9727 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1432004 (1.3 MiB) TX bytes:6365028 (6.0 MiB)
> Memory:fb680000-fb6a0000
>
> eth3 Link encap:Ethernet HWaddr 00:90:0b:14:ac:3b
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:9689 errors:0 dropped:0 overruns:0 frame:0
> TX packets:19228 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:6372426 (6.0 MiB) TX bytes:1367600 (1.3 MiB)
> Memory:fb6e0000-fb700000
> eth4 Link encap:Ethernet HWaddr 00:90:0b:15:79:18
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:9252 errors:0 dropped:0 overruns:0 frame:0
> TX packets:18354 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:6085260 (5.8 MiB) TX bytes:1305576 (1.2 MiB)
> Memory:fb8e0000-fb900000
>
> eth5 Link encap:Ethernet HWaddr 00:90:0b:15:79:19
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:18316 errors:0 dropped:0 overruns:0 frame:0
> TX packets:9290 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1366920 (1.3 MiB) TX bytes:6078300 (5.7 MiB)
> Memory:fb9e0000-fba00000
>
> eth6 Link encap:Ethernet HWaddr 00:90:0b:15:79:1a
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:18709 errors:0 dropped:0 overruns:0 frame:0
> TX packets:9484 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:1396126 (1.3 MiB) TX bytes:6204656 (5.9 MiB)
> Memory:fbae0000-fbb00000
>
> eth7 Link encap:Ethernet HWaddr 00:90:0b:15:79:1b
> UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
> RX packets:9446 errors:0 dropped:0 overruns:0 frame:0
> TX packets:18746 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:1000
> RX bytes:6211812 (5.9 MiB) TX bytes:1333280 (1.2 MiB)
> Memory:fbbe0000-fbc00000
>
> imq0 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> UP RUNNING NOARP MTU:16000 Metric:1
> RX packets:129112 errors:0 dropped:0 overruns:0 frame:0
> TX packets:129114 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:31060964 (29.6 MiB) TX bytes:31062207 (29.6 MiB)
>
> imq1 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> imq2 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> imq3 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> imq4 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> imq5 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> imq6 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
> imq7 Link encap:UNSPEC HWaddr
> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
> NOARP MTU:16000 Metric:1
> RX packets:0 errors:0 dropped:0 overruns:0 frame:0
> TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:11000
> RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
>
> lo Link encap:Local Loopback
> inet addr:127.0.0.1 Mask:255.0.0.0
> UP LOOPBACK RUNNING MTU:16436 Metric:1
> RX packets:14454 errors:0 dropped:0 overruns:0 frame:0
> TX packets:14454 errors:0 dropped:0 overruns:0 carrier:0
> collisions:0 txqueuelen:0
> RX bytes:1962561 (1.8 MiB) TX bytes:1962561 (1.8 MiB)
>
>
>
> > 4) cat /proc/sys/vm/min_free_kbytes
> 3806
>
> > 5) cat /proc/meminfo
>
> MemTotal: 12462064 kB
> MemFree: 12265628 kB
> Buffers: 29672 kB
> Cached: 64800 kB
> SwapCached: 0 kB
> Active: 87748 kB
> Inactive: 75756 kB
> Active(anon): 69040 kB
> Inactive(anon): 288 kB
> Active(file): 18708 kB
> Inactive(file): 75468 kB
> Unevictable: 0 kB
> Mlocked: 0 kB
> HighTotal: 11660872 kB
> HighFree: 11515792 kB
> LowTotal: 801192 kB
> LowFree: 749836 kB
> SwapTotal: 27226684 kB
> SwapFree: 27226684 kB
> Dirty: 120 kB
> Writeback: 0 kB
> AnonPages: 69032 kB
> Mapped: 14508 kB
> Shmem: 296 kB
> Slab: 15728 kB
> SReclaimable: 6044 kB
> SUnreclaim: 9684 kB
> KernelStack: 1376 kB
> PageTables: 996 kB
> NFS_Unstable: 0 kB
> Bounce: 0 kB
> WritebackTmp: 0 kB
> CommitLimit: 33457716 kB
> Committed_AS: 848336 kB
> VmallocTotal: 122880 kB
> VmallocUsed: 9468 kB
> VmallocChunk: 106708 kB
> DirectMap4k: 6136 kB
> DirectMap2M: 907264 kB
>
>
> > for each eth*
> > ethtool -g eth*
>
> e5570#ethtool -g eth0 - eth7 ( have the same config)
> Ring parameters for eth0:
> Pre-set maximums:
> RX: 4096
> RX Mini: 0
> RX Jumbo: 0
> TX: 4096
> Current hardware settings:
> RX: 256
> RX Mini: 0
> RX Jumbo: 0
> TX: 256
Thanks
You are running a 32bit kernel, so kernel can _not_ use more than 800
Mbytes of memory (Lowmem) for his own needs (including all network
buffers)
If you cannot switch to 64bit kernel, then you are forced to use lower
queue lengths (I see your imq devices use insane 11000 txqueuelen)
Each frame use 4K, maybe 16K, it depends on MTU.
even if we dont take into account other needs :
11000 * 16K = 170 Mbytes per imqX
1000 * 4K = 4Mbytes per ethX
170M * 8 -> memory overflow
Also /proc/sys/vm/min_free_kbytes limit is probably too small.
Really, using a 32bit kernel on this machine is not wise.
Even reducing physical ram to 4GB would be better, you would get more
Lowmem memory ;)
Since vmalloc is not used very much on your machine you can reduce
vmalloc window from 128M to 32M to save 96MB of low mem
(boot parameter : vmalloc=32M )
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-08 14:45 ` Eric Dumazet
@ 2010-09-08 15:02 ` Krzysztof Olędzki
2010-09-08 17:25 ` Jarek Poplawski
1 sibling, 0 replies; 8+ messages in thread
From: Krzysztof Olędzki @ 2010-09-08 15:02 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Anand Raj Manickam, netdev, netfilter-devel, shemminger
On 2010-09-08 16:45, Eric Dumazet wrote:
<CUT>
> Thanks
>
> You are running a 32bit kernel, so kernel can _not_ use more than 800
> Mbytes of memory (Lowmem) for his own needs (including all network
> buffers)
>
> If you cannot switch to 64bit kernel, then you are forced to use lower
> queue lengths (I see your imq devices use insane 11000 txqueuelen)
>
> Each frame use 4K, maybe 16K, it depends on MTU.
>
> even if we dont take into account other needs :
> 11000 * 16K = 170 Mbytes per imqX
> 1000 * 4K = 4Mbytes per ethX
>
> 170M * 8 -> memory overflow
>
> Also /proc/sys/vm/min_free_kbytes limit is probably too small.
>
> Really, using a 32bit kernel on this machine is not wise.
>
> Even reducing physical ram to 4GB would be better, you would get more
> Lowmem memory ;)
>
> Since vmalloc is not used very much on your machine you can reduce
> vmalloc window from 128M to 32M to save 96MB of low mem
> (boot parameter : vmalloc=32M )
It is also possible to build a kernel using 2G/2G MEM SPLIT or even
1G/3G if there is small userspace activity. But indeed, x86-64 is the
proper solution to choose.
Best regards,
Krzysztof Olędzki
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-08 14:45 ` Eric Dumazet
2010-09-08 15:02 ` Krzysztof Olędzki
@ 2010-09-08 17:25 ` Jarek Poplawski
1 sibling, 0 replies; 8+ messages in thread
From: Jarek Poplawski @ 2010-09-08 17:25 UTC (permalink / raw)
To: Eric Dumazet; +Cc: Anand Raj Manickam, netdev, netfilter-devel, shemminger
Eric Dumazet wrote, On 09/08/2010 04:45 PM:
> Le mercredi 08 septembre 2010 à 19:39 +0530, Anand Raj Manickam a
> écrit :
>>
>> imq0 Link encap:UNSPEC HWaddr
>> 00-00-00-00-00-00-00-00-00-00-00-00-00-00-00-00
>> UP RUNNING NOARP MTU:16000 Metric:1
>> RX packets:129112 errors:0 dropped:0 overruns:0 frame:0
>> TX packets:129114 errors:0 dropped:0 overruns:0 carrier:0
>> collisions:0 txqueuelen:11000
>> RX bytes:31060964 (29.6 MiB) TX bytes:31062207 (29.6 MiB)
>>
> If you cannot switch to 64bit kernel, then you are forced to use lower
> queue lengths (I see your imq devices use insane 11000 txqueuelen)
>
> Each frame use 4K, maybe 16K, it depends on MTU.
>
> even if we dont take into account other needs :
> 11000 * 16K = 170 Mbytes per imqX
> 1000 * 4K = 4Mbytes per ethX
>
> 170M * 8 -> memory overflow
>
You should consider that htb creates by default one queue per class
with txqueuelen limit each. This should probably explain why your
problems start when you classify to many classes.
Jarek P.
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-08 12:42 ` Eric Dumazet
2010-09-08 14:09 ` Anand Raj Manickam
@ 2010-09-16 9:35 ` Patrick McHardy
2010-09-16 12:20 ` Anand Raj Manickam
1 sibling, 1 reply; 8+ messages in thread
From: Patrick McHardy @ 2010-09-16 9:35 UTC (permalink / raw)
To: Anand Raj Manickam; +Cc: Eric Dumazet, netdev, netfilter-devel, shemminger
On 08.09.2010 14:42, Eric Dumazet wrote:
> Le mercredi 08 septembre 2010 à 16:22 +0530, Anand Raj Manickam a
> écrit :
>> I m struck with
>> Kernel Panic - Out of Memory Issue when traffic is more than 100k
>> pps and have around 10 tc rules.
>>
>> There is NO kernel panic with 1 tc rule .
>>
>> I using a 2.6.34.6 ( latest stable kernel) with SMP enabled
>>
>> The rules are simple -
>> 10 iptable rules :
>> iptables -t mangle -I FORWARD -s x.x.x.x -j CLASSIFY --set-class 0xz:0xy
>>
>> 10 tc rules :
>> tc class add dev imq0 parent 1:z classid 1:y htb rate 2kbit ceil 32768kbps
>>
>> I m using Intel Xeon Quad core x5570, 12GB RAM , with 8 e1000e , irq
>> affined ( 1 pair per core)
>>
>> There is NO issue / CRASH with multiple iptable rules marking to the
>> same tc rule/HTB class , i can sucessfully pass more than 200k pps /
>> 800Mbps traffic .
>>
>> But when each iptable rule is marking to seprate tc rule/HTB class and
>> when the traffic exceeds 100K pps / 20000 connections per second , we
>> hit this Out of Memory - Kernel panic condition .
>> Any help is appreciated . Do let me know if any more further
>> information is need.
>
> Hi Anand
>
> Could you give :
>
> 1) A complete stack trace / panic report
> 2) cat /proc/interrupts
> 3) ifconfig -a
> 4) cat /proc/sys/vm/min_free_kbytes
> 5) cat /proc/meminfo
>
> for each eth*
> ethtool -g eth*
Please try to reproduce without the imq patch.
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
* Re: Kernel Panic on OOM with 10 HTB rules
2010-09-16 9:35 ` Patrick McHardy
@ 2010-09-16 12:20 ` Anand Raj Manickam
0 siblings, 0 replies; 8+ messages in thread
From: Anand Raj Manickam @ 2010-09-16 12:20 UTC (permalink / raw)
To: Patrick McHardy; +Cc: Eric Dumazet, netdev, netfilter-devel
Things seem to be fine now after i reduced imq devs txqueuelen to 32
and MTU to 1600 and no of imqs to 2.
Thank you all for the Suggestions .
Anand
On Thu, Sep 16, 2010 at 3:05 PM, Patrick McHardy <kaber@trash.net> wrote:
> On 08.09.2010 14:42, Eric Dumazet wrote:
>> Le mercredi 08 septembre 2010 à 16:22 +0530, Anand Raj Manickam a
>> écrit :
>>> I m struck with
>>> Kernel Panic - Out of Memory Issue when traffic is more than 100k
>>> pps and have around 10 tc rules.
>>>
>>> There is NO kernel panic with 1 tc rule .
>>>
>>> I using a 2.6.34.6 ( latest stable kernel) with SMP enabled
>>>
>>> The rules are simple -
>>> 10 iptable rules :
>>> iptables -t mangle -I FORWARD -s x.x.x.x -j CLASSIFY --set-class 0xz:0xy
>>>
>>> 10 tc rules :
>>> tc class add dev imq0 parent 1:z classid 1:y htb rate 2kbit ceil 32768kbps
>>>
>>> I m using Intel Xeon Quad core x5570, 12GB RAM , with 8 e1000e , irq
>>> affined ( 1 pair per core)
>>>
>>> There is NO issue / CRASH with multiple iptable rules marking to the
>>> same tc rule/HTB class , i can sucessfully pass more than 200k pps /
>>> 800Mbps traffic .
>>>
>>> But when each iptable rule is marking to seprate tc rule/HTB class and
>>> when the traffic exceeds 100K pps / 20000 connections per second , we
>>> hit this Out of Memory - Kernel panic condition .
>>> Any help is appreciated . Do let me know if any more further
>>> information is need.
>>
>> Hi Anand
>>
>> Could you give :
>>
>> 1) A complete stack trace / panic report
>> 2) cat /proc/interrupts
>> 3) ifconfig -a
>> 4) cat /proc/sys/vm/min_free_kbytes
>> 5) cat /proc/meminfo
>>
>> for each eth*
>> ethtool -g eth*
>
> Please try to reproduce without the imq patch.
>
--
To unsubscribe from this list: send the line "unsubscribe netfilter-devel" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 8+ messages in thread
end of thread, other threads:[~2010-09-16 12:20 UTC | newest]
Thread overview: 8+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-09-08 10:52 Kernel Panic on OOM with 10 HTB rules Anand Raj Manickam
2010-09-08 12:42 ` Eric Dumazet
2010-09-08 14:09 ` Anand Raj Manickam
2010-09-08 14:45 ` Eric Dumazet
2010-09-08 15:02 ` Krzysztof Olędzki
2010-09-08 17:25 ` Jarek Poplawski
2010-09-16 9:35 ` Patrick McHardy
2010-09-16 12:20 ` Anand Raj Manickam
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.