* [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) @ 2010-01-23 17:46 Linus Lüssing 2010-01-23 18:10 ` Andrew Lunn ` (2 more replies) 0 siblings, 3 replies; 37+ messages in thread From: Linus Lüssing @ 2010-01-23 17:46 UTC (permalink / raw) To: b.a.t.m.a.n [-- Attachment #1: Type: text/plain, Size: 690 bytes --] Hey guys, I've been installing a 9 node setup here in our cellar. They are all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current batman-adv maintance version in OpenWRT). The result over night: - 1x: bat_events: page allocation failure - 3x: WARNING: at net/sched/sch_generic.c:226 0x801c3554() (those 3 had TQ values of about 10-15 in batctl) For the last error, I'm not sure if this is related to batman-adv or to the madwifi driver. I was using OpenWRT trunk 18405 on all routers. I'm attaching some backtraces. Sorry again I can't get more out of them than that with ksymoops. The first two ones are the bat_events problem, the others the scheduling thingy. Cheers, Linus [-- Attachment #2: kernel-panic-1752-23-01-10.log --] [-- Type: text/plain, Size: 14940 bytes --] root@OpenWrt:~# dmesg 01a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<8022329c>] 0x8022329c [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<8022329c>] 0x8022329c [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared wifi0: ath_uapsd_processtriggers: Dropping; skb is NULL in received ath_buf. wifi0: ath_uapsd_processtriggers: Dropping; skb is NULL in received ath_buf. bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<8007d3e4>] 0x8007d3e4 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<80041444>] 0x80041444 [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<8018c538>] 0x8018c538 [<800bcf78>] 0x800bcf78 [<800bd450>] 0x800bd450 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<80074174>] 0x80074174 [<80b42070>] 0x80b42070 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:149 active_file:564 inactive_anon:159 inactive_file:828 dirty:0 writeback:0 unstable:0 free:48 slab:1126 mapped:279 pagetables:55 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 1417 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 699 pages reserved 1395 pages shared 2630 pages non-shared device VPN entered promiscuous mode br-wan_vpn: topology change detected, propagating br-wan_vpn: port 2(VPN) entering forwarding state root@OpenWrt:~# [-- Attachment #3: kernel-panic-1752-23-01-10.ksymoops --] [-- Type: text/plain, Size: 22019 bytes --] ksymoops 2.4.11 on x86_64 2.6.31-1-amd64. Options used -v /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/vmlinux (specified) -K (specified) -L (specified) -o /home/linus/lff-test/kamikaze/build_dir/linux-atheros -o /home/linus/lff-test/kamikaze/build_dir/target-mips_uClibc-0.9.30.1 (specified) -m /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/System.map (specified) No modules in ksyms, skipping objects 01a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<8022329c>] 0x8022329c [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80074b5c <queue_delayed_work_on+f8/108> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 8051c178 <END_OF_CODE+266a48/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<8022329c>] 0x8022329c [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80074b5c <queue_delayed_work_on+f8/108> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 8051c178 <END_OF_CODE+266a48/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80074b5c <queue_delayed_work_on+f8/108> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 8051c178 <END_OF_CODE+266a48/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 800bd450 <__kmalloc+b4/124> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80074b5c <queue_delayed_work_on+f8/108> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 8051c178 <END_OF_CODE+266a48/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 800bd450 <__kmalloc+b4/124> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80074b5c <queue_delayed_work_on+f8/108> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 8051c178 <END_OF_CODE+266a48/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 800bd450 <__kmalloc+b4/124> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80074b5c>] 0x80074b5c [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<8051c178>] 0x8051c178 [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<8022329c>] 0x8022329c [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<8022329c>] 0x8022329c [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80074b5c <queue_delayed_work_on+f8/108> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 8051c178 <END_OF_CODE+266a48/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 800bd450 <__kmalloc+b4/124> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 8022329c <br_handle_frame_finish+0/1a4> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<8007d3e4>] 0x8007d3e4 [<800bcf78>] 0x800bcf78 [<800bd538>] 0x800bd538 [<80b41a58>] 0x80b41a58 [<80b42674>] 0x80b42674 [<80b479f8>] 0x80b479f8 [<8052129c>] 0x8052129c [<80525a20>] 0x80525a20 [<800bd450>] 0x800bd450 [<80041444>] 0x80041444 [<805269e0>] 0x805269e0 [<80525ed8>] 0x80525ed8 [<80503f50>] 0x80503f50 [<8051910c>] 0x8051910c [<801b0fb8>] 0x801b0fb8 [<801ae580>] 0x801ae580 [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<8009ac90>] 0x8009ac90 [<80041444>] 0x80041444 [<801784a0>] 0x801784a0 [<800bd450>] 0x800bd450 [<800bd470>] 0x800bd470 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<8018c4b8>] 0x8018c4b8 [<80b42088>] 0x80b42088 [<80b42070>] 0x80b42070 [<80b419d4>] 0x80b419d4 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 8007d3e4 <ktime_get+18/3c> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd538 <kmem_cache_alloc+78/e8> Trace; 80b41a58 <END_OF_CODE+88c328/????> Trace; 80b42674 <END_OF_CODE+88cf44/????> Trace; 80b479f8 <END_OF_CODE+8922c8/????> Trace; 8052129c <END_OF_CODE+26bb6c/????> Trace; 80525a20 <END_OF_CODE+2702f0/????> Trace; 800bd450 <__kmalloc+b4/124> Trace; 80041444 <ret_from_irq+0/4> Trace; 805269e0 <END_OF_CODE+2712b0/????> Trace; 80525ed8 <END_OF_CODE+2707a8/????> Trace; 80503f50 <END_OF_CODE+24e820/????> Trace; 8051910c <END_OF_CODE+2639dc/????> Trace; 801b0fb8 <net_rx_action+a8/24c> Trace; 801ae580 <net_tx_action+1c8/1f0> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 8009ac90 <__alloc_pages_internal+3f4/434> Trace; 80041444 <ret_from_irq+0/4> Trace; 801784a0 <mem_serial_out+0/20> Trace; 800bd450 <__kmalloc+b4/124> Trace; 800bd470 <__kmalloc+d4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 80b42088 <END_OF_CODE+88c958/????> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 80b419d4 <END_OF_CODE+88c2a4/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<8009ac88>] 0x8009ac88 [<8018c538>] 0x8018c538 [<800bcf78>] 0x800bcf78 [<800bd450>] 0x800bd450 [<801a8664>] 0x801a8664 [<801a9120>] 0x801a9120 [<80074174>] 0x80074174 [<80b42070>] 0x80b42070 [<8007425c>] 0x8007425c [<800745c8>] 0x800745c8 [<800789b4>] 0x800789b4 [<800790a8>] 0x800790a8 [<8007441c>] 0x8007441c [<80078a10>] 0x80078a10 [<8004e988>] 0x8004e988 CPU 0: hi: 0, btch: 1 usd: 0 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 8009ac88 <__alloc_pages_internal+3ec/434> Trace; 8018c538 <ar231x_start_xmit+80/d0> Trace; 800bcf78 <cache_alloc_refill+2d8/6fc> Trace; 800bd450 <__kmalloc+b4/124> Trace; 801a8664 <__alloc_skb+70/148> Trace; 801a9120 <skb_copy+40/c8> Trace; 80074174 <__queue_work+7c/b8> Trace; 80b42070 <END_OF_CODE+88c940/????> Trace; 8007425c <queue_work_on+90/a0> Trace; 800745c8 <worker_thread+1ac/260> Trace; 800789b4 <kthread+0/a0> Trace; 800790a8 <autoremove_wake_function+0/4c> Trace; 8007441c <worker_thread+0/260> Trace; 80078a10 <kthread+5c/a0> Trace; 8004e988 <kernel_thread_helper+10/18> 1417 total pagecache pages 4096 pages RAM 1395 pages shared 2630 pages non-shared 8 warnings issued. Results may not be reliable. [-- Attachment #4: kernel-panic-1825-23-01-10.log --] [-- Type: text/plain, Size: 10285 bytes --] BusyBox v1.14.4 (2010-01-22 23:22:23 CET) built-in shell (ash) Enter 'help' for a list of built-in commands. _______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M KAMIKAZE (bleeding edge, r18405) ------------------ * 10 oz Vodka Shake well with ice and strain * 10 oz Triple sec mixture into 10 shot glasses. * 10 oz lime juice Salute! --------------------------------------------------- root@OpenWrt:~# batctl o -b Originator (#/255) Nexthop [outgoingIF]: Potential nexthops ... [B.A.T.M.A.N. adv 0.2.1-beta r1545, MainIF/MAC: ath2/0a:24:01:b7:6a:d1] 06:22:b0:98:87:f9 ( 3) 0a:24:01:b7:61:1b [ ath2]: 06:22:b0:98:87:f9 ( 0) 0a:24:01:b7:61:1b ( 3) 06:24:01:b7:69:c1 ( 1) 06:24:01:b7:69:c1 ( 11) 06:24:01:b7:69:c1 [ ath2]: 06:24:01:b7:69:c1 ( 11) 0a:24:01:b7:61:19 ( 3) 0a:24:01:b7:6a:fd [ ath2]: 0a:24:01:b7:61:19 ( 0) 0a:24:01:b7:6a:fd ( 3) 0a:24:01:b7:61:1b ( 19) 0a:24:01:b7:61:1b [ ath2]: 0a:24:01:b7:61:1b ( 19) 06:24:01:b7:69:c1 ( 0) 06:22:b0:98:87:f9 ( 0) 0a:24:01:b7:6a:fd ( 10) 0a:24:01:b7:6a:fd [ ath2]: 0a:24:01:b7:6a:fd ( 10) 0a:24:01:b7:61:49 ( 0) 0a:24:01:b7:61:49 ( 10) 0a:24:01:b7:61:49 [ ath2]: 0a:24:01:b7:61:49 ( 10) 0a:24:01:b7:61:1b ( 1) root@OpenWrt:~# dmesg Linux version 2.6.30.9 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #56 Fri Jan 22 23:24:25 CET 2010 CPU revision is: 00019064 (MIPS 4KEc) Determined physical RAM map: memory: 01000000 @ 00000000 (usable) Initrd not found or empty - disabling initrd Zone PFN ranges: Normal 0x00000000 -> 0x00001000 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x00000000 -> 0x00001000 On node 0 totalpages: 4096 free_area_init_node: node 0, pgdat 8027ef40, node_mem_map 802b7000 Normal zone: 32 pages used for memmap Normal zone: 0 pages reserved Normal zone: 4064 pages, LIFO batch:0 Built 1 zonelists in Zone order, mobility grouping off. Total pages: 4064 Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2 Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes. Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes NR_IRQS:128 PID hash table entries: 64 (order: 6, 256 bytes) console [ttyS0] enabled Dentry cache hash table entries: 2048 (order: 1, 8192 bytes) Inode-cache hash table entries: 1024 (order: 0, 4096 bytes) Memory: 13460k/16384k available (1976k kernel code, 2924k reserved, 324k data, 128k init, 0k highmem) Calibrating delay loop... 183.50 BogoMIPS (lpj=917504) Mount-cache hash table entries: 512 net_namespace: 732 bytes NET: Registered protocol family 16 bio: create slab <bio-0> at 0 Switched to high resolution mode on CPU 0 NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 512 (order: 0, 4096 bytes) TCP bind hash table entries: 512 (order: -1, 2048 bytes) TCP: Hash tables configured (established 512 bind 512) TCP reno registered NET: Registered protocol family 1 Radio config found at offset 0xf8(0x1f8) squashfs: version 4.0 (2009/01/31) Phillip Lougher Registering mini_fo version $Id$ JFFS2 version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc. msgmni has been set to 26 io scheduler noop registered io scheduler deadline registered (default) gpiodev: gpio device registered with major 254 gpiodev: gpio platform device registered with access mask FFFFFFFF Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A eth0: Atheros AR231x: 00:24:01:b7:6a:d2, irq 4 ar231x_eth_mii: probed eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00) cmdlinepart partition parsing not available Searching for RedBoot partition table in spiflash at offset 0x3d0000 Searching for RedBoot partition table in spiflash at offset 0x3e0000 6 RedBoot partitions found on MTD device spiflash Creating 6 MTD partitions on "spiflash": 0x000000000000-0x000000030000 : "RedBoot" 0x000000030000-0x0000002f0000 : "rootfs" mtd: partition "rootfs" set to be root filesystem mtd: partition "rootfs_data" created automatically, ofs=250000, len=A0000 0x000000250000-0x0000002f0000 : "rootfs_data" 0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7" 0x0000003e0000-0x0000003ef000 : "FIS directory" 0x0000003ef000-0x0000003f0000 : "RedBoot config" 0x0000003f0000-0x000000400000 : "boardconfig" TCP westwood registered NET: Registered protocol family 17 Bridge firewalling registered 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFS: Mounted root (squashfs filesystem) readonly on device 31:1. Freeing unused kernel memory: 128k freed Please be patient, while OpenWrt loads ... mini_fo: using base directory: / mini_fo: using storage directory: /tmp/root NET: Registered protocol family 10 lo: Disabled Privacy Extensions tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> device eth0.1 entered promiscuous mode device eth0 entered promiscuous mode br-mesh: port 1(eth0.1) entering forwarding state ip_tables: (C) 2000-2006 Netfilter Core Team Ebtables v2.0 registered ip6_tables: (C) 2000-2006 Netfilter Core Team batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded device eth0.4 entered promiscuous mode br-wan_vpn: port 1(eth0.4) entering forwarding state ath_hal: module license 'Proprietary' taints kernel. Disabling lock debugging due to kernel taint ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR) br-wan_vpn: starting userspace STP failed, starting kernel STP eth0.1: no IPv6 routers present ath_ahb: trunk wlan: trunk wlan: mac acl policy registered ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk) br-mesh: no IPv6 routers present ath_rate_minstrel: look around rate set to 10% ath_rate_minstrel: EWMA rolloff level set to 75% ath_rate_minstrel: max segment size in the mrr set to 6000 us Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: H/W encryption support: WEP AES AES_CCM TKIP ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3 IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs eth0.2: no IPv6 routers present device bat0 entered promiscuous mode br-mesh: port 2(bat0) entering forwarding state eth0.3: no IPv6 routers present eth0.4: no IPv6 routers present device ath0 entered promiscuous mode br-mesh: port 3(ath0) entering forwarding state device ath0 left promiscuous mode br-mesh: port 3(ath0) entering disabled state device ath0 entered promiscuous mode br-mesh: port 3(ath0) entering forwarding state bat0: no IPv6 routers present device ath1 entered promiscuous mode br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 2(ath1) entering forwarding state device ath1 left promiscuous mode br-wan_vpn: port 2(ath1) entering disabled state device ath1 entered promiscuous mode br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 2(ath1) entering forwarding state ath0: no IPv6 routers present ath1: no IPv6 routers present br-wan_vpn: port 2(ath1) entering disabled state br-wan_vpn: port 1(eth0.4) entering disabled state br-wan_vpn: topology change detected, propagating br-wan_vpn: topology change detected, propagating br-wan_vpn: port 2(ath1) entering forwarding state br-wan_vpn: topology change detected, propagating br-wan_vpn: port 1(eth0.4) entering forwarding state br-mesh: port 3(ath0) entering disabled state br-mesh: port 2(bat0) entering disabled state br-mesh: port 1(eth0.1) entering disabled state br-mesh: port 3(ath0) entering forwarding state br-mesh: port 2(bat0) entering forwarding state br-mesh: port 1(eth0.1) entering forwarding state ath2: no IPv6 routers present br-mesh: no IPv6 routers present ------------[ cut here ]------------ WARNING: at net/sched/sch_generic.c:226 0x801c3554() NETDEV WATCHDOG: wifi0 (): transmit timed out Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<80060914>] 0x80060914 [<801c3554>] 0x801c3554 [<80060994>] 0x80060994 [<800790bc>] 0x800790bc [<801c3554>] 0x801c3554 [<8005aaf8>] 0x8005aaf8 [<800741b0>] 0x800741b0 [<80222220>] 0x80222220 [<8007d674>] 0x8007d674 [<80222138>] 0x80222138 [<801c33b8>] 0x801c33b8 [<8006c01c>] 0x8006c01c [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<80041444>] 0x80041444 ---[ end trace 31db24b68cfbba59 ]--- batman-adv:Adding interface: ath2 batman-adv:Interface activated: ath2 batman-adv:Adding interface: eth0.2 batman-adv:Interface activated: eth0.2 batman-adv:Adding interface: eth0.3 batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2 batman-adv:It is strongly recommended to keep mac addresses unique to avoid problems! batman-adv:Interface activated: eth0.3 jffs2_scan_eraseblock(): End of filesystem marker found at 0x0 jffs2_build_filesystem(): unlocking the mtd device... done. jffs2_build_filesystem(): erasing all blocks after the end marker... done. mini_fo: using base directory: / mini_fo: using storage directory: /jffs device VPN entered promiscuous mode br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 3(VPN) entering forwarding state br-wan_vpn: neighbor 8000.00:22:b0:98:8c:34 lost on port 1(eth0.4) br-wan_vpn: topology change detected, propagating root@OpenWrt:~# [-- Attachment #5: kernel-panic-1825-23-01-10.ksymoops --] [-- Type: text/plain, Size: 1787 bytes --] ksymoops 2.4.11 on x86_64 2.6.31-1-amd64. Options used -v /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/vmlinux (specified) -K (specified) -L (specified) -o /home/linus/lff-test/kamikaze/build_dir/linux-atheros -o /home/linus/lff-test/kamikaze/build_dir/target-mips_uClibc-0.9.30.1 (specified) -m /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/System.map (specified) No modules in ksyms, skipping objects lo: Disabled Privacy Extensions WARNING: at net/sched/sch_generic.c:226 0x801c3554() Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<80060914>] 0x80060914 [<801c3554>] 0x801c3554 [<80060994>] 0x80060994 [<800790bc>] 0x800790bc [<801c3554>] 0x801c3554 [<8005aaf8>] 0x8005aaf8 [<800741b0>] 0x800741b0 [<80222220>] 0x80222220 [<8007d674>] 0x8007d674 [<80222138>] 0x80222138 [<801c33b8>] 0x801c33b8 [<8006c01c>] 0x8006c01c [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<80041444>] 0x80041444 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 80060914 <warn_slowpath_common+70/b0> Trace; 801c3554 <dev_watchdog+19c/338> Trace; 80060994 <warn_slowpath_fmt+24/30> Trace; 800790bc <autoremove_wake_function+14/4c> Trace; 801c3554 <dev_watchdog+19c/338> Trace; 8005aaf8 <__wake_up_common+58/c0> Trace; 800741b0 <delayed_work_timer_fn+0/1c> Trace; 80222220 <br_fdb_cleanup+e8/124> Trace; 8007d674 <hrtimer_interrupt+26c/2a4> Trace; 80222138 <br_fdb_cleanup+0/124> Trace; 801c33b8 <dev_watchdog+0/338> Trace; 8006c01c <run_timer_softirq+17c/234> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 80041444 <ret_from_irq+0/4> 1 warning issued. Results may not be reliable. [-- Attachment #6: kernel-panic-1831-23-01-10.log --] [-- Type: text/plain, Size: 9718 bytes --] root@OpenWrt:~# batctl o -b Originator (#/255) Nexthop [outgoingIF]: Potential nexthops ... [B.A.T.M.A.N. adv 0.2.1-beta r1545, MainIF/MAC: ath2/0a:24:01:b7:61:1b] 06:22:b0:98:87:f9 ( 23) 06:22:b0:98:87:f9 [ ath2]: 06:22:b0:98:87:f9 ( 23) 06:24:01:b7:69:c1 ( 21) 06:24:01:b7:69:c1 [ ath2]: 06:24:01:b7:69:c1 ( 21) 0a:24:01:b7:6a:d1 ( 1) 0a:24:01:b7:61:19 ( 4) 0a:24:01:b7:61:49 [ ath2]: 0a:24:01:b7:61:19 ( 0) 0a:24:01:b7:6a:d1 ( 1) 0a:24:01:b7:61:49 ( 4) 0a:24:01:b7:6a:d1 ( 35) 0a:24:01:b7:6a:d1 [ ath2]: 0a:24:01:b7:6a:d1 ( 35) 06:24:01:b7:69:c1 ( 0) 0a:24:01:b7:6a:fd ( 10) 0a:24:01:b7:6a:fd [ ath2]: 0a:24:01:b7:6a:fd ( 10) 0a:24:01:b7:6a:d1 ( 0) 0a:24:01:b7:61:49 ( 0) 0a:24:01:b7:61:49 ( 10) 0a:24:01:b7:61:49 [ ath2]: 0a:24:01:b7:61:49 ( 10) 0a:24:01:b7:6a:fd ( 6) root@OpenWrt:~# dmesg Linux version 2.6.30.9 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #49 Fri Jan 22 18:48:49 CET 2010 CPU revision is: 00019064 (MIPS 4KEc) Determined physical RAM map: memory: 01000000 @ 00000000 (usable) Initrd not found or empty - disabling initrd Zone PFN ranges: Normal 0x00000000 -> 0x00001000 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x00000000 -> 0x00001000 On node 0 totalpages: 4096 free_area_init_node: node 0, pgdat 8027ef40, node_mem_map 802b7000 Normal zone: 32 pages used for memmap Normal zone: 0 pages reserved Normal zone: 4064 pages, LIFO batch:0 Built 1 zonelists in Zone order, mobility grouping off. Total pages: 4064 Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2 Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes. Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes NR_IRQS:128 PID hash table entries: 64 (order: 6, 256 bytes) console [ttyS0] enabled Dentry cache hash table entries: 2048 (order: 1, 8192 bytes) Inode-cache hash table entries: 1024 (order: 0, 4096 bytes) Memory: 13460k/16384k available (1976k kernel code, 2924k reserved, 324k data, 128k init, 0k highmem) Calibrating delay loop... 183.50 BogoMIPS (lpj=917504) Mount-cache hash table entries: 512 net_namespace: 732 bytes NET: Registered protocol family 16 bio: create slab <bio-0> at 0 Switched to high resolution mode on CPU 0 NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 512 (order: 0, 4096 bytes) TCP bind hash table entries: 512 (order: -1, 2048 bytes) TCP: Hash tables configured (established 512 bind 512) TCP reno registered NET: Registered protocol family 1 Radio config found at offset 0xf8(0x1f8) squashfs: version 4.0 (2009/01/31) Phillip Lougher Registering mini_fo version $Id$ JFFS2 version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc. msgmni has been set to 26 io scheduler noop registered io scheduler deadline registered (default) gpiodev: gpio device registered with major 254 gpiodev: gpio platform device registered with access mask FFFFFFFF Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A eth0: Atheros AR231x: 00:24:01:b7:61:1c, irq 4 ar231x_eth_mii: probed eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00) cmdlinepart partition parsing not available Searching for RedBoot partition table in spiflash at offset 0x3d0000 Searching for RedBoot partition table in spiflash at offset 0x3e0000 6 RedBoot partitions found on MTD device spiflash Creating 6 MTD partitions on "spiflash": 0x000000000000-0x000000030000 : "RedBoot" 0x000000030000-0x0000002f0000 : "rootfs" mtd: partition "rootfs" set to be root filesystem mtd: partition "rootfs_data" created automatically, ofs=250000, len=A0000 0x000000250000-0x0000002f0000 : "rootfs_data" 0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7" 0x0000003e0000-0x0000003ef000 : "FIS directory" 0x0000003ef000-0x0000003f0000 : "RedBoot config" 0x0000003f0000-0x000000400000 : "boardconfig" TCP westwood registered NET: Registered protocol family 17 Bridge firewalling registered 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFS: Mounted root (squashfs filesystem) readonly on device 31:1. Freeing unused kernel memory: 128k freed Please be patient, while OpenWrt loads ... mini_fo: using base directory: / mini_fo: using storage directory: /jffs NET: Registered protocol family 10 lo: Disabled Privacy Extensions tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> device eth0.1 entered promiscuous mode device eth0 entered promiscuous mode br-mesh: port 1(eth0.1) entering forwarding state ip_tables: (C) 2000-2006 Netfilter Core Team Ebtables v2.0 registered ip6_tables: (C) 2000-2006 Netfilter Core Team batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded device eth0.4 entered promiscuous mode br-wan_vpn: port 1(eth0.4) entering forwarding state br-wan_vpn: starting userspace STP failed, starting kernel STP ath_hal: module license 'Proprietary' taints kernel. Disabling lock debugging due to kernel taint ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR) eth0.1: no IPv6 routers present br-mesh: no IPv6 routers present ath_ahb: trunk wlan: trunk wlan: mac acl policy registered ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk) ath_rate_minstrel: look around rate set to 10% ath_rate_minstrel: EWMA rolloff level set to 75% ath_rate_minstrel: max segment size in the mrr set to 6000 us Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: H/W encryption support: WEP AES AES_CCM TKIP ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3 IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs eth0.2: no IPv6 routers present device bat0 entered promiscuous mode br-mesh: port 2(bat0) entering forwarding state eth0.3: no IPv6 routers present br-wan_vpn: no IPv6 routers present eth0.4: no IPv6 routers present device ath0 entered promiscuous mode br-mesh: port 3(ath0) entering forwarding state bat0: no IPv6 routers present device ath0 left promiscuous mode br-mesh: port 3(ath0) entering disabled state device ath0 entered promiscuous mode br-mesh: port 3(ath0) entering forwarding state device ath1 entered promiscuous mode br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 2(ath1) entering forwarding state device ath1 left promiscuous mode br-wan_vpn: port 2(ath1) entering disabled state device ath1 entered promiscuous mode br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 2(ath1) entering forwarding state ath0: no IPv6 routers present br-wan_vpn: port 2(ath1) entering disabled state br-wan_vpn: port 1(eth0.4) entering disabled state br-wan_vpn: topology change detected, propagating br-wan_vpn: topology change detected, propagating br-wan_vpn: port 2(ath1) entering forwarding state br-wan_vpn: topology change detected, propagating br-wan_vpn: port 1(eth0.4) entering forwarding state br-mesh: port 3(ath0) entering disabled state br-mesh: port 2(bat0) entering disabled state br-mesh: port 1(eth0.1) entering disabled state br-mesh: port 3(ath0) entering forwarding state br-mesh: port 2(bat0) entering forwarding state br-mesh: port 1(eth0.1) entering forwarding state ath1: no IPv6 routers present ath2: no IPv6 routers present br-mesh: no IPv6 routers present batman-adv:Adding interface: ath2 batman-adv:Interface activated: ath2 batman-adv:Adding interface: eth0.2 batman-adv:Interface activated: eth0.2 batman-adv:Adding interface: eth0.3 batman-adv:The newly added mac address (00:24:01:b7:61:1c) already exists on: eth0.2 batman-adv:It is strongly recommended to keep mac addresses unique to avoid problems! batman-adv:Interface activated: eth0.3 br-wan_vpn: port 2(ath1) entering disabled state device VPN entered promiscuous mode br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 3(VPN) entering forwarding state br-wan_vpn: port 3(VPN) entering disabled state br-wan_vpn: topology change detected, sending tcn bpdu br-wan_vpn: port 3(VPN) entering forwarding state br-wan_vpn: neighbor 8000.00:22:b0:98:8c:34 lost on port 1(eth0.4) br-wan_vpn: topology change detected, propagating ------------[ cut here ]------------ WARNING: at net/sched/sch_generic.c:226 0x801c3554() NETDEV WATCHDOG: wifi0 (): transmit timed out Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<80060914>] 0x80060914 [<801c3554>] 0x801c3554 [<80060994>] 0x80060994 [<801c3554>] 0x801c3554 [<80086bf4>] 0x80086bf4 [<8007d674>] 0x8007d674 [<801b0838>] 0x801b0838 [<801c33b8>] 0x801c33b8 [<8006c01c>] 0x8006c01c [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<80041444>] 0x80041444 [<80041660>] 0x80041660 [<8018c4b8>] 0x8018c4b8 [<8004e85c>] 0x8004e85c [<80041680>] 0x80041680 [<80281a4c>] 0x80281a4c [<80281370>] 0x80281370 ---[ end trace bf8b652a122b3db4 ]--- root@OpenWrt:~# [-- Attachment #7: kernel-panic-1831-23-01-10.ksymoops --] [-- Type: text/plain, Size: 1950 bytes --] ksymoops 2.4.11 on x86_64 2.6.31-1-amd64. Options used -v /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/vmlinux (specified) -K (specified) -L (specified) -o /home/linus/lff-test/kamikaze/build_dir/linux-atheros -o /home/linus/lff-test/kamikaze/build_dir/target-mips_uClibc-0.9.30.1 (specified) -m /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/System.map (specified) No modules in ksyms, skipping objects lo: Disabled Privacy Extensions WARNING: at net/sched/sch_generic.c:226 0x801c3554() Call Trace:[<800493bc>] 0x800493bc [<800493bc>] 0x800493bc [<80060914>] 0x80060914 [<801c3554>] 0x801c3554 [<80060994>] 0x80060994 [<801c3554>] 0x801c3554 [<80086bf4>] 0x80086bf4 [<8007d674>] 0x8007d674 [<801b0838>] 0x801b0838 [<801c33b8>] 0x801c33b8 [<8006c01c>] 0x8006c01c [<80066d84>] 0x80066d84 [<80066e90>] 0x80066e90 [<80041444>] 0x80041444 [<80041660>] 0x80041660 [<8018c4b8>] 0x8018c4b8 [<8004e85c>] 0x8004e85c [<80041680>] 0x80041680 [<80281a4c>] 0x80281a4c [<80281370>] 0x80281370 Warning (Oops_read): Code line not seen, dumping what data is available Trace; 800493bc <dump_stack+20/50> Trace; 800493bc <dump_stack+20/50> Trace; 80060914 <warn_slowpath_common+70/b0> Trace; 801c3554 <dev_watchdog+19c/338> Trace; 80060994 <warn_slowpath_fmt+24/30> Trace; 801c3554 <dev_watchdog+19c/338> Trace; 80086bf4 <tick_program_event+28/34> Trace; 8007d674 <hrtimer_interrupt+26c/2a4> Trace; 801b0838 <process_backlog+d4/14c> Trace; 801c33b8 <dev_watchdog+0/338> Trace; 8006c01c <run_timer_softirq+17c/234> Trace; 80066d84 <__do_softirq+a4/154> Trace; 80066e90 <do_softirq+5c/98> Trace; 80041444 <ret_from_irq+0/4> Trace; 80041660 <r4k_wait+0/40> Trace; 8018c4b8 <ar231x_start_xmit+0/d0> Trace; 8004e85c <cpu_idle+24/4c> Trace; 80041680 <r4k_wait+20/40> Trace; 80281a4c <start_kernel+36c/384> Trace; 80281370 <unknown_bootoption+0/2fc> 1 warning issued. Results may not be reliable. ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing @ 2010-01-23 18:10 ` Andrew Lunn 2010-01-23 23:30 ` Linus Lüssing 2010-01-24 20:42 ` Linus Lüssing 2010-01-24 4:24 ` Linus Lüssing 2010-01-26 6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing 2 siblings, 2 replies; 37+ messages in thread From: Andrew Lunn @ 2010-01-23 18:10 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Sat, Jan 23, 2010 at 06:46:16PM +0100, Linus L??ssing wrote: > Hey guys, > > I've been installing a 9 node setup here in our cellar. They are > all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current > batman-adv maintance version in OpenWRT). > > The result over night: > - 1x: bat_events: page allocation failure This looks like a memory leak somewhere. It could be anywhere, batman or some other part of the kernel. Can you build your kernel with the kernel memory leak detector enabled? It is under the Kernel Hacking options. You probably also need to build the kernel with symbols, which will help with ksymopps anyway. Documentation for kmemleak is in Documentation/kmemleak.txt. I've never used it myself, so i've no idea how good it actually is... Interestingly, did you notice the warnings: batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2 batman-adv:It is strongly recommended to keep mac addresses unique to avoid +problems! I guess this is because you are using VLANs. Have you seen these problems without using VLANs? Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-23 18:10 ` Andrew Lunn @ 2010-01-23 23:30 ` Linus Lüssing 2010-01-24 20:42 ` Linus Lüssing 1 sibling, 0 replies; 37+ messages in thread From: Linus Lüssing @ 2010-01-23 23:30 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 1455 bytes --] Ok, this is with symbols enabled and I changed the mac addresses, so they are defintely different now. Here is the new backtrace for the second issue. It occured after about 1-2 hours. Cheers, Linus On Sat, Jan 23, 2010 at 07:10:48PM +0100, Andrew Lunn wrote: > On Sat, Jan 23, 2010 at 06:46:16PM +0100, Linus L??ssing wrote: > > Hey guys, > > > > I've been installing a 9 node setup here in our cellar. They are > > all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current > > batman-adv maintance version in OpenWRT). > > > > The result over night: > > - 1x: bat_events: page allocation failure > > This looks like a memory leak somewhere. It could be anywhere, batman > or some other part of the kernel. > > Can you build your kernel with the kernel memory leak detector > enabled? It is under the Kernel Hacking options. You probably also > need to build the kernel with symbols, which will help with ksymopps > anyway. Documentation for kmemleak is in Documentation/kmemleak.txt. > I've never used it myself, so i've no idea how good it actually is... > > Interestingly, did you notice the warnings: > > batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2 > batman-adv:It is strongly recommended to keep mac addresses unique to avoid > +problems! > > I guess this is because you are using VLANs. Have you seen these > problems without using VLANs? > > Andrew > [-- Attachment #2: call-trace-0027-24-01-10.log --] [-- Type: text/plain, Size: 1127 bytes --] root@OpenWrt:/# ------------[ cut here ]------------ WARNING: at net/sched/sch_generic.c:226 dev_watchdog+0x19c/0x338() NETDEV WATCHDOG: wifi0 (): transmit timed out Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<80062a84>] warn_slowpath_common+0x70/0xb0 [<80062b04>] warn_slowpath_fmt+0x24/0x30 [<801c9c98>] dev_watchdog+0x19c/0x338 [<8006e1b8>] run_timer_softirq+0x17c/0x234 [<80068f20>] __do_softirq+0xa4/0x154 [<8006902c>] do_softirq+0x5c/0x98 [<80041444>] ret_from_irq+0x0/0x4 [<80041680>] r4k_wait+0x20/0x40 [<8004e910>] cpu_idle+0x24/0x4c [<802a9a4c>] start_kernel+0x36c/0x384 ---[ end trace 57451ac66d72b45d ]-- ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-23 18:10 ` Andrew Lunn 2010-01-23 23:30 ` Linus Lüssing @ 2010-01-24 20:42 ` Linus Lüssing 2010-01-24 21:00 ` Andrew Lunn 2010-01-25 6:47 ` Andrew Lunn 1 sibling, 2 replies; 37+ messages in thread From: Linus Lüssing @ 2010-01-24 20:42 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 1905 bytes --] Hi Andrew, Here is the output with the symbols inserted. I can reproduce this memory leak very reliable now, by just activating the vis server (didn't do that with the logs I've posted before) in a couple of minutes. free is showing a nice countdown then :). For kmemleak... I'm using 2.6.30.10 here and I think Torvalds merged it into 2.6.33 a month ago. Anyone having a 2.6.33 kernel running at the moment, patches for kmemleak in 2.6.30 or another idea of how to debug this in 2.6.30? "Debug slab memory allocations" (CONFIG_DEBUG_SLAB) and "Memory leak debugging" (DEBUG_SLAB_LEAK) are activated on the kernel I'm using here, but they are not of any use, are they? Cheers, Linus On Sat, Jan 23, 2010 at 07:10:48PM +0100, Andrew Lunn wrote: > On Sat, Jan 23, 2010 at 06:46:16PM +0100, Linus L??ssing wrote: > > Hey guys, > > > > I've been installing a 9 node setup here in our cellar. They are > > all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current > > batman-adv maintance version in OpenWRT). > > > > The result over night: > > - 1x: bat_events: page allocation failure > > This looks like a memory leak somewhere. It could be anywhere, batman > or some other part of the kernel. > > Can you build your kernel with the kernel memory leak detector > enabled? It is under the Kernel Hacking options. You probably also > need to build the kernel with symbols, which will help with ksymopps > anyway. Documentation for kmemleak is in Documentation/kmemleak.txt. > I've never used it myself, so i've no idea how good it actually is... > > Interestingly, did you notice the warnings: > > batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2 > batman-adv:It is strongly recommended to keep mac addresses unique to avoid > +problems! > > I guess this is because you are using VLANs. Have you seen these > problems without using VLANs? > > Andrew > [-- Attachment #2: 2010-01-24-20-53-memory-leak.log --] [-- Type: text/plain, Size: 19567 bytes --] +Ethernet eth0: MAC address 00:22:b0:98:87:de IP: 192.168.1.1/255.255.255.0, Gateway: 0.0.0.0 Default server: 192.168.1.2 RedBoot(tm) bootstrap and debug environment [ROMRAM] production release, version "2.1.3" - built 18:43:19, Sep 20 2007 Platform: ap61 (Atheros WiSOC) Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc. Copyright (C) 2007, NewMedia-NET GmbH. Board: DLINK DIR-300 RAM: 0x80000000-0x81000000, [0x80040580-0x80fe1000] available FLASH: 0xbfc00000 - 0xbfff0000, 64 blocks of 0x00010000 bytes each. == Executing boot script in 5.000 seconds - enter ^C to abort DD-WRT> fis load -l vmlinux.bin.l7 Image loaded from 0x80041000-0x802c8200 DD-WRT> exec Now booting linux kernel: Base address 0x80030000 Entry 0x80041000 Cmdline : Linux version 2.6.30.10 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #2 Sun Jan 24 04:12:11 CET 2010 CPU revision is: 00019064 (MIPS 4KEc) Determined physical RAM map: memory: 01000000 @ 00000000 (usable) Initrd not found or empty - disabling initrd Zone PFN ranges: Normal 0x00000000 -> 0x00001000 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x00000000 -> 0x00001000 Built 1 zonelists in Zone order, mobility grouping off. Total pages: 4064 Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2 Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes. Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes NR_IRQS:128 PID hash table entries: 64 (order: 6, 256 bytes) console [ttyS0] enabled Dentry cache hash table entries: 2048 (order: 1, 8192 bytes) Inode-cache hash table entries: 1024 (order: 0, 4096 bytes) Memory: 13300k/16384k available (2002k kernel code, 3084k reserved, 458k data, 128k init, 0k highmem) Calibrating delay loop... 183.50 BogoMIPS (lpj=917504) Mount-cache hash table entries: 512 net_namespace: 732 bytes NET: Registered protocol family 16 bio: create slab <bio-0> at 0 NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 512 (order: 0, 4096 bytes) TCP bind hash table entries: 512 (order: -1, 2048 bytes) TCP: Hash tables configured (established 512 bind 512) TCP reno registered NET: Registered protocol family 1 Radio config found at offset 0xf8(0x1f8) squashfs: version 4.0 (2009/01/31) Phillip Lougher Registering mini_fo version $Id$ JFFS2 version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc. msgmni has been set to 25 io scheduler noop registered io scheduler deadline registered (default) gpiodev: gpio device registered with major 254 gpiodev: gpio platform device registered with access mask FFFFFFFF Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A eth0: Atheros AR231x: 00:22:b0:98:87:de, irq 4 ar231x_eth_mii: probed eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00) cmdlinepart partition parsing not available Searching for RedBoot partition table in spiflash at offset 0x3d0000 Searching for RedBoot partition table in spiflash at offset 0x3e0000 6 RedBoot partitions found on MTD device spiflash Creating 6 MTD partitions on "spiflash": 0x000000000000-0x000000030000 : "RedBoot" 0x000000030000-0x0000002f0000 : "rootfs" mtd: partition "rootfs" set to be root filesystem mtd: partition "rootfs_data" created automatically, ofs=230000, len=C0000 0x000000230000-0x0000002f0000 : "rootfs_data" 0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7" 0x0000003e0000-0x0000003ef000 : "FIS directory" 0x0000003ef000-0x0000003f0000 : "RedBoot config" 0x0000003f0000-0x000000400000 : "boardconfig" TCP westwood registered NET: Registered protocol family 17 Bridge firewalling registered 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFS: Mounted root (squashfs filesystem) readonly on device 31:1. Freeing unused kernel memory: 128k freed Please be patient, while OpenWrt loads ... - preinit - Press CTRL-C for failsafe switching to jffs2 mini_fo: using base directory: / mini_fo: using storage directory: /jffs - init - Please press Enter to activate this console. NET: Registered protocol family 10 lo: Disabled Privacy Extensions tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> device eth0.1 entered promiscuous mode device eth0 entered promiscuous mode br-mesh: port 1(eth0.1) entering forwarding state ip_tables: (C) 2000-2006 Netfilter Core Team Ebtables v2.0 registered ip6_tables: (C) 2000-2006 Netfilter Core Team device eth0.4 entered promiscuous mode br-wan_vpn: port 1(eth0.4) entering forwarding state br-wan_vpn: starting userspace STP failed, starting kernel STP batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded ath_hal: module license 'Proprietary' taints kernel. Disabling lock debugging due to kernel taint ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR) ath_ahb: trunk wlan: trunk wlan: mac acl policy registered ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk) ath_rate_minstrel: look around rate set to 10% ath_rate_minstrel: EWMA rolloff level set to 75% ath_rate_minstrel: max segment size in the mrr set to 6000 us Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: H/W encryption support: WEP AES AES_CCM TKIP ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3 IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs device bat0 entered promiscuous mode br-mesh: port 2(bat0) entering forwarding state br-wan_vpn: port 1(eth0.4) entering disabled state br-wan_vpn: topology change detected, propagating br-wan_vpn: port 1(eth0.4) entering forwarding state br-mesh: port 2(bat0) entering disabled state br-mesh: port 1(eth0.1) entering disabled state br-mesh: port 2(bat0) entering forwarding state br-mesh: port 1(eth0.1) entering forwarding state batman-adv:Adding interface: ath0 batman-adv:Interface activated: ath0 batman-adv:Adding interface: eth0.2 batman-adv:Interface activated: eth0.2 batman-adv:Adding interface: eth0.3 batman-adv:Interface activated: eth0.3 device VPN entered promiscuous mode br-wan_vpn: topology change detected, propagating br-wan_vpn: port 2(VPN) entering forwarding state BusyBox v1.15.3 (2010-01-24 04:59:20 CET) built-in shell (ash) Enter 'help' for a list of built-in commands. _______ ________ __ | |.-----.-----.-----.| | | |.----.| |_ | - || _ | -__| || | | || _|| _| |_______|| __|_____|__|__||________||__| |____| |__| W I R E L E S S F R E E D O M KAMIKAZE (bleeding edge, r19304) ------------------ * 10 oz Vodka Shake well with ice and strain * 10 oz Triple sec mixture into 10 shot glasses. * 10 oz lime juice Salute! --------------------------------------------------- root@OpenWrt:/# batctl Usage: batctl [options] commands commands: interface|if [none|interface] display or modify the interface settings originators|o display the originator table interval|it [orig_interval] display or modify the originator interval in ms loglevel|ll [level] display or modify the log level log|l read the log produced by the kernel module translocal|tl display the local translation table transglobal|tg display the global translation table vis_server|vs [enable|disable] display or modify the status of the VIS server vis_data|vd [dot|JSON] display the VIS data in dot or JSON format aggregation|ag [0|1] display or modify the packet aggregation setting ping|p <destination> ping another batman adv host via layer 2 traceroute|tr <destination> traceroute another batman adv host via layer 2 tcpdump|td <interface> tcpdump layer 2 traffic on the given interface bisect <file1> .. <fileN> analyze given log files for routing stability options: -h print this help (or 'batctl <command> -h' for the command specific help) -v print version root@OpenWrt:/# batctl vs enabled batman-adv:Setting VIS mode to server (enabling vis server) root@OpenWrt:/# watchdog invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009b644>] oom_kill_process+0x6c/0x1fc [<8009bd08>] __out_of_memory+0x184/0x1bc [<8009bdb0>] out_of_memory+0x70/0xa0 [<8009f03c>] __alloc_pages_internal+0x338/0x434 [<800a1e64>] __do_page_cache_readahead+0xe0/0x278 [<800a206c>] do_page_cache_readahead+0x70/0x8c [<80098880>] filemap_fault+0x19c/0x464 [<800ae778>] __do_fault+0x74/0x4e8 [<800b0e18>] handle_mm_fault+0x314/0x6fc [<80055d3c>] do_page_fault+0x110/0x30c [<80041420>] ret_from_exception+0x0/0x24 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:0 inactive_anon:164 inactive_file:10 dirty:0 writeback:0 unstable:0 free:126 slab:2222 mapped:0 pagetables:49 bounce:0 Normal free:500kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:0kB inactive_file:40kB present:16256kB pages_scanned:1533 all_unreclaimable? no lowmem_reserve[]: 0 0 Normal: 1*4kB 0*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 500kB 27 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 10 pages shared 3105 pages non-shared Out of memory: kill process 967 (dnsmasq) score 225 or a child Killed process 967 (dnsmasq) swapper: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb] [<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb] Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared swapper: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb] [<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb] Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared wifi0: ath_reset: Unable to start receive logic. bat_events: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<80bc2144>] send_raw_packet+0x3c/0xc0 [batman_adv] [<80bc23bc>] send_outstanding_bat_packet+0x1f4/0x250 [batman_adv] [<800767b8>] worker_thread+0x1b0/0x278 [<8007ac14>] kthread+0x5c/0xa0 [<8004ec88>] kernel_thread_helper+0x10/0x18 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb] [<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb] Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared wifi0: ath_reset: Unable to start receive logic. bat_events: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<8022bb7c>] br_send_bpdu+0x2c/0x150 [<8022beb8>] br_send_config_bpdu+0x1d0/0x1ec [<8022b398>] br_transmit_config+0x148/0x180 [<8022b4b4>] br_config_bpdu_generation+0x60/0x8c [<8022cc58>] br_hello_timer_expired+0x28/0x64 [<8006e204>] run_timer_softirq+0x17c/0x234 [<80068f6c>] __do_softirq+0xa4/0x154 [<80069078>] do_softirq+0x5c/0x98 [<80041444>] ret_from_irq+0x0/0x4 [<800c30e8>] __kmalloc_track_caller+0x1a4/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<80bc2144>] send_raw_packet+0x3c/0xc0 [batman_adv] [<80bc23bc>] send_outstanding_bat_packet+0x1f4/0x250 [batman_adv] [<800767b8>] worker_thread+0x1b0/0x278 [<8007ac14>] kthread+0x5c/0xa0 [<8004ec88>] kernel_thread_helper+0x10/0x18 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<8022bb7c>] br_send_bpdu+0x2c/0x150 [<8022beb8>] br_send_config_bpdu+0x1d0/0x1ec [<8022b398>] br_transmit_config+0x148/0x180 [<8022b4b4>] br_config_bpdu_generation+0x60/0x8c [<8022cc58>] br_hello_timer_expired+0x28/0x64 [<8006e204>] run_timer_softirq+0x17c/0x234 [<80068f6c>] __do_softirq+0xa4/0x154 [<80069078>] do_softirq+0x5c/0x98 [<80041444>] ret_from_irq+0x0/0x4 [<800c30e8>] __kmalloc_track_caller+0x1a4/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<80bc2144>] send_raw_packet+0x3c/0xc0 [batman_adv] [<80bc23bc>] send_outstanding_bat_packet+0x1f4/0x250 [batman_adv] [<800767b8>] worker_thread+0x1b0/0x278 [<8007ac14>] kthread+0x5c/0xa0 [<8004ec88>] kernel_thread_helper+0x10/0x18 Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared bat_events: page allocation failure. order:0, mode:0x20 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<8009f0f0>] __alloc_pages_internal+0x3ec/0x434 [<800c2990>] cache_alloc_refill+0x39c/0x950 [<800c30c8>] __kmalloc_track_caller+0x184/0x208 [<801aee18>] __alloc_skb+0x7c/0x158 [<801afd2c>] dev_alloc_skb+0x1c/0x4c [<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb] [<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb] Mem-Info: Normal per-cpu: CPU 0: hi: 0, btch: 1 usd: 0 Active_anon:160 active_file:3 inactive_anon:164 inactive_file:3 dirty:0 writeback:0 unstable:0 free:48 slab:2304 mapped:0 pagetables:49 bounce:0 Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes lowmem_reserve[]: 0 0 Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB 23 total pagecache pages 0 pages in swap cache Swap cache stats: add 0, delete 0, find 0/0 Free swap = 0kB Total swap = 0kB 4096 pages RAM 739 pages reserved 6 pages shared 3185 pages non-shared wifi0: ath_reset: Unable to start receive logic. watchdog expired, rebooting system + ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-24 20:42 ` Linus Lüssing @ 2010-01-24 21:00 ` Andrew Lunn 2010-01-25 6:46 ` Andrew Lunn 2010-01-25 6:47 ` Andrew Lunn 1 sibling, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-01-24 21:00 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Sun, Jan 24, 2010 at 09:42:12PM +0100, Linus L??ssing wrote: > Hi Andrew, > > Here is the output with the symbols inserted. I can reproduce this > memory leak very reliable now, by just activating the vis server > (didn't do that with the logs I've posted before) in a couple of > minutes. free is showing a nice countdown then :). So it is vis server functionality which is leaking? If vis server is not enabled, there is no leak? If so, we don't need kmemcheck. I expect just looking at the vis code is probably enough to find the missing free. Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-24 21:00 ` Andrew Lunn @ 2010-01-25 6:46 ` Andrew Lunn 0 siblings, 0 replies; 37+ messages in thread From: Andrew Lunn @ 2010-01-25 6:46 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Sun, Jan 24, 2010 at 10:00:10PM +0100, Andrew Lunn wrote: > On Sun, Jan 24, 2010 at 09:42:12PM +0100, Linus L??ssing wrote: > > Hi Andrew, > > > > Here is the output with the symbols inserted. I can reproduce this > > memory leak very reliable now, by just activating the vis server > > (didn't do that with the logs I've posted before) in a couple of > > minutes. free is showing a nice countdown then :). > > So it is vis server functionality which is leaking? If vis server is > not enabled, there is no leak? > > If so, we don't need kmemcheck. I expect just looking at the vis code > is probably enough to find the missing free. It looks pretty obvious: Working our way via the call stack: In batman_skb_recv() we have: case BAT_VIS: ret = recv_vis_packet(skb); break; default: ret = NET_RX_DROP; } if (ret == NET_RX_DROP) kfree_skb(skb); i.e. the packet is only freed if recv_vis_packet() returns NET_RX_DROP In recv_vis_packet() we have: switch (vis_packet->vis_type) { case VIS_TYPE_SERVER_SYNC: /* TODO: handle fragmented skbs properly */ receive_server_sync_packet(vis_packet, skb_headlen(skb)); ret = NET_RX_SUCCESS; break; case VIS_TYPE_CLIENT_UPDATE: /* TODO: handle fragmented skbs properly */ receive_client_update_packet(vis_packet, skb_headlen(skb)); ret = NET_RX_SUCCESS; break; default: /* ignore unknown packet */ ret = NET_RX_DROP; break; } return ret; i.e. receive_client_update_packet() and receive_server_sync_packet() need to sometime result in the packet being freed. batman_skb_recv() will not free the packet. void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len) { struct vis_info *info; int is_new; unsigned long flags; int vis_server = atomic_read(&vis_mode); spin_lock_irqsave(&vis_hash_lock, flags); info = add_packet(vis_packet, vis_info_len, &is_new); if (info == NULL) goto end; /* only if we are server ourselves and packet is newer than the one in * hash.*/ if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) { memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); if (list_empty(&info->send_list)) list_add_tail(&info->send_list, &send_list); } end: spin_unlock_irqrestore(&vis_hash_lock, flags); } No free here. vis_packet is a pointer into the skbuf, but we never add vis_packet to any queue here, just info is queued. add_packet(): info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC); if (info == NULL) return NULL; INIT_LIST_HEAD(&info->send_list); INIT_LIST_HEAD(&info->recv_list); info->first_seen = jiffies; memcpy(&info->packet, vis_packet, sizeof(struct vis_packet) + vis_info_len); We allocate memory can copy the contents of the packet into the allocated memory. So again, the skbuf is not queued. As far as i can see, the skbuf is never put on a queue. We just copy the needed data out of it. This means the skbuf does need freeing. I had a very quick look at receive_client_update_packet(). It seems to have the same structure. I will submit a fix for this soon. Is there any other similar code which might have the same bug? Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-24 20:42 ` Linus Lüssing 2010-01-24 21:00 ` Andrew Lunn @ 2010-01-25 6:47 ` Andrew Lunn 2010-01-25 8:21 ` Marek Lindner 2010-01-26 1:48 ` Linus Lüssing 1 sibling, 2 replies; 37+ messages in thread From: Andrew Lunn @ 2010-01-25 6:47 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Staging: batman-adv: Fix skbuff leak in VIS code. The vis code takes a copy of the data inside the skbuf if it is interesting for us, so we always need to release the skbuf. Reported-by: Linus Lüssing <linus.luessing@web.de> Signed-off-by: Andrew Lunn <andrew@lunn.ch> Index: routing.c =================================================================== --- routing.c (revision 1563) +++ routing.c (working copy) @@ -1135,21 +1135,22 @@ if (is_my_mac(vis_packet->sender_orig)) return NET_RX_DROP; + /* We take a copy of the data in the packet, so we should + always free the skbuf. */ + ret = NET_RX_DROP; + switch (vis_packet->vis_type) { case VIS_TYPE_SERVER_SYNC: /* TODO: handle fragmented skbs properly */ receive_server_sync_packet(vis_packet, skb_headlen(skb)); - ret = NET_RX_SUCCESS; break; case VIS_TYPE_CLIENT_UPDATE: /* TODO: handle fragmented skbs properly */ receive_client_update_packet(vis_packet, skb_headlen(skb)); - ret = NET_RX_SUCCESS; break; default: /* ignore unknown packet */ - ret = NET_RX_DROP; break; } return ret; ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-25 6:47 ` Andrew Lunn @ 2010-01-25 8:21 ` Marek Lindner 2010-01-26 1:48 ` Linus Lüssing 1 sibling, 0 replies; 37+ messages in thread From: Marek Lindner @ 2010-01-25 8:21 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Monday 25 January 2010 14:47:47 Andrew Lunn wrote: > Staging: batman-adv: Fix skbuff leak in VIS code. > > The vis code takes a copy of the data inside the skbuf if it is > interesting for us, so we always need to release the skbuf. Applied in rev 1564. Thanks, Marek ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-25 6:47 ` Andrew Lunn 2010-01-25 8:21 ` Marek Lindner @ 2010-01-26 1:48 ` Linus Lüssing 1 sibling, 0 replies; 37+ messages in thread From: Linus Lüssing @ 2010-01-26 1:48 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Just tried the patch on the routers and it works perfectly, no more memory leak caused by vis! Thanks for the quick fix. Cheers, Linus ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) 2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing 2010-01-23 18:10 ` Andrew Lunn @ 2010-01-24 4:24 ` Linus Lüssing 2010-01-26 6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing 2 siblings, 0 replies; 37+ messages in thread From: Linus Lüssing @ 2010-01-24 4:24 UTC (permalink / raw) To: b.a.t.m.a.n [-- Attachment #1: Type: text/plain, Size: 170 bytes --] And another (different) one. Which occured right after flashing and booting causing a kernel panic. Sorry couldn't find a method for reproducing them yet. Cheers, Linus [-- Attachment #2: kernel-panic-0512-24-01-10.log --] [-- Type: text/plain, Size: 9295 bytes --] root@OpenWrt:/# n+Warning: IP address 192.168.1.1 in use Ethernet eth0: MAC address 00:22:b0:98:87:de IP: 192.168.1.1/255.255.255.0, Gateway: 0.0.0.0 Default server: 192.168.1.2 RedBoot(tm) bootstrap and debug environment [ROMRAM] production release, version "2.1.3" - built 18:43:19, Sep 20 2007 Platform: ap61 (Atheros WiSOC) Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc. Copyright (C) 2007, NewMedia-NET GmbH. Board: DLINK DIR-300 RAM: 0x80000000-0x81000000, [0x80040580-0x80fe1000] available FLASH: 0xbfc00000 - 0xbfff0000, 64 blocks of 0x00010000 bytes each. == Executing boot script in 5.000 seconds - enter ^C to abort +Warning: IP address 192.168.1.1 in use Ethernet eth0: MAC address 00:22:b0:98:87:de IP: 192.168.1.1/255.255.255.0, Gateway: 0.0.0.0 Default server: 192.168.1.2 RedBoot(tm) bootstrap and debug environment [ROMRAM] production release, version "2.1.3" - built 18:43:19, Sep 20 2007 Platform: ap61 (Atheros WiSOC) Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc. Copyright (C) 2007, NewMedia-NET GmbH. Board: DLINK DIR-300 RAM: 0x80000000-0x81000000, [0x80040580-0x80fe1000] available FLASH: 0xbfc00000 - 0xbfff0000, 64 blocks of 0x00010000 bytes each. == Executing boot script in 5.000 seconds - enter ^C to abort DD-WRT> fis load -l vmlinux.bin.l7 Image loaded from 0x80041000-0x802c8200 DD-WRT> exec Now booting linux kernel: Base address 0x80030000 Entry 0x80041000 Cmdline : Linux version 2.6.30.10 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #2 Sun Jan 24 04:12:11 CET 2010 CPU revision is: 00019064 (MIPS 4KEc) Determined physical RAM map: memory: 01000000 @ 00000000 (usable) Initrd not found or empty - disabling initrd Zone PFN ranges: Normal 0x00000000 -> 0x00001000 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x00000000 -> 0x00001000 Built 1 zonelists in Zone order, mobility grouping off. Total pages: 4064 Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2 Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes. Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes NR_IRQS:128 PID hash table entries: 64 (order: 6, 256 bytes) console [ttyS0] enabled Dentry cache hash table entries: 2048 (order: 1, 8192 bytes) Inode-cache hash table entries: 1024 (order: 0, 4096 bytes) Memory: 13300k/16384k available (2002k kernel code, 3084k reserved, 458k data, 128k init, 0k highmem) Calibrating delay loop... 183.50 BogoMIPS (lpj=917504) Mount-cache hash table entries: 512 net_namespace: 732 bytes NET: Registered protocol family 16 bio: create slab <bio-0> at 0 NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 512 (order: 0, 4096 bytes) TCP bind hash table entries: 512 (order: -1, 2048 bytes) TCP: Hash tables configured (established 512 bind 512) TCP reno registered NET: Registered protocol family 1 Radio config found at offset 0xf8(0x1f8) squashfs: version 4.0 (2009/01/31) Phillip Lougher Registering mini_fo version $Id$ JFFS2 version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc. msgmni has been set to 25 io scheduler noop registered io scheduler deadline registered (default) gpiodev: gpio device registered with major 254 gpiodev: gpio platform device registered with access mask FFFFFFFF Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A eth0: Atheros AR231x: 00:22:b0:98:87:de, irq 4 ar231x_eth_mii: probed eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00) cmdlinepart partition parsing not available Searching for RedBoot partition table in spiflash at offset 0x3d0000 Searching for RedBoot partition table in spiflash at offset 0x3e0000 6 RedBoot partitions found on MTD device spiflash Creating 6 MTD partitions on "spiflash": 0x000000000000-0x000000030000 : "RedBoot" 0x000000030000-0x0000002f0000 : "rootfs" mtd: partition "rootfs" set to be root filesystem mtd: partition "rootfs_data" created automatically, ofs=230000, len=C0000 0x000000230000-0x0000002f0000 : "rootfs_data" 0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7" 0x0000003e0000-0x0000003ef000 : "FIS directory" 0x0000003ef000-0x0000003f0000 : "RedBoot config" 0x0000003f0000-0x000000400000 : "boardconfig" TCP westwood registered NET: Registered protocol family 17 Bridge firewalling registered 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFS: Mounted root (squashfs filesystem) readonly on device 31:1. Freeing unused kernel memory: 128k freed Please be patient, while OpenWrt loads ... - preinit - Press CTRL-C for failsafe jffs2 not ready yet; using ramdisk mini_fo: using base directory: / mini_fo: using storage directory: /tmp/root - init - Please press Enter to activate this console. NET: Registered protocol family 10 lo: Disabled Privacy Extensions tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> device eth0.1 entered promiscuous mode device eth0 entered promiscuous mode br-mesh: port 1(eth0.1) entering forwarding state ip_tables: (C) 2000-2006 Netfilter Core Team Ebtables v2.0 registered ip6_tables: (C) 2000-2006 Netfilter Core Team device eth0.4 entered promiscuous mode br-wan_vpn: port 1(eth0.4) entering forwarding state br-wan_vpn: starting userspace STP failed, starting kernel STP batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded ath_hal: module license 'Proprietary' taints kernel. Disabling lock debugging due to kernel taint ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR) ath_ahb: trunk wlan: trunk wlan: mac acl policy registered ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk) ath_rate_minstrel: look around rate set to 10% ath_rate_minstrel: EWMA rolloff level set to 75% ath_rate_minstrel: max segment size in the mrr set to 6000 us Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: H/W encryption support: WEP AES AES_CCM TKIP ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3 IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs device bat0 entered promiscuous mode br-mesh: port 2(bat0) entering forwarding state br-wan_vpn: port 1(eth0.4) entering disabled state br-wan_vpn: topology change detected, propagating br-wan_vpn: port 1(eth0.4) entering forwarding state br-mesh: port 2(bat0) entering disabled state br-mesh: port 1(eth0.1) entering disabled state br-mesh: port 2(bat0) entering forwarding state br-mesh: port 1(eth0.1) entering forwarding state batman-adv:Adding interface: ath0 batman-adv:Interface activated: ath0 CPU 0 Unable to handle kernel paging request at virtual address 00000010, epc == 80be3584, ra == 80be34e4 Oops[#1]: Cpu 0 $ 0 : 00000000 10009c00 00000010 80afd800 $ 4 : 00000000 00000000 00000000 80be85f8 $ 8 : 0000611b 80afd68e 00000010 00000000 $12 : 00000000 feced300 ffffffff 61636520 $16 : 8064ca40 8064ca00 80afd680 8064ca40 $20 : 8064ca00 00000000 00000001 80640060 $24 : 00000010 8069c74c $28 : 8046c000 8046d9a8 802a0000 80be34e4 Hi : 00000000 Lo : 0000000b epc : 80be3584 receive_bat_packet+0x3d8/0x6ec [batman_adv] Tainted: P ra : 80be34e4 receive_bat_packet+0x338/0x6ec [batman_adv] Status: 10009c02 KERNEL EXL Cause : 10800008 BadVA : 00000010 PrId : 00019064 (MIPS 4KEc) Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 Process batctl (pid: 1140, threadinfo=8046c000, task=8052bad0, tls=00000000) Stack : 00000000 806a2aac 10009c01 80303620 802a0000 80f38a90 8064006c 00000001 80640058 00000000 00000002 80640052 80640074 0000000c 80afd800 00000000 80640060 00000000 80640060 00000040 80640052 80afd800 00004305 00000000 80640040 80be8094 80f38a90 8005cd6c 00000001 80c27798 80afd800 80c27798 80303620 80c176b8 80640052 80afd800 10009c01 80547000 80547000 80be2ee4 ... Call Trace: [<80be3584>] receive_bat_packet+0x3d8/0x6ec [batman_adv] [<80be8094>] receive_aggr_bat_packet+0x7c/0xbc [batman_adv] [<80be2ee4>] recv_bat_packet+0x94/0x24c [batman_adv] [<80be79c8>] batman_skb_recv+0x128/0x1dc [batman_adv] [<806a1494>] ieee80211_saveath+0xb24/0xb80 [ath_ahb] Code: 9245000e 84640008 00441021 <90440000> 00a4102b 00a2200b 10800003 00000000 14a00003 Kernel panic - not syncing: Fatal exception in interrupt ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing 2010-01-23 18:10 ` Andrew Lunn 2010-01-24 4:24 ` Linus Lüssing @ 2010-01-26 6:13 ` Linus Lüssing 2010-01-26 7:16 ` Marek Lindner 2010-01-29 8:25 ` Andrew Lunn 2 siblings, 2 replies; 37+ messages in thread From: Linus Lüssing @ 2010-01-26 6:13 UTC (permalink / raw) To: b.a.t.m.a.n [-- Attachment #1: Type: text/plain, Size: 350 bytes --] I'm now able to reproduce the second problem as well: Activating the vis-server on one node causes the vis-clients to throw this call trace with its slow path warning imediately. Nevertheless, vis output works anyway, clients are sending vis-packets and the server is drawing nice graphs. See the attachment for an example call trace. Cheers, Linus [-- Attachment #2: slowpath-warning.log --] [-- Type: text/plain, Size: 7966 bytes --] Linux version 2.6.30.10 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #3 Tue Jan 26 02:18:56 CET 2010 CPU revision is: 00019064 (MIPS 4KEc) Determined physical RAM map: memory: 01000000 @ 00000000 (usable) Initrd not found or empty - disabling initrd Zone PFN ranges: Normal 0x00000000 -> 0x00001000 Movable zone start PFN for each node early_node_map[1] active PFN ranges 0: 0x00000000 -> 0x00001000 On node 0 totalpages: 4096 free_area_init_node: node 0, pgdat 802a0f40, node_mem_map 802d9000 Normal zone: 32 pages used for memmap Normal zone: 0 pages reserved Normal zone: 4064 pages, LIFO batch:0 Built 1 zonelists in Zone order, mobility grouping off. Total pages: 4064 Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2 Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes. Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes NR_IRQS:128 PID hash table entries: 64 (order: 6, 256 bytes) console [ttyS0] enabled Dentry cache hash table entries: 2048 (order: 1, 8192 bytes) Inode-cache hash table entries: 1024 (order: 0, 4096 bytes) Memory: 13324k/16384k available (1985k kernel code, 3060k reserved, 452k data, 128k init, 0k highmem) Calibrating delay loop... 183.50 BogoMIPS (lpj=917504) Mount-cache hash table entries: 512 net_namespace: 732 bytes NET: Registered protocol family 16 bio: create slab <bio-0> at 0 Switched to high resolution mode on CPU 0 NET: Registered protocol family 2 IP route cache hash table entries: 1024 (order: 0, 4096 bytes) TCP established hash table entries: 512 (order: 0, 4096 bytes) TCP bind hash table entries: 512 (order: -1, 2048 bytes) TCP: Hash tables configured (established 512 bind 512) TCP reno registered NET: Registered protocol family 1 Radio config found at offset 0xf8(0x1f8) squashfs: version 4.0 (2009/01/31) Phillip Lougher Registering mini_fo version $Id$ JFFS2 version 2.2. (NAND) (SUMMARY) © 2001-2006 Red Hat, Inc. msgmni has been set to 26 io scheduler noop registered io scheduler deadline registered (default) gpiodev: gpio device registered with major 254 gpiodev: gpio platform device registered with access mask FFFFFFFF Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A eth0: Atheros AR231x: 00:22:b0:98:87:fa, irq 4 ar231x_eth_mii: probed eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00) cmdlinepart partition parsing not available Searching for RedBoot partition table in spiflash at offset 0x3d0000 Searching for RedBoot partition table in spiflash at offset 0x3e0000 6 RedBoot partitions found on MTD device spiflash Creating 6 MTD partitions on "spiflash": 0x000000000000-0x000000030000 : "RedBoot" 0x000000030000-0x0000002f0000 : "rootfs" mtd: partition "rootfs" set to be root filesystem mtd: partition "rootfs_data" created automatically, ofs=230000, len=C0000 0x000000230000-0x0000002f0000 : "rootfs_data" 0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7" 0x0000003e0000-0x0000003ef000 : "FIS directory" 0x0000003ef000-0x0000003f0000 : "RedBoot config" 0x0000003f0000-0x000000400000 : "boardconfig" TCP westwood registered NET: Registered protocol family 17 Bridge firewalling registered 802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com> All bugs added by David S. Miller <davem@redhat.com> VFS: Mounted root (squashfs filesystem) readonly on device 31:1. Freeing unused kernel memory: 128k freed Please be patient, while OpenWrt loads ... mini_fo: using base directory: / mini_fo: using storage directory: /tmp/root NET: Registered protocol family 10 lo: Disabled Privacy Extensions tun: Universal TUN/TAP device driver, 1.6 tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com> ip_tables: (C) 2000-2006 Netfilter Core Team device eth0.1 entered promiscuous mode device eth0 entered promiscuous mode br-mesh: port 1(eth0.1) entering forwarding state Ebtables v2.0 registered ip6_tables: (C) 2000-2006 Netfilter Core Team batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1564 (compatibility version 8) loaded ath_hal: module license 'Proprietary' taints kernel. Disabling lock debugging due to kernel taint ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR) device eth0.4 entered promiscuous mode br-wan_vpn: port 1(eth0.4) entering forwarding state br-wan_vpn: starting userspace STP failed, starting kernel STP eth0.1: no IPv6 routers present br-mesh: no IPv6 routers present ath_ahb: trunk wlan: trunk wlan: mac acl policy registered ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk) ath_rate_minstrel: look around rate set to 10% ath_rate_minstrel: EWMA rolloff level set to 75% ath_rate_minstrel: max segment size in the mrr set to 6000 us Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps wifi0: H/W encryption support: WEP AES AES_CCM TKIP ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3 IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs eth0.2: no IPv6 routers present device bat0 entered promiscuous mode br-mesh: port 2(bat0) entering forwarding state eth0.3: no IPv6 routers present eth0.4: no IPv6 routers present bat0: no IPv6 routers present br-wan_vpn: port 1(eth0.4) entering disabled state br-wan_vpn: topology change detected, propagating br-wan_vpn: port 1(eth0.4) entering forwarding state br-mesh: port 2(bat0) entering disabled state br-mesh: port 1(eth0.1) entering disabled state br-mesh: port 2(bat0) entering forwarding state br-mesh: port 1(eth0.1) entering forwarding state br-wan_vpn: received tcn bpdu on port 1(eth0.4) br-wan_vpn: topology change detected, propagating ath0: no IPv6 routers present br-mesh: no IPv6 routers present batman-adv:Adding interface: ath0 batman-adv:Interface activated: ath0 batman-adv:Adding interface: eth0.2 batman-adv:Interface activated: eth0.2 batman-adv:Adding interface: eth0.3 batman-adv:Interface activated: eth0.3 jffs2_scan_eraseblock(): End of filesystem marker found at 0x0 jffs2_build_filesystem(): unlocking the mtd device... done. jffs2_build_filesystem(): erasing all blocks after the end marker... <6>device VPN entered promiscuous mode br-wan_vpn: topology change detected, propagating br-wan_vpn: port 2(VPN) entering forwarding state done. mini_fo: using base directory: / mini_fo: using storage directory: /jffs br-wan_vpn: received tcn bpdu on port 1(eth0.4) br-wan_vpn: topology change detected, propagating br-wan_vpn: received tcn bpdu on port 1(eth0.4) br-wan_vpn: topology change detected, propagating br-wan_vpn: received tcn bpdu on port 1(eth0.4) br-wan_vpn: topology change detected, propagating ------------[ cut here ]------------ WARNING: at kernel/softirq.c:141 local_bh_enable+0x48/0xa0() Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 Call Trace: [<800493a4>] dump_stack+0x8/0x34 [<80060cd8>] warn_slowpath_common+0x70/0xb0 [<800674ac>] local_bh_enable+0x48/0xa0 [<801b3ef8>] dev_queue_xmit+0x388/0x3dc [<8062c454>] ieee80211_parent_queue_xmit+0x8c/0xb4 [ath_ahb] [<8062c778>] ieee80211_hardstart+0x2fc/0x34c [ath_ahb] [<801b3e5c>] dev_queue_xmit+0x2ec/0x3dc [<80bb68d8>] vis_quit+0x7c4/0x950 [batman_adv] ---[ end trace e96b16c908cf7c40 ]--- br-wan_vpn: received tcn bpdu on port 1(eth0.4) br-wan_vpn: topology change detected, propagating root@OpenWrt:~# ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-26 6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing @ 2010-01-26 7:16 ` Marek Lindner 2010-01-27 0:10 ` Linus Lüssing 2010-01-29 8:25 ` Andrew Lunn 1 sibling, 1 reply; 37+ messages in thread From: Marek Lindner @ 2010-01-26 7:16 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: Text/Plain, Size: 484 bytes --] On Tuesday 26 January 2010 14:13:11 Linus Lüssing wrote: > I'm now able to reproduce the second problem as well: > Activating the vis-server on one node causes the vis-clients to > throw this call trace with its slow path warning imediately. > Nevertheless, vis output works anyway, clients are sending > vis-packets and the server is drawing nice graphs. > See the attachment for an example call trace. Please try the attached patch and see if it helps. Regards, Marek [-- Attachment #2: no_lock_while_send.patch --] [-- Type: text/x-patch, Size: 3998 bytes --] diff --git a/batman-adv-kernelland/vis.c b/batman-adv-kernelland/vis.c index b118d1e..a6c235f 100644 --- a/batman-adv-kernelland/vis.c +++ b/batman-adv-kernelland/vis.c @@ -405,6 +405,9 @@ static void purge_vis_packets(void) { HASHIT(hashit); struct vis_info *info; + unsigned long flags; + + spin_lock_irqsave(&vis_hash_lock, flags); while (hash_iterate(vis_hash, &hashit)) { info = hashit.bucket->data; @@ -416,13 +419,17 @@ static void purge_vis_packets(void) free_info(info); } } + + spin_unlock_irqrestore(&vis_hash_lock, flags); } static void broadcast_vis_packet(struct vis_info *info, int packet_length) { HASHIT(hashit); struct orig_node *orig_node; + struct batman_if *batman_if; unsigned long flags; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); @@ -431,45 +438,60 @@ static void broadcast_vis_packet(struct vis_info *info, int packet_length) orig_node = hashit.bucket->data; /* if it's a vis server and reachable, send it. */ - if (orig_node && - (orig_node->flags & VIS_SERVER) && - orig_node->batman_if && - orig_node->router) { + if ((!orig_node) || (!orig_node->batman_if) || + (!orig_node->router)) + continue; - /* don't send it if we already received the packet from - * this node. */ - if (recv_list_is_in(&info->recv_list, orig_node->orig)) - continue; + if (!(orig_node->flags & VIS_SERVER)) + continue; - memcpy(info->packet.target_orig, - orig_node->orig, ETH_ALEN); + /* don't send it if we already received the packet from + * this node. */ + if (recv_list_is_in(&info->recv_list, orig_node->orig)) + continue; - send_raw_packet((unsigned char *) &info->packet, - packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + spin_unlock_irqrestore(&orig_hash_lock, flags); + + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); + + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); + + spin_lock_irqsave(&orig_hash_lock, flags); } - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); + spin_unlock_irqrestore(&orig_hash_lock, flags); + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); } static void unicast_vis_packet(struct vis_info *info, int packet_length) { struct orig_node *orig_node; + struct batman_if *batman_if; unsigned long flags; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); orig_node = ((struct orig_node *) hash_find(orig_hash, info->packet.target_orig)); - if ((orig_node != NULL) && - (orig_node->batman_if != NULL) && - (orig_node->router != NULL)) { - send_raw_packet((unsigned char *) &info->packet, packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) + goto out; + + /* don't lock while sending the packets ... we therefore + * copy the required data before sending */ + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); + spin_unlock_irqrestore(&orig_hash_lock, flags); + + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); + + return; + +out: spin_unlock_irqrestore(&orig_hash_lock, flags); } @@ -502,17 +524,18 @@ static void send_vis_packets(struct work_struct *work) struct vis_info *info, *temp; unsigned long flags; - spin_lock_irqsave(&vis_hash_lock, flags); purge_vis_packets(); if (generate_vis_packet() == 0) /* schedule if generation was successful */ list_add_tail(&my_vis_info->send_list, &send_list); + spin_lock_irqsave(&vis_hash_lock, flags); list_for_each_entry_safe(info, temp, &send_list, send_list) { list_del_init(&info->send_list); send_vis_packet(info); } + spin_unlock_irqrestore(&vis_hash_lock, flags); start_vis_timer(); } ^ permalink raw reply related [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-26 7:16 ` Marek Lindner @ 2010-01-27 0:10 ` Linus Lüssing 2010-01-28 0:09 ` Marek Lindner 0 siblings, 1 reply; 37+ messages in thread From: Linus Lüssing @ 2010-01-27 0:10 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 5083 bytes --] Hi Marek, nope, does not seem to work, still the same issue I also tried it on my laptop here with the patch installed and it is a very similar call trace, although here it is send_vis_packets instead of vis_quit in the call trace. Cheers, Linus On Tue, Jan 26, 2010 at 03:16:38PM +0800, Marek Lindner wrote: > On Tuesday 26 January 2010 14:13:11 Linus Lüssing wrote: > > I'm now able to reproduce the second problem as well: > > Activating the vis-server on one node causes the vis-clients to > > throw this call trace with its slow path warning imediately. > > Nevertheless, vis output works anyway, clients are sending > > vis-packets and the server is drawing nice graphs. > > See the attachment for an example call trace. > > Please try the attached patch and see if it helps. > > Regards, > Marek > diff --git a/batman-adv-kernelland/vis.c b/batman-adv-kernelland/vis.c > index b118d1e..a6c235f 100644 > --- a/batman-adv-kernelland/vis.c > +++ b/batman-adv-kernelland/vis.c > @@ -405,6 +405,9 @@ static void purge_vis_packets(void) > { > HASHIT(hashit); > struct vis_info *info; > + unsigned long flags; > + > + spin_lock_irqsave(&vis_hash_lock, flags); > > while (hash_iterate(vis_hash, &hashit)) { > info = hashit.bucket->data; > @@ -416,13 +419,17 @@ static void purge_vis_packets(void) > free_info(info); > } > } > + > + spin_unlock_irqrestore(&vis_hash_lock, flags); > } > > static void broadcast_vis_packet(struct vis_info *info, int packet_length) > { > HASHIT(hashit); > struct orig_node *orig_node; > + struct batman_if *batman_if; > unsigned long flags; > + uint8_t dstaddr[ETH_ALEN]; > > spin_lock_irqsave(&orig_hash_lock, flags); > > @@ -431,45 +438,60 @@ static void broadcast_vis_packet(struct vis_info *info, int packet_length) > orig_node = hashit.bucket->data; > > /* if it's a vis server and reachable, send it. */ > - if (orig_node && > - (orig_node->flags & VIS_SERVER) && > - orig_node->batman_if && > - orig_node->router) { > + if ((!orig_node) || (!orig_node->batman_if) || > + (!orig_node->router)) > + continue; > > - /* don't send it if we already received the packet from > - * this node. */ > - if (recv_list_is_in(&info->recv_list, orig_node->orig)) > - continue; > + if (!(orig_node->flags & VIS_SERVER)) > + continue; > > - memcpy(info->packet.target_orig, > - orig_node->orig, ETH_ALEN); > + /* don't send it if we already received the packet from > + * this node. */ > + if (recv_list_is_in(&info->recv_list, orig_node->orig)) > + continue; > > - send_raw_packet((unsigned char *) &info->packet, > - packet_length, > - orig_node->batman_if, > - orig_node->router->addr); > - } > + spin_unlock_irqrestore(&orig_hash_lock, flags); > + > + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); > + batman_if = orig_node->batman_if; > + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); > + > + send_raw_packet((unsigned char *)&info->packet, > + packet_length, batman_if, dstaddr); > + > + spin_lock_irqsave(&orig_hash_lock, flags); > } > - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); > + > spin_unlock_irqrestore(&orig_hash_lock, flags); > + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); > } > > static void unicast_vis_packet(struct vis_info *info, int packet_length) > { > struct orig_node *orig_node; > + struct batman_if *batman_if; > unsigned long flags; > + uint8_t dstaddr[ETH_ALEN]; > > spin_lock_irqsave(&orig_hash_lock, flags); > orig_node = ((struct orig_node *) > hash_find(orig_hash, info->packet.target_orig)); > > - if ((orig_node != NULL) && > - (orig_node->batman_if != NULL) && > - (orig_node->router != NULL)) { > - send_raw_packet((unsigned char *) &info->packet, packet_length, > - orig_node->batman_if, > - orig_node->router->addr); > - } > + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) > + goto out; > + > + /* don't lock while sending the packets ... we therefore > + * copy the required data before sending */ > + batman_if = orig_node->batman_if; > + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); > + spin_unlock_irqrestore(&orig_hash_lock, flags); > + > + send_raw_packet((unsigned char *)&info->packet, > + packet_length, batman_if, dstaddr); > + > + return; > + > +out: > spin_unlock_irqrestore(&orig_hash_lock, flags); > } > > @@ -502,17 +524,18 @@ static void send_vis_packets(struct work_struct *work) > struct vis_info *info, *temp; > unsigned long flags; > > - spin_lock_irqsave(&vis_hash_lock, flags); > purge_vis_packets(); > > if (generate_vis_packet() == 0) > /* schedule if generation was successful */ > list_add_tail(&my_vis_info->send_list, &send_list); > > + spin_lock_irqsave(&vis_hash_lock, flags); > list_for_each_entry_safe(info, temp, &send_list, send_list) { > list_del_init(&info->send_list); > send_vis_packet(info); > } > + > spin_unlock_irqrestore(&vis_hash_lock, flags); > start_vis_timer(); > } [-- Attachment #2: call-trace-0104-27-01-10.log --] [-- Type: text/plain, Size: 2770 bytes --] [22933.613178] ------------[ cut here ]------------ [22933.613199] WARNING: at /tmp/buildd/linux-2.6-2.6.31/debian/build/source_i386_none/kernel/softirq.c:143 _local_bh_enable_ip+0x41/0x97() [22933.613206] Hardware name: ��������������� [22933.613211] Modules linked in: batman_adv isofs udf nvidia(P) ppdev lp parport sco bridge stp bnep rfcomm kvm_intel kvm acpi_cpufreq cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_stats l2cap nfsd exportfs nfs lockd fscache nfs_acl auth_rpcgss sunrpc fuse dm_snapshot dm_mirror dm_region_hash dm_log firewire_sbp2 loop snd_hda_codec_realtek snd_hda_intel snd_hda_codec arc4 ecb acer_wmi iwlagn snd_hwdep iwlcore snd_pcm_oss snd_mixer_oss snd_pcm mac80211 snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device uvcvideo videodev snd soundcore i2c_i801 cp210x v4l1_compat btusb snd_page_alloc cfg80211 v4l2_compat_ioctl32 psmouse bluetooth wmi serio_raw rfkill i2c_core usbserial evdev pcspkr ac processor button battery ext3 jbd mbcache sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod sd_mod crc_t10dif ide_cd_mod cdrom ata_generic ide_pci_generic ahci sdhci_pci sdhci uhci_hcd mmc_core libata tg3 led_class libphy firewire_ohci firewire_core crc_itu_t ricoh_mmc video output piix ide_core scsi_mod ehci_hcd intel_agp thermal fan thermal_sys [last unloaded: batman_adv] [22933.613436] Pid: 14882, comm: bat_events Tainted: P 2.6.31-1-amd64 #1 [22933.613442] Call Trace: [22933.613453] [<ffffffff810503fa>] ? _local_bh_enable_ip+0x41/0x97 [22933.613462] [<ffffffff810503fa>] ? _local_bh_enable_ip+0x41/0x97 [22933.613471] [<ffffffff8104af08>] ? warn_slowpath_common+0x77/0xa3 [22933.613480] [<ffffffff810503fa>] ? _local_bh_enable_ip+0x41/0x97 [22933.613490] [<ffffffff812c821f>] ? run_filter+0x33/0x3a [22933.613498] [<ffffffff812c8c56>] ? packet_rcv+0xa9/0x2fb [22933.613511] [<ffffffff812389e5>] ? dev_hard_start_xmit+0x143/0x2ba [22933.613521] [<ffffffff8124ac7e>] ? __qdisc_run+0xe2/0x1ec [22933.613529] [<ffffffff81238e8f>] ? dev_queue_xmit+0x232/0x32c [22933.613551] [<ffffffffa0fe8945>] ? send_vis_packets+0x4f5/0x5d5 [batman_adv] [22933.613571] [<ffffffffa0fe8450>] ? send_vis_packets+0x0/0x5d5 [batman_adv] [22933.613581] [<ffffffff8105b43b>] ? worker_thread+0x174/0x211 [22933.613590] [<ffffffff8105f39e>] ? autoremove_wake_function+0x0/0x2e [22933.613599] [<ffffffff8105b2c7>] ? worker_thread+0x0/0x211 [22933.613606] [<ffffffff8105f042>] ? kthread+0x8b/0x93 [22933.613616] [<ffffffff81011baa>] ? child_rip+0xa/0x20 [22933.613623] [<ffffffff8105efb7>] ? kthread+0x0/0x93 [22933.613631] [<ffffffff81011ba0>] ? child_rip+0x0/0x20 [22933.613636] ---[ end trace 1758b3e01bf5c445 ]--- ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-27 0:10 ` Linus Lüssing @ 2010-01-28 0:09 ` Marek Lindner 2010-01-28 6:29 ` Andrew Lunn 0 siblings, 1 reply; 37+ messages in thread From: Marek Lindner @ 2010-01-28 0:09 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Hi, > nope, does not seem to work, still the same issue > I also tried it on my laptop here with the patch installed > and it is a very similar call trace, although here it is > send_vis_packets instead of vis_quit in the call trace. you should take the stack trace with a grain of salt. Stack tracing such an issue is a quite tricky thing, hence can not be fully trusted. Since the skb changes the code runs in interrupt context which probably introduced this bug. All previous warnings of this kind were related to holding a lock while sending packets. My patch unlocked the problematic orig_hash - maybe that was not enough ? By adding retrun statements at the beginning of the vis send function you might be able to get down to the problem. Regards, Marek ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-28 0:09 ` Marek Lindner @ 2010-01-28 6:29 ` Andrew Lunn 0 siblings, 0 replies; 37+ messages in thread From: Andrew Lunn @ 2010-01-28 6:29 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Thu, Jan 28, 2010 at 08:09:57AM +0800, Marek Lindner wrote: > > Hi, > > > nope, does not seem to work, still the same issue > > I also tried it on my laptop here with the patch installed > > and it is a very similar call trace, although here it is > > send_vis_packets instead of vis_quit in the call trace. > > you should take the stack trace with a grain of salt. Stack tracing such an > issue is a quite tricky thing, hence can not be fully trusted. Since the skb > changes the code runs in interrupt context which probably introduced this bug. > All previous warnings of this kind were related to holding a lock while > sending packets. My patch unlocked the problematic orig_hash - maybe that was > not enough ? By adding retrun statements at the beginning of the vis send > function you might be able to get down to the problem. It might also be worth running lockdep on the code. Normally you don^t need the actually lockup, you just need to execute the code path that would lockup under whatever conditions are required for it to lockup. Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-26 6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing 2010-01-26 7:16 ` Marek Lindner @ 2010-01-29 8:25 ` Andrew Lunn 2010-01-29 8:59 ` Marek Lindner 1 sibling, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-01-29 8:25 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking > WARNING: at kernel/softirq.c:141 local_bh_enable+0x48/0xa0() > Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 > Call Trace: > [<800493a4>] dump_stack+0x8/0x34 > [<80060cd8>] warn_slowpath_common+0x70/0xb0 > [<800674ac>] local_bh_enable+0x48/0xa0 > [<801b3ef8>] dev_queue_xmit+0x388/0x3dc > [<8062c454>] ieee80211_parent_queue_xmit+0x8c/0xb4 [ath_ahb] > [<8062c778>] ieee80211_hardstart+0x2fc/0x34c [ath_ahb] > [<801b3e5c>] dev_queue_xmit+0x2ec/0x3dc > [<80bb68d8>] vis_quit+0x7c4/0x950 [batman_adv] > > ---[ end trace e96b16c908cf7c40 ]--- > br-wan_vpn: received tcn bpdu on port 1(eth0.4) > br-wan_vpn: topology change detected, propagating > root@OpenWrt:~# I dug a little deeper into what his means. It means we are in interrupt context and interrupts are disabled. This seems to be not allowed when calling local_bh_enable(), which probably means it is not allowed when calling dev_queue_xmit. What i don't understand yet is the call path. The Call Trace above does not make much sense. vis_quit() does not send any packets. Looking at Marek's patch it is going in the right direction. send_skb_packet() cannot be called with interrupts disabled. which means send_raw_packet() cannot be called with interrupts disabled. which means broadcast_vis_packet() needs to spin_unlock_irqrestore() before sending. which means unicast_vis_packet() needs to spin_unlock_irqrestore() before sending. This is what Mareks patch does. However, this part of Marek's patch looks wrong: struct vis_info *info, *temp; unsigned long flags; - spin_lock_irqsave(&vis_hash_lock, flags); purge_vis_packets(); if (generate_vis_packet() == 0) /* schedule if generation was successful */ list_add_tail(&my_vis_info->send_list, &send_list); + spin_lock_irqsave(&vis_hash_lock, flags); list_for_each_entry_safe(info, temp, &send_list, send_list) { list_del_init(&info->send_list); send_vis_packet(info); } + spin_unlock_irqrestore(&vis_hash_lock, flags); start_vis_timer(); } You are disable interrupts and then call send_vis_packet(). This will not work since all calls under send_vis_packet will have interrupts disabled. Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-29 8:25 ` Andrew Lunn @ 2010-01-29 8:59 ` Marek Lindner 2010-01-30 16:50 ` Andrew Lunn 0 siblings, 1 reply; 37+ messages in thread From: Marek Lindner @ 2010-01-29 8:59 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Friday 29 January 2010 16:25:45 Andrew Lunn wrote: > This is what Mareks patch does. However, this part of Marek's patch looks > wrong: I'd like to point out that this lock wasn't introduced by my previously posted patch. I managed to reduce the effect of this lock but you can't remove it entirely without changing the whole sending mechanism. > You are disable interrupts and then call send_vis_packet(). This will > not work since all calls under send_vis_packet will have interrupts > disabled. Yes, that is correct. The reason for this is simple: The interrupt flags are stored in send_vis_packets() and without passing them on we can't re-enable the interrupts. That did not appear to be a good solution. ;) As far as I understand the vis_lock's only purpose is to avoid that the "info" pointer which is passed to send_vis_packet() can't be deleted while we use it. That might be better solved using reference counting. Cheers, Marek ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-29 8:59 ` Marek Lindner @ 2010-01-30 16:50 ` Andrew Lunn 2010-01-31 19:37 ` Linus Lüssing 2010-02-11 9:46 ` Andrew Lunn 0 siblings, 2 replies; 37+ messages in thread From: Andrew Lunn @ 2010-01-30 16:50 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Hi Linus Please could you try this patch and see if it fixes the vis problem. It compiles cleanly with 2.6.32. It is checkpatch clean, it is sparse clean. However, since i've not actually tried running the code i would not be too surprised if it deadlocked, leaked memory, oopsed, ... Andrew Staging: batman-adv: Don't have interrupts disabled while sending. send_vis_packets() would disable interrupts before calling dev_queue_xmit() which resulting in a backtrace in local_bh_enable(). Fix this by using kref on the vis_info object so that we can call send_vis_packets() without holding vis_hash_lock. vis_hash_lock also used to protect recv_list, so we now need a new lock to protect that instead of vis_hash_lock. Also a few checkpatch cleanups. Reported-by: Linus Lüssing <linus.luessing@web.de> Signed-off-by: Andrew Lunn <andrew@lunn.ch> Index: vis.c =================================================================== --- vis.c (revision 1568) +++ vis.c (working copy) @@ -30,22 +30,26 @@ struct hashtable_t *vis_hash; DEFINE_SPINLOCK(vis_hash_lock); +static DEFINE_SPINLOCK(recv_list_lock); static struct vis_info *my_vis_info; static struct list_head send_list; /* always locked with vis_hash_lock */ static void start_vis_timer(void); /* free the info */ -static void free_info(void *data) +static void free_info(struct kref *ref) { - struct vis_info *info = data; + struct vis_info *info = container_of(ref, struct vis_info, refcount); struct recvlist_node *entry, *tmp; + unsigned long flags; list_del_init(&info->send_list); + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry_safe(entry, tmp, &info->recv_list, list) { list_del(&entry->list); kfree(entry); } + spin_unlock_irqrestore(&recv_list_lock, flags); kfree(info); } @@ -147,32 +151,41 @@ static void recv_list_add(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC); if (!entry) return; memcpy(entry->mac, mac, ETH_ALEN); + spin_lock_irqsave(&recv_list_lock, flags); list_add_tail(&entry->list, recv_list); + spin_unlock_irqrestore(&recv_list_lock, flags); } /* returns 1 if this mac is in the recv_list */ static int recv_list_is_in(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry(entry, recv_list, list) { - if (memcmp(entry->mac, mac, ETH_ALEN) == 0) + if (memcmp(entry->mac, mac, ETH_ALEN) == 0) { + spin_unlock_irqrestore(&recv_list_lock, flags); return 1; + } } - + spin_unlock_irqrestore(&recv_list_lock, flags); return 0; } /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old, - * broken.. ). vis hash must be locked outside. is_new is set when the packet + * broken.. ). vis hash must be locked outside. is_new is set when the packet * is newer than old entries in the hash. */ static struct vis_info *add_packet(struct vis_packet *vis_packet, - int vis_info_len, int *is_new) + int vis_info_len, int *is_new, + int make_broadcast) { struct vis_info *info, *old_info; struct vis_info search_elem; @@ -199,7 +212,7 @@ } /* remove old entry */ hash_remove(vis_hash, old_info); - free_info(old_info); + kref_put(&old_info->refcount, free_info); } info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC); @@ -208,6 +221,7 @@ INIT_LIST_HEAD(&info->send_list); INIT_LIST_HEAD(&info->recv_list); + kref_init(&info->refcount); info->first_seen = jiffies; memcpy(&info->packet, vis_packet, sizeof(struct vis_packet) + vis_info_len); @@ -215,16 +229,21 @@ /* initialize and add new packet. */ *is_new = 1; + /* Make it a broadcast packet, if required */ + if (make_broadcast) + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); + /* repair if entries is longer than packet. */ if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len) - info->packet.entries = vis_info_len / sizeof(struct vis_info_entry); + info->packet.entries = vis_info_len / + sizeof(struct vis_info_entry); recv_list_add(&info->recv_list, info->packet.sender_orig); /* try to add it */ if (hash_add(vis_hash, info) < 0) { /* did not work (for some reason) */ - free_info(info); + kref_put(&old_info->refcount, free_info); info = NULL; } @@ -235,19 +254,20 @@ void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len) { struct vis_info *info; - int is_new; + int is_new, make_broadcast; unsigned long flags; int vis_server = atomic_read(&vis_mode); + make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC); + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast); if (info == NULL) goto end; /* only if we are server ourselves and packet is newer than the one in * hash.*/ if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) { - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); if (list_empty(&info->send_list)) list_add_tail(&info->send_list, &send_list); } @@ -263,24 +283,27 @@ int is_new; unsigned long flags; int vis_server = atomic_read(&vis_mode); + int are_target = 0; /* clients shall not broadcast. */ if (is_bcast(vis_packet->target_orig)) return; + /* Are we the target for this VIS packet? */ + if (vis_server == VIS_TYPE_SERVER_SYNC && + is_my_mac(info->packet.target_orig)) + are_target = 1; + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, are_target); if (info == NULL) goto end; /* note that outdated packets will be dropped at this point. */ /* send only if we're the target server or ... */ - if (vis_server == VIS_TYPE_SERVER_SYNC && - is_my_mac(info->packet.target_orig) && - is_new) { + if (are_target && is_new) { info->packet.vis_type = VIS_TYPE_SERVER_SYNC; /* upgrade! */ - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); if (list_empty(&info->send_list)) list_add_tail(&info->send_list, &send_list); @@ -362,14 +385,17 @@ while (hash_iterate(orig_hash, &hashit_global)) { orig_node = hashit_global.bucket->data; if (orig_node->router != NULL - && compare_orig(orig_node->router->addr, orig_node->orig) + && compare_orig(orig_node->router->addr, + orig_node->orig) && orig_node->batman_if && (orig_node->batman_if->if_active == IF_ACTIVE) && orig_node->router->tq_avg > 0) { /* fill one entry into buffer. */ entry = &entry_array[info->packet.entries]; - memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN); + memcpy(entry->src, + orig_node->batman_if->net_dev->dev_addr, + ETH_ALEN); memcpy(entry->dest, orig_node->orig, ETH_ALEN); entry->quality = orig_node->router->tq_avg; info->packet.entries++; @@ -401,6 +427,8 @@ return 0; } +/* free old vis packets. Must be called with this vis_hash_lock + * held */ static void purge_vis_packets(void) { HASHIT(hashit); @@ -413,7 +441,7 @@ if (time_after(jiffies, info->first_seen + (VIS_TIMEOUT*HZ)/1000)) { hash_remove_bucket(vis_hash, &hashit); - free_info(info); + kref_put(&info->refcount, free_info); } } } @@ -423,6 +451,8 @@ HASHIT(hashit); struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); @@ -431,46 +461,57 @@ orig_node = hashit.bucket->data; /* if it's a vis server and reachable, send it. */ - if (orig_node && - (orig_node->flags & VIS_SERVER) && - orig_node->batman_if && - orig_node->router) { + if ((!orig_node) || (!orig_node->batman_if) || + (!orig_node->router)) + continue; + if (!(orig_node->flags & VIS_SERVER)) + continue; + /* don't send it if we already received the packet from + * this node. */ + if (recv_list_is_in(&info->recv_list, orig_node->orig)) + continue; - /* don't send it if we already received the packet from - * this node. */ - if (recv_list_is_in(&info->recv_list, orig_node->orig)) - continue; + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); + spin_unlock_irqrestore(&orig_hash_lock, flags); - memcpy(info->packet.target_orig, - orig_node->orig, ETH_ALEN); + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); - send_raw_packet((unsigned char *) &info->packet, - packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + spin_lock_irqsave(&orig_hash_lock, flags); + } + spin_unlock_irqrestore(&orig_hash_lock, flags); memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - spin_unlock_irqrestore(&orig_hash_lock, flags); } static void unicast_vis_packet(struct vis_info *info, int packet_length) { struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); orig_node = ((struct orig_node *) hash_find(orig_hash, info->packet.target_orig)); - if ((orig_node != NULL) && - (orig_node->batman_if != NULL) && - (orig_node->router != NULL)) { - send_raw_packet((unsigned char *) &info->packet, packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) + goto out; + + /* don't lock while sending the packets ... we therefore + * copy the required data before sending */ + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); spin_unlock_irqrestore(&orig_hash_lock, flags); + + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); + return; + +out: + spin_unlock_irqrestore(&orig_hash_lock, flags); } /* only send one vis packet. called from send_vis_packets() */ @@ -503,6 +544,7 @@ unsigned long flags; spin_lock_irqsave(&vis_hash_lock, flags); + purge_vis_packets(); if (generate_vis_packet() == 0) @@ -511,7 +553,11 @@ list_for_each_entry_safe(info, temp, &send_list, send_list) { list_del_init(&info->send_list); + kref_get(&info->refcount); + spin_unlock_irqrestore(&vis_hash_lock, flags); send_vis_packet(info); + spin_lock_irqsave(&vis_hash_lock, flags); + kref_put(&info->refcount, free_info); } spin_unlock_irqrestore(&vis_hash_lock, flags); start_vis_timer(); @@ -544,6 +590,7 @@ my_vis_info->first_seen = jiffies - atomic_read(&vis_interval); INIT_LIST_HEAD(&my_vis_info->recv_list); INIT_LIST_HEAD(&my_vis_info->send_list); + kref_init(&my_vis_info->refcount); my_vis_info->packet.version = COMPAT_VERSION; my_vis_info->packet.packet_type = BAT_VIS; my_vis_info->packet.ttl = TTL; @@ -557,9 +604,9 @@ if (hash_add(vis_hash, my_vis_info) < 0) { printk(KERN_ERR - "batman-adv:Can't add own vis packet into hash\n"); - free_info(my_vis_info); /* not in hash, need to remove it - * manually. */ + "batman-adv:Can't add own vis packet into hash\n"); + /* not in hash, need to remove it manually. */ + kref_put(&my_vis_info->refcount, free_info); goto err; } @@ -573,6 +620,13 @@ return 0; } +/* Decrease the reference count on a hash item info */ +static void free_info_ref(void *data) +{ + struct vis_info *info = data; + kref_put(&info->refcount, free_info); +} + /* shutdown vis-server */ void vis_quit(void) { @@ -584,7 +638,7 @@ spin_lock_irqsave(&vis_hash_lock, flags); /* properly remove, kill timers ... */ - hash_delete(vis_hash, free_info); + hash_delete(vis_hash, free_info_ref); vis_hash = NULL; my_vis_info = NULL; spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -594,5 +648,5 @@ static void start_vis_timer(void) { queue_delayed_work(bat_event_workqueue, &vis_timer_wq, - (atomic_read(&vis_interval) * HZ ) / 1000); + (atomic_read(&vis_interval) * HZ) / 1000); } Index: vis.h =================================================================== --- vis.h (revision 1568) +++ vis.h (working copy) @@ -29,6 +29,7 @@ /* list of server-neighbors we received a vis-packet * from. we should not reply to them. */ struct list_head send_list; + struct kref refcount; /* this packet might be part of the vis send queue. */ struct vis_packet packet; /* vis_info may follow here*/ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-30 16:50 ` Andrew Lunn @ 2010-01-31 19:37 ` Linus Lüssing 2010-01-31 20:56 ` Andrew Lunn 2010-02-11 9:46 ` Andrew Lunn 1 sibling, 1 reply; 37+ messages in thread From: Linus Lüssing @ 2010-01-31 19:37 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Warning: decoded text below may be mangled, UTF-8 assumed --] [-- Attachment #1.1: Type: text/plain; charset=utf-8, Size: 14179 bytes --] Hi Andrew, Yes, ehm it does indeed cause a kernel panic on the server side immediately :). See the attachment for the full call trace. Cheers, Linus On Sat, Jan 30, 2010 at 05:50:59PM +0100, Andrew Lunn wrote: > Hi Linus > > Please could you try this patch and see if it fixes the vis problem. > It compiles cleanly with 2.6.32. It is checkpatch clean, it is sparse > clean. However, since i've not actually tried running the code i would > not be too surprised if it deadlocked, leaked memory, oopsed, ... > > Andrew > > Staging: batman-adv: Don't have interrupts disabled while sending. > > send_vis_packets() would disable interrupts before calling > dev_queue_xmit() which resulting in a backtrace in local_bh_enable(). > Fix this by using kref on the vis_info object so that we can call > send_vis_packets() without holding vis_hash_lock. vis_hash_lock also > used to protect recv_list, so we now need a new lock to protect that > instead of vis_hash_lock. > > Also a few checkpatch cleanups. > > Reported-by: Linus Lüssing <linus.luessing@web.de> > Signed-off-by: Andrew Lunn <andrew@lunn.ch> > > Index: vis.c > =================================================================== > --- vis.c (revision 1568) > +++ vis.c (working copy) > @@ -30,22 +30,26 @@ > > struct hashtable_t *vis_hash; > DEFINE_SPINLOCK(vis_hash_lock); > +static DEFINE_SPINLOCK(recv_list_lock); > static struct vis_info *my_vis_info; > static struct list_head send_list; /* always locked with vis_hash_lock */ > > static void start_vis_timer(void); > > /* free the info */ > -static void free_info(void *data) > +static void free_info(struct kref *ref) > { > - struct vis_info *info = data; > + struct vis_info *info = container_of(ref, struct vis_info, refcount); > struct recvlist_node *entry, *tmp; > + unsigned long flags; > > list_del_init(&info->send_list); > + spin_lock_irqsave(&recv_list_lock, flags); > list_for_each_entry_safe(entry, tmp, &info->recv_list, list) { > list_del(&entry->list); > kfree(entry); > } > + spin_unlock_irqrestore(&recv_list_lock, flags); > kfree(info); > } > > @@ -147,32 +151,41 @@ > static void recv_list_add(struct list_head *recv_list, char *mac) > { > struct recvlist_node *entry; > + unsigned long flags; > + > entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC); > if (!entry) > return; > > memcpy(entry->mac, mac, ETH_ALEN); > + spin_lock_irqsave(&recv_list_lock, flags); > list_add_tail(&entry->list, recv_list); > + spin_unlock_irqrestore(&recv_list_lock, flags); > } > > /* returns 1 if this mac is in the recv_list */ > static int recv_list_is_in(struct list_head *recv_list, char *mac) > { > struct recvlist_node *entry; > + unsigned long flags; > > + spin_lock_irqsave(&recv_list_lock, flags); > list_for_each_entry(entry, recv_list, list) { > - if (memcmp(entry->mac, mac, ETH_ALEN) == 0) > + if (memcmp(entry->mac, mac, ETH_ALEN) == 0) { > + spin_unlock_irqrestore(&recv_list_lock, flags); > return 1; > + } > } > - > + spin_unlock_irqrestore(&recv_list_lock, flags); > return 0; > } > > /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old, > - * broken.. ). vis hash must be locked outside. is_new is set when the packet > + * broken.. ). vis hash must be locked outside. is_new is set when the packet > * is newer than old entries in the hash. */ > static struct vis_info *add_packet(struct vis_packet *vis_packet, > - int vis_info_len, int *is_new) > + int vis_info_len, int *is_new, > + int make_broadcast) > { > struct vis_info *info, *old_info; > struct vis_info search_elem; > @@ -199,7 +212,7 @@ > } > /* remove old entry */ > hash_remove(vis_hash, old_info); > - free_info(old_info); > + kref_put(&old_info->refcount, free_info); > } > > info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC); > @@ -208,6 +221,7 @@ > > INIT_LIST_HEAD(&info->send_list); > INIT_LIST_HEAD(&info->recv_list); > + kref_init(&info->refcount); > info->first_seen = jiffies; > memcpy(&info->packet, vis_packet, > sizeof(struct vis_packet) + vis_info_len); > @@ -215,16 +229,21 @@ > /* initialize and add new packet. */ > *is_new = 1; > > + /* Make it a broadcast packet, if required */ > + if (make_broadcast) > + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); > + > /* repair if entries is longer than packet. */ > if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len) > - info->packet.entries = vis_info_len / sizeof(struct vis_info_entry); > + info->packet.entries = vis_info_len / > + sizeof(struct vis_info_entry); > > recv_list_add(&info->recv_list, info->packet.sender_orig); > > /* try to add it */ > if (hash_add(vis_hash, info) < 0) { > /* did not work (for some reason) */ > - free_info(info); > + kref_put(&old_info->refcount, free_info); > info = NULL; > } > > @@ -235,19 +254,20 @@ > void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len) > { > struct vis_info *info; > - int is_new; > + int is_new, make_broadcast; > unsigned long flags; > int vis_server = atomic_read(&vis_mode); > > + make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC); > + > spin_lock_irqsave(&vis_hash_lock, flags); > - info = add_packet(vis_packet, vis_info_len, &is_new); > + info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast); > if (info == NULL) > goto end; > > /* only if we are server ourselves and packet is newer than the one in > * hash.*/ > if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) { > - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); > if (list_empty(&info->send_list)) > list_add_tail(&info->send_list, &send_list); > } > @@ -263,24 +283,27 @@ > int is_new; > unsigned long flags; > int vis_server = atomic_read(&vis_mode); > + int are_target = 0; > > /* clients shall not broadcast. */ > if (is_bcast(vis_packet->target_orig)) > return; > > + /* Are we the target for this VIS packet? */ > + if (vis_server == VIS_TYPE_SERVER_SYNC && > + is_my_mac(info->packet.target_orig)) > + are_target = 1; > + > spin_lock_irqsave(&vis_hash_lock, flags); > - info = add_packet(vis_packet, vis_info_len, &is_new); > + info = add_packet(vis_packet, vis_info_len, &is_new, are_target); > if (info == NULL) > goto end; > /* note that outdated packets will be dropped at this point. */ > > > /* send only if we're the target server or ... */ > - if (vis_server == VIS_TYPE_SERVER_SYNC && > - is_my_mac(info->packet.target_orig) && > - is_new) { > + if (are_target && is_new) { > info->packet.vis_type = VIS_TYPE_SERVER_SYNC; /* upgrade! */ > - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); > if (list_empty(&info->send_list)) > list_add_tail(&info->send_list, &send_list); > > @@ -362,14 +385,17 @@ > while (hash_iterate(orig_hash, &hashit_global)) { > orig_node = hashit_global.bucket->data; > if (orig_node->router != NULL > - && compare_orig(orig_node->router->addr, orig_node->orig) > + && compare_orig(orig_node->router->addr, > + orig_node->orig) > && orig_node->batman_if > && (orig_node->batman_if->if_active == IF_ACTIVE) > && orig_node->router->tq_avg > 0) { > > /* fill one entry into buffer. */ > entry = &entry_array[info->packet.entries]; > - memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN); > + memcpy(entry->src, > + orig_node->batman_if->net_dev->dev_addr, > + ETH_ALEN); > memcpy(entry->dest, orig_node->orig, ETH_ALEN); > entry->quality = orig_node->router->tq_avg; > info->packet.entries++; > @@ -401,6 +427,8 @@ > return 0; > } > > +/* free old vis packets. Must be called with this vis_hash_lock > + * held */ > static void purge_vis_packets(void) > { > HASHIT(hashit); > @@ -413,7 +441,7 @@ > if (time_after(jiffies, > info->first_seen + (VIS_TIMEOUT*HZ)/1000)) { > hash_remove_bucket(vis_hash, &hashit); > - free_info(info); > + kref_put(&info->refcount, free_info); > } > } > } > @@ -423,6 +451,8 @@ > HASHIT(hashit); > struct orig_node *orig_node; > unsigned long flags; > + struct batman_if *batman_if; > + uint8_t dstaddr[ETH_ALEN]; > > spin_lock_irqsave(&orig_hash_lock, flags); > > @@ -431,46 +461,57 @@ > orig_node = hashit.bucket->data; > > /* if it's a vis server and reachable, send it. */ > - if (orig_node && > - (orig_node->flags & VIS_SERVER) && > - orig_node->batman_if && > - orig_node->router) { > + if ((!orig_node) || (!orig_node->batman_if) || > + (!orig_node->router)) > + continue; > + if (!(orig_node->flags & VIS_SERVER)) > + continue; > + /* don't send it if we already received the packet from > + * this node. */ > + if (recv_list_is_in(&info->recv_list, orig_node->orig)) > + continue; > > - /* don't send it if we already received the packet from > - * this node. */ > - if (recv_list_is_in(&info->recv_list, orig_node->orig)) > - continue; > + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); > + batman_if = orig_node->batman_if; > + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); > + spin_unlock_irqrestore(&orig_hash_lock, flags); > > - memcpy(info->packet.target_orig, > - orig_node->orig, ETH_ALEN); > + send_raw_packet((unsigned char *)&info->packet, > + packet_length, batman_if, dstaddr); > > - send_raw_packet((unsigned char *) &info->packet, > - packet_length, > - orig_node->batman_if, > - orig_node->router->addr); > - } > + spin_lock_irqsave(&orig_hash_lock, flags); > + > } > + spin_unlock_irqrestore(&orig_hash_lock, flags); > memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); > - spin_unlock_irqrestore(&orig_hash_lock, flags); > } > > static void unicast_vis_packet(struct vis_info *info, int packet_length) > { > struct orig_node *orig_node; > unsigned long flags; > + struct batman_if *batman_if; > + uint8_t dstaddr[ETH_ALEN]; > > spin_lock_irqsave(&orig_hash_lock, flags); > orig_node = ((struct orig_node *) > hash_find(orig_hash, info->packet.target_orig)); > > - if ((orig_node != NULL) && > - (orig_node->batman_if != NULL) && > - (orig_node->router != NULL)) { > - send_raw_packet((unsigned char *) &info->packet, packet_length, > - orig_node->batman_if, > - orig_node->router->addr); > - } > + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) > + goto out; > + > + /* don't lock while sending the packets ... we therefore > + * copy the required data before sending */ > + batman_if = orig_node->batman_if; > + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); > spin_unlock_irqrestore(&orig_hash_lock, flags); > + > + send_raw_packet((unsigned char *)&info->packet, > + packet_length, batman_if, dstaddr); > + return; > + > +out: > + spin_unlock_irqrestore(&orig_hash_lock, flags); > } > > /* only send one vis packet. called from send_vis_packets() */ > @@ -503,6 +544,7 @@ > unsigned long flags; > > spin_lock_irqsave(&vis_hash_lock, flags); > + > purge_vis_packets(); > > if (generate_vis_packet() == 0) > @@ -511,7 +553,11 @@ > > list_for_each_entry_safe(info, temp, &send_list, send_list) { > list_del_init(&info->send_list); > + kref_get(&info->refcount); > + spin_unlock_irqrestore(&vis_hash_lock, flags); > send_vis_packet(info); > + spin_lock_irqsave(&vis_hash_lock, flags); > + kref_put(&info->refcount, free_info); > } > spin_unlock_irqrestore(&vis_hash_lock, flags); > start_vis_timer(); > @@ -544,6 +590,7 @@ > my_vis_info->first_seen = jiffies - atomic_read(&vis_interval); > INIT_LIST_HEAD(&my_vis_info->recv_list); > INIT_LIST_HEAD(&my_vis_info->send_list); > + kref_init(&my_vis_info->refcount); > my_vis_info->packet.version = COMPAT_VERSION; > my_vis_info->packet.packet_type = BAT_VIS; > my_vis_info->packet.ttl = TTL; > @@ -557,9 +604,9 @@ > > if (hash_add(vis_hash, my_vis_info) < 0) { > printk(KERN_ERR > - "batman-adv:Can't add own vis packet into hash\n"); > - free_info(my_vis_info); /* not in hash, need to remove it > - * manually. */ > + "batman-adv:Can't add own vis packet into hash\n"); > + /* not in hash, need to remove it manually. */ > + kref_put(&my_vis_info->refcount, free_info); > goto err; > } > > @@ -573,6 +620,13 @@ > return 0; > } > > +/* Decrease the reference count on a hash item info */ > +static void free_info_ref(void *data) > +{ > + struct vis_info *info = data; > + kref_put(&info->refcount, free_info); > +} > + > /* shutdown vis-server */ > void vis_quit(void) > { > @@ -584,7 +638,7 @@ > > spin_lock_irqsave(&vis_hash_lock, flags); > /* properly remove, kill timers ... */ > - hash_delete(vis_hash, free_info); > + hash_delete(vis_hash, free_info_ref); > vis_hash = NULL; > my_vis_info = NULL; > spin_unlock_irqrestore(&vis_hash_lock, flags); > @@ -594,5 +648,5 @@ > static void start_vis_timer(void) > { > queue_delayed_work(bat_event_workqueue, &vis_timer_wq, > - (atomic_read(&vis_interval) * HZ ) / 1000); > + (atomic_read(&vis_interval) * HZ) / 1000); > } > Index: vis.h > =================================================================== > --- vis.h (revision 1568) > +++ vis.h (working copy) > @@ -29,6 +29,7 @@ > /* list of server-neighbors we received a vis-packet > * from. we should not reply to them. */ > struct list_head send_list; > + struct kref refcount; > /* this packet might be part of the vis send queue. */ > struct vis_packet packet; > /* vis_info may follow here*/ > [-- Attachment #1.2: 2010-01-31-2030-vis-patch-kernelpanic.log --] [-- Type: text/plain, Size: 2279 bytes --] root@OpenWrt:/# batctl vs [ ] client mode (server disabled) [x] server mode (server enabled) root@OpenWrt:/# CPU 0 Unable to handle kernel paging request at virtual address 00000024, epc == 8014d58c, ra == 80410154 Oops[#1]: Cpu 0 $ 0 : 00000000 10009c00 8070e140 00000006 $ 4 : 8070e140 00000024 00000006 00000001 $ 8 : 00000024 00000000 00000000 00000000 $12 : 00000000 7fd89ed8 00000413 00430000 $16 : 80fef500 8041cdc0 00000024 8070e000 $20 : 0000004c 80d63000 00004305 80c05040 $24 : 00000002 8061c874 $28 : 80290000 80291ca8 00000000 80410154 Hi : 000000f5 Lo : 15a32180 epc : 8014d58c memcmp+0xc/0x38 Tainted: P ra : 80410154 is_my_mac+0x3c/0x78 [batman_adv] Status: 10009c03 KERNEL EXL IE Cause : 10800008 BadVA : 00000024 PrId : 00019064 (MIPS 4KEc) Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6 Process swapper (pid: 0, threadinfo=80290000, task=80292170, tls=00000000) Stack : 80225540 00000000 80a11c00 8022b330 00000000 00000000 80c05060 80417264 00000000 8022a728 80000000 806c0052 c02e4000 c02e7000 80f49a40 80c05060 80c05052 8070e000 80d63000 804124e4 80225540 00000000 80a11c00 00000000 80c14e40 80fef500 80f49a40 80417bf4 8070e2c0 80c05052 8070e000 80d63000 80d63000 80f49a40 8070e2c0 806215c4 00000000 80292170 80291db0 000332b6 ... Call Trace: [<8014d58c>] memcmp+0xc/0x38 [<80410154>] is_my_mac+0x3c/0x78 [batman_adv] [<80417264>] receive_client_update_packet+0x4c/0x154 [batman_adv] [<804124e4>] recv_vis_packet+0xa4/0xbc [batman_adv] [<80417bf4>] batman_skb_recv+0x168/0x1dc [batman_adv] [<806215c4>] ieee80211_saveath+0xb24/0xb80 [ath_ahb] Code: 08053567 00003821 90430000 <91020000> 00621023 14400006 24c6ffff 00871021 00a74021 Kernel panic - not syncing: Fatal exception in interrupt Rebooting in 3 seconds..+ [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-31 19:37 ` Linus Lüssing @ 2010-01-31 20:56 ` Andrew Lunn 0 siblings, 0 replies; 37+ messages in thread From: Andrew Lunn @ 2010-01-31 20:56 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Sun, Jan 31, 2010 at 08:37:29PM +0100, Linus L??ssing wrote: > Hi Andrew, > > Yes, ehm it does indeed cause a kernel panic on the server side > immediately :). See the attachment for the full call trace. Oh yes, dumb error. I will give you a new patch tomorrow. I might even try it first this time! Thanks for testing, Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-01-30 16:50 ` Andrew Lunn 2010-01-31 19:37 ` Linus Lüssing @ 2010-02-11 9:46 ` Andrew Lunn 2010-02-11 10:01 ` Andrew Lunn 1 sibling, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-02-11 9:46 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Hi Linus Here is a new version of the patch. I've tested it this time using five UML machines. It should not immediately opps now. Please could you test it with your test setup. Thanks Andrew Staging: batman-adv: Don't have interrupts disabled while sending. send_vis_packets() would disable interrupts before calling dev_queue_xmit() which resulting in a backtrace in local_bh_enable(). Fix this by using kref on the vis_info object so that we can call send_vis_packets() without holding vis_hash_lock. vis_hash_lock also used to protect recv_list, so we now need a new lock to protect that instead of vis_hash_lock. Also a few checkpatch cleanups. Reported-by: Linus L\374ssing <linus.luessing@web.de> Signed-off-by: Andrew Lunn <andrew@lunn.ch> Index: vis.c Index: vis.c =================================================================== --- vis.c (revision 1568) +++ vis.c (working copy) @@ -30,22 +30,26 @@ struct hashtable_t *vis_hash; DEFINE_SPINLOCK(vis_hash_lock); +static DEFINE_SPINLOCK(recv_list_lock); static struct vis_info *my_vis_info; static struct list_head send_list; /* always locked with vis_hash_lock */ static void start_vis_timer(void); /* free the info */ -static void free_info(void *data) +static void free_info(struct kref *ref) { - struct vis_info *info = data; + struct vis_info *info = container_of(ref, struct vis_info, refcount); struct recvlist_node *entry, *tmp; + unsigned long flags; list_del_init(&info->send_list); + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry_safe(entry, tmp, &info->recv_list, list) { list_del(&entry->list); kfree(entry); } + spin_unlock_irqrestore(&recv_list_lock, flags); kfree(info); } @@ -147,32 +151,41 @@ static void recv_list_add(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC); if (!entry) return; memcpy(entry->mac, mac, ETH_ALEN); + spin_lock_irqsave(&recv_list_lock, flags); list_add_tail(&entry->list, recv_list); + spin_unlock_irqrestore(&recv_list_lock, flags); } /* returns 1 if this mac is in the recv_list */ static int recv_list_is_in(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry(entry, recv_list, list) { - if (memcmp(entry->mac, mac, ETH_ALEN) == 0) + if (memcmp(entry->mac, mac, ETH_ALEN) == 0) { + spin_unlock_irqrestore(&recv_list_lock, flags); return 1; + } } - + spin_unlock_irqrestore(&recv_list_lock, flags); return 0; } /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old, - * broken.. ). vis hash must be locked outside. is_new is set when the packet + * broken.. ). vis hash must be locked outside. is_new is set when the packet * is newer than old entries in the hash. */ static struct vis_info *add_packet(struct vis_packet *vis_packet, - int vis_info_len, int *is_new) + int vis_info_len, int *is_new, + int make_broadcast) { struct vis_info *info, *old_info; struct vis_info search_elem; @@ -199,7 +212,7 @@ } /* remove old entry */ hash_remove(vis_hash, old_info); - free_info(old_info); + kref_put(&old_info->refcount, free_info); } info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC); @@ -208,6 +221,7 @@ INIT_LIST_HEAD(&info->send_list); INIT_LIST_HEAD(&info->recv_list); + kref_init(&info->refcount); info->first_seen = jiffies; memcpy(&info->packet, vis_packet, sizeof(struct vis_packet) + vis_info_len); @@ -215,16 +229,21 @@ /* initialize and add new packet. */ *is_new = 1; + /* Make it a broadcast packet, if required */ + if (make_broadcast) + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); + /* repair if entries is longer than packet. */ if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len) - info->packet.entries = vis_info_len / sizeof(struct vis_info_entry); + info->packet.entries = vis_info_len / + sizeof(struct vis_info_entry); recv_list_add(&info->recv_list, info->packet.sender_orig); /* try to add it */ if (hash_add(vis_hash, info) < 0) { /* did not work (for some reason) */ - free_info(info); + kref_put(&old_info->refcount, free_info); info = NULL; } @@ -235,19 +254,20 @@ void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len) { struct vis_info *info; - int is_new; + int is_new, make_broadcast; unsigned long flags; int vis_server = atomic_read(&vis_mode); + make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC); + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast); if (info == NULL) goto end; /* only if we are server ourselves and packet is newer than the one in * hash.*/ if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) { - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); if (list_empty(&info->send_list)) list_add_tail(&info->send_list, &send_list); } @@ -263,24 +283,27 @@ int is_new; unsigned long flags; int vis_server = atomic_read(&vis_mode); + int are_target = 0; /* clients shall not broadcast. */ if (is_bcast(vis_packet->target_orig)) return; + /* Are we the target for this VIS packet? */ + if (vis_server == VIS_TYPE_SERVER_SYNC && + is_my_mac(vis_packet->target_orig)) + are_target = 1; + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, are_target); if (info == NULL) goto end; /* note that outdated packets will be dropped at this point. */ /* send only if we're the target server or ... */ - if (vis_server == VIS_TYPE_SERVER_SYNC && - is_my_mac(info->packet.target_orig) && - is_new) { + if (are_target && is_new) { info->packet.vis_type = VIS_TYPE_SERVER_SYNC; /* upgrade! */ - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); if (list_empty(&info->send_list)) list_add_tail(&info->send_list, &send_list); @@ -362,14 +385,17 @@ while (hash_iterate(orig_hash, &hashit_global)) { orig_node = hashit_global.bucket->data; if (orig_node->router != NULL - && compare_orig(orig_node->router->addr, orig_node->orig) + && compare_orig(orig_node->router->addr, + orig_node->orig) && orig_node->batman_if && (orig_node->batman_if->if_active == IF_ACTIVE) && orig_node->router->tq_avg > 0) { /* fill one entry into buffer. */ entry = &entry_array[info->packet.entries]; - memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN); + memcpy(entry->src, + orig_node->batman_if->net_dev->dev_addr, + ETH_ALEN); memcpy(entry->dest, orig_node->orig, ETH_ALEN); entry->quality = orig_node->router->tq_avg; info->packet.entries++; @@ -401,6 +427,8 @@ return 0; } +/* free old vis packets. Must be called with this vis_hash_lock + * held */ static void purge_vis_packets(void) { HASHIT(hashit); @@ -413,7 +441,7 @@ if (time_after(jiffies, info->first_seen + (VIS_TIMEOUT*HZ)/1000)) { hash_remove_bucket(vis_hash, &hashit); - free_info(info); + kref_put(&info->refcount, free_info); } } } @@ -423,6 +451,8 @@ HASHIT(hashit); struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); @@ -431,46 +461,57 @@ orig_node = hashit.bucket->data; /* if it's a vis server and reachable, send it. */ - if (orig_node && - (orig_node->flags & VIS_SERVER) && - orig_node->batman_if && - orig_node->router) { + if ((!orig_node) || (!orig_node->batman_if) || + (!orig_node->router)) + continue; + if (!(orig_node->flags & VIS_SERVER)) + continue; + /* don't send it if we already received the packet from + * this node. */ + if (recv_list_is_in(&info->recv_list, orig_node->orig)) + continue; - /* don't send it if we already received the packet from - * this node. */ - if (recv_list_is_in(&info->recv_list, orig_node->orig)) - continue; + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); + spin_unlock_irqrestore(&orig_hash_lock, flags); - memcpy(info->packet.target_orig, - orig_node->orig, ETH_ALEN); + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); - send_raw_packet((unsigned char *) &info->packet, - packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + spin_lock_irqsave(&orig_hash_lock, flags); + } + spin_unlock_irqrestore(&orig_hash_lock, flags); memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - spin_unlock_irqrestore(&orig_hash_lock, flags); } static void unicast_vis_packet(struct vis_info *info, int packet_length) { struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); orig_node = ((struct orig_node *) hash_find(orig_hash, info->packet.target_orig)); - if ((orig_node != NULL) && - (orig_node->batman_if != NULL) && - (orig_node->router != NULL)) { - send_raw_packet((unsigned char *) &info->packet, packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) + goto out; + + /* don't lock while sending the packets ... we therefore + * copy the required data before sending */ + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); spin_unlock_irqrestore(&orig_hash_lock, flags); + + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); + return; + +out: + spin_unlock_irqrestore(&orig_hash_lock, flags); } /* only send one vis packet. called from send_vis_packets() */ @@ -503,6 +544,7 @@ unsigned long flags; spin_lock_irqsave(&vis_hash_lock, flags); + purge_vis_packets(); if (generate_vis_packet() == 0) @@ -511,7 +553,11 @@ list_for_each_entry_safe(info, temp, &send_list, send_list) { list_del_init(&info->send_list); + kref_get(&info->refcount); + spin_unlock_irqrestore(&vis_hash_lock, flags); send_vis_packet(info); + spin_lock_irqsave(&vis_hash_lock, flags); + kref_put(&info->refcount, free_info); } spin_unlock_irqrestore(&vis_hash_lock, flags); start_vis_timer(); @@ -544,6 +590,7 @@ my_vis_info->first_seen = jiffies - atomic_read(&vis_interval); INIT_LIST_HEAD(&my_vis_info->recv_list); INIT_LIST_HEAD(&my_vis_info->send_list); + kref_init(&my_vis_info->refcount); my_vis_info->packet.version = COMPAT_VERSION; my_vis_info->packet.packet_type = BAT_VIS; my_vis_info->packet.ttl = TTL; @@ -557,9 +604,9 @@ if (hash_add(vis_hash, my_vis_info) < 0) { printk(KERN_ERR - "batman-adv:Can't add own vis packet into hash\n"); - free_info(my_vis_info); /* not in hash, need to remove it - * manually. */ + "batman-adv:Can't add own vis packet into hash\n"); + /* not in hash, need to remove it manually. */ + kref_put(&my_vis_info->refcount, free_info); goto err; } @@ -573,6 +620,13 @@ return 0; } +/* Decrease the reference count on a hash item info */ +static void free_info_ref(void *data) +{ + struct vis_info *info = data; + kref_put(&info->refcount, free_info); +} + /* shutdown vis-server */ void vis_quit(void) { @@ -584,7 +638,7 @@ spin_lock_irqsave(&vis_hash_lock, flags); /* properly remove, kill timers ... */ - hash_delete(vis_hash, free_info); + hash_delete(vis_hash, free_info_ref); vis_hash = NULL; my_vis_info = NULL; spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -594,5 +648,5 @@ static void start_vis_timer(void) { queue_delayed_work(bat_event_workqueue, &vis_timer_wq, - (atomic_read(&vis_interval) * HZ ) / 1000); + (atomic_read(&vis_interval) * HZ) / 1000); } Index: vis.h =================================================================== --- vis.h (revision 1568) +++ vis.h (working copy) @@ -29,6 +29,7 @@ /* list of server-neighbors we received a vis-packet * from. we should not reply to them. */ struct list_head send_list; + struct kref refcount; /* this packet might be part of the vis send queue. */ struct vis_packet packet; /* vis_info may follow here*/ ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-02-11 9:46 ` Andrew Lunn @ 2010-02-11 10:01 ` Andrew Lunn 2010-02-19 17:19 ` Linus Lüssing 2010-02-28 16:34 ` Simon Wunderlich 0 siblings, 2 replies; 37+ messages in thread From: Andrew Lunn @ 2010-02-11 10:01 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Thu, Feb 11, 2010 at 10:46:59AM +0100, Andrew Lunn wrote: > Hi Linus > > Here is a new version of the patch. I've tested it this time using > five UML machines. It should not immediately opps now. Instead is will leak memory and crash after a while... I will try to find the memory leak. Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-02-11 10:01 ` Andrew Lunn @ 2010-02-19 17:19 ` Linus Lüssing 2010-02-20 18:04 ` Andrew Lunn 2010-02-28 16:34 ` Simon Wunderlich 1 sibling, 1 reply; 37+ messages in thread From: Linus Lüssing @ 2010-02-19 17:19 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 3065 bytes --] Hi Andrew, Sorry, didn't have the time to try your patch any earlier, I'm right in the middle of my exams :). Your patch already looks quite good, I couldn't reproduce any memory leaks or crashes here (tried that with three routers and 1 or 2 vis servers activated, also activating/deactivating them a lot, no problems with that). And yes, the slow-path warning has gone with your patch. However, I'm having some weird output when connecting two routers over wifi _and_ over ethernet cable. The setup: Before plugging in the cable: r1-ath1 <-- wifi --> r2-ath1 ------------ root@OpenWrt:~# batctl vd dot digraph { "r1-ath1" -> "r2-ath1" [label="1.32"] "r1-ath1" -> "r1-hna" [label="HNA"] "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] subgraph "cluster_r1-ath1" { "r1-ath1" [peripheries=2] } "r2-ath1" -> "r1-ath1" [label="1.11"] "r2-ath1" -> "r2-hna" [label="HNA"] "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] subgraph "cluster_r2-ath1" { "r2-ath1" [peripheries=2] } } ------------ After plugging in the cable: r1-ath1 <-- wifi --> r2-ath1 + r1-eth0.3 <-- cable --> r2-eth0.3 ------------ root@OpenWrt:~# batctl vd dot digraph { "r1-ath1" -> "r2-ath1" [label="1.0"] "r1-ath1" -> "r2-eth0.3" [label="1.66"] "r1-ath1" -> "r1-hna" [label="HNA"] "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] subgraph "cluster_r1-ath1" { "r1-ath1" [peripheries=2] "r1-eth0.3" } subgraph "cluster_r1-ath1" { "r1-ath1" [peripheries=2] } "r2-ath1" -> "r1-ath1" [label="1.0"] "r2-ath1" -> "r1-eth0.3" [label="1.15"] "r2-ath1" -> "r2-hna" [label="HNA"] "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] subgraph "cluster_r2-ath1" { "r2-ath1" [peripheries=2] "r2-eth0.3" } subgraph "cluster_r2-ath1" { "r2-ath1" [peripheries=2] } } root@OpenWrt:~# cat /proc/net/batman-adv/vis_data 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA 5a:2e:1e:1f:64:6b, PRIMARY, SEC 04:22:b0:98:87:de, 06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA 00:22:b0:98:87:f9, HNA 82:31:95:f9:14:6f, SEC 04:22:b0:98:87:fa, PRIMARY, ---------- So the second 'subgraph "cluster_r1-ath1"' is obviously unnecessary. Also "r1-ath1" -> "r2-eth0.3" looks wrong, should be "r1-eth0.3" -> "r2-eth0.3" instead (and the same with r2 a few lines later). Cheers, Linus On Thu, Feb 11, 2010 at 11:01:56AM +0100, Andrew Lunn wrote: > On Thu, Feb 11, 2010 at 10:46:59AM +0100, Andrew Lunn wrote: > > Hi Linus > > > > Here is a new version of the patch. I've tested it this time using > > five UML machines. It should not immediately opps now. > > Instead is will leak memory and crash after a while... > > I will try to find the memory leak. > > Andrew > [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-02-19 17:19 ` Linus Lüssing @ 2010-02-20 18:04 ` Andrew Lunn 2010-02-21 13:10 ` Linus Lüssing 0 siblings, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-02-20 18:04 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Fri, Feb 19, 2010 at 06:19:05PM +0100, Linus L??ssing wrote: > Hi Andrew, > > Sorry, didn't have the time to try your patch any earlier, I'm > right in the middle of my exams :). Hi Linus Marek told me. No problems. I remember what its like studying for exams. However, it is nice to sometimes take a break and do something totally different. > Your patch already looks quite good, I couldn't reproduce any > memory leaks or crashes here (tried that with three routers and 1 > or 2 vis servers activated, also activating/deactivating them a > lot, no problems with that). And yes, the slow-path warning has > gone with your patch. Great. So we are on the right tracks. > However, I'm having some weird output when connecting two routers > over wifi _and_ over ethernet cable. The setup: > > Before plugging in the cable: > r1-ath1 <-- wifi --> r2-ath1 > ------------ > root@OpenWrt:~# batctl vd dot > digraph { > "r1-ath1" -> "r2-ath1" [label="1.32"] > "r1-ath1" -> "r1-hna" [label="HNA"] > "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] > subgraph "cluster_r1-ath1" { > "r1-ath1" [peripheries=2] > } > "r2-ath1" -> "r1-ath1" [label="1.11"] > "r2-ath1" -> "r2-hna" [label="HNA"] > "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] > subgraph "cluster_r2-ath1" { > "r2-ath1" [peripheries=2] > } > } > ------------ > After plugging in the cable: > r1-ath1 <-- wifi --> r2-ath1 + > r1-eth0.3 <-- cable --> r2-eth0.3 > ------------ > root@OpenWrt:~# batctl vd dot > digraph { > "r1-ath1" -> "r2-ath1" [label="1.0"] > "r1-ath1" -> "r2-eth0.3" [label="1.66"] > "r1-ath1" -> "r1-hna" [label="HNA"] > "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] > subgraph "cluster_r1-ath1" { > "r1-ath1" [peripheries=2] > "r1-eth0.3" > } > subgraph "cluster_r1-ath1" { > "r1-ath1" [peripheries=2] > } > "r2-ath1" -> "r1-ath1" [label="1.0"] > "r2-ath1" -> "r1-eth0.3" [label="1.15"] > "r2-ath1" -> "r2-hna" [label="HNA"] > "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] > subgraph "cluster_r2-ath1" { > "r2-ath1" [peripheries=2] > "r2-eth0.3" > } > subgraph "cluster_r2-ath1" { > "r2-ath1" [peripheries=2] > } > } > root@OpenWrt:~# cat /proc/net/batman-adv/vis_data > 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA 5a:2e:1e:1f:64:6b, PRIMARY, SEC 04:22:b0:98:87:de, > 06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA 00:22:b0:98:87:f9, HNA 82:31:95:f9:14:6f, SEC 04:22:b0:98:87:fa, PRIMARY, Actually, this vis_data to does not map to the dot above! There are the wrong number of HNA, wrong order etc. Here is what i think your bat-host file contains: 06:22:b0:98:87:dd r1-ath1 06:22:b0:98:87:f9 r2-ath1 00:22:b0:98:87:dd r1-hna 04:22:b0:98:87:de r1-eth0.3 00:22:b0:98:87:f9 r2-hna 04:22:b0:98:87:fa r2-eth0.3 and this is what i get, assuming i got the MAC->name mapping correct: digraph { "r1-ath1" -> "r2-eth0.3" [label="1.15"] "r1-ath1" -> "r1-hna" [label="HNA"] "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] subgraph "cluster_r1-ath1" { "r1-ath1" [peripheries=2] } subgraph "cluster_r1-ath1" { "r1-ath1" [peripheries=2] "r1-eth0.3" } "r2-ath1" -> "r1-ath1" [label="1.0"] "r2-ath1" -> "r1-eth0.3" [label="1.15"] "r2-ath1" -> "r2-hna" [label="HNA"] "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] subgraph "cluster_r2-ath1" { "r2-ath1" [peripheries=2] "r2-eth0.3" } subgraph "cluster_r2-ath1" { "r2-ath1" [peripheries=2] } } batctl parses top-to-bottom, left-to-right. It does not consolidate the PRIMARY and the SECONDARY into one cluster. It leaves DOT to do that. Hence there are two cluster statements for each cluster actually drawn. > So the second 'subgraph "cluster_r1-ath1"' is obviously > unnecessary. Yes, unnecessary, but makes the batctl code easier. Also "r1-ath1" -> "r2-eth0.3" looks wrong, should be > "r1-eth0.3" -> "r2-eth0.3" instead (and the same with r2 a few > lines later). These comments i agree with. A wireless and a wired device should not be neighbours. We don't have any records which originate from the secondary MAC address. That is guess is the major problem here. So, did my/Mareks patch break it, or was it broken before? First i suggest you go back to just before Simon's patch which introduced receiving using skbufs: http://open-mesh.org/changeset/1517 That will tell us if we need to go back further, or our patch broke it. If you need to go back further, i would suggest just before: http://open-mesh.org/changeset/1510 However, if it is our patch then we can chop the patch into two: Use Mareks patch: https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2010-January/002261.html and Index: vis.c =================================================================== --- vis.c (revision 1575) +++ vis.c (working copy) @@ -444,10 +444,15 @@ memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); +spin_unlock_irqrestore(&orig_hash_lock, flags); + send_raw_packet((unsigned char *) &info->packet, packet_length, orig_node->batman_if, orig_node->router->addr); + +spin_lock_irqsave(&orig_hash_lock, flags); + } } memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); This adds a race condition, which i hope if O.K. for debugging purposes, but i hope allows the send to happen without the slowpath errors. If so, we can test Marek's part of the patch. I'm on vacation for a week now. I will have Internet access some time, but not much. Have fun debugging. Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-02-20 18:04 ` Andrew Lunn @ 2010-02-21 13:10 ` Linus Lüssing 0 siblings, 0 replies; 37+ messages in thread From: Linus Lüssing @ 2010-02-21 13:10 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 6172 bytes --] On Sat, Feb 20, 2010 at 07:04:11PM +0100, Andrew Lunn wrote: > On Fri, Feb 19, 2010 at 06:19:05PM +0100, Linus L??ssing wrote: > > Hi Andrew, > > > > Sorry, didn't have the time to try your patch any earlier, I'm > > right in the middle of my exams :). > > Hi Linus > > Marek told me. No problems. I remember what its like studying for > exams. However, it is nice to sometimes take a break and do something > totally different. > > > Your patch already looks quite good, I couldn't reproduce any > > memory leaks or crashes here (tried that with three routers and 1 > > or 2 vis servers activated, also activating/deactivating them a > > lot, no problems with that). And yes, the slow-path warning has > > gone with your patch. > > Great. So we are on the right tracks. > > > However, I'm having some weird output when connecting two routers > > over wifi _and_ over ethernet cable. The setup: > > > > Before plugging in the cable: > > r1-ath1 <-- wifi --> r2-ath1 > > ------------ > > root@OpenWrt:~# batctl vd dot > > digraph { > > "r1-ath1" -> "r2-ath1" [label="1.32"] > > "r1-ath1" -> "r1-hna" [label="HNA"] > > "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] > > subgraph "cluster_r1-ath1" { > > "r1-ath1" [peripheries=2] > > } > > "r2-ath1" -> "r1-ath1" [label="1.11"] > > "r2-ath1" -> "r2-hna" [label="HNA"] > > "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] > > subgraph "cluster_r2-ath1" { > > "r2-ath1" [peripheries=2] > > } > > } > > ------------ > > After plugging in the cable: > > r1-ath1 <-- wifi --> r2-ath1 + > > r1-eth0.3 <-- cable --> r2-eth0.3 > > ------------ > > root@OpenWrt:~# batctl vd dot > > digraph { > > "r1-ath1" -> "r2-ath1" [label="1.0"] > > "r1-ath1" -> "r2-eth0.3" [label="1.66"] > > "r1-ath1" -> "r1-hna" [label="HNA"] > > "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] > > subgraph "cluster_r1-ath1" { > > "r1-ath1" [peripheries=2] > > "r1-eth0.3" > > } > > subgraph "cluster_r1-ath1" { > > "r1-ath1" [peripheries=2] > > } > > "r2-ath1" -> "r1-ath1" [label="1.0"] > > "r2-ath1" -> "r1-eth0.3" [label="1.15"] > > "r2-ath1" -> "r2-hna" [label="HNA"] > > "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] > > subgraph "cluster_r2-ath1" { > > "r2-ath1" [peripheries=2] > > "r2-eth0.3" > > } > > subgraph "cluster_r2-ath1" { > > "r2-ath1" [peripheries=2] > > } > > } > > root@OpenWrt:~# cat /proc/net/batman-adv/vis_data > > 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA 5a:2e:1e:1f:64:6b, PRIMARY, SEC 04:22:b0:98:87:de, > > 06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA 00:22:b0:98:87:f9, HNA 82:31:95:f9:14:6f, SEC 04:22:b0:98:87:fa, PRIMARY, > > Actually, this vis_data to does not map to the dot above! There are > the wrong number of HNA, wrong order etc. Hmm, just noticed, the output also seems to be flapping between those two from time to time: ------------------ root@OpenWrt:~# cat /proc/net/batman-adv/vis 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA f6:ae:97:b3:9a:5c, PRIMARY, SEC 04:22:b0:98:87:de, 06:22:b0:98:87:f9,TQ 04:22:b0:98:87:de 251, HNA da:3e:79:2c:d3:3e, HNA 00:22:b0:98:87:f9, PRIMARY, SEC 04:22:b0:98:87:fa, root@OpenWrt:~# cat /proc/net/batman-adv/vis 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA f6:ae:97:b3:9a:5c, PRIMARY, SEC 04:22:b0:98:87:de, 06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA da:3e:79:2c:d3:3e, HNA 00:22:b0:98:87:f9, SEC 04:22:b0:98:87:fa, PRIMARY, ------------------ > > Here is what i think your bat-host file contains: > 06:22:b0:98:87:dd r1-ath1 > 06:22:b0:98:87:f9 r2-ath1 > 00:22:b0:98:87:dd r1-hna > 04:22:b0:98:87:de r1-eth0.3 > 00:22:b0:98:87:f9 r2-hna > 04:22:b0:98:87:fa r2-eth0.3 > > and this is what i get, assuming i got the MAC->name mapping correct: Yes, correct mapping :). > > digraph { > "r1-ath1" -> "r2-eth0.3" [label="1.15"] > "r1-ath1" -> "r1-hna" [label="HNA"] > "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"] > subgraph "cluster_r1-ath1" { > "r1-ath1" [peripheries=2] > } > subgraph "cluster_r1-ath1" { > "r1-ath1" [peripheries=2] > "r1-eth0.3" > } > "r2-ath1" -> "r1-ath1" [label="1.0"] > "r2-ath1" -> "r1-eth0.3" [label="1.15"] > "r2-ath1" -> "r2-hna" [label="HNA"] > "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"] > subgraph "cluster_r2-ath1" { > "r2-ath1" [peripheries=2] > "r2-eth0.3" > } > subgraph "cluster_r2-ath1" { > "r2-ath1" [peripheries=2] > } > } > > batctl parses top-to-bottom, left-to-right. It does not consolidate > the PRIMARY and the SECONDARY into one cluster. It leaves DOT to do > that. Hence there are two cluster statements for each cluster actually > drawn. > > > So the second 'subgraph "cluster_r1-ath1"' is obviously > > unnecessary. > > Yes, unnecessary, but makes the batctl code easier. > > Also "r1-ath1" -> "r2-eth0.3" looks wrong, should be > > "r1-eth0.3" -> "r2-eth0.3" instead (and the same with r2 a few > > lines later). > > These comments i agree with. A wireless and a wired device should not > be neighbours. We don't have any records which originate from the > secondary MAC address. That is guess is the major problem here. > > So, did my/Mareks patch break it, or was it broken before? > > First i suggest you go back to just before Simon's patch which > introduced receiving using skbufs: > > http://open-mesh.org/changeset/1517 > > That will tell us if we need to go back further, or our patch broke > it. > > If you need to go back further, i would suggest just before: > > http://open-mesh.org/changeset/1510 Okay, just checked, this got introduced with 1510 already, yes. I might have a closer look at this next week. Cheers, Linus [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-02-11 10:01 ` Andrew Lunn 2010-02-19 17:19 ` Linus Lüssing @ 2010-02-28 16:34 ` Simon Wunderlich 2010-03-01 5:59 ` Andrew Lunn 1 sibling, 1 reply; 37+ messages in thread From: Simon Wunderlich @ 2010-02-28 16:34 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1.1: Type: text/plain, Size: 4979 bytes --] Hello, i can reproduce the slowpath warning in the trunk within my qemu setup. When i start a server, some of the clients show this slow path warning. I've tried Andrews patch, and i can not see any memory leaks - it varies in a 100k window. However some of the servers crash after some time (30 minutes) without any special trigger i am aware of. Stack trace is this: root@OpenWrt:/# ------------[ cut here ]------------ WARNING: at lib/kref.c:43 kref_get+0x18/0x30() Hardware name: Modules linked in: via_velocity via_rhine tg3 sis900 r8169 pcnet32 ne2k_pci 8390 e1000 e100 batman_adv 8139too 3c59x nf_nat_tftp nf_conntrack_tftp nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_conntrack_ftp ipt_MASQUERADE iptable_nat nf_nat xt_NOTRACK iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack pppoe pppox libphy ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ppp_async ppp_generic slhc natsemi crc_ccitt ipv6 Pid: 570, comm: bat_events Not tainted 2.6.31.1 #32 Call Trace: [<c10dd608>] ? kref_get+0x18/0x30 [<c101f55f>] ? warn_slowpath_common+0x7f/0xb0 [<c10dd608>] ? kref_get+0x18/0x30 [<c101f5a3>] ? warn_slowpath_null+0x13/0x20 [<c10dd608>] ? kref_get+0x18/0x30 [<c2ac51a2>] ? proc_vis_read_prim_sec+0x352/0x5a0 [batman_adv] [<c2ac4ee0>] ? proc_vis_read_prim_sec+0x90/0x5a0 [batman_adv] [<c102d23a>] ? worker_thread+0xca/0x150 [<c102fd80>] ? autoremove_wake_function+0x0/0x50 [<c102d170>] ? worker_thread+0x0/0x150 [<c102fbc3>] ? kthread+0x73/0x90 [<c102fb50>] ? kthread+0x0/0x90 [<c1003813>] ? kernel_thread_helper+0x7/0x14 ---[ end trace 4f770856ce7e0712 ]--- ------------[ cut here ]------------ kernel BUG at mm/slub.c:2929! invalid opcode: 0000 [#1] last sysfs file: /sys/kernel/uevent_seqnum Modules linked in: via_velocity via_rhine tg3 sis900 r8169 pcnet32 ne2k_pci 8390 e1000 e100 batman_adv 8139too 3c59x nf_nat_tftp nf_conntrack_tftp nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_conntrack_ftp ipt_MASQUERADE iptable_nat nf_nat xt_NOTRACK iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack pppoe pppox libphy ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ppp_async ppp_generic slhc natsemi crc_ccitt ipv6 Pid: 570, comm: bat_events Tainted: G W (2.6.31.1 #32) EIP: 0060:[<c1064669>] EFLAGS: 00010046 CPU: 0 EIP is at kfree+0x49/0xc0 EAX: 00000000 EBX: c1f5dd94 ECX: 40080000 EDX: c133cba0 ESI: c2ac5540 EDI: c1f5dd80 EBP: 00000216 ESP: c0d06eec DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068 Process bat_events (pid: 570, ti=c0d06000 task=c18d0000 task.ti=c0d06000) Stack: 00000002 c1f5dd94 c1f5dd94 c2ac5540 c1f5dda4 c10dd5cf c1f5dda4 00000000 <0> c2ac538b c0d06f68 c0e0504a c1f5dd84 c1f5dd94 c1f5dd80 c1f5dd80 000022a0 <0> 00000046 c0e05024 c0e05030 00000286 00000086 c1f5dda4 0000003f 00000080 Call Trace: [<c2ac5540>] ? vis_init+0x150/0x1b0 [batman_adv] [<c10dd5cf>] ? kref_put+0x4f/0x70 [<c2ac538b>] ? proc_vis_read_prim_sec+0x53b/0x5a0 [batman_adv] [<c2ac4ee0>] ? proc_vis_read_prim_sec+0x90/0x5a0 [batman_adv] [<c102d23a>] ? worker_thread+0xca/0x150 [<c102fd80>] ? autoremove_wake_function+0x0/0x50 [<c102d170>] ? worker_thread+0x0/0x150 [<c102fbc3>] ? kthread+0x73/0x90 [<c102fb50>] ? kthread+0x0/0x90 [<c1003813>] ? kernel_thread_helper+0x7/0x14 Code: 00 40 a1 80 cb 2e c1 c1 ea 0c c1 e2 05 01 c2 8b 0a 89 c8 25 00 80 00 00 66 85 c0 74 05 8b 52 0c 8b 0a 84 c9 78 24 f6 c5 c0 75 09 <0f> 0b 90 8d 74 26 00 eb fe 8b 5c 24 08 89 d0 8b 74 24 0c 8b 7c EIP: [<c1064669>] kfree+0x49/0xc0 SS:ESP 0068:c0d06eec ---[ end trace 4f770856ce7e0713 ]--- Please note that in this moment, i have not queried the vis_data file, so the proc_vis_read_prim_sec call is probably not correctly reported. Furthermore there are some more stack traces in vfs/mount and then the boxes reboot - we probably write at some bad memory position. I've changed Andrews patch a little bit, and could not crash my 9 kvm instances when running it some hours last night. I've added kref_get() calls when the info packets get referenced from the send_list, this reference is removed with kref_put() in the send_vis_packets() after sending the packet. The disadvantage of this solution is that in a race condition, we might (unnecessarily) send out an old packet. No memory leaks as far as i could see after running some hours. Please give it a try (patch is attached to this e-mail)! best regards, Simon On Thu, Feb 11, 2010 at 11:01:56AM +0100, Andrew Lunn wrote: > On Thu, Feb 11, 2010 at 10:46:59AM +0100, Andrew Lunn wrote: > > Hi Linus > > > > Here is a new version of the patch. I've tested it this time using > > five UML machines. It should not immediately opps now. > > Instead is will leak memory and crash after a while... > > I will try to find the memory leak. > > Andrew > [-- Attachment #1.2: vis_refcount.patch --] [-- Type: text/x-diff, Size: 13632 bytes --] taging: batman-adv: Don't have interrupts disabled while sending. send_vis_packets() would disable interrupts before calling dev_queue_xmit() which resulting in a backtrace in local_bh_enable(). Fix this by using kref on the vis_info object so that we can call send_vis_packets() without holding vis_hash_lock. vis_hash_lock also used to protect recv_list, so we now need a new lock to protect that instead of vis_hash_lock. Also a few checkpatch cleanups. Reported-by: Linus Luessing <linus.luessing@web.de> Signed-off-by: Andrew Lunn <andrew@lunn.ch> Signed-off-by: Simon Wunderlich <siwu@hrz.tu-chemnitz.de> =================================================================== --- batman-adv-kernelland/vis.c (revision 1578) +++ batman-adv-kernelland/vis.c (working copy) @@ -30,22 +30,26 @@ struct hashtable_t *vis_hash; DEFINE_SPINLOCK(vis_hash_lock); +static DEFINE_SPINLOCK(recv_list_lock); static struct vis_info *my_vis_info; static struct list_head send_list; /* always locked with vis_hash_lock */ static void start_vis_timer(void); /* free the info */ -static void free_info(void *data) +static void free_info(struct kref *ref) { - struct vis_info *info = data; + struct vis_info *info = container_of(ref, struct vis_info, refcount); struct recvlist_node *entry, *tmp; + unsigned long flags; list_del_init(&info->send_list); + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry_safe(entry, tmp, &info->recv_list, list) { list_del(&entry->list); kfree(entry); } + spin_unlock_irqrestore(&recv_list_lock, flags); kfree(info); } @@ -147,32 +151,41 @@ static void recv_list_add(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC); if (!entry) return; memcpy(entry->mac, mac, ETH_ALEN); + spin_lock_irqsave(&recv_list_lock, flags); list_add_tail(&entry->list, recv_list); + spin_unlock_irqrestore(&recv_list_lock, flags); } /* returns 1 if this mac is in the recv_list */ static int recv_list_is_in(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry(entry, recv_list, list) { - if (memcmp(entry->mac, mac, ETH_ALEN) == 0) + if (memcmp(entry->mac, mac, ETH_ALEN) == 0) { + spin_unlock_irqrestore(&recv_list_lock, flags); return 1; + } } - + spin_unlock_irqrestore(&recv_list_lock, flags); return 0; } /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old, - * broken.. ). vis hash must be locked outside. is_new is set when the packet + * broken.. ). vis hash must be locked outside. is_new is set when the packet * is newer than old entries in the hash. */ static struct vis_info *add_packet(struct vis_packet *vis_packet, - int vis_info_len, int *is_new) + int vis_info_len, int *is_new, + int make_broadcast) { struct vis_info *info, *old_info; struct vis_info search_elem; @@ -199,13 +212,14 @@ } /* remove old entry */ hash_remove(vis_hash, old_info); - free_info(old_info); + kref_put(&old_info->refcount, free_info); } info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC); if (info == NULL) return NULL; + kref_init(&info->refcount); INIT_LIST_HEAD(&info->send_list); INIT_LIST_HEAD(&info->recv_list); info->first_seen = jiffies; @@ -215,16 +229,21 @@ /* initialize and add new packet. */ *is_new = 1; + /* Make it a broadcast packet, if required */ + if (make_broadcast) + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); + /* repair if entries is longer than packet. */ if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len) - info->packet.entries = vis_info_len / sizeof(struct vis_info_entry); + info->packet.entries = vis_info_len / + sizeof(struct vis_info_entry); recv_list_add(&info->recv_list, info->packet.sender_orig); /* try to add it */ if (hash_add(vis_hash, info) < 0) { /* did not work (for some reason) */ - free_info(info); + kref_put(&old_info->refcount, free_info); info = NULL; } @@ -235,21 +254,24 @@ void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len) { struct vis_info *info; - int is_new; + int is_new, make_broadcast; unsigned long flags; int vis_server = atomic_read(&vis_mode); + make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC); + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast); if (info == NULL) goto end; /* only if we are server ourselves and packet is newer than the one in * hash.*/ if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) { - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - if (list_empty(&info->send_list)) + if (list_empty(&info->send_list)) { + kref_get(&info->refcount); list_add_tail(&info->send_list, &send_list); + } } end: spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -263,31 +285,38 @@ int is_new; unsigned long flags; int vis_server = atomic_read(&vis_mode); + int are_target = 0; /* clients shall not broadcast. */ if (is_bcast(vis_packet->target_orig)) return; + /* Are we the target for this VIS packet? */ + if (vis_server == VIS_TYPE_SERVER_SYNC && + is_my_mac(vis_packet->target_orig)) + are_target = 1; + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, are_target); if (info == NULL) goto end; /* note that outdated packets will be dropped at this point. */ /* send only if we're the target server or ... */ - if (vis_server == VIS_TYPE_SERVER_SYNC && - is_my_mac(info->packet.target_orig) && - is_new) { + if (are_target && is_new) { info->packet.vis_type = VIS_TYPE_SERVER_SYNC; /* upgrade! */ - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - if (list_empty(&info->send_list)) + if (list_empty(&info->send_list)) { + kref_get(&info->refcount); list_add_tail(&info->send_list, &send_list); + } /* ... we're not the recipient (and thus need to forward). */ } else if (!is_my_mac(info->packet.target_orig)) { - if (list_empty(&info->send_list)) + if (list_empty(&info->send_list)) { + kref_get(&info->refcount); list_add_tail(&info->send_list, &send_list); + } } end: spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -362,14 +391,17 @@ while (hash_iterate(orig_hash, &hashit_global)) { orig_node = hashit_global.bucket->data; if (orig_node->router != NULL - && compare_orig(orig_node->router->addr, orig_node->orig) + && compare_orig(orig_node->router->addr, + orig_node->orig) && orig_node->batman_if && (orig_node->batman_if->if_active == IF_ACTIVE) && orig_node->router->tq_avg > 0) { /* fill one entry into buffer. */ entry = &entry_array[info->packet.entries]; - memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN); + memcpy(entry->src, + orig_node->batman_if->net_dev->dev_addr, + ETH_ALEN); memcpy(entry->dest, orig_node->orig, ETH_ALEN); entry->quality = orig_node->router->tq_avg; info->packet.entries++; @@ -401,6 +433,8 @@ return 0; } +/* free old vis packets. Must be called with this vis_hash_lock + * held */ static void purge_vis_packets(void) { HASHIT(hashit); @@ -413,7 +447,7 @@ if (time_after(jiffies, info->first_seen + (VIS_TIMEOUT*HZ)/1000)) { hash_remove_bucket(vis_hash, &hashit); - free_info(info); + kref_put(&info->refcount, free_info); } } } @@ -423,6 +457,8 @@ HASHIT(hashit); struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); @@ -431,46 +467,57 @@ orig_node = hashit.bucket->data; /* if it's a vis server and reachable, send it. */ - if (orig_node && - (orig_node->flags & VIS_SERVER) && - orig_node->batman_if && - orig_node->router) { + if ((!orig_node) || (!orig_node->batman_if) || + (!orig_node->router)) + continue; + if (!(orig_node->flags & VIS_SERVER)) + continue; + /* don't send it if we already received the packet from + * this node. */ + if (recv_list_is_in(&info->recv_list, orig_node->orig)) + continue; - /* don't send it if we already received the packet from - * this node. */ - if (recv_list_is_in(&info->recv_list, orig_node->orig)) - continue; + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); + spin_unlock_irqrestore(&orig_hash_lock, flags); - memcpy(info->packet.target_orig, - orig_node->orig, ETH_ALEN); + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); - send_raw_packet((unsigned char *) &info->packet, - packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + spin_lock_irqsave(&orig_hash_lock, flags); + } + spin_unlock_irqrestore(&orig_hash_lock, flags); memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - spin_unlock_irqrestore(&orig_hash_lock, flags); } static void unicast_vis_packet(struct vis_info *info, int packet_length) { struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); orig_node = ((struct orig_node *) hash_find(orig_hash, info->packet.target_orig)); - if ((orig_node != NULL) && - (orig_node->batman_if != NULL) && - (orig_node->router != NULL)) { - send_raw_packet((unsigned char *) &info->packet, packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) + goto out; + + /* don't lock while sending the packets ... we therefore + * copy the required data before sending */ + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); spin_unlock_irqrestore(&orig_hash_lock, flags); + + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); + return; + +out: + spin_unlock_irqrestore(&orig_hash_lock, flags); } /* only send one vis packet. called from send_vis_packets() */ @@ -503,15 +550,23 @@ unsigned long flags; spin_lock_irqsave(&vis_hash_lock, flags); + purge_vis_packets(); - if (generate_vis_packet() == 0) + if (generate_vis_packet() == 0) { /* schedule if generation was successful */ + kref_get(&my_vis_info->refcount); list_add_tail(&my_vis_info->send_list, &send_list); + } list_for_each_entry_safe(info, temp, &send_list, send_list) { + spin_unlock_irqrestore(&vis_hash_lock, flags); + + send_vis_packet(info); + + spin_lock_irqsave(&vis_hash_lock, flags); list_del_init(&info->send_list); - send_vis_packet(info); + kref_put(&info->refcount, free_info); } spin_unlock_irqrestore(&vis_hash_lock, flags); start_vis_timer(); @@ -544,6 +599,7 @@ my_vis_info->first_seen = jiffies - atomic_read(&vis_interval); INIT_LIST_HEAD(&my_vis_info->recv_list); INIT_LIST_HEAD(&my_vis_info->send_list); + kref_init(&my_vis_info->refcount); my_vis_info->packet.version = COMPAT_VERSION; my_vis_info->packet.packet_type = BAT_VIS; my_vis_info->packet.ttl = TTL; @@ -557,9 +613,9 @@ if (hash_add(vis_hash, my_vis_info) < 0) { printk(KERN_ERR - "batman-adv:Can't add own vis packet into hash\n"); - free_info(my_vis_info); /* not in hash, need to remove it - * manually. */ + "batman-adv:Can't add own vis packet into hash\n"); + /* not in hash, need to remove it manually. */ + kref_put(&my_vis_info->refcount, free_info); goto err; } @@ -573,6 +629,13 @@ return 0; } +/* Decrease the reference count on a hash item info */ +static void free_info_ref(void *data) +{ + struct vis_info *info = data; + kref_put(&info->refcount, free_info); +} + /* shutdown vis-server */ void vis_quit(void) { @@ -584,7 +647,7 @@ spin_lock_irqsave(&vis_hash_lock, flags); /* properly remove, kill timers ... */ - hash_delete(vis_hash, free_info); + hash_delete(vis_hash, free_info_ref); vis_hash = NULL; my_vis_info = NULL; spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -594,5 +657,5 @@ static void start_vis_timer(void) { queue_delayed_work(bat_event_workqueue, &vis_timer_wq, - (atomic_read(&vis_interval) * HZ ) / 1000); + (atomic_read(&vis_interval) * HZ) / 1000); } Index: batman-adv-kernelland/vis.h =================================================================== --- batman-adv-kernelland/vis.h (revision 1578) +++ batman-adv-kernelland/vis.h (working copy) @@ -29,6 +29,7 @@ /* list of server-neighbors we received a vis-packet * from. we should not reply to them. */ struct list_head send_list; + struct kref refcount; /* this packet might be part of the vis send queue. */ struct vis_packet packet; /* vis_info may follow here*/ [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-02-28 16:34 ` Simon Wunderlich @ 2010-03-01 5:59 ` Andrew Lunn 2010-03-01 16:57 ` Simon Wunderlich 0 siblings, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-03-01 5:59 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Hi Simon > @@ -503,15 +550,23 @@ > unsigned long flags; > > spin_lock_irqsave(&vis_hash_lock, flags); > + > purge_vis_packets(); > > - if (generate_vis_packet() == 0) > + if (generate_vis_packet() == 0) { > /* schedule if generation was successful */ > + kref_get(&my_vis_info->refcount); > list_add_tail(&my_vis_info->send_list, &send_list); > + } > > list_for_each_entry_safe(info, temp, &send_list, send_list) { > + spin_unlock_irqrestore(&vis_hash_lock, flags); > + > + send_vis_packet(info); > + > + spin_lock_irqsave(&vis_hash_lock, flags); > list_del_init(&info->send_list); > - send_vis_packet(info); > + kref_put(&info->refcount, free_info); > } > spin_unlock_irqrestore(&vis_hash_lock, flags); > start_vis_timer(); Although you say this works, i just looks wrong. It looks unbalanced. Where is the kref_put for my_vis_info->refcount? There is a kref_put inside what looks like a loop, but no kref_get inside the loop. Can you explain this is more detail? Can we make it look correctly balanced? Thanks Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-01 5:59 ` Andrew Lunn @ 2010-03-01 16:57 ` Simon Wunderlich 2010-03-02 6:43 ` Andrew Lunn 0 siblings, 1 reply; 37+ messages in thread From: Simon Wunderlich @ 2010-03-01 16:57 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1.1: Type: text/plain, Size: 1850 bytes --] Hey Andrew, all list-adds and list-removes do a kref_get or kref_put respectively, but that probably was not very clear. As soon as a refence is created (by linking into the hash or a list), we should also kref_get() it. Please find attached a new version of this patch with separate send_list_add/remove functions which include the put/get functions. I've also added the kref_put/get call to the send_list loop again. It should look more balanced now. Tested in my 9 qemu setup again, no memory leaks or crashes but i only did a very quick test (15 minutes) best regards, Simon On Mon, Mar 01, 2010 at 06:59:55AM +0100, Andrew Lunn wrote: > Hi Simon > > > @@ -503,15 +550,23 @@ > > unsigned long flags; > > > > spin_lock_irqsave(&vis_hash_lock, flags); > > + > > purge_vis_packets(); > > > > - if (generate_vis_packet() == 0) > > + if (generate_vis_packet() == 0) { > > /* schedule if generation was successful */ > > + kref_get(&my_vis_info->refcount); > > list_add_tail(&my_vis_info->send_list, &send_list); > > + } > > > > list_for_each_entry_safe(info, temp, &send_list, send_list) { > > + spin_unlock_irqrestore(&vis_hash_lock, flags); > > + > > + send_vis_packet(info); > > + > > + spin_lock_irqsave(&vis_hash_lock, flags); > > list_del_init(&info->send_list); > > - send_vis_packet(info); > > + kref_put(&info->refcount, free_info); > > } > > spin_unlock_irqrestore(&vis_hash_lock, flags); > > start_vis_timer(); > > Although you say this works, i just looks wrong. It looks > unbalanced. Where is the kref_put for my_vis_info->refcount? There is > a kref_put inside what looks like a loop, but no kref_get inside the > loop. > > Can you explain this is more detail? Can we make it look correctly > balanced? > > Thanks > Andrew > [-- Attachment #1.2: vis_refcount2.patch --] [-- Type: text/x-diff, Size: 14283 bytes --] taging: batman-adv: Don't have interrupts disabled while sending. send_vis_packets() would disable interrupts before calling dev_queue_xmit() which resulting in a backtrace in local_bh_enable(). Fix this by using kref on the vis_info object so that we can call send_vis_packets() without holding vis_hash_lock. vis_hash_lock also used to protect recv_list, so we now need a new lock to protect that instead of vis_hash_lock. Also a few checkpatch cleanups. Reported-by: Linus Luessing <linus.luessing@web.de> Signed-off-by: Andrew Lunn <andrew@lunn.ch> Signed-off-by: Simon Wunderlich <siwu@hrz.tu-chemnitz.de> Index: a/batman-adv-kernelland/vis.c =================================================================== --- a/batman-adv-kernelland/vis.c (revision 1578) +++ a/batman-adv-kernelland/vis.c (working copy) @@ -30,22 +30,26 @@ struct hashtable_t *vis_hash; DEFINE_SPINLOCK(vis_hash_lock); +static DEFINE_SPINLOCK(recv_list_lock); static struct vis_info *my_vis_info; static struct list_head send_list; /* always locked with vis_hash_lock */ static void start_vis_timer(void); /* free the info */ -static void free_info(void *data) +static void free_info(struct kref *ref) { - struct vis_info *info = data; + struct vis_info *info = container_of(ref, struct vis_info, refcount); struct recvlist_node *entry, *tmp; + unsigned long flags; list_del_init(&info->send_list); + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry_safe(entry, tmp, &info->recv_list, list) { list_del(&entry->list); kfree(entry); } + spin_unlock_irqrestore(&recv_list_lock, flags); kfree(info); } @@ -143,36 +147,65 @@ } } +/* add the info packet to the send list, if it was not + * already linked in. */ +static void send_list_add(struct vis_info *info) +{ + if (list_empty(&info->send_list)) { + kref_get(&info->refcount); + list_add_tail(&info->send_list, &send_list); + } +} + +/* delete the info packet from the send list, if it was + * linked in. */ +static void send_list_del(struct vis_info *info) +{ + if (!list_empty(&info->send_list)) { + list_del_init(&info->send_list); + kref_put(&info->refcount, free_info); + } +} + /* tries to add one entry to the receive list. */ static void recv_list_add(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC); if (!entry) return; memcpy(entry->mac, mac, ETH_ALEN); + spin_lock_irqsave(&recv_list_lock, flags); list_add_tail(&entry->list, recv_list); + spin_unlock_irqrestore(&recv_list_lock, flags); } /* returns 1 if this mac is in the recv_list */ static int recv_list_is_in(struct list_head *recv_list, char *mac) { struct recvlist_node *entry; + unsigned long flags; + spin_lock_irqsave(&recv_list_lock, flags); list_for_each_entry(entry, recv_list, list) { - if (memcmp(entry->mac, mac, ETH_ALEN) == 0) + if (memcmp(entry->mac, mac, ETH_ALEN) == 0) { + spin_unlock_irqrestore(&recv_list_lock, flags); return 1; + } } - + spin_unlock_irqrestore(&recv_list_lock, flags); return 0; } /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old, - * broken.. ). vis hash must be locked outside. is_new is set when the packet + * broken.. ). vis hash must be locked outside. is_new is set when the packet * is newer than old entries in the hash. */ static struct vis_info *add_packet(struct vis_packet *vis_packet, - int vis_info_len, int *is_new) + int vis_info_len, int *is_new, + int make_broadcast) { struct vis_info *info, *old_info; struct vis_info search_elem; @@ -199,13 +232,15 @@ } /* remove old entry */ hash_remove(vis_hash, old_info); - free_info(old_info); + send_list_del(old_info); + kref_put(&old_info->refcount, free_info); } info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC); if (info == NULL) return NULL; + kref_init(&info->refcount); INIT_LIST_HEAD(&info->send_list); INIT_LIST_HEAD(&info->recv_list); info->first_seen = jiffies; @@ -215,16 +250,21 @@ /* initialize and add new packet. */ *is_new = 1; + /* Make it a broadcast packet, if required */ + if (make_broadcast) + memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); + /* repair if entries is longer than packet. */ if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len) - info->packet.entries = vis_info_len / sizeof(struct vis_info_entry); + info->packet.entries = vis_info_len / + sizeof(struct vis_info_entry); recv_list_add(&info->recv_list, info->packet.sender_orig); /* try to add it */ if (hash_add(vis_hash, info) < 0) { /* did not work (for some reason) */ - free_info(info); + kref_put(&old_info->refcount, free_info); info = NULL; } @@ -235,22 +275,21 @@ void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len) { struct vis_info *info; - int is_new; + int is_new, make_broadcast; unsigned long flags; int vis_server = atomic_read(&vis_mode); + make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC); + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast); if (info == NULL) goto end; /* only if we are server ourselves and packet is newer than the one in * hash.*/ - if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) { - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - if (list_empty(&info->send_list)) - list_add_tail(&info->send_list, &send_list); - } + if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) + send_list_add(info); end: spin_unlock_irqrestore(&vis_hash_lock, flags); } @@ -263,31 +302,32 @@ int is_new; unsigned long flags; int vis_server = atomic_read(&vis_mode); + int are_target = 0; /* clients shall not broadcast. */ if (is_bcast(vis_packet->target_orig)) return; + /* Are we the target for this VIS packet? */ + if (vis_server == VIS_TYPE_SERVER_SYNC && + is_my_mac(vis_packet->target_orig)) + are_target = 1; + spin_lock_irqsave(&vis_hash_lock, flags); - info = add_packet(vis_packet, vis_info_len, &is_new); + info = add_packet(vis_packet, vis_info_len, &is_new, are_target); if (info == NULL) goto end; /* note that outdated packets will be dropped at this point. */ /* send only if we're the target server or ... */ - if (vis_server == VIS_TYPE_SERVER_SYNC && - is_my_mac(info->packet.target_orig) && - is_new) { + if (are_target && is_new) { info->packet.vis_type = VIS_TYPE_SERVER_SYNC; /* upgrade! */ - memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - if (list_empty(&info->send_list)) - list_add_tail(&info->send_list, &send_list); + send_list_add(info); /* ... we're not the recipient (and thus need to forward). */ } else if (!is_my_mac(info->packet.target_orig)) { - if (list_empty(&info->send_list)) - list_add_tail(&info->send_list, &send_list); + send_list_add(info); } end: spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -362,14 +402,17 @@ while (hash_iterate(orig_hash, &hashit_global)) { orig_node = hashit_global.bucket->data; if (orig_node->router != NULL - && compare_orig(orig_node->router->addr, orig_node->orig) + && compare_orig(orig_node->router->addr, + orig_node->orig) && orig_node->batman_if && (orig_node->batman_if->if_active == IF_ACTIVE) && orig_node->router->tq_avg > 0) { /* fill one entry into buffer. */ entry = &entry_array[info->packet.entries]; - memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN); + memcpy(entry->src, + orig_node->batman_if->net_dev->dev_addr, + ETH_ALEN); memcpy(entry->dest, orig_node->orig, ETH_ALEN); entry->quality = orig_node->router->tq_avg; info->packet.entries++; @@ -401,6 +444,8 @@ return 0; } +/* free old vis packets. Must be called with this vis_hash_lock + * held */ static void purge_vis_packets(void) { HASHIT(hashit); @@ -413,7 +458,8 @@ if (time_after(jiffies, info->first_seen + (VIS_TIMEOUT*HZ)/1000)) { hash_remove_bucket(vis_hash, &hashit); - free_info(info); + send_list_del(info); + kref_put(&info->refcount, free_info); } } } @@ -423,6 +469,8 @@ HASHIT(hashit); struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); @@ -431,46 +479,57 @@ orig_node = hashit.bucket->data; /* if it's a vis server and reachable, send it. */ - if (orig_node && - (orig_node->flags & VIS_SERVER) && - orig_node->batman_if && - orig_node->router) { + if ((!orig_node) || (!orig_node->batman_if) || + (!orig_node->router)) + continue; + if (!(orig_node->flags & VIS_SERVER)) + continue; + /* don't send it if we already received the packet from + * this node. */ + if (recv_list_is_in(&info->recv_list, orig_node->orig)) + continue; - /* don't send it if we already received the packet from - * this node. */ - if (recv_list_is_in(&info->recv_list, orig_node->orig)) - continue; + memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN); + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); + spin_unlock_irqrestore(&orig_hash_lock, flags); - memcpy(info->packet.target_orig, - orig_node->orig, ETH_ALEN); + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); - send_raw_packet((unsigned char *) &info->packet, - packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + spin_lock_irqsave(&orig_hash_lock, flags); + } + spin_unlock_irqrestore(&orig_hash_lock, flags); memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN); - spin_unlock_irqrestore(&orig_hash_lock, flags); } static void unicast_vis_packet(struct vis_info *info, int packet_length) { struct orig_node *orig_node; unsigned long flags; + struct batman_if *batman_if; + uint8_t dstaddr[ETH_ALEN]; spin_lock_irqsave(&orig_hash_lock, flags); orig_node = ((struct orig_node *) hash_find(orig_hash, info->packet.target_orig)); - if ((orig_node != NULL) && - (orig_node->batman_if != NULL) && - (orig_node->router != NULL)) { - send_raw_packet((unsigned char *) &info->packet, packet_length, - orig_node->batman_if, - orig_node->router->addr); - } + if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router)) + goto out; + + /* don't lock while sending the packets ... we therefore + * copy the required data before sending */ + batman_if = orig_node->batman_if; + memcpy(dstaddr, orig_node->router->addr, ETH_ALEN); spin_unlock_irqrestore(&orig_hash_lock, flags); + + send_raw_packet((unsigned char *)&info->packet, + packet_length, batman_if, dstaddr); + return; + +out: + spin_unlock_irqrestore(&orig_hash_lock, flags); } /* only send one vis packet. called from send_vis_packets() */ @@ -503,15 +562,24 @@ unsigned long flags; spin_lock_irqsave(&vis_hash_lock, flags); + purge_vis_packets(); - if (generate_vis_packet() == 0) + if (generate_vis_packet() == 0) { /* schedule if generation was successful */ - list_add_tail(&my_vis_info->send_list, &send_list); + send_list_add(my_vis_info); + } list_for_each_entry_safe(info, temp, &send_list, send_list) { - list_del_init(&info->send_list); + + kref_get(&info->refcount); + spin_unlock_irqrestore(&vis_hash_lock, flags); + send_vis_packet(info); + + spin_lock_irqsave(&vis_hash_lock, flags); + send_list_del(info); + kref_put(&info->refcount, free_info); } spin_unlock_irqrestore(&vis_hash_lock, flags); start_vis_timer(); @@ -544,6 +612,7 @@ my_vis_info->first_seen = jiffies - atomic_read(&vis_interval); INIT_LIST_HEAD(&my_vis_info->recv_list); INIT_LIST_HEAD(&my_vis_info->send_list); + kref_init(&my_vis_info->refcount); my_vis_info->packet.version = COMPAT_VERSION; my_vis_info->packet.packet_type = BAT_VIS; my_vis_info->packet.ttl = TTL; @@ -557,9 +626,9 @@ if (hash_add(vis_hash, my_vis_info) < 0) { printk(KERN_ERR - "batman-adv:Can't add own vis packet into hash\n"); - free_info(my_vis_info); /* not in hash, need to remove it - * manually. */ + "batman-adv:Can't add own vis packet into hash\n"); + /* not in hash, need to remove it manually. */ + kref_put(&my_vis_info->refcount, free_info); goto err; } @@ -573,6 +642,15 @@ return 0; } +/* Decrease the reference count on a hash item info */ +static void free_info_ref(void *data) +{ + struct vis_info *info = data; + + send_list_del(info); + kref_put(&info->refcount, free_info); +} + /* shutdown vis-server */ void vis_quit(void) { @@ -584,7 +662,7 @@ spin_lock_irqsave(&vis_hash_lock, flags); /* properly remove, kill timers ... */ - hash_delete(vis_hash, free_info); + hash_delete(vis_hash, free_info_ref); vis_hash = NULL; my_vis_info = NULL; spin_unlock_irqrestore(&vis_hash_lock, flags); @@ -594,5 +672,5 @@ static void start_vis_timer(void) { queue_delayed_work(bat_event_workqueue, &vis_timer_wq, - (atomic_read(&vis_interval) * HZ ) / 1000); + (atomic_read(&vis_interval) * HZ) / 1000); } Index: a/batman-adv-kernelland/vis.h =================================================================== --- a/batman-adv-kernelland/vis.h (revision 1578) +++ a/batman-adv-kernelland/vis.h (working copy) @@ -29,6 +29,7 @@ /* list of server-neighbors we received a vis-packet * from. we should not reply to them. */ struct list_head send_list; + struct kref refcount; /* this packet might be part of the vis send queue. */ struct vis_packet packet; /* vis_info may follow here*/ [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-01 16:57 ` Simon Wunderlich @ 2010-03-02 6:43 ` Andrew Lunn 2010-03-02 21:13 ` Simon Wunderlich 0 siblings, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-03-02 6:43 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Mon, Mar 01, 2010 at 05:57:06PM +0100, Simon Wunderlich wrote: > Hey Andrew, > > all list-adds and list-removes do a kref_get or kref_put respectively, > but that probably was not very clear. As soon as a refence is created > (by linking into the hash or a list), we should also kref_get() it. > > Please find attached a new version of this patch with separate > send_list_add/remove functions which include the put/get functions. > I've also added the kref_put/get call to the send_list loop again. > It should look more balanced now. Tested in my 9 qemu setup again, > no memory leaks or crashes but i only did a very quick test > (15 minutes) Hi Simon This looks better. Did it survive longer testing? Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-02 6:43 ` Andrew Lunn @ 2010-03-02 21:13 ` Simon Wunderlich 2010-03-02 21:26 ` elektra 2010-03-02 21:44 ` Linus Lüssing 0 siblings, 2 replies; 37+ messages in thread From: Simon Wunderlich @ 2010-03-02 21:13 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 1132 bytes --] Hello Andrew, no, sorry, and i won't be able to do it this week as i am at the cebit this whole week (if anyone else is there, please tell me. ;). I would therefore like to ask you and Linus to test it as well, i will perform more tests on sunday. Thanks, Simon On Tue, Mar 02, 2010 at 07:43:08AM +0100, Andrew Lunn wrote: > On Mon, Mar 01, 2010 at 05:57:06PM +0100, Simon Wunderlich wrote: > > Hey Andrew, > > > > all list-adds and list-removes do a kref_get or kref_put respectively, > > but that probably was not very clear. As soon as a refence is created > > (by linking into the hash or a list), we should also kref_get() it. > > > > Please find attached a new version of this patch with separate > > send_list_add/remove functions which include the put/get functions. > > I've also added the kref_put/get call to the send_list loop again. > > It should look more balanced now. Tested in my 9 qemu setup again, > > no memory leaks or crashes but i only did a very quick test > > (15 minutes) > > Hi Simon > > This looks better. Did it survive longer testing? > > Andrew > [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 197 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-02 21:13 ` Simon Wunderlich @ 2010-03-02 21:26 ` elektra 2010-03-02 21:44 ` Linus Lüssing 1 sibling, 0 replies; 37+ messages in thread From: elektra @ 2010-03-02 21:26 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking Hello Simon - I'll be in hall 13 at the booth of Atcom E42 to present the Mesh-Potato router. (With B.A.T.M.A.N. inside, of course) Will arrive tomorrow afternoon and be there until Saturday 18:00. Cheers, Elektra > no, sorry, and i won't be able to do it this week as i am at the cebit > this whole week (if anyone else is there, please tell me. ;). ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-02 21:13 ` Simon Wunderlich 2010-03-02 21:26 ` elektra @ 2010-03-02 21:44 ` Linus Lüssing 2010-03-04 0:26 ` Linus Lüssing 1 sibling, 1 reply; 37+ messages in thread From: Linus Lüssing @ 2010-03-02 21:44 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 1959 bytes --] Hi Simon and Andrew, I already tried both Andrew's intial and Simon's latest patch yesterday and had both of them running for about 1h+, also with alternate vis-intervals (1s, 50ms, 30s) but so far I wasn't able to reproduce any crashes or leaks on two Debian stable VMs connected with each other. I'll run Andrew's patch version again this night for some more hours to make sure that I'm able to reproduce the crash at all. Any other hints for reproducing it more quickly welcome :). Cheers, Linus PS: With how many nodes and what kind of systems could you reproduce the crash? Were all those nodes connected directly with each other? How many vis-servers were running? On Tue, Mar 02, 2010 at 10:13:24PM +0100, Simon Wunderlich wrote: > Hello Andrew, > > no, sorry, and i won't be able to do it this week as i am at the cebit > this whole week (if anyone else is there, please tell me. ;). > > I would therefore like to ask you and Linus to test it as well, i will > perform more tests on sunday. > > Thanks, > Simon > > On Tue, Mar 02, 2010 at 07:43:08AM +0100, Andrew Lunn wrote: > > On Mon, Mar 01, 2010 at 05:57:06PM +0100, Simon Wunderlich wrote: > > > Hey Andrew, > > > > > > all list-adds and list-removes do a kref_get or kref_put respectively, > > > but that probably was not very clear. As soon as a refence is created > > > (by linking into the hash or a list), we should also kref_get() it. > > > > > > Please find attached a new version of this patch with separate > > > send_list_add/remove functions which include the put/get functions. > > > I've also added the kref_put/get call to the send_list loop again. > > > It should look more balanced now. Tested in my 9 qemu setup again, > > > no memory leaks or crashes but i only did a very quick test > > > (15 minutes) > > > > Hi Simon > > > > This looks better. Did it survive longer testing? > > > > Andrew > > [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-02 21:44 ` Linus Lüssing @ 2010-03-04 0:26 ` Linus Lüssing 2010-03-04 8:57 ` Andrew Lunn 0 siblings, 1 reply; 37+ messages in thread From: Linus Lüssing @ 2010-03-04 0:26 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking [-- Attachment #1: Type: text/plain, Size: 306 bytes --] I've been running 3 qemu x86 instances with Debian stable here (2.6.26 kernel) as well as 3 Atheros ap61 / mips based routers with OpenWRT (rev. 19468, 2.6.30.10 kernel) for 22 hours. None of them crashed with Andrew's patch... so, sorry, still can't reproduce the bug you're talking about. Cheers, Linus [-- Attachment #2: Digital signature --] [-- Type: application/pgp-signature, Size: 836 bytes --] ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-04 0:26 ` Linus Lüssing @ 2010-03-04 8:57 ` Andrew Lunn 2010-03-04 9:19 ` Marek Lindner 0 siblings, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-03-04 8:57 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Thu, Mar 04, 2010 at 01:26:20AM +0100, Linus L??ssing wrote: > I've been running 3 qemu x86 instances with Debian stable here > (2.6.26 kernel) as well as 3 Atheros ap61 / mips based routers with > OpenWRT (rev. 19468, 2.6.30.10 kernel) for 22 hours. None of them > crashed with Andrew's patch... so, sorry, still can't reproduce the > bug you're talking about. If you cannot reproduce it, its probably fixed :-) Marek, could you commit the patch. Thanks Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-04 8:57 ` Andrew Lunn @ 2010-03-04 9:19 ` Marek Lindner 2010-03-04 9:49 ` Andrew Lunn 0 siblings, 1 reply; 37+ messages in thread From: Marek Lindner @ 2010-03-04 9:19 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Thursday 04 March 2010 16:57:24 Andrew Lunn wrote: > On Thu, Mar 04, 2010 at 01:26:20AM +0100, Linus L??ssing wrote: > > I've been running 3 qemu x86 instances with Debian stable here > > (2.6.26 kernel) as well as 3 Atheros ap61 / mips based routers with > > OpenWRT (rev. 19468, 2.6.30.10 kernel) for 22 hours. None of them > > crashed with Andrew's patch... so, sorry, still can't reproduce the > > bug you're talking about. > > If you cannot reproduce it, its probably fixed :-) > > Marek, could you commit the patch. Sure but which one ? I have to admit I lost the overview (there were too many patches flying around) .. :-) Cheers, Marek ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-04 9:19 ` Marek Lindner @ 2010-03-04 9:49 ` Andrew Lunn 2010-03-04 10:00 ` Marek Lindner 0 siblings, 1 reply; 37+ messages in thread From: Andrew Lunn @ 2010-03-04 9:49 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking > Sure but which one ? I have to admit I lost the overview (there were > too many patches flying around) .. :-) https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2010-March/002354.html Andrew ^ permalink raw reply [flat|nested] 37+ messages in thread
* Re: [B.A.T.M.A.N.] slowpath warning 2010-03-04 9:49 ` Andrew Lunn @ 2010-03-04 10:00 ` Marek Lindner 0 siblings, 0 replies; 37+ messages in thread From: Marek Lindner @ 2010-03-04 10:00 UTC (permalink / raw) To: The list for a Better Approach To Mobile Ad-hoc Networking On Thursday 04 March 2010 17:49:41 Andrew Lunn wrote: > > Sure but which one ? I have to admit I lost the overview (there were > > too many patches flying around) .. :-) > > https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2010-March/002354.html Applied in rev 1579, Regards, Marek ^ permalink raw reply [flat|nested] 37+ messages in thread
end of thread, other threads:[~2010-03-04 10:00 UTC | newest] Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing 2010-01-23 18:10 ` Andrew Lunn 2010-01-23 23:30 ` Linus Lüssing 2010-01-24 20:42 ` Linus Lüssing 2010-01-24 21:00 ` Andrew Lunn 2010-01-25 6:46 ` Andrew Lunn 2010-01-25 6:47 ` Andrew Lunn 2010-01-25 8:21 ` Marek Lindner 2010-01-26 1:48 ` Linus Lüssing 2010-01-24 4:24 ` Linus Lüssing 2010-01-26 6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing 2010-01-26 7:16 ` Marek Lindner 2010-01-27 0:10 ` Linus Lüssing 2010-01-28 0:09 ` Marek Lindner 2010-01-28 6:29 ` Andrew Lunn 2010-01-29 8:25 ` Andrew Lunn 2010-01-29 8:59 ` Marek Lindner 2010-01-30 16:50 ` Andrew Lunn 2010-01-31 19:37 ` Linus Lüssing 2010-01-31 20:56 ` Andrew Lunn 2010-02-11 9:46 ` Andrew Lunn 2010-02-11 10:01 ` Andrew Lunn 2010-02-19 17:19 ` Linus Lüssing 2010-02-20 18:04 ` Andrew Lunn 2010-02-21 13:10 ` Linus Lüssing 2010-02-28 16:34 ` Simon Wunderlich 2010-03-01 5:59 ` Andrew Lunn 2010-03-01 16:57 ` Simon Wunderlich 2010-03-02 6:43 ` Andrew Lunn 2010-03-02 21:13 ` Simon Wunderlich 2010-03-02 21:26 ` elektra 2010-03-02 21:44 ` Linus Lüssing 2010-03-04 0:26 ` Linus Lüssing 2010-03-04 8:57 ` Andrew Lunn 2010-03-04 9:19 ` Marek Lindner 2010-03-04 9:49 ` Andrew Lunn 2010-03-04 10:00 ` Marek Lindner
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).