b.a.t.m.a.n.lists.open-mesh.org archive mirror
 help / color / mirror / Atom feed
* [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
@ 2010-01-23 17:46 Linus Lüssing
  2010-01-23 18:10 ` Andrew Lunn
                   ` (2 more replies)
  0 siblings, 3 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-01-23 17:46 UTC (permalink / raw)
  To: b.a.t.m.a.n

[-- Attachment #1: Type: text/plain, Size: 690 bytes --]

Hey guys,

I've been installing a 9 node setup here in our cellar. They are
all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current
batman-adv maintance version in OpenWRT).

The result over night:
- 1x: bat_events: page allocation failure
- 3x: WARNING: at net/sched/sch_generic.c:226 0x801c3554()
	(those 3 had TQ values of about 10-15 in batctl)

For the last error, I'm not sure if this is related to batman-adv
or to the madwifi driver. I was using OpenWRT trunk 18405 on all
routers.

I'm attaching some backtraces. Sorry again I can't get more out of
them than that with ksymoops. The first two ones are the
bat_events problem, the others the scheduling thingy.

Cheers, Linus

[-- Attachment #2: kernel-panic-1752-23-01-10.log --]
[-- Type: text/plain, Size: 14940 bytes --]

root@OpenWrt:~# dmesg
01a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu: 
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM  
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<8022329c>] 0x8022329c
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu: 
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM  
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<8022329c>] 0x8022329c
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu: 
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM  
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu: 
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM  
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
699 pages reserved
1395 pages shared
2630 pages non-shared
wifi0: ath_uapsd_processtriggers: Dropping; skb is NULL in received ath_buf.
wifi0: ath_uapsd_processtriggers: Dropping; skb is NULL in received ath_buf.
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<8007d3e4>] 0x8007d3e4
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<80041444>] 0x80041444
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
699 pages reserved
1395 pages shared
2630 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<8018c538>] 0x8018c538
[<800bcf78>] 0x800bcf78
[<800bd450>] 0x800bd450
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<80074174>] 0x80074174
[<80b42070>] 0x80b42070
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:149 active_file:564 inactive_anon:159
 inactive_file:828 dirty:0 writeback:0 unstable:0
 free:48 slab:1126 mapped:279 pagetables:55 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:596kB inactive_anon:636kB active_file:2256kB inactive_file:3312kB present:16256kB pages_scanned:0 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
1417 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
699 pages reserved
1395 pages shared
2630 pages non-shared
device VPN entered promiscuous mode
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 2(VPN) entering forwarding state
root@OpenWrt:~#

[-- Attachment #3: kernel-panic-1752-23-01-10.ksymoops --]
[-- Type: text/plain, Size: 22019 bytes --]

ksymoops 2.4.11 on x86_64 2.6.31-1-amd64.  Options used
     -v /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/vmlinux (specified)
     -K (specified)
     -L (specified)
     -o /home/linus/lff-test/kamikaze/build_dir/linux-atheros -o /home/linus/lff-test/kamikaze/build_dir/target-mips_uClibc-0.9.30.1 (specified)
     -m /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/System.map (specified)

No modules in ksyms, skipping objects
01a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
1417 total pagecache pages
4096 pages RAM  
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<8022329c>] 0x8022329c
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80074b5c <queue_delayed_work_on+f8/108>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 8051c178 <END_OF_CODE+266a48/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM  
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<8022329c>] 0x8022329c
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80074b5c <queue_delayed_work_on+f8/108>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 8051c178 <END_OF_CODE+266a48/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM  
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80074b5c <queue_delayed_work_on+f8/108>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 8051c178 <END_OF_CODE+266a48/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM  
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80074b5c <queue_delayed_work_on+f8/108>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 8051c178 <END_OF_CODE+266a48/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80074b5c <queue_delayed_work_on+f8/108>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 8051c178 <END_OF_CODE+266a48/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80074b5c>] 0x80074b5c
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<8051c178>] 0x8051c178
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<8022329c>] 0x8022329c
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<8022329c>] 0x8022329c
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80074b5c <queue_delayed_work_on+f8/108>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 8051c178 <END_OF_CODE+266a48/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 8022329c <br_handle_frame_finish+0/1a4>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<8007d3e4>] 0x8007d3e4
[<800bcf78>] 0x800bcf78
[<800bd538>] 0x800bd538
[<80b41a58>] 0x80b41a58
[<80b42674>] 0x80b42674
[<80b479f8>] 0x80b479f8
[<8052129c>] 0x8052129c
[<80525a20>] 0x80525a20
[<800bd450>] 0x800bd450
[<80041444>] 0x80041444
[<805269e0>] 0x805269e0
[<80525ed8>] 0x80525ed8
[<80503f50>] 0x80503f50
[<8051910c>] 0x8051910c
[<801b0fb8>] 0x801b0fb8
[<801ae580>] 0x801ae580
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<8009ac90>] 0x8009ac90
[<80041444>] 0x80041444
[<801784a0>] 0x801784a0
[<800bd450>] 0x800bd450
[<800bd470>] 0x800bd470
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<8018c4b8>] 0x8018c4b8
[<80b42088>] 0x80b42088
[<80b42070>] 0x80b42070
[<80b419d4>] 0x80b419d4
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 8007d3e4 <ktime_get+18/3c>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd538 <kmem_cache_alloc+78/e8>
Trace; 80b41a58 <END_OF_CODE+88c328/????>
Trace; 80b42674 <END_OF_CODE+88cf44/????>
Trace; 80b479f8 <END_OF_CODE+8922c8/????>
Trace; 8052129c <END_OF_CODE+26bb6c/????>
Trace; 80525a20 <END_OF_CODE+2702f0/????>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 805269e0 <END_OF_CODE+2712b0/????>
Trace; 80525ed8 <END_OF_CODE+2707a8/????>
Trace; 80503f50 <END_OF_CODE+24e820/????>
Trace; 8051910c <END_OF_CODE+2639dc/????>
Trace; 801b0fb8 <net_rx_action+a8/24c>
Trace; 801ae580 <net_tx_action+1c8/1f0>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 8009ac90 <__alloc_pages_internal+3f4/434>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 801784a0 <mem_serial_out+0/20>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 800bd470 <__kmalloc+d4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 80b42088 <END_OF_CODE+88c958/????>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 80b419d4 <END_OF_CODE+88c2a4/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM
1395 pages shared
2630 pages non-shared
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<8009ac88>] 0x8009ac88
[<8018c538>] 0x8018c538
[<800bcf78>] 0x800bcf78
[<800bd450>] 0x800bd450
[<801a8664>] 0x801a8664
[<801a9120>] 0x801a9120
[<80074174>] 0x80074174
[<80b42070>] 0x80b42070
[<8007425c>] 0x8007425c
[<800745c8>] 0x800745c8
[<800789b4>] 0x800789b4
[<800790a8>] 0x800790a8
[<8007441c>] 0x8007441c
[<80078a10>] 0x80078a10
[<8004e988>] 0x8004e988
CPU    0: hi:    0, btch:   1 usd:   0
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 8009ac88 <__alloc_pages_internal+3ec/434>
Trace; 8018c538 <ar231x_start_xmit+80/d0>
Trace; 800bcf78 <cache_alloc_refill+2d8/6fc>
Trace; 800bd450 <__kmalloc+b4/124>
Trace; 801a8664 <__alloc_skb+70/148>
Trace; 801a9120 <skb_copy+40/c8>
Trace; 80074174 <__queue_work+7c/b8>
Trace; 80b42070 <END_OF_CODE+88c940/????>
Trace; 8007425c <queue_work_on+90/a0>
Trace; 800745c8 <worker_thread+1ac/260>
Trace; 800789b4 <kthread+0/a0>
Trace; 800790a8 <autoremove_wake_function+0/4c>
Trace; 8007441c <worker_thread+0/260>
Trace; 80078a10 <kthread+5c/a0>
Trace; 8004e988 <kernel_thread_helper+10/18>

1417 total pagecache pages
4096 pages RAM
1395 pages shared
2630 pages non-shared

8 warnings issued.  Results may not be reliable.

[-- Attachment #4: kernel-panic-1825-23-01-10.log --]
[-- Type: text/plain, Size: 10285 bytes --]

BusyBox v1.14.4 (2010-01-22 23:22:23 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 KAMIKAZE (bleeding edge, r18405) ------------------
  * 10 oz Vodka       Shake well with ice and strain
  * 10 oz Triple sec  mixture into 10 shot glasses.
  * 10 oz lime juice  Salute!
 ---------------------------------------------------
root@OpenWrt:~# batctl o -b
  Originator     (#/255)           Nexthop [outgoingIF]:   Potential nexthops ... [B.A.T.M.A.N. adv 0.2.1-beta r1545, MainIF/MAC: ath2/0a:24:01:b7:6a:d1]
06:22:b0:98:87:f9  (  3) 0a:24:01:b7:61:1b [      ath2]: 06:22:b0:98:87:f9 (  0) 0a:24:01:b7:61:1b (  3) 06:24:01:b7:69:c1 (  1)
06:24:01:b7:69:c1  ( 11) 06:24:01:b7:69:c1 [      ath2]: 06:24:01:b7:69:c1 ( 11)
0a:24:01:b7:61:19  (  3) 0a:24:01:b7:6a:fd [      ath2]: 0a:24:01:b7:61:19 (  0) 0a:24:01:b7:6a:fd (  3)
0a:24:01:b7:61:1b  ( 19) 0a:24:01:b7:61:1b [      ath2]: 0a:24:01:b7:61:1b ( 19) 06:24:01:b7:69:c1 (  0) 06:22:b0:98:87:f9 (  0)
0a:24:01:b7:6a:fd  ( 10) 0a:24:01:b7:6a:fd [      ath2]: 0a:24:01:b7:6a:fd ( 10) 0a:24:01:b7:61:49 (  0)
0a:24:01:b7:61:49  ( 10) 0a:24:01:b7:61:49 [      ath2]: 0a:24:01:b7:61:49 ( 10) 0a:24:01:b7:61:1b (  1)
root@OpenWrt:~# dmesg
Linux version 2.6.30.9 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #56 Fri Jan 22 23:24:25 CET 2010
CPU revision is: 00019064 (MIPS 4KEc)
Determined physical RAM map:
 memory: 01000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal   0x00000000 -> 0x00001000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00001000
On node 0 totalpages: 4096
free_area_init_node: node 0, pgdat 8027ef40, node_mem_map 802b7000
  Normal zone: 32 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 4064 pages, LIFO batch:0
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2
Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
NR_IRQS:128
PID hash table entries: 64 (order: 6, 256 bytes)
console [ttyS0] enabled
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 13460k/16384k available (1976k kernel code, 2924k reserved, 324k data, 128k init, 0k highmem)
Calibrating delay loop... 183.50 BogoMIPS (lpj=917504)
Mount-cache hash table entries: 512
net_namespace: 732 bytes
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
Switched to high resolution mode on CPU 0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
NET: Registered protocol family 1
Radio config found at offset 0xf8(0x1f8)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
msgmni has been set to 26
io scheduler noop registered
io scheduler deadline registered (default)
gpiodev: gpio device registered with major 254
gpiodev: gpio platform device registered with access mask FFFFFFFF
Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A
eth0: Atheros AR231x: 00:24:01:b7:6a:d2, irq 4
ar231x_eth_mii: probed
eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00)
cmdlinepart partition parsing not available
Searching for RedBoot partition table in spiflash at offset 0x3d0000
Searching for RedBoot partition table in spiflash at offset 0x3e0000
6 RedBoot partitions found on MTD device spiflash
Creating 6 MTD partitions on "spiflash":
0x000000000000-0x000000030000 : "RedBoot"
0x000000030000-0x0000002f0000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
mtd: partition "rootfs_data" created automatically, ofs=250000, len=A0000
0x000000250000-0x0000002f0000 : "rootfs_data"
0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7"
0x0000003e0000-0x0000003ef000 : "FIS directory"
0x0000003ef000-0x0000003f0000 : "RedBoot config"
0x0000003f0000-0x000000400000 : "boardconfig"
TCP westwood registered
NET: Registered protocol family 17
Bridge firewalling registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:1.
Freeing unused kernel memory: 128k freed
Please be patient, while OpenWrt loads ...
mini_fo: using base directory: /
mini_fo: using storage directory: /tmp/root
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
device eth0.1 entered promiscuous mode
device eth0 entered promiscuous mode
br-mesh: port 1(eth0.1) entering forwarding state
ip_tables: (C) 2000-2006 Netfilter Core Team
Ebtables v2.0 registered
ip6_tables: (C) 2000-2006 Netfilter Core Team
batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded
device eth0.4 entered promiscuous mode
br-wan_vpn: port 1(eth0.4) entering forwarding state
ath_hal: module license 'Proprietary' taints kernel.
Disabling lock debugging due to kernel taint
ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR)
br-wan_vpn: starting userspace STP failed, starting kernel STP
eth0.1: no IPv6 routers present
ath_ahb: trunk
wlan: trunk
wlan: mac acl policy registered
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
br-mesh: no IPv6 routers present
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3
IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs
eth0.2: no IPv6 routers present
device bat0 entered promiscuous mode
br-mesh: port 2(bat0) entering forwarding state
eth0.3: no IPv6 routers present
eth0.4: no IPv6 routers present
device ath0 entered promiscuous mode
br-mesh: port 3(ath0) entering forwarding state
device ath0 left promiscuous mode
br-mesh: port 3(ath0) entering disabled state
device ath0 entered promiscuous mode
br-mesh: port 3(ath0) entering forwarding state
bat0: no IPv6 routers present
device ath1 entered promiscuous mode
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 2(ath1) entering forwarding state
device ath1 left promiscuous mode
br-wan_vpn: port 2(ath1) entering disabled state
device ath1 entered promiscuous mode
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 2(ath1) entering forwarding state
ath0: no IPv6 routers present
ath1: no IPv6 routers present
br-wan_vpn: port 2(ath1) entering disabled state
br-wan_vpn: port 1(eth0.4) entering disabled state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 2(ath1) entering forwarding state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-mesh: port 3(ath0) entering disabled state
br-mesh: port 2(bat0) entering disabled state
br-mesh: port 1(eth0.1) entering disabled state
br-mesh: port 3(ath0) entering forwarding state
br-mesh: port 2(bat0) entering forwarding state
br-mesh: port 1(eth0.1) entering forwarding state
ath2: no IPv6 routers present
br-mesh: no IPv6 routers present
------------[ cut here ]------------
WARNING: at net/sched/sch_generic.c:226 0x801c3554()
NETDEV WATCHDOG: wifi0 (): transmit timed out
Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<80060914>] 0x80060914
[<801c3554>] 0x801c3554
[<80060994>] 0x80060994
[<800790bc>] 0x800790bc
[<801c3554>] 0x801c3554
[<8005aaf8>] 0x8005aaf8
[<800741b0>] 0x800741b0
[<80222220>] 0x80222220
[<8007d674>] 0x8007d674
[<80222138>] 0x80222138
[<801c33b8>] 0x801c33b8
[<8006c01c>] 0x8006c01c
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<80041444>] 0x80041444

---[ end trace 31db24b68cfbba59 ]---
batman-adv:Adding interface: ath2
batman-adv:Interface activated: ath2
batman-adv:Adding interface: eth0.2
batman-adv:Interface activated: eth0.2
batman-adv:Adding interface: eth0.3
batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2
batman-adv:It is strongly recommended to keep mac addresses unique to avoid problems!
batman-adv:Interface activated: eth0.3
jffs2_scan_eraseblock(): End of filesystem marker found at 0x0
jffs2_build_filesystem(): unlocking the mtd device... done.
jffs2_build_filesystem(): erasing all blocks after the end marker... done.
mini_fo: using base directory: /
mini_fo: using storage directory: /jffs
device VPN entered promiscuous mode
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 3(VPN) entering forwarding state
br-wan_vpn: neighbor 8000.00:22:b0:98:8c:34 lost on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
root@OpenWrt:~#

[-- Attachment #5: kernel-panic-1825-23-01-10.ksymoops --]
[-- Type: text/plain, Size: 1787 bytes --]

ksymoops 2.4.11 on x86_64 2.6.31-1-amd64.  Options used
     -v /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/vmlinux (specified)
     -K (specified)
     -L (specified)
     -o /home/linus/lff-test/kamikaze/build_dir/linux-atheros -o /home/linus/lff-test/kamikaze/build_dir/target-mips_uClibc-0.9.30.1 (specified)
     -m /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/System.map (specified)

No modules in ksyms, skipping objects
lo: Disabled Privacy Extensions
WARNING: at net/sched/sch_generic.c:226 0x801c3554()
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<80060914>] 0x80060914
[<801c3554>] 0x801c3554
[<80060994>] 0x80060994
[<800790bc>] 0x800790bc
[<801c3554>] 0x801c3554
[<8005aaf8>] 0x8005aaf8
[<800741b0>] 0x800741b0
[<80222220>] 0x80222220
[<8007d674>] 0x8007d674
[<80222138>] 0x80222138
[<801c33b8>] 0x801c33b8
[<8006c01c>] 0x8006c01c
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<80041444>] 0x80041444
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 80060914 <warn_slowpath_common+70/b0>
Trace; 801c3554 <dev_watchdog+19c/338>
Trace; 80060994 <warn_slowpath_fmt+24/30>
Trace; 800790bc <autoremove_wake_function+14/4c>
Trace; 801c3554 <dev_watchdog+19c/338>
Trace; 8005aaf8 <__wake_up_common+58/c0>
Trace; 800741b0 <delayed_work_timer_fn+0/1c>
Trace; 80222220 <br_fdb_cleanup+e8/124>
Trace; 8007d674 <hrtimer_interrupt+26c/2a4>
Trace; 80222138 <br_fdb_cleanup+0/124>
Trace; 801c33b8 <dev_watchdog+0/338>
Trace; 8006c01c <run_timer_softirq+17c/234>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 80041444 <ret_from_irq+0/4>


1 warning issued.  Results may not be reliable.

[-- Attachment #6: kernel-panic-1831-23-01-10.log --]
[-- Type: text/plain, Size: 9718 bytes --]

root@OpenWrt:~# batctl o -b
  Originator     (#/255)           Nexthop [outgoingIF]:   Potential nexthops ... [B.A.T.M.A.N. adv 0.2.1-beta r1545, MainIF/MAC: ath2/0a:24:01:b7:61:1b]
06:22:b0:98:87:f9  ( 23) 06:22:b0:98:87:f9 [      ath2]: 06:22:b0:98:87:f9 ( 23)
06:24:01:b7:69:c1  ( 21) 06:24:01:b7:69:c1 [      ath2]: 06:24:01:b7:69:c1 ( 21) 0a:24:01:b7:6a:d1 (  1)
0a:24:01:b7:61:19  (  4) 0a:24:01:b7:61:49 [      ath2]: 0a:24:01:b7:61:19 (  0) 0a:24:01:b7:6a:d1 (  1) 0a:24:01:b7:61:49 (  4)
0a:24:01:b7:6a:d1  ( 35) 0a:24:01:b7:6a:d1 [      ath2]: 0a:24:01:b7:6a:d1 ( 35) 06:24:01:b7:69:c1 (  0)
0a:24:01:b7:6a:fd  ( 10) 0a:24:01:b7:6a:fd [      ath2]: 0a:24:01:b7:6a:fd ( 10) 0a:24:01:b7:6a:d1 (  0) 0a:24:01:b7:61:49 (  0)
0a:24:01:b7:61:49  ( 10) 0a:24:01:b7:61:49 [      ath2]: 0a:24:01:b7:61:49 ( 10) 0a:24:01:b7:6a:fd (  6)
root@OpenWrt:~# dmesg
Linux version 2.6.30.9 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #49 Fri Jan 22 18:48:49 CET 2010
CPU revision is: 00019064 (MIPS 4KEc)
Determined physical RAM map:
 memory: 01000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal   0x00000000 -> 0x00001000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00001000
On node 0 totalpages: 4096
free_area_init_node: node 0, pgdat 8027ef40, node_mem_map 802b7000
  Normal zone: 32 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 4064 pages, LIFO batch:0
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2
Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
NR_IRQS:128
PID hash table entries: 64 (order: 6, 256 bytes)
console [ttyS0] enabled
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 13460k/16384k available (1976k kernel code, 2924k reserved, 324k data, 128k init, 0k highmem)
Calibrating delay loop... 183.50 BogoMIPS (lpj=917504)
Mount-cache hash table entries: 512
net_namespace: 732 bytes
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
Switched to high resolution mode on CPU 0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
NET: Registered protocol family 1
Radio config found at offset 0xf8(0x1f8)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
msgmni has been set to 26
io scheduler noop registered
io scheduler deadline registered (default)
gpiodev: gpio device registered with major 254
gpiodev: gpio platform device registered with access mask FFFFFFFF
Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A
eth0: Atheros AR231x: 00:24:01:b7:61:1c, irq 4
ar231x_eth_mii: probed
eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00)
cmdlinepart partition parsing not available
Searching for RedBoot partition table in spiflash at offset 0x3d0000
Searching for RedBoot partition table in spiflash at offset 0x3e0000
6 RedBoot partitions found on MTD device spiflash
Creating 6 MTD partitions on "spiflash":
0x000000000000-0x000000030000 : "RedBoot"
0x000000030000-0x0000002f0000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
mtd: partition "rootfs_data" created automatically, ofs=250000, len=A0000
0x000000250000-0x0000002f0000 : "rootfs_data"
0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7"
0x0000003e0000-0x0000003ef000 : "FIS directory"
0x0000003ef000-0x0000003f0000 : "RedBoot config"
0x0000003f0000-0x000000400000 : "boardconfig"
TCP westwood registered
NET: Registered protocol family 17
Bridge firewalling registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:1.
Freeing unused kernel memory: 128k freed
Please be patient, while OpenWrt loads ...
mini_fo: using base directory: /
mini_fo: using storage directory: /jffs
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
device eth0.1 entered promiscuous mode
device eth0 entered promiscuous mode
br-mesh: port 1(eth0.1) entering forwarding state
ip_tables: (C) 2000-2006 Netfilter Core Team
Ebtables v2.0 registered
ip6_tables: (C) 2000-2006 Netfilter Core Team
batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded
device eth0.4 entered promiscuous mode
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-wan_vpn: starting userspace STP failed, starting kernel STP
ath_hal: module license 'Proprietary' taints kernel.
Disabling lock debugging due to kernel taint
ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR)
eth0.1: no IPv6 routers present
br-mesh: no IPv6 routers present
ath_ahb: trunk
wlan: trunk
wlan: mac acl policy registered
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3
IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs
eth0.2: no IPv6 routers present
device bat0 entered promiscuous mode
br-mesh: port 2(bat0) entering forwarding state
eth0.3: no IPv6 routers present
br-wan_vpn: no IPv6 routers present
eth0.4: no IPv6 routers present
device ath0 entered promiscuous mode
br-mesh: port 3(ath0) entering forwarding state
bat0: no IPv6 routers present
device ath0 left promiscuous mode
br-mesh: port 3(ath0) entering disabled state
device ath0 entered promiscuous mode
br-mesh: port 3(ath0) entering forwarding state
device ath1 entered promiscuous mode
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 2(ath1) entering forwarding state
device ath1 left promiscuous mode
br-wan_vpn: port 2(ath1) entering disabled state
device ath1 entered promiscuous mode
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 2(ath1) entering forwarding state
ath0: no IPv6 routers present
br-wan_vpn: port 2(ath1) entering disabled state
br-wan_vpn: port 1(eth0.4) entering disabled state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 2(ath1) entering forwarding state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-mesh: port 3(ath0) entering disabled state
br-mesh: port 2(bat0) entering disabled state
br-mesh: port 1(eth0.1) entering disabled state
br-mesh: port 3(ath0) entering forwarding state
br-mesh: port 2(bat0) entering forwarding state
br-mesh: port 1(eth0.1) entering forwarding state
ath1: no IPv6 routers present
ath2: no IPv6 routers present
br-mesh: no IPv6 routers present
batman-adv:Adding interface: ath2
batman-adv:Interface activated: ath2
batman-adv:Adding interface: eth0.2
batman-adv:Interface activated: eth0.2
batman-adv:Adding interface: eth0.3
batman-adv:The newly added mac address (00:24:01:b7:61:1c) already exists on: eth0.2
batman-adv:It is strongly recommended to keep mac addresses unique to avoid problems!
batman-adv:Interface activated: eth0.3
br-wan_vpn: port 2(ath1) entering disabled state
device VPN entered promiscuous mode
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 3(VPN) entering forwarding state
br-wan_vpn: port 3(VPN) entering disabled state
br-wan_vpn: topology change detected, sending tcn bpdu
br-wan_vpn: port 3(VPN) entering forwarding state
br-wan_vpn: neighbor 8000.00:22:b0:98:8c:34 lost on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
------------[ cut here ]------------
WARNING: at net/sched/sch_generic.c:226 0x801c3554()
NETDEV WATCHDOG: wifi0 (): transmit timed out
Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<80060914>] 0x80060914
[<801c3554>] 0x801c3554
[<80060994>] 0x80060994
[<801c3554>] 0x801c3554
[<80086bf4>] 0x80086bf4
[<8007d674>] 0x8007d674
[<801b0838>] 0x801b0838
[<801c33b8>] 0x801c33b8
[<8006c01c>] 0x8006c01c
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<80041444>] 0x80041444
[<80041660>] 0x80041660
[<8018c4b8>] 0x8018c4b8
[<8004e85c>] 0x8004e85c
[<80041680>] 0x80041680
[<80281a4c>] 0x80281a4c
[<80281370>] 0x80281370

---[ end trace bf8b652a122b3db4 ]---
root@OpenWrt:~#

[-- Attachment #7: kernel-panic-1831-23-01-10.ksymoops --]
[-- Type: text/plain, Size: 1950 bytes --]

ksymoops 2.4.11 on x86_64 2.6.31-1-amd64.  Options used
     -v /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/vmlinux (specified)
     -K (specified)
     -L (specified)
     -o /home/linus/lff-test/kamikaze/build_dir/linux-atheros -o /home/linus/lff-test/kamikaze/build_dir/target-mips_uClibc-0.9.30.1 (specified)
     -m /home/linus/lff-test/kamikaze/build_dir/linux-atheros/linux-2.6.30.9/System.map (specified)

No modules in ksyms, skipping objects
lo: Disabled Privacy Extensions
WARNING: at net/sched/sch_generic.c:226 0x801c3554()
Call Trace:[<800493bc>] 0x800493bc
[<800493bc>] 0x800493bc
[<80060914>] 0x80060914
[<801c3554>] 0x801c3554
[<80060994>] 0x80060994
[<801c3554>] 0x801c3554
[<80086bf4>] 0x80086bf4
[<8007d674>] 0x8007d674
[<801b0838>] 0x801b0838
[<801c33b8>] 0x801c33b8
[<8006c01c>] 0x8006c01c
[<80066d84>] 0x80066d84
[<80066e90>] 0x80066e90
[<80041444>] 0x80041444
[<80041660>] 0x80041660
[<8018c4b8>] 0x8018c4b8
[<8004e85c>] 0x8004e85c
[<80041680>] 0x80041680
[<80281a4c>] 0x80281a4c
[<80281370>] 0x80281370
Warning (Oops_read): Code line not seen, dumping what data is available


Trace; 800493bc <dump_stack+20/50>
Trace; 800493bc <dump_stack+20/50>
Trace; 80060914 <warn_slowpath_common+70/b0>
Trace; 801c3554 <dev_watchdog+19c/338>
Trace; 80060994 <warn_slowpath_fmt+24/30>
Trace; 801c3554 <dev_watchdog+19c/338>
Trace; 80086bf4 <tick_program_event+28/34>
Trace; 8007d674 <hrtimer_interrupt+26c/2a4>
Trace; 801b0838 <process_backlog+d4/14c>
Trace; 801c33b8 <dev_watchdog+0/338>
Trace; 8006c01c <run_timer_softirq+17c/234>
Trace; 80066d84 <__do_softirq+a4/154>
Trace; 80066e90 <do_softirq+5c/98>
Trace; 80041444 <ret_from_irq+0/4>
Trace; 80041660 <r4k_wait+0/40>
Trace; 8018c4b8 <ar231x_start_xmit+0/d0>
Trace; 8004e85c <cpu_idle+24/4c>
Trace; 80041680 <r4k_wait+20/40>
Trace; 80281a4c <start_kernel+36c/384>
Trace; 80281370 <unknown_bootoption+0/2fc>


1 warning issued.  Results may not be reliable.

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing
@ 2010-01-23 18:10 ` Andrew Lunn
  2010-01-23 23:30   ` Linus Lüssing
  2010-01-24 20:42   ` Linus Lüssing
  2010-01-24  4:24 ` Linus Lüssing
  2010-01-26  6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing
  2 siblings, 2 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-01-23 18:10 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Sat, Jan 23, 2010 at 06:46:16PM +0100, Linus L??ssing wrote:
> Hey guys,
> 
> I've been installing a 9 node setup here in our cellar. They are
> all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current
> batman-adv maintance version in OpenWRT).
> 
> The result over night:
> - 1x: bat_events: page allocation failure

This looks like a memory leak somewhere. It could be anywhere, batman
or some other part of the kernel.

Can you build your kernel with the kernel memory leak detector
enabled? It is under the Kernel Hacking options. You probably also
need to build the kernel with symbols, which will help with ksymopps
anyway. Documentation for kmemleak is in Documentation/kmemleak.txt.
I've never used it myself, so i've no idea how good it actually is...

Interestingly, did you notice the warnings:

batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2
batman-adv:It is strongly recommended to keep mac addresses unique to avoid
+problems!

I guess this is because you are using VLANs. Have you seen these
problems without using VLANs? 

	 Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-23 18:10 ` Andrew Lunn
@ 2010-01-23 23:30   ` Linus Lüssing
  2010-01-24 20:42   ` Linus Lüssing
  1 sibling, 0 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-01-23 23:30 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 1455 bytes --]

Ok, this is with symbols enabled and I changed the mac
addresses, so they are defintely different now. Here is the new
backtrace for the second issue. It occured after about 1-2 hours.

Cheers, Linus

On Sat, Jan 23, 2010 at 07:10:48PM +0100, Andrew Lunn wrote:
> On Sat, Jan 23, 2010 at 06:46:16PM +0100, Linus L??ssing wrote:
> > Hey guys,
> > 
> > I've been installing a 9 node setup here in our cellar. They are
> > all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current
> > batman-adv maintance version in OpenWRT).
> > 
> > The result over night:
> > - 1x: bat_events: page allocation failure
> 
> This looks like a memory leak somewhere. It could be anywhere, batman
> or some other part of the kernel.
> 
> Can you build your kernel with the kernel memory leak detector
> enabled? It is under the Kernel Hacking options. You probably also
> need to build the kernel with symbols, which will help with ksymopps
> anyway. Documentation for kmemleak is in Documentation/kmemleak.txt.
> I've never used it myself, so i've no idea how good it actually is...
> 
> Interestingly, did you notice the warnings:
> 
> batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2
> batman-adv:It is strongly recommended to keep mac addresses unique to avoid
> +problems!
> 
> I guess this is because you are using VLANs. Have you seen these
> problems without using VLANs? 
> 
> 	 Andrew
> 

[-- Attachment #2: call-trace-0027-24-01-10.log --]
[-- Type: text/plain, Size: 1127 bytes --]

root@OpenWrt:/# ------------[ cut here ]------------
WARNING: at net/sched/sch_generic.c:226 dev_watchdog+0x19c/0x338()
NETDEV WATCHDOG: wifi0 (): transmit timed out
Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<80062a84>] warn_slowpath_common+0x70/0xb0
[<80062b04>] warn_slowpath_fmt+0x24/0x30
[<801c9c98>] dev_watchdog+0x19c/0x338
[<8006e1b8>] run_timer_softirq+0x17c/0x234
[<80068f20>] __do_softirq+0xa4/0x154
[<8006902c>] do_softirq+0x5c/0x98
[<80041444>] ret_from_irq+0x0/0x4
[<80041680>] r4k_wait+0x20/0x40
[<8004e910>] cpu_idle+0x24/0x4c
[<802a9a4c>] start_kernel+0x36c/0x384

---[ end trace 57451ac66d72b45d ]--

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing
  2010-01-23 18:10 ` Andrew Lunn
@ 2010-01-24  4:24 ` Linus Lüssing
  2010-01-26  6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing
  2 siblings, 0 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-01-24  4:24 UTC (permalink / raw)
  To: b.a.t.m.a.n

[-- Attachment #1: Type: text/plain, Size: 170 bytes --]

And another (different) one. Which occured right after flashing
and booting causing a kernel panic.
Sorry couldn't find a method for reproducing them yet.

Cheers, Linus

[-- Attachment #2: kernel-panic-0512-24-01-10.log --]
[-- Type: text/plain, Size: 9295 bytes --]

root@OpenWrt:/# n+Warning: IP address 192.168.1.1 in use
Ethernet eth0: MAC address 00:22:b0:98:87:de
IP: 192.168.1.1/255.255.255.0, Gateway: 0.0.0.0
Default server: 192.168.1.2

RedBoot(tm) bootstrap and debug environment [ROMRAM]
production release, version "2.1.3" - built 18:43:19, Sep 20 2007

Platform: ap61 (Atheros WiSOC)
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2007, NewMedia-NET GmbH.

Board: DLINK DIR-300
RAM: 0x80000000-0x81000000, [0x80040580-0x80fe1000] available
FLASH: 0xbfc00000 - 0xbfff0000, 64 blocks of 0x00010000 bytes each.
== Executing boot script in 5.000 seconds - enter ^C to abort
+Warning: IP address 192.168.1.1 in use
Ethernet eth0: MAC address 00:22:b0:98:87:de
IP: 192.168.1.1/255.255.255.0, Gateway: 0.0.0.0
Default server: 192.168.1.2

RedBoot(tm) bootstrap and debug environment [ROMRAM]
production release, version "2.1.3" - built 18:43:19, Sep 20 2007

Platform: ap61 (Atheros WiSOC)
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2007, NewMedia-NET GmbH.

Board: DLINK DIR-300
RAM: 0x80000000-0x81000000, [0x80040580-0x80fe1000] available
FLASH: 0xbfc00000 - 0xbfff0000, 64 blocks of 0x00010000 bytes each.
== Executing boot script in 5.000 seconds - enter ^C to abort
DD-WRT> fis load -l vmlinux.bin.l7
Image loaded from 0x80041000-0x802c8200
DD-WRT> exec
Now booting linux kernel:
 Base address 0x80030000 Entry 0x80041000
 Cmdline :
Linux version 2.6.30.10 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #2 Sun Jan 24 04:12:11 CET 2010
CPU revision is: 00019064 (MIPS 4KEc)
Determined physical RAM map:
 memory: 01000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal   0x00000000 -> 0x00001000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00001000
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2
Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
NR_IRQS:128
PID hash table entries: 64 (order: 6, 256 bytes)
console [ttyS0] enabled
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 13300k/16384k available (2002k kernel code, 3084k reserved, 458k data, 128k init, 0k highmem)
Calibrating delay loop... 183.50 BogoMIPS (lpj=917504)
Mount-cache hash table entries: 512
net_namespace: 732 bytes
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
NET: Registered protocol family 1
Radio config found at offset 0xf8(0x1f8)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
msgmni has been set to 25
io scheduler noop registered
io scheduler deadline registered (default)
gpiodev: gpio device registered with major 254
gpiodev: gpio platform device registered with access mask FFFFFFFF
Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A
eth0: Atheros AR231x: 00:22:b0:98:87:de, irq 4
ar231x_eth_mii: probed
eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00)
cmdlinepart partition parsing not available
Searching for RedBoot partition table in spiflash at offset 0x3d0000
Searching for RedBoot partition table in spiflash at offset 0x3e0000
6 RedBoot partitions found on MTD device spiflash
Creating 6 MTD partitions on "spiflash":
0x000000000000-0x000000030000 : "RedBoot"
0x000000030000-0x0000002f0000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
mtd: partition "rootfs_data" created automatically, ofs=230000, len=C0000
0x000000230000-0x0000002f0000 : "rootfs_data"
0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7"
0x0000003e0000-0x0000003ef000 : "FIS directory"
0x0000003ef000-0x0000003f0000 : "RedBoot config"
0x0000003f0000-0x000000400000 : "boardconfig"
TCP westwood registered
NET: Registered protocol family 17
Bridge firewalling registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:1.
Freeing unused kernel memory: 128k freed
Please be patient, while OpenWrt loads ...
- preinit -
Press CTRL-C for failsafe
jffs2 not ready yet; using ramdisk
mini_fo: using base directory: /
mini_fo: using storage directory: /tmp/root
- init -

Please press Enter to activate this console. NET: Registered protocol family 10
lo: Disabled Privacy Extensions
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
device eth0.1 entered promiscuous mode
device eth0 entered promiscuous mode
br-mesh: port 1(eth0.1) entering forwarding state
ip_tables: (C) 2000-2006 Netfilter Core Team
Ebtables v2.0 registered
ip6_tables: (C) 2000-2006 Netfilter Core Team
device eth0.4 entered promiscuous mode
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-wan_vpn: starting userspace STP failed, starting kernel STP
batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded
ath_hal: module license 'Proprietary' taints kernel.
Disabling lock debugging due to kernel taint
ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR)
ath_ahb: trunk
wlan: trunk
wlan: mac acl policy registered
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3
IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs
device bat0 entered promiscuous mode
br-mesh: port 2(bat0) entering forwarding state
br-wan_vpn: port 1(eth0.4) entering disabled state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-mesh: port 2(bat0) entering disabled state
br-mesh: port 1(eth0.1) entering disabled state
br-mesh: port 2(bat0) entering forwarding state
br-mesh: port 1(eth0.1) entering forwarding state
batman-adv:Adding interface: ath0
batman-adv:Interface activated: ath0
CPU 0 Unable to handle kernel paging request at virtual address 00000010, epc == 80be3584, ra == 80be34e4
Oops[#1]:
Cpu 0
$ 0   : 00000000 10009c00 00000010 80afd800
$ 4   : 00000000 00000000 00000000 80be85f8
$ 8   : 0000611b 80afd68e 00000010 00000000
$12   : 00000000 feced300 ffffffff 61636520
$16   : 8064ca40 8064ca00 80afd680 8064ca40
$20   : 8064ca00 00000000 00000001 80640060
$24   : 00000010 8069c74c
$28   : 8046c000 8046d9a8 802a0000 80be34e4
Hi    : 00000000
Lo    : 0000000b
epc   : 80be3584 receive_bat_packet+0x3d8/0x6ec [batman_adv]
    Tainted: P
ra    : 80be34e4 receive_bat_packet+0x338/0x6ec [batman_adv]
Status: 10009c02    KERNEL EXL
Cause : 10800008
BadVA : 00000010
PrId  : 00019064 (MIPS 4KEc)
Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
Process batctl (pid: 1140, threadinfo=8046c000, task=8052bad0, tls=00000000)
Stack : 00000000 806a2aac 10009c01 80303620 802a0000 80f38a90 8064006c 00000001
        80640058 00000000 00000002 80640052 80640074 0000000c 80afd800 00000000
        80640060 00000000 80640060 00000040 80640052 80afd800 00004305 00000000
        80640040 80be8094 80f38a90 8005cd6c 00000001 80c27798 80afd800 80c27798
        80303620 80c176b8 80640052 80afd800 10009c01 80547000 80547000 80be2ee4
        ...
Call Trace:
[<80be3584>] receive_bat_packet+0x3d8/0x6ec [batman_adv]
[<80be8094>] receive_aggr_bat_packet+0x7c/0xbc [batman_adv]
[<80be2ee4>] recv_bat_packet+0x94/0x24c [batman_adv]
[<80be79c8>] batman_skb_recv+0x128/0x1dc [batman_adv]
[<806a1494>] ieee80211_saveath+0xb24/0xb80 [ath_ahb]


Code: 9245000e  84640008  00441021 <90440000> 00a4102b  00a2200b  10800003  00000000  14a00003
Kernel panic - not syncing: Fatal exception in interrupt

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-23 18:10 ` Andrew Lunn
  2010-01-23 23:30   ` Linus Lüssing
@ 2010-01-24 20:42   ` Linus Lüssing
  2010-01-24 21:00     ` Andrew Lunn
  2010-01-25  6:47     ` Andrew Lunn
  1 sibling, 2 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-01-24 20:42 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 1905 bytes --]

Hi Andrew,

Here is the output with the symbols inserted. I can reproduce this
memory leak very reliable now, by just activating the vis server
(didn't do that with the logs I've posted before) in a couple of
minutes. free is showing a nice countdown then :).

For kmemleak... I'm using 2.6.30.10 here and I think Torvalds
merged it into 2.6.33 a month ago. Anyone having a 2.6.33 kernel
running at the moment, patches for kmemleak in 2.6.30 or another
idea of how to debug this in 2.6.30?

"Debug slab memory allocations" (CONFIG_DEBUG_SLAB) and "Memory
leak debugging" (DEBUG_SLAB_LEAK) are activated on the kernel I'm
using here, but they are not of any use, are they?

Cheers, Linus

On Sat, Jan 23, 2010 at 07:10:48PM +0100, Andrew Lunn wrote:
> On Sat, Jan 23, 2010 at 06:46:16PM +0100, Linus L??ssing wrote:
> > Hey guys,
> > 
> > I've been installing a 9 node setup here in our cellar. They are
> > all running B.A.T.M.A.N. adv 0.2.1-beta r1545 (so the current
> > batman-adv maintance version in OpenWRT).
> > 
> > The result over night:
> > - 1x: bat_events: page allocation failure
> 
> This looks like a memory leak somewhere. It could be anywhere, batman
> or some other part of the kernel.
> 
> Can you build your kernel with the kernel memory leak detector
> enabled? It is under the Kernel Hacking options. You probably also
> need to build the kernel with symbols, which will help with ksymopps
> anyway. Documentation for kmemleak is in Documentation/kmemleak.txt.
> I've never used it myself, so i've no idea how good it actually is...
> 
> Interestingly, did you notice the warnings:
> 
> batman-adv:The newly added mac address (00:24:01:b7:6a:d2) already exists on: eth0.2
> batman-adv:It is strongly recommended to keep mac addresses unique to avoid
> +problems!
> 
> I guess this is because you are using VLANs. Have you seen these
> problems without using VLANs? 
> 
> 	 Andrew
> 

[-- Attachment #2: 2010-01-24-20-53-memory-leak.log --]
[-- Type: text/plain, Size: 19567 bytes --]

+Ethernet eth0: MAC address 00:22:b0:98:87:de
IP: 192.168.1.1/255.255.255.0, Gateway: 0.0.0.0
Default server: 192.168.1.2

RedBoot(tm) bootstrap and debug environment [ROMRAM]
production release, version "2.1.3" - built 18:43:19, Sep 20 2007

Platform: ap61 (Atheros WiSOC)
Copyright (C) 2000, 2001, 2002, 2003, 2004 Red Hat, Inc.
Copyright (C) 2007, NewMedia-NET GmbH.

Board: DLINK DIR-300
RAM: 0x80000000-0x81000000, [0x80040580-0x80fe1000] available
FLASH: 0xbfc00000 - 0xbfff0000, 64 blocks of 0x00010000 bytes each.
== Executing boot script in 5.000 seconds - enter ^C to abort
DD-WRT> fis load -l vmlinux.bin.l7
Image loaded from 0x80041000-0x802c8200
DD-WRT> exec
Now booting linux kernel:
 Base address 0x80030000 Entry 0x80041000
 Cmdline :
Linux version 2.6.30.10 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #2 Sun Jan 24 04:12:11 CET 2010
CPU revision is: 00019064 (MIPS 4KEc)
Determined physical RAM map:
 memory: 01000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal   0x00000000 -> 0x00001000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00001000
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2
Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
NR_IRQS:128
PID hash table entries: 64 (order: 6, 256 bytes)
console [ttyS0] enabled
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 13300k/16384k available (2002k kernel code, 3084k reserved, 458k data, 128k init, 0k highmem)
Calibrating delay loop... 183.50 BogoMIPS (lpj=917504)
Mount-cache hash table entries: 512
net_namespace: 732 bytes
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
NET: Registered protocol family 1
Radio config found at offset 0xf8(0x1f8)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
msgmni has been set to 25
io scheduler noop registered
io scheduler deadline registered (default)
gpiodev: gpio device registered with major 254
gpiodev: gpio platform device registered with access mask FFFFFFFF
Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A
eth0: Atheros AR231x: 00:22:b0:98:87:de, irq 4
ar231x_eth_mii: probed
eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00)
cmdlinepart partition parsing not available
Searching for RedBoot partition table in spiflash at offset 0x3d0000
Searching for RedBoot partition table in spiflash at offset 0x3e0000
6 RedBoot partitions found on MTD device spiflash
Creating 6 MTD partitions on "spiflash":
0x000000000000-0x000000030000 : "RedBoot"
0x000000030000-0x0000002f0000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
mtd: partition "rootfs_data" created automatically, ofs=230000, len=C0000
0x000000230000-0x0000002f0000 : "rootfs_data"
0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7"
0x0000003e0000-0x0000003ef000 : "FIS directory"
0x0000003ef000-0x0000003f0000 : "RedBoot config"
0x0000003f0000-0x000000400000 : "boardconfig"
TCP westwood registered
NET: Registered protocol family 17
Bridge firewalling registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:1.
Freeing unused kernel memory: 128k freed
Please be patient, while OpenWrt loads ...
- preinit -
Press CTRL-C for failsafe
switching to jffs2
mini_fo: using base directory: /
mini_fo: using storage directory: /jffs
- init -

Please press Enter to activate this console. NET: Registered protocol family 10
lo: Disabled Privacy Extensions
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
device eth0.1 entered promiscuous mode
device eth0 entered promiscuous mode
br-mesh: port 1(eth0.1) entering forwarding state
ip_tables: (C) 2000-2006 Netfilter Core Team
Ebtables v2.0 registered
ip6_tables: (C) 2000-2006 Netfilter Core Team
device eth0.4 entered promiscuous mode
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-wan_vpn: starting userspace STP failed, starting kernel STP
batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1545 (compatibility version 8) loaded
ath_hal: module license 'Proprietary' taints kernel.
Disabling lock debugging due to kernel taint
ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR)
ath_ahb: trunk
wlan: trunk
wlan: mac acl policy registered
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3
IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs
device bat0 entered promiscuous mode
br-mesh: port 2(bat0) entering forwarding state
br-wan_vpn: port 1(eth0.4) entering disabled state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-mesh: port 2(bat0) entering disabled state
br-mesh: port 1(eth0.1) entering disabled state
br-mesh: port 2(bat0) entering forwarding state
br-mesh: port 1(eth0.1) entering forwarding state
batman-adv:Adding interface: ath0
batman-adv:Interface activated: ath0
batman-adv:Adding interface: eth0.2
batman-adv:Interface activated: eth0.2
batman-adv:Adding interface: eth0.3
batman-adv:Interface activated: eth0.3
device VPN entered promiscuous mode
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 2(VPN) entering forwarding state



BusyBox v1.15.3 (2010-01-24 04:59:20 CET) built-in shell (ash)
Enter 'help' for a list of built-in commands.

  _______                     ________        __
 |       |.-----.-----.-----.|  |  |  |.----.|  |_
 |   -   ||  _  |  -__|     ||  |  |  ||   _||   _|
 |_______||   __|_____|__|__||________||__|  |____|
          |__| W I R E L E S S   F R E E D O M
 KAMIKAZE (bleeding edge, r19304) ------------------
  * 10 oz Vodka       Shake well with ice and strain
  * 10 oz Triple sec  mixture into 10 shot glasses.
  * 10 oz lime juice  Salute!
 ---------------------------------------------------
root@OpenWrt:/# batctl
Usage: batctl [options] commands
commands:
        interface|if  [none|interface]          display or modify the interface settings
        originators|o                           display the originator table
        interval|it   [orig_interval]           display or modify the originator interval in ms
        loglevel|ll   [level]                   display or modify the log level
        log|l                                   read the log produced by the kernel module
        translocal|tl                           display the local translation table
        transglobal|tg                          display the global translation table
        vis_server|vs [enable|disable]          display or modify the status of the VIS server
        vis_data|vd [dot|JSON]                  display the VIS data in dot or JSON format
        aggregation|ag   [0|1]                  display or modify the packet aggregation setting

        ping|p        <destination>             ping another batman adv host via layer 2
        traceroute|tr <destination>             traceroute another batman adv host via layer 2
        tcpdump|td    <interface>               tcpdump layer 2 traffic on the given interface
        bisect        <file1> .. <fileN>        analyze given log files for routing stability
options:
        -h print this help (or 'batctl <command> -h' for the command specific help)
        -v print version
root@OpenWrt:/# batctl vs enabled
batman-adv:Setting VIS mode to server (enabling vis server)
root@OpenWrt:/# watchdog invoked oom-killer: gfp_mask=0x1201d2, order=0, oomkilladj=0
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009b644>] oom_kill_process+0x6c/0x1fc
[<8009bd08>] __out_of_memory+0x184/0x1bc
[<8009bdb0>] out_of_memory+0x70/0xa0
[<8009f03c>] __alloc_pages_internal+0x338/0x434
[<800a1e64>] __do_page_cache_readahead+0xe0/0x278
[<800a206c>] do_page_cache_readahead+0x70/0x8c
[<80098880>] filemap_fault+0x19c/0x464
[<800ae778>] __do_fault+0x74/0x4e8
[<800b0e18>] handle_mm_fault+0x314/0x6fc
[<80055d3c>] do_page_fault+0x110/0x30c
[<80041420>] ret_from_exception+0x0/0x24

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:0 inactive_anon:164
 inactive_file:10 dirty:0 writeback:0 unstable:0
 free:126 slab:2222 mapped:0 pagetables:49 bounce:0
Normal free:500kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:0kB inactive_file:40kB present:16256kB pages_scanned:1533 all_unreclaimable? no
lowmem_reserve[]: 0 0
Normal: 1*4kB 0*8kB 1*16kB 1*32kB 1*64kB 1*128kB 1*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 500kB
27 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
10 pages shared
3105 pages non-shared
Out of memory: kill process 967 (dnsmasq) score 225 or a child
Killed process 967 (dnsmasq)
swapper: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb]
[<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb]

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
swapper: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb]
[<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb]

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
wifi0: ath_reset: Unable to start receive logic.
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<80bc2144>] send_raw_packet+0x3c/0xc0 [batman_adv]
[<80bc23bc>] send_outstanding_bat_packet+0x1f4/0x250 [batman_adv]
[<800767b8>] worker_thread+0x1b0/0x278
[<8007ac14>] kthread+0x5c/0xa0
[<8004ec88>] kernel_thread_helper+0x10/0x18

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb]
[<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb]

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
wifi0: ath_reset: Unable to start receive logic.
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<8022bb7c>] br_send_bpdu+0x2c/0x150
[<8022beb8>] br_send_config_bpdu+0x1d0/0x1ec
[<8022b398>] br_transmit_config+0x148/0x180
[<8022b4b4>] br_config_bpdu_generation+0x60/0x8c
[<8022cc58>] br_hello_timer_expired+0x28/0x64
[<8006e204>] run_timer_softirq+0x17c/0x234
[<80068f6c>] __do_softirq+0xa4/0x154
[<80069078>] do_softirq+0x5c/0x98
[<80041444>] ret_from_irq+0x0/0x4
[<800c30e8>] __kmalloc_track_caller+0x1a4/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<80bc2144>] send_raw_packet+0x3c/0xc0 [batman_adv]
[<80bc23bc>] send_outstanding_bat_packet+0x1f4/0x250 [batman_adv]
[<800767b8>] worker_thread+0x1b0/0x278
[<8007ac14>] kthread+0x5c/0xa0
[<8004ec88>] kernel_thread_helper+0x10/0x18

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<8022bb7c>] br_send_bpdu+0x2c/0x150
[<8022beb8>] br_send_config_bpdu+0x1d0/0x1ec
[<8022b398>] br_transmit_config+0x148/0x180
[<8022b4b4>] br_config_bpdu_generation+0x60/0x8c
[<8022cc58>] br_hello_timer_expired+0x28/0x64
[<8006e204>] run_timer_softirq+0x17c/0x234
[<80068f6c>] __do_softirq+0xa4/0x154
[<80069078>] do_softirq+0x5c/0x98
[<80041444>] ret_from_irq+0x0/0x4
[<800c30e8>] __kmalloc_track_caller+0x1a4/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<80bc2144>] send_raw_packet+0x3c/0xc0 [batman_adv]
[<80bc23bc>] send_outstanding_bat_packet+0x1f4/0x250 [batman_adv]
[<800767b8>] worker_thread+0x1b0/0x278
[<8007ac14>] kthread+0x5c/0xa0
[<8004ec88>] kernel_thread_helper+0x10/0x18

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
bat_events: page allocation failure. order:0, mode:0x20
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<8009f0f0>] __alloc_pages_internal+0x3ec/0x434
[<800c2990>] cache_alloc_refill+0x39c/0x950
[<800c30c8>] __kmalloc_track_caller+0x184/0x208
[<801aee18>] __alloc_skb+0x7c/0x158
[<801afd2c>] dev_alloc_skb+0x1c/0x4c
[<8069c2c8>] ieee80211_dev_alloc_skb+0x18/0x60 [ath_ahb]
[<80683f98>] ath_sysctl_unregister+0x2680/0x6ecc [ath_ahb]

Mem-Info:
Normal per-cpu:
CPU    0: hi:    0, btch:   1 usd:   0
Active_anon:160 active_file:3 inactive_anon:164
 inactive_file:3 dirty:0 writeback:0 unstable:0
 free:48 slab:2304 mapped:0 pagetables:49 bounce:0
Normal free:192kB min:508kB low:632kB high:760kB active_anon:640kB inactive_anon:656kB active_file:12kB inactive_file:12kB present:16256kB pages_scanned:2290 all_unreclaimable? yes
lowmem_reserve[]: 0 0
Normal: 0*4kB 0*8kB 0*16kB 0*32kB 1*64kB 1*128kB 0*256kB 0*512kB 0*1024kB 0*2048kB 0*4096kB = 192kB
23 total pagecache pages
0 pages in swap cache
Swap cache stats: add 0, delete 0, find 0/0
Free swap  = 0kB
Total swap = 0kB
4096 pages RAM
739 pages reserved
6 pages shared
3185 pages non-shared
wifi0: ath_reset: Unable to start receive logic.
watchdog expired, rebooting system
+

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-24 20:42   ` Linus Lüssing
@ 2010-01-24 21:00     ` Andrew Lunn
  2010-01-25  6:46       ` Andrew Lunn
  2010-01-25  6:47     ` Andrew Lunn
  1 sibling, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-01-24 21:00 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Sun, Jan 24, 2010 at 09:42:12PM +0100, Linus L??ssing wrote:
> Hi Andrew,
> 
> Here is the output with the symbols inserted. I can reproduce this
> memory leak very reliable now, by just activating the vis server
> (didn't do that with the logs I've posted before) in a couple of
> minutes. free is showing a nice countdown then :).

So it is vis server functionality which is leaking? If vis server is
not enabled, there is no leak?

If so, we don't need kmemcheck. I expect just looking at the vis code
is probably enough to find the missing free.

   Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-24 21:00     ` Andrew Lunn
@ 2010-01-25  6:46       ` Andrew Lunn
  0 siblings, 0 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-01-25  6:46 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Sun, Jan 24, 2010 at 10:00:10PM +0100, Andrew Lunn wrote:
> On Sun, Jan 24, 2010 at 09:42:12PM +0100, Linus L??ssing wrote:
> > Hi Andrew,
> > 
> > Here is the output with the symbols inserted. I can reproduce this
> > memory leak very reliable now, by just activating the vis server
> > (didn't do that with the logs I've posted before) in a couple of
> > minutes. free is showing a nice countdown then :).
> 
> So it is vis server functionality which is leaking? If vis server is
> not enabled, there is no leak?
> 
> If so, we don't need kmemcheck. I expect just looking at the vis code
> is probably enough to find the missing free.

It looks pretty obvious:

Working our way via the call stack:

In batman_skb_recv() we have:

        case BAT_VIS:
                ret = recv_vis_packet(skb);
                break;
        default:
                ret = NET_RX_DROP;
        }
        if (ret == NET_RX_DROP)
                kfree_skb(skb);

i.e. the packet is only freed if recv_vis_packet() returns NET_RX_DROP


In recv_vis_packet() we have:
       switch (vis_packet->vis_type) {
        case VIS_TYPE_SERVER_SYNC:
                /* TODO: handle fragmented skbs properly */
                receive_server_sync_packet(vis_packet, skb_headlen(skb));
                ret = NET_RX_SUCCESS;
                break;

        case VIS_TYPE_CLIENT_UPDATE:
                /* TODO: handle fragmented skbs properly */
                receive_client_update_packet(vis_packet, skb_headlen(skb));
                ret = NET_RX_SUCCESS;
                break;

        default:        /* ignore unknown packet */
                ret = NET_RX_DROP;
                break;
        }
        return ret;

i.e. receive_client_update_packet() and receive_server_sync_packet()
need to sometime result in the packet being freed. batman_skb_recv()
will not free the packet.

void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len)
{
        struct vis_info *info;
        int is_new;
        unsigned long flags;
        int vis_server = atomic_read(&vis_mode);

        spin_lock_irqsave(&vis_hash_lock, flags);
        info = add_packet(vis_packet, vis_info_len, &is_new);
        if (info == NULL)
                goto end;

        /* only if we are server ourselves and packet is newer than the one in
         * hash.*/
        if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) {
                memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
                if (list_empty(&info->send_list))
                        list_add_tail(&info->send_list, &send_list);
        }
end:
        spin_unlock_irqrestore(&vis_hash_lock, flags);
}

No free here. vis_packet is a pointer into the skbuf, but we never add
vis_packet to any queue here, just info is queued.

add_packet():

        info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC);
        if (info == NULL)
                return NULL;

        INIT_LIST_HEAD(&info->send_list);
        INIT_LIST_HEAD(&info->recv_list);
        info->first_seen = jiffies;
        memcpy(&info->packet, vis_packet,
               sizeof(struct vis_packet) + vis_info_len);

We allocate memory can copy the contents of the packet into the
allocated memory. So again, the skbuf is not queued.

As far as i can see, the skbuf is never put on a queue. We just copy
the needed data out of it. This means the skbuf does need freeing.

I had a very quick look at receive_client_update_packet(). It seems to
have the same structure.

I will submit a fix for this soon. Is there any other similar code
which might have the same bug?

      Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-24 20:42   ` Linus Lüssing
  2010-01-24 21:00     ` Andrew Lunn
@ 2010-01-25  6:47     ` Andrew Lunn
  2010-01-25  8:21       ` Marek Lindner
  2010-01-26  1:48       ` Linus Lüssing
  1 sibling, 2 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-01-25  6:47 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

Staging: batman-adv: Fix skbuff leak in VIS code.

The vis code takes a copy of the data inside the skbuf if it is interesting 
for us, so we always need to release the skbuf.

Reported-by: Linus Lüssing <linus.luessing@web.de>
Signed-off-by: Andrew Lunn <andrew@lunn.ch>

Index: routing.c
===================================================================
--- routing.c	(revision 1563)
+++ routing.c	(working copy)
@@ -1135,21 +1135,22 @@
 	if (is_my_mac(vis_packet->sender_orig))
 		return NET_RX_DROP;
 
+	/* We take a copy of the data in the packet, so we should
+	   always free the skbuf. */
+	ret = NET_RX_DROP;
+	
 	switch (vis_packet->vis_type) {
 	case VIS_TYPE_SERVER_SYNC:
 		/* TODO: handle fragmented skbs properly */
 		receive_server_sync_packet(vis_packet, skb_headlen(skb));
-		ret = NET_RX_SUCCESS;
 		break;
 
 	case VIS_TYPE_CLIENT_UPDATE:
 		/* TODO: handle fragmented skbs properly */
 		receive_client_update_packet(vis_packet, skb_headlen(skb));
-		ret = NET_RX_SUCCESS;
 		break;
 
 	default:	/* ignore unknown packet */
-		ret = NET_RX_DROP;
 		break;
 	}
 	return ret;

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-25  6:47     ` Andrew Lunn
@ 2010-01-25  8:21       ` Marek Lindner
  2010-01-26  1:48       ` Linus Lüssing
  1 sibling, 0 replies; 37+ messages in thread
From: Marek Lindner @ 2010-01-25  8:21 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Monday 25 January 2010 14:47:47 Andrew Lunn wrote:
> Staging: batman-adv: Fix skbuff leak in VIS code.
> 
> The vis code takes a copy of the data inside the skbuf if it is
>  interesting  for us, so we always need to release the skbuf.

Applied in rev 1564.

Thanks,
Marek

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint)
  2010-01-25  6:47     ` Andrew Lunn
  2010-01-25  8:21       ` Marek Lindner
@ 2010-01-26  1:48       ` Linus Lüssing
  1 sibling, 0 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-01-26  1:48 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

Just tried the patch on the routers and it works perfectly, no
more memory leak caused by vis! Thanks for the quick fix.

Cheers, Linus

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing
  2010-01-23 18:10 ` Andrew Lunn
  2010-01-24  4:24 ` Linus Lüssing
@ 2010-01-26  6:13 ` Linus Lüssing
  2010-01-26  7:16   ` Marek Lindner
  2010-01-29  8:25   ` Andrew Lunn
  2 siblings, 2 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-01-26  6:13 UTC (permalink / raw)
  To: b.a.t.m.a.n

[-- Attachment #1: Type: text/plain, Size: 350 bytes --]

I'm now able to reproduce the second problem as well:
Activating the vis-server on one node causes the vis-clients to
throw this call trace with its slow path warning imediately.
Nevertheless, vis output works anyway, clients are sending
vis-packets and the server is drawing nice graphs.
See the attachment for an example call trace.

Cheers, Linus

[-- Attachment #2: slowpath-warning.log --]
[-- Type: text/plain, Size: 7966 bytes --]

Linux version 2.6.30.10 (linus@Linus-Debian) (gcc version 4.3.3 (GCC) ) #3 Tue Jan 26 02:18:56 CET 2010
CPU revision is: 00019064 (MIPS 4KEc)
Determined physical RAM map:
 memory: 01000000 @ 00000000 (usable)
Initrd not found or empty - disabling initrd
Zone PFN ranges:
  Normal   0x00000000 -> 0x00001000
Movable zone start PFN for each node
early_node_map[1] active PFN ranges
    0: 0x00000000 -> 0x00001000
On node 0 totalpages: 4096
free_area_init_node: node 0, pgdat 802a0f40, node_mem_map 802d9000
  Normal zone: 32 pages used for memmap
  Normal zone: 0 pages reserved
  Normal zone: 4064 pages, LIFO batch:0
Built 1 zonelists in Zone order, mobility grouping off.  Total pages: 4064
Kernel command line: console=ttyS0,9600 rootfstype=squashfs,jffs2
Primary instruction cache 16kB, VIPT, 4-way, linesize 16 bytes.
Primary data cache 16kB, 4-way, VIPT, no aliases, linesize 16 bytes
NR_IRQS:128
PID hash table entries: 64 (order: 6, 256 bytes)
console [ttyS0] enabled
Dentry cache hash table entries: 2048 (order: 1, 8192 bytes)
Inode-cache hash table entries: 1024 (order: 0, 4096 bytes)
Memory: 13324k/16384k available (1985k kernel code, 3060k reserved, 452k data, 128k init, 0k highmem)
Calibrating delay loop... 183.50 BogoMIPS (lpj=917504)
Mount-cache hash table entries: 512
net_namespace: 732 bytes
NET: Registered protocol family 16
bio: create slab <bio-0> at 0
Switched to high resolution mode on CPU 0
NET: Registered protocol family 2
IP route cache hash table entries: 1024 (order: 0, 4096 bytes)
TCP established hash table entries: 512 (order: 0, 4096 bytes)
TCP bind hash table entries: 512 (order: -1, 2048 bytes)
TCP: Hash tables configured (established 512 bind 512)
TCP reno registered
NET: Registered protocol family 1
Radio config found at offset 0xf8(0x1f8)
squashfs: version 4.0 (2009/01/31) Phillip Lougher
Registering mini_fo version $Id$
JFFS2 version 2.2. (NAND) (SUMMARY)  © 2001-2006 Red Hat, Inc.
msgmni has been set to 26
io scheduler noop registered
io scheduler deadline registered (default)
gpiodev: gpio device registered with major 254
gpiodev: gpio platform device registered with access mask FFFFFFFF
Serial: 8250/16550 driver, 1 ports, IRQ sharing disabled
serial8250: ttyS0 at MMIO 0xb1100003 (irq = 37) is a 16550A
eth0: Atheros AR231x: 00:22:b0:98:87:fa, irq 4
ar231x_eth_mii: probed
eth0: attached PHY driver [IC+ IP175C] (mii_bus:phy_addr=0:00)
cmdlinepart partition parsing not available
Searching for RedBoot partition table in spiflash at offset 0x3d0000
Searching for RedBoot partition table in spiflash at offset 0x3e0000
6 RedBoot partitions found on MTD device spiflash
Creating 6 MTD partitions on "spiflash":
0x000000000000-0x000000030000 : "RedBoot"
0x000000030000-0x0000002f0000 : "rootfs"
mtd: partition "rootfs" set to be root filesystem
mtd: partition "rootfs_data" created automatically, ofs=230000, len=C0000
0x000000230000-0x0000002f0000 : "rootfs_data"
0x0000002f0000-0x0000003d0000 : "vmlinux.bin.l7"
0x0000003e0000-0x0000003ef000 : "FIS directory"
0x0000003ef000-0x0000003f0000 : "RedBoot config"
0x0000003f0000-0x000000400000 : "boardconfig"
TCP westwood registered
NET: Registered protocol family 17
Bridge firewalling registered
802.1Q VLAN Support v1.8 Ben Greear <greearb@candelatech.com>
All bugs added by David S. Miller <davem@redhat.com>
VFS: Mounted root (squashfs filesystem) readonly on device 31:1.
Freeing unused kernel memory: 128k freed
Please be patient, while OpenWrt loads ...
mini_fo: using base directory: /
mini_fo: using storage directory: /tmp/root
NET: Registered protocol family 10
lo: Disabled Privacy Extensions
tun: Universal TUN/TAP device driver, 1.6
tun: (C) 1999-2004 Max Krasnyansky <maxk@qualcomm.com>
ip_tables: (C) 2000-2006 Netfilter Core Team
device eth0.1 entered promiscuous mode
device eth0 entered promiscuous mode
br-mesh: port 1(eth0.1) entering forwarding state
Ebtables v2.0 registered
ip6_tables: (C) 2000-2006 Netfilter Core Team
batman-adv:B.A.T.M.A.N. advanced 0.2.1-beta r1564 (compatibility version 8) loaded
ath_hal: module license 'Proprietary' taints kernel.
Disabling lock debugging due to kernel taint
ath_hal: 2009-05-08 (AR5212, AR5312, RF5111, RF5112, RF2316, RF2317, REGOPS_FUNC, TX_DESC_SWAP, XR)
device eth0.4 entered promiscuous mode
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-wan_vpn: starting userspace STP failed, starting kernel STP
eth0.1: no IPv6 routers present
br-mesh: no IPv6 routers present
ath_ahb: trunk
wlan: trunk
wlan: mac acl policy registered
ath_rate_minstrel: Minstrel automatic rate control algorithm 1.2 (trunk)
ath_rate_minstrel: look around rate set to 10%
ath_rate_minstrel: EWMA rolloff level set to 75%
ath_rate_minstrel: max segment size in the mrr set to 6000 us
Atheros HAL provided by OpenWrt, DD-WRT and MakSat Technologies
wifi0: 11b rates: 1Mbps 2Mbps 5.5Mbps 11Mbps
wifi0: 11g rates: 1Mbps 2Mbps 5.5Mbps 11Mbps 6Mbps 9Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: turboG rates: 6Mbps 12Mbps 18Mbps 24Mbps 36Mbps 48Mbps 54Mbps
wifi0: H/W encryption support: WEP AES AES_CCM TKIP
ath_ahb: wifi0: Atheros 2317 WiSoC REV1: mem=0xb0000000, irq=3
IRQ 3/wifi0: IRQF_DISABLED is not guaranteed on shared IRQs
eth0.2: no IPv6 routers present
device bat0 entered promiscuous mode
br-mesh: port 2(bat0) entering forwarding state
eth0.3: no IPv6 routers present
eth0.4: no IPv6 routers present
bat0: no IPv6 routers present
br-wan_vpn: port 1(eth0.4) entering disabled state
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 1(eth0.4) entering forwarding state
br-mesh: port 2(bat0) entering disabled state
br-mesh: port 1(eth0.1) entering disabled state
br-mesh: port 2(bat0) entering forwarding state
br-mesh: port 1(eth0.1) entering forwarding state
br-wan_vpn: received tcn bpdu on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
ath0: no IPv6 routers present
br-mesh: no IPv6 routers present
batman-adv:Adding interface: ath0
batman-adv:Interface activated: ath0
batman-adv:Adding interface: eth0.2
batman-adv:Interface activated: eth0.2
batman-adv:Adding interface: eth0.3
batman-adv:Interface activated: eth0.3
jffs2_scan_eraseblock(): End of filesystem marker found at 0x0
jffs2_build_filesystem(): unlocking the mtd device... done.
jffs2_build_filesystem(): erasing all blocks after the end marker... <6>device VPN entered promiscuous mode
br-wan_vpn: topology change detected, propagating
br-wan_vpn: port 2(VPN) entering forwarding state
done.
mini_fo: using base directory: /
mini_fo: using storage directory: /jffs
br-wan_vpn: received tcn bpdu on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
br-wan_vpn: received tcn bpdu on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
br-wan_vpn: received tcn bpdu on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
------------[ cut here ]------------
WARNING: at kernel/softirq.c:141 local_bh_enable+0x48/0xa0()
Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
Call Trace:
[<800493a4>] dump_stack+0x8/0x34
[<80060cd8>] warn_slowpath_common+0x70/0xb0
[<800674ac>] local_bh_enable+0x48/0xa0
[<801b3ef8>] dev_queue_xmit+0x388/0x3dc
[<8062c454>] ieee80211_parent_queue_xmit+0x8c/0xb4 [ath_ahb]
[<8062c778>] ieee80211_hardstart+0x2fc/0x34c [ath_ahb]
[<801b3e5c>] dev_queue_xmit+0x2ec/0x3dc
[<80bb68d8>] vis_quit+0x7c4/0x950 [batman_adv]

---[ end trace e96b16c908cf7c40 ]---
br-wan_vpn: received tcn bpdu on port 1(eth0.4)
br-wan_vpn: topology change detected, propagating
root@OpenWrt:~#

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-26  6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing
@ 2010-01-26  7:16   ` Marek Lindner
  2010-01-27  0:10     ` Linus Lüssing
  2010-01-29  8:25   ` Andrew Lunn
  1 sibling, 1 reply; 37+ messages in thread
From: Marek Lindner @ 2010-01-26  7:16 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: Text/Plain, Size: 484 bytes --]

On Tuesday 26 January 2010 14:13:11 Linus Lüssing wrote:
> I'm now able to reproduce the second problem as well:
> Activating the vis-server on one node causes the vis-clients to
> throw this call trace with its slow path warning imediately.
> Nevertheless, vis output works anyway, clients are sending
> vis-packets and the server is drawing nice graphs.
> See the attachment for an example call trace.

Please try the attached patch and see if it helps.

Regards,
Marek

[-- Attachment #2: no_lock_while_send.patch --]
[-- Type: text/x-patch, Size: 3998 bytes --]

diff --git a/batman-adv-kernelland/vis.c b/batman-adv-kernelland/vis.c
index b118d1e..a6c235f 100644
--- a/batman-adv-kernelland/vis.c
+++ b/batman-adv-kernelland/vis.c
@@ -405,6 +405,9 @@ static void purge_vis_packets(void)
 {
 	HASHIT(hashit);
 	struct vis_info *info;
+	unsigned long flags;
+
+	spin_lock_irqsave(&vis_hash_lock, flags);
 
 	while (hash_iterate(vis_hash, &hashit)) {
 		info = hashit.bucket->data;
@@ -416,13 +419,17 @@ static void purge_vis_packets(void)
 			free_info(info);
 		}
 	}
+
+	spin_unlock_irqrestore(&vis_hash_lock, flags);
 }
 
 static void broadcast_vis_packet(struct vis_info *info, int packet_length)
 {
 	HASHIT(hashit);
 	struct orig_node *orig_node;
+	struct batman_if *batman_if;
 	unsigned long flags;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 
@@ -431,45 +438,60 @@ static void broadcast_vis_packet(struct vis_info *info, int packet_length)
 		orig_node = hashit.bucket->data;
 
 		/* if it's a vis server and reachable, send it. */
-		if (orig_node &&
-		    (orig_node->flags & VIS_SERVER) &&
-		    orig_node->batman_if &&
-		    orig_node->router) {
+		if ((!orig_node) || (!orig_node->batman_if) ||
+		    (!orig_node->router))
+			continue;
 
-			/* don't send it if we already received the packet from
-			 * this node. */
-			if (recv_list_is_in(&info->recv_list, orig_node->orig))
-				continue;
+		if (!(orig_node->flags & VIS_SERVER))
+			continue;
 
-			memcpy(info->packet.target_orig,
-			       orig_node->orig, ETH_ALEN);
+		/* don't send it if we already received the packet from
+		 * this node. */
+		if (recv_list_is_in(&info->recv_list, orig_node->orig))
+			continue;
 
-			send_raw_packet((unsigned char *) &info->packet,
-					packet_length,
-					orig_node->batman_if,
-					orig_node->router->addr);
-		}
+		spin_unlock_irqrestore(&orig_hash_lock, flags);
+
+		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
+		batman_if = orig_node->batman_if;
+		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
+
+		send_raw_packet((unsigned char *)&info->packet,
+				packet_length, batman_if, dstaddr);
+
+		spin_lock_irqsave(&orig_hash_lock, flags);
 	}
-	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
+
 	spin_unlock_irqrestore(&orig_hash_lock, flags);
+	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
 }
 
 static void unicast_vis_packet(struct vis_info *info, int packet_length)
 {
 	struct orig_node *orig_node;
+	struct batman_if *batman_if;
 	unsigned long flags;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 	orig_node = ((struct orig_node *)
 		     hash_find(orig_hash, info->packet.target_orig));
 
-	if ((orig_node != NULL) &&
-	    (orig_node->batman_if != NULL) &&
-	    (orig_node->router != NULL)) {
-		send_raw_packet((unsigned char *) &info->packet, packet_length,
-				orig_node->batman_if,
-				orig_node->router->addr);
-	}
+	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
+		goto out;
+
+	/* don't lock while sending the packets ... we therefore
+	 * copy the required data before sending */
+	batman_if = orig_node->batman_if;
+	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
+
+	send_raw_packet((unsigned char *)&info->packet,
+			packet_length, batman_if, dstaddr);
+
+	return;
+
+out:
 	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
@@ -502,17 +524,18 @@ static void send_vis_packets(struct work_struct *work)
 	struct vis_info *info, *temp;
 	unsigned long flags;
 
-	spin_lock_irqsave(&vis_hash_lock, flags);
 	purge_vis_packets();
 
 	if (generate_vis_packet() == 0)
 		/* schedule if generation was successful */
 		list_add_tail(&my_vis_info->send_list, &send_list);
 
+	spin_lock_irqsave(&vis_hash_lock, flags);
 	list_for_each_entry_safe(info, temp, &send_list, send_list) {
 		list_del_init(&info->send_list);
 		send_vis_packet(info);
 	}
+
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
 	start_vis_timer();
 }

^ permalink raw reply related	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-26  7:16   ` Marek Lindner
@ 2010-01-27  0:10     ` Linus Lüssing
  2010-01-28  0:09       ` Marek Lindner
  0 siblings, 1 reply; 37+ messages in thread
From: Linus Lüssing @ 2010-01-27  0:10 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 5083 bytes --]

Hi Marek,

nope, does not seem to work, still the same issue
I also tried it on my laptop here with the patch installed 
and it is a very similar call trace, although here it is 
send_vis_packets instead of vis_quit in the call trace.

Cheers, Linus

On Tue, Jan 26, 2010 at 03:16:38PM +0800, Marek Lindner wrote:
> On Tuesday 26 January 2010 14:13:11 Linus Lüssing wrote:
> > I'm now able to reproduce the second problem as well:
> > Activating the vis-server on one node causes the vis-clients to
> > throw this call trace with its slow path warning imediately.
> > Nevertheless, vis output works anyway, clients are sending
> > vis-packets and the server is drawing nice graphs.
> > See the attachment for an example call trace.
> 
> Please try the attached patch and see if it helps.
> 
> Regards,
> Marek

> diff --git a/batman-adv-kernelland/vis.c b/batman-adv-kernelland/vis.c
> index b118d1e..a6c235f 100644
> --- a/batman-adv-kernelland/vis.c
> +++ b/batman-adv-kernelland/vis.c
> @@ -405,6 +405,9 @@ static void purge_vis_packets(void)
>  {
>  	HASHIT(hashit);
>  	struct vis_info *info;
> +	unsigned long flags;
> +
> +	spin_lock_irqsave(&vis_hash_lock, flags);
>  
>  	while (hash_iterate(vis_hash, &hashit)) {
>  		info = hashit.bucket->data;
> @@ -416,13 +419,17 @@ static void purge_vis_packets(void)
>  			free_info(info);
>  		}
>  	}
> +
> +	spin_unlock_irqrestore(&vis_hash_lock, flags);
>  }
>  
>  static void broadcast_vis_packet(struct vis_info *info, int packet_length)
>  {
>  	HASHIT(hashit);
>  	struct orig_node *orig_node;
> +	struct batman_if *batman_if;
>  	unsigned long flags;
> +	uint8_t dstaddr[ETH_ALEN];
>  
>  	spin_lock_irqsave(&orig_hash_lock, flags);
>  
> @@ -431,45 +438,60 @@ static void broadcast_vis_packet(struct vis_info *info, int packet_length)
>  		orig_node = hashit.bucket->data;
>  
>  		/* if it's a vis server and reachable, send it. */
> -		if (orig_node &&
> -		    (orig_node->flags & VIS_SERVER) &&
> -		    orig_node->batman_if &&
> -		    orig_node->router) {
> +		if ((!orig_node) || (!orig_node->batman_if) ||
> +		    (!orig_node->router))
> +			continue;
>  
> -			/* don't send it if we already received the packet from
> -			 * this node. */
> -			if (recv_list_is_in(&info->recv_list, orig_node->orig))
> -				continue;
> +		if (!(orig_node->flags & VIS_SERVER))
> +			continue;
>  
> -			memcpy(info->packet.target_orig,
> -			       orig_node->orig, ETH_ALEN);
> +		/* don't send it if we already received the packet from
> +		 * this node. */
> +		if (recv_list_is_in(&info->recv_list, orig_node->orig))
> +			continue;
>  
> -			send_raw_packet((unsigned char *) &info->packet,
> -					packet_length,
> -					orig_node->batman_if,
> -					orig_node->router->addr);
> -		}
> +		spin_unlock_irqrestore(&orig_hash_lock, flags);
> +
> +		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
> +		batman_if = orig_node->batman_if;
> +		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
> +
> +		send_raw_packet((unsigned char *)&info->packet,
> +				packet_length, batman_if, dstaddr);
> +
> +		spin_lock_irqsave(&orig_hash_lock, flags);
>  	}
> -	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
> +
>  	spin_unlock_irqrestore(&orig_hash_lock, flags);
> +	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
>  }
>  
>  static void unicast_vis_packet(struct vis_info *info, int packet_length)
>  {
>  	struct orig_node *orig_node;
> +	struct batman_if *batman_if;
>  	unsigned long flags;
> +	uint8_t dstaddr[ETH_ALEN];
>  
>  	spin_lock_irqsave(&orig_hash_lock, flags);
>  	orig_node = ((struct orig_node *)
>  		     hash_find(orig_hash, info->packet.target_orig));
>  
> -	if ((orig_node != NULL) &&
> -	    (orig_node->batman_if != NULL) &&
> -	    (orig_node->router != NULL)) {
> -		send_raw_packet((unsigned char *) &info->packet, packet_length,
> -				orig_node->batman_if,
> -				orig_node->router->addr);
> -	}
> +	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
> +		goto out;
> +
> +	/* don't lock while sending the packets ... we therefore
> +	 * copy the required data before sending */
> +	batman_if = orig_node->batman_if;
> +	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
> +	spin_unlock_irqrestore(&orig_hash_lock, flags);
> +
> +	send_raw_packet((unsigned char *)&info->packet,
> +			packet_length, batman_if, dstaddr);
> +
> +	return;
> +
> +out:
>  	spin_unlock_irqrestore(&orig_hash_lock, flags);
>  }
>  
> @@ -502,17 +524,18 @@ static void send_vis_packets(struct work_struct *work)
>  	struct vis_info *info, *temp;
>  	unsigned long flags;
>  
> -	spin_lock_irqsave(&vis_hash_lock, flags);
>  	purge_vis_packets();
>  
>  	if (generate_vis_packet() == 0)
>  		/* schedule if generation was successful */
>  		list_add_tail(&my_vis_info->send_list, &send_list);
>  
> +	spin_lock_irqsave(&vis_hash_lock, flags);
>  	list_for_each_entry_safe(info, temp, &send_list, send_list) {
>  		list_del_init(&info->send_list);
>  		send_vis_packet(info);
>  	}
> +
>  	spin_unlock_irqrestore(&vis_hash_lock, flags);
>  	start_vis_timer();
>  }


[-- Attachment #2: call-trace-0104-27-01-10.log --]
[-- Type: text/plain, Size: 2770 bytes --]

[22933.613178] ------------[ cut here ]------------
[22933.613199] WARNING: at /tmp/buildd/linux-2.6-2.6.31/debian/build/source_i386_none/kernel/softirq.c:143 _local_bh_enable_ip+0x41/0x97()
[22933.613206] Hardware name: ���������������
[22933.613211] Modules linked in: batman_adv isofs udf nvidia(P) ppdev lp parport sco bridge stp bnep rfcomm kvm_intel kvm acpi_cpufreq cpufreq_powersave cpufreq_conservative cpufreq_userspace cpufreq_stats l2cap nfsd exportfs nfs lockd fscache nfs_acl auth_rpcgss sunrpc fuse dm_snapshot dm_mirror dm_region_hash dm_log firewire_sbp2 loop snd_hda_codec_realtek snd_hda_intel snd_hda_codec arc4 ecb acer_wmi iwlagn snd_hwdep iwlcore snd_pcm_oss snd_mixer_oss snd_pcm mac80211 snd_seq_midi snd_rawmidi snd_seq_midi_event snd_seq snd_timer snd_seq_device uvcvideo videodev snd soundcore i2c_i801 cp210x v4l1_compat btusb snd_page_alloc cfg80211 v4l2_compat_ioctl32 psmouse bluetooth wmi serio_raw rfkill i2c_core usbserial evdev pcspkr ac processor button battery ext3 jbd mbcache sha256_generic cryptd aes_x86_64 aes_generic cbc dm_crypt dm_mod sd_mod crc_t10dif ide_cd_mod cdrom ata_generic ide_pci_generic ahci sdhci_pci sdhci uhci_hcd mmc_core libata tg3 led_class libphy firewire_ohci firewire_core crc_itu_t ricoh_mmc video output piix ide_core scsi_mod ehci_hcd intel_agp thermal fan thermal_sys [last unloaded: batman_adv]
[22933.613436] Pid: 14882, comm: bat_events Tainted: P           2.6.31-1-amd64 #1
[22933.613442] Call Trace:
[22933.613453]  [<ffffffff810503fa>] ? _local_bh_enable_ip+0x41/0x97
[22933.613462]  [<ffffffff810503fa>] ? _local_bh_enable_ip+0x41/0x97
[22933.613471]  [<ffffffff8104af08>] ? warn_slowpath_common+0x77/0xa3
[22933.613480]  [<ffffffff810503fa>] ? _local_bh_enable_ip+0x41/0x97
[22933.613490]  [<ffffffff812c821f>] ? run_filter+0x33/0x3a
[22933.613498]  [<ffffffff812c8c56>] ? packet_rcv+0xa9/0x2fb
[22933.613511]  [<ffffffff812389e5>] ? dev_hard_start_xmit+0x143/0x2ba
[22933.613521]  [<ffffffff8124ac7e>] ? __qdisc_run+0xe2/0x1ec
[22933.613529]  [<ffffffff81238e8f>] ? dev_queue_xmit+0x232/0x32c
[22933.613551]  [<ffffffffa0fe8945>] ? send_vis_packets+0x4f5/0x5d5 [batman_adv]
[22933.613571]  [<ffffffffa0fe8450>] ? send_vis_packets+0x0/0x5d5 [batman_adv]
[22933.613581]  [<ffffffff8105b43b>] ? worker_thread+0x174/0x211
[22933.613590]  [<ffffffff8105f39e>] ? autoremove_wake_function+0x0/0x2e
[22933.613599]  [<ffffffff8105b2c7>] ? worker_thread+0x0/0x211
[22933.613606]  [<ffffffff8105f042>] ? kthread+0x8b/0x93
[22933.613616]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
[22933.613623]  [<ffffffff8105efb7>] ? kthread+0x0/0x93
[22933.613631]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20
[22933.613636] ---[ end trace 1758b3e01bf5c445 ]---

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-27  0:10     ` Linus Lüssing
@ 2010-01-28  0:09       ` Marek Lindner
  2010-01-28  6:29         ` Andrew Lunn
  0 siblings, 1 reply; 37+ messages in thread
From: Marek Lindner @ 2010-01-28  0:09 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking


Hi,

> nope, does not seem to work, still the same issue
> I also tried it on my laptop here with the patch installed 
> and it is a very similar call trace, although here it is 
> send_vis_packets instead of vis_quit in the call trace.

you should take the stack trace with a grain of salt. Stack tracing such an 
issue is a quite tricky thing, hence can not be fully trusted. Since the skb 
changes the code runs in interrupt context which probably introduced this bug. 
All previous warnings of this kind were related to holding a lock while 
sending packets. My patch unlocked the problematic orig_hash - maybe that was 
not enough ? By adding retrun statements at the beginning of the vis send 
function you might be able to get down to the problem.

Regards,
Marek

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-28  0:09       ` Marek Lindner
@ 2010-01-28  6:29         ` Andrew Lunn
  0 siblings, 0 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-01-28  6:29 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Thu, Jan 28, 2010 at 08:09:57AM +0800, Marek Lindner wrote:
> 
> Hi,
> 
> > nope, does not seem to work, still the same issue
> > I also tried it on my laptop here with the patch installed 
> > and it is a very similar call trace, although here it is 
> > send_vis_packets instead of vis_quit in the call trace.
> 
> you should take the stack trace with a grain of salt. Stack tracing such an 
> issue is a quite tricky thing, hence can not be fully trusted. Since the skb 
> changes the code runs in interrupt context which probably introduced this bug. 
> All previous warnings of this kind were related to holding a lock while 
> sending packets. My patch unlocked the problematic orig_hash - maybe that was 
> not enough ? By adding retrun statements at the beginning of the vis send 
> function you might be able to get down to the problem.

It might also be worth running lockdep on the code. Normally you don^t
need the actually lockup, you just need to execute the code path that
would lockup under whatever conditions are required for it to lockup.

      Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-26  6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing
  2010-01-26  7:16   ` Marek Lindner
@ 2010-01-29  8:25   ` Andrew Lunn
  2010-01-29  8:59     ` Marek Lindner
  1 sibling, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-01-29  8:25 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

> WARNING: at kernel/softirq.c:141 local_bh_enable+0x48/0xa0()
> Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
> Call Trace:
> [<800493a4>] dump_stack+0x8/0x34
> [<80060cd8>] warn_slowpath_common+0x70/0xb0
> [<800674ac>] local_bh_enable+0x48/0xa0
> [<801b3ef8>] dev_queue_xmit+0x388/0x3dc
> [<8062c454>] ieee80211_parent_queue_xmit+0x8c/0xb4 [ath_ahb]
> [<8062c778>] ieee80211_hardstart+0x2fc/0x34c [ath_ahb]
> [<801b3e5c>] dev_queue_xmit+0x2ec/0x3dc
> [<80bb68d8>] vis_quit+0x7c4/0x950 [batman_adv]
> 
> ---[ end trace e96b16c908cf7c40 ]---
> br-wan_vpn: received tcn bpdu on port 1(eth0.4)
> br-wan_vpn: topology change detected, propagating
> root@OpenWrt:~#

I dug a little deeper into what his means. It means we are in
interrupt context and interrupts are disabled. This seems to be not
allowed when calling local_bh_enable(), which probably means it is not
allowed when calling dev_queue_xmit. 

What i don't understand yet is the call path. The Call Trace above
does not make much sense. vis_quit() does not send any packets.

Looking at Marek's patch it is going in the right direction.
send_skb_packet() cannot be called with interrupts disabled.
which means send_raw_packet() cannot be called with interrupts disabled.
which means broadcast_vis_packet() needs to spin_unlock_irqrestore() before sending.
which means unicast_vis_packet() needs to spin_unlock_irqrestore() before sending.

This is what Mareks patch does. However, this part of Marek's patch looks wrong:

        struct vis_info *info, *temp;
        unsigned long flags;

-       spin_lock_irqsave(&vis_hash_lock, flags);
        purge_vis_packets();

        if (generate_vis_packet() == 0)
                /* schedule if generation was successful */
                list_add_tail(&my_vis_info->send_list, &send_list);

+       spin_lock_irqsave(&vis_hash_lock, flags);
        list_for_each_entry_safe(info, temp, &send_list, send_list) {
                list_del_init(&info->send_list);
                send_vis_packet(info);
        }
+
        spin_unlock_irqrestore(&vis_hash_lock, flags);
        start_vis_timer();
 }

You are disable interrupts and then call send_vis_packet(). This will
not work since all calls under send_vis_packet will have interrupts
disabled.

	Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-29  8:25   ` Andrew Lunn
@ 2010-01-29  8:59     ` Marek Lindner
  2010-01-30 16:50       ` Andrew Lunn
  0 siblings, 1 reply; 37+ messages in thread
From: Marek Lindner @ 2010-01-29  8:59 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Friday 29 January 2010 16:25:45 Andrew Lunn wrote:
> This is what Mareks patch does. However, this part of Marek's patch looks
>  wrong:

I'd like to point out that this lock wasn't introduced by my previously posted 
patch. I managed to reduce the effect of this lock but you can't remove it 
entirely without changing the whole sending mechanism.


> You are disable interrupts and then call send_vis_packet(). This will
> not work since all calls under send_vis_packet will have interrupts
> disabled.

Yes, that is correct. The reason for this is simple: The interrupt flags are 
stored in send_vis_packets() and without passing them on we can't re-enable 
the interrupts. That did not appear to be a good solution.  ;)
As far as I understand the vis_lock's only purpose is to avoid that the "info" 
pointer which is passed to send_vis_packet() can't be deleted while we use it. 
That might be better solved using reference counting. 

Cheers,
Marek

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-29  8:59     ` Marek Lindner
@ 2010-01-30 16:50       ` Andrew Lunn
  2010-01-31 19:37         ` Linus Lüssing
  2010-02-11  9:46         ` Andrew Lunn
  0 siblings, 2 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-01-30 16:50 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

Hi Linus

Please could you try this patch and see if it fixes the vis problem.
It compiles cleanly with 2.6.32. It is checkpatch clean, it is sparse
clean. However, since i've not actually tried running the code i would
not be too surprised if it deadlocked, leaked memory, oopsed, ...

    Andrew

Staging: batman-adv: Don't have interrupts disabled while sending.

send_vis_packets() would disable interrupts before calling
dev_queue_xmit() which resulting in a backtrace in local_bh_enable().
Fix this by using kref on the vis_info object so that we can call
send_vis_packets() without holding vis_hash_lock. vis_hash_lock also
used to protect recv_list, so we now need a new lock to protect that
instead of vis_hash_lock.

Also a few checkpatch cleanups.

Reported-by: Linus Lüssing <linus.luessing@web.de>
Signed-off-by: Andrew Lunn <andrew@lunn.ch>

Index: vis.c
===================================================================
--- vis.c	(revision 1568)
+++ vis.c	(working copy)
@@ -30,22 +30,26 @@
 
 struct hashtable_t *vis_hash;
 DEFINE_SPINLOCK(vis_hash_lock);
+static DEFINE_SPINLOCK(recv_list_lock);
 static struct vis_info *my_vis_info;
 static struct list_head send_list;	/* always locked with vis_hash_lock */
 
 static void start_vis_timer(void);
 
 /* free the info */
-static void free_info(void *data)
+static void free_info(struct kref *ref)
 {
-	struct vis_info *info = data;
+	struct vis_info *info = container_of(ref, struct vis_info, refcount);
 	struct recvlist_node *entry, *tmp;
+	unsigned long flags;
 
 	list_del_init(&info->send_list);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry_safe(entry, tmp, &info->recv_list, list) {
 		list_del(&entry->list);
 		kfree(entry);
 	}
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	kfree(info);
 }
 
@@ -147,32 +151,41 @@
 static void recv_list_add(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
+
 	entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC);
 	if (!entry)
 		return;
 
 	memcpy(entry->mac, mac, ETH_ALEN);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_add_tail(&entry->list, recv_list);
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 }
 
 /* returns 1 if this mac is in the recv_list */
 static int recv_list_is_in(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
 
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry(entry, recv_list, list) {
-		if (memcmp(entry->mac, mac, ETH_ALEN) == 0)
+		if (memcmp(entry->mac, mac, ETH_ALEN) == 0) {
+			spin_unlock_irqrestore(&recv_list_lock, flags);
 			return 1;
+		}
 	}
-
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	return 0;
 }
 
 /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old,
- * broken.. ).  vis hash must be locked outside.  is_new is set when the packet
+ * broken.. ).	vis hash must be locked outside.  is_new is set when the packet
  * is newer than old entries in the hash. */
 static struct vis_info *add_packet(struct vis_packet *vis_packet,
-				   int vis_info_len, int *is_new)
+				   int vis_info_len, int *is_new,
+				   int make_broadcast)
 {
 	struct vis_info *info, *old_info;
 	struct vis_info search_elem;
@@ -199,7 +212,7 @@
 		}
 		/* remove old entry */
 		hash_remove(vis_hash, old_info);
-		free_info(old_info);
+		kref_put(&old_info->refcount, free_info);
 	}
 
 	info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC);
@@ -208,6 +221,7 @@
 
 	INIT_LIST_HEAD(&info->send_list);
 	INIT_LIST_HEAD(&info->recv_list);
+	kref_init(&info->refcount);
 	info->first_seen = jiffies;
 	memcpy(&info->packet, vis_packet,
 	       sizeof(struct vis_packet) + vis_info_len);
@@ -215,16 +229,21 @@
 	/* initialize and add new packet. */
 	*is_new = 1;
 
+	/* Make it a broadcast packet, if required */
+	if (make_broadcast)
+		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
+
 	/* repair if entries is longer than packet. */
 	if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len)
-		info->packet.entries = vis_info_len / sizeof(struct vis_info_entry);
+		info->packet.entries = vis_info_len /
+			sizeof(struct vis_info_entry);
 
 	recv_list_add(&info->recv_list, info->packet.sender_orig);
 
 	/* try to add it */
 	if (hash_add(vis_hash, info) < 0) {
 		/* did not work (for some reason) */
-		free_info(info);
+		kref_put(&old_info->refcount, free_info);
 		info = NULL;
 	}
 
@@ -235,19 +254,20 @@
 void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len)
 {
 	struct vis_info *info;
-	int is_new;
+	int is_new, make_broadcast;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
 
+	make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC);
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast);
 	if (info == NULL)
 		goto end;
 
 	/* only if we are server ourselves and packet is newer than the one in
 	 * hash.*/
 	if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) {
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
 		if (list_empty(&info->send_list))
 			list_add_tail(&info->send_list, &send_list);
 	}
@@ -263,24 +283,27 @@
 	int is_new;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
+	int are_target = 0;
 
 	/* clients shall not broadcast. */
 	if (is_bcast(vis_packet->target_orig))
 		return;
 
+	/* Are we the target for this VIS packet? */
+	if (vis_server == VIS_TYPE_SERVER_SYNC	&&
+	    is_my_mac(info->packet.target_orig))
+		are_target = 1;
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, are_target);
 	if (info == NULL)
 		goto end;
 	/* note that outdated packets will be dropped at this point. */
 
 
 	/* send only if we're the target server or ... */
-	if (vis_server == VIS_TYPE_SERVER_SYNC  &&
-	    is_my_mac(info->packet.target_orig) &&
-	    is_new) {
+	if (are_target && is_new) {
 		info->packet.vis_type = VIS_TYPE_SERVER_SYNC;	/* upgrade! */
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
 		if (list_empty(&info->send_list))
 			list_add_tail(&info->send_list, &send_list);
 
@@ -362,14 +385,17 @@
 	while (hash_iterate(orig_hash, &hashit_global)) {
 		orig_node = hashit_global.bucket->data;
 		if (orig_node->router != NULL
-			&& compare_orig(orig_node->router->addr, orig_node->orig)
+			&& compare_orig(orig_node->router->addr,
+					orig_node->orig)
 			&& orig_node->batman_if
 			&& (orig_node->batman_if->if_active == IF_ACTIVE)
 		    && orig_node->router->tq_avg > 0) {
 
 			/* fill one entry into buffer. */
 			entry = &entry_array[info->packet.entries];
-			memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN);
+			memcpy(entry->src,
+			       orig_node->batman_if->net_dev->dev_addr,
+			       ETH_ALEN);
 			memcpy(entry->dest, orig_node->orig, ETH_ALEN);
 			entry->quality = orig_node->router->tq_avg;
 			info->packet.entries++;
@@ -401,6 +427,8 @@
 	return 0;
 }
 
+/* free old vis packets. Must be called with this vis_hash_lock
+ * held */
 static void purge_vis_packets(void)
 {
 	HASHIT(hashit);
@@ -413,7 +441,7 @@
 		if (time_after(jiffies,
 			       info->first_seen + (VIS_TIMEOUT*HZ)/1000)) {
 			hash_remove_bucket(vis_hash, &hashit);
-			free_info(info);
+			kref_put(&info->refcount, free_info);
 		}
 	}
 }
@@ -423,6 +451,8 @@
 	HASHIT(hashit);
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 
@@ -431,46 +461,57 @@
 		orig_node = hashit.bucket->data;
 
 		/* if it's a vis server and reachable, send it. */
-		if (orig_node &&
-		    (orig_node->flags & VIS_SERVER) &&
-		    orig_node->batman_if &&
-		    orig_node->router) {
+		if ((!orig_node) || (!orig_node->batman_if) ||
+		    (!orig_node->router))
+			continue;
+		if (!(orig_node->flags & VIS_SERVER))
+			continue;
+		/* don't send it if we already received the packet from
+		 * this node. */
+		if (recv_list_is_in(&info->recv_list, orig_node->orig))
+			continue;
 
-			/* don't send it if we already received the packet from
-			 * this node. */
-			if (recv_list_is_in(&info->recv_list, orig_node->orig))
-				continue;
+		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
+		batman_if = orig_node->batman_if;
+		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
+		spin_unlock_irqrestore(&orig_hash_lock, flags);
 
-			memcpy(info->packet.target_orig,
-			       orig_node->orig, ETH_ALEN);
+		send_raw_packet((unsigned char *)&info->packet,
+				packet_length, batman_if, dstaddr);
 
-			send_raw_packet((unsigned char *) &info->packet,
-					packet_length,
-					orig_node->batman_if,
-					orig_node->router->addr);
-		}
+		spin_lock_irqsave(&orig_hash_lock, flags);
+
 	}
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 static void unicast_vis_packet(struct vis_info *info, int packet_length)
 {
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 	orig_node = ((struct orig_node *)
 		     hash_find(orig_hash, info->packet.target_orig));
 
-	if ((orig_node != NULL) &&
-	    (orig_node->batman_if != NULL) &&
-	    (orig_node->router != NULL)) {
-		send_raw_packet((unsigned char *) &info->packet, packet_length,
-				orig_node->batman_if,
-				orig_node->router->addr);
-	}
+	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
+		goto out;
+
+	/* don't lock while sending the packets ... we therefore
+	 * copy the required data before sending */
+	batman_if = orig_node->batman_if;
+	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
 	spin_unlock_irqrestore(&orig_hash_lock, flags);
+
+	send_raw_packet((unsigned char *)&info->packet,
+			packet_length, batman_if, dstaddr);
+	return;
+
+out:
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 /* only send one vis packet. called from send_vis_packets() */
@@ -503,6 +544,7 @@
 	unsigned long flags;
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
+
 	purge_vis_packets();
 
 	if (generate_vis_packet() == 0)
@@ -511,7 +553,11 @@
 
 	list_for_each_entry_safe(info, temp, &send_list, send_list) {
 		list_del_init(&info->send_list);
+		kref_get(&info->refcount);
+		spin_unlock_irqrestore(&vis_hash_lock, flags);
 		send_vis_packet(info);
+		spin_lock_irqsave(&vis_hash_lock, flags);
+		kref_put(&info->refcount, free_info);
 	}
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
 	start_vis_timer();
@@ -544,6 +590,7 @@
 	my_vis_info->first_seen = jiffies - atomic_read(&vis_interval);
 	INIT_LIST_HEAD(&my_vis_info->recv_list);
 	INIT_LIST_HEAD(&my_vis_info->send_list);
+	kref_init(&my_vis_info->refcount);
 	my_vis_info->packet.version = COMPAT_VERSION;
 	my_vis_info->packet.packet_type = BAT_VIS;
 	my_vis_info->packet.ttl = TTL;
@@ -557,9 +604,9 @@
 
 	if (hash_add(vis_hash, my_vis_info) < 0) {
 		printk(KERN_ERR
-			  "batman-adv:Can't add own vis packet into hash\n");
-		free_info(my_vis_info);	/* not in hash, need to remove it
-					 * manually. */
+		       "batman-adv:Can't add own vis packet into hash\n");
+		/* not in hash, need to remove it manually. */
+		kref_put(&my_vis_info->refcount, free_info);
 		goto err;
 	}
 
@@ -573,6 +620,13 @@
 	return 0;
 }
 
+/* Decrease the reference count on a hash item info */
+static void free_info_ref(void *data)
+{
+	struct vis_info *info = data;
+	kref_put(&info->refcount, free_info);
+}
+
 /* shutdown vis-server */
 void vis_quit(void)
 {
@@ -584,7 +638,7 @@
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
 	/* properly remove, kill timers ... */
-	hash_delete(vis_hash, free_info);
+	hash_delete(vis_hash, free_info_ref);
 	vis_hash = NULL;
 	my_vis_info = NULL;
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -594,5 +648,5 @@
 static void start_vis_timer(void)
 {
 	queue_delayed_work(bat_event_workqueue, &vis_timer_wq,
-			   (atomic_read(&vis_interval) * HZ ) / 1000);
+			   (atomic_read(&vis_interval) * HZ) / 1000);
 }
Index: vis.h
===================================================================
--- vis.h	(revision 1568)
+++ vis.h	(working copy)
@@ -29,6 +29,7 @@
 			    /* list of server-neighbors we received a vis-packet
 			     * from.  we should not reply to them. */
 	struct list_head send_list;
+	struct kref refcount;
 	/* this packet might be part of the vis send queue. */
 	struct vis_packet packet;
 	/* vis_info may follow here*/


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-30 16:50       ` Andrew Lunn
@ 2010-01-31 19:37         ` Linus Lüssing
  2010-01-31 20:56           ` Andrew Lunn
  2010-02-11  9:46         ` Andrew Lunn
  1 sibling, 1 reply; 37+ messages in thread
From: Linus Lüssing @ 2010-01-31 19:37 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking


[-- Warning: decoded text below may be mangled, UTF-8 assumed --]
[-- Attachment #1.1: Type: text/plain; charset=utf-8, Size: 14179 bytes --]

Hi Andrew,

Yes, ehm it does indeed cause a kernel panic on the server side
immediately :). See the attachment for the full call trace.

Cheers, Linus

On Sat, Jan 30, 2010 at 05:50:59PM +0100, Andrew Lunn wrote:
> Hi Linus
> 
> Please could you try this patch and see if it fixes the vis problem.
> It compiles cleanly with 2.6.32. It is checkpatch clean, it is sparse
> clean. However, since i've not actually tried running the code i would
> not be too surprised if it deadlocked, leaked memory, oopsed, ...
> 
>     Andrew
> 
> Staging: batman-adv: Don't have interrupts disabled while sending.
> 
> send_vis_packets() would disable interrupts before calling
> dev_queue_xmit() which resulting in a backtrace in local_bh_enable().
> Fix this by using kref on the vis_info object so that we can call
> send_vis_packets() without holding vis_hash_lock. vis_hash_lock also
> used to protect recv_list, so we now need a new lock to protect that
> instead of vis_hash_lock.
> 
> Also a few checkpatch cleanups.
> 
> Reported-by: Linus Lüssing <linus.luessing@web.de>
> Signed-off-by: Andrew Lunn <andrew@lunn.ch>
> 
> Index: vis.c
> ===================================================================
> --- vis.c	(revision 1568)
> +++ vis.c	(working copy)
> @@ -30,22 +30,26 @@
>  
>  struct hashtable_t *vis_hash;
>  DEFINE_SPINLOCK(vis_hash_lock);
> +static DEFINE_SPINLOCK(recv_list_lock);
>  static struct vis_info *my_vis_info;
>  static struct list_head send_list;	/* always locked with vis_hash_lock */
>  
>  static void start_vis_timer(void);
>  
>  /* free the info */
> -static void free_info(void *data)
> +static void free_info(struct kref *ref)
>  {
> -	struct vis_info *info = data;
> +	struct vis_info *info = container_of(ref, struct vis_info, refcount);
>  	struct recvlist_node *entry, *tmp;
> +	unsigned long flags;
>  
>  	list_del_init(&info->send_list);
> +	spin_lock_irqsave(&recv_list_lock, flags);
>  	list_for_each_entry_safe(entry, tmp, &info->recv_list, list) {
>  		list_del(&entry->list);
>  		kfree(entry);
>  	}
> +	spin_unlock_irqrestore(&recv_list_lock, flags);
>  	kfree(info);
>  }
>  
> @@ -147,32 +151,41 @@
>  static void recv_list_add(struct list_head *recv_list, char *mac)
>  {
>  	struct recvlist_node *entry;
> +	unsigned long flags;
> +
>  	entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC);
>  	if (!entry)
>  		return;
>  
>  	memcpy(entry->mac, mac, ETH_ALEN);
> +	spin_lock_irqsave(&recv_list_lock, flags);
>  	list_add_tail(&entry->list, recv_list);
> +	spin_unlock_irqrestore(&recv_list_lock, flags);
>  }
>  
>  /* returns 1 if this mac is in the recv_list */
>  static int recv_list_is_in(struct list_head *recv_list, char *mac)
>  {
>  	struct recvlist_node *entry;
> +	unsigned long flags;
>  
> +	spin_lock_irqsave(&recv_list_lock, flags);
>  	list_for_each_entry(entry, recv_list, list) {
> -		if (memcmp(entry->mac, mac, ETH_ALEN) == 0)
> +		if (memcmp(entry->mac, mac, ETH_ALEN) == 0) {
> +			spin_unlock_irqrestore(&recv_list_lock, flags);
>  			return 1;
> +		}
>  	}
> -
> +	spin_unlock_irqrestore(&recv_list_lock, flags);
>  	return 0;
>  }
>  
>  /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old,
> - * broken.. ).  vis hash must be locked outside.  is_new is set when the packet
> + * broken.. ).	vis hash must be locked outside.  is_new is set when the packet
>   * is newer than old entries in the hash. */
>  static struct vis_info *add_packet(struct vis_packet *vis_packet,
> -				   int vis_info_len, int *is_new)
> +				   int vis_info_len, int *is_new,
> +				   int make_broadcast)
>  {
>  	struct vis_info *info, *old_info;
>  	struct vis_info search_elem;
> @@ -199,7 +212,7 @@
>  		}
>  		/* remove old entry */
>  		hash_remove(vis_hash, old_info);
> -		free_info(old_info);
> +		kref_put(&old_info->refcount, free_info);
>  	}
>  
>  	info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC);
> @@ -208,6 +221,7 @@
>  
>  	INIT_LIST_HEAD(&info->send_list);
>  	INIT_LIST_HEAD(&info->recv_list);
> +	kref_init(&info->refcount);
>  	info->first_seen = jiffies;
>  	memcpy(&info->packet, vis_packet,
>  	       sizeof(struct vis_packet) + vis_info_len);
> @@ -215,16 +229,21 @@
>  	/* initialize and add new packet. */
>  	*is_new = 1;
>  
> +	/* Make it a broadcast packet, if required */
> +	if (make_broadcast)
> +		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
> +
>  	/* repair if entries is longer than packet. */
>  	if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len)
> -		info->packet.entries = vis_info_len / sizeof(struct vis_info_entry);
> +		info->packet.entries = vis_info_len /
> +			sizeof(struct vis_info_entry);
>  
>  	recv_list_add(&info->recv_list, info->packet.sender_orig);
>  
>  	/* try to add it */
>  	if (hash_add(vis_hash, info) < 0) {
>  		/* did not work (for some reason) */
> -		free_info(info);
> +		kref_put(&old_info->refcount, free_info);
>  		info = NULL;
>  	}
>  
> @@ -235,19 +254,20 @@
>  void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len)
>  {
>  	struct vis_info *info;
> -	int is_new;
> +	int is_new, make_broadcast;
>  	unsigned long flags;
>  	int vis_server = atomic_read(&vis_mode);
>  
> +	make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC);
> +
>  	spin_lock_irqsave(&vis_hash_lock, flags);
> -	info = add_packet(vis_packet, vis_info_len, &is_new);
> +	info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast);
>  	if (info == NULL)
>  		goto end;
>  
>  	/* only if we are server ourselves and packet is newer than the one in
>  	 * hash.*/
>  	if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) {
> -		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
>  		if (list_empty(&info->send_list))
>  			list_add_tail(&info->send_list, &send_list);
>  	}
> @@ -263,24 +283,27 @@
>  	int is_new;
>  	unsigned long flags;
>  	int vis_server = atomic_read(&vis_mode);
> +	int are_target = 0;
>  
>  	/* clients shall not broadcast. */
>  	if (is_bcast(vis_packet->target_orig))
>  		return;
>  
> +	/* Are we the target for this VIS packet? */
> +	if (vis_server == VIS_TYPE_SERVER_SYNC	&&
> +	    is_my_mac(info->packet.target_orig))
> +		are_target = 1;
> +
>  	spin_lock_irqsave(&vis_hash_lock, flags);
> -	info = add_packet(vis_packet, vis_info_len, &is_new);
> +	info = add_packet(vis_packet, vis_info_len, &is_new, are_target);
>  	if (info == NULL)
>  		goto end;
>  	/* note that outdated packets will be dropped at this point. */
>  
>  
>  	/* send only if we're the target server or ... */
> -	if (vis_server == VIS_TYPE_SERVER_SYNC  &&
> -	    is_my_mac(info->packet.target_orig) &&
> -	    is_new) {
> +	if (are_target && is_new) {
>  		info->packet.vis_type = VIS_TYPE_SERVER_SYNC;	/* upgrade! */
> -		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
>  		if (list_empty(&info->send_list))
>  			list_add_tail(&info->send_list, &send_list);
>  
> @@ -362,14 +385,17 @@
>  	while (hash_iterate(orig_hash, &hashit_global)) {
>  		orig_node = hashit_global.bucket->data;
>  		if (orig_node->router != NULL
> -			&& compare_orig(orig_node->router->addr, orig_node->orig)
> +			&& compare_orig(orig_node->router->addr,
> +					orig_node->orig)
>  			&& orig_node->batman_if
>  			&& (orig_node->batman_if->if_active == IF_ACTIVE)
>  		    && orig_node->router->tq_avg > 0) {
>  
>  			/* fill one entry into buffer. */
>  			entry = &entry_array[info->packet.entries];
> -			memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN);
> +			memcpy(entry->src,
> +			       orig_node->batman_if->net_dev->dev_addr,
> +			       ETH_ALEN);
>  			memcpy(entry->dest, orig_node->orig, ETH_ALEN);
>  			entry->quality = orig_node->router->tq_avg;
>  			info->packet.entries++;
> @@ -401,6 +427,8 @@
>  	return 0;
>  }
>  
> +/* free old vis packets. Must be called with this vis_hash_lock
> + * held */
>  static void purge_vis_packets(void)
>  {
>  	HASHIT(hashit);
> @@ -413,7 +441,7 @@
>  		if (time_after(jiffies,
>  			       info->first_seen + (VIS_TIMEOUT*HZ)/1000)) {
>  			hash_remove_bucket(vis_hash, &hashit);
> -			free_info(info);
> +			kref_put(&info->refcount, free_info);
>  		}
>  	}
>  }
> @@ -423,6 +451,8 @@
>  	HASHIT(hashit);
>  	struct orig_node *orig_node;
>  	unsigned long flags;
> +	struct batman_if *batman_if;
> +	uint8_t dstaddr[ETH_ALEN];
>  
>  	spin_lock_irqsave(&orig_hash_lock, flags);
>  
> @@ -431,46 +461,57 @@
>  		orig_node = hashit.bucket->data;
>  
>  		/* if it's a vis server and reachable, send it. */
> -		if (orig_node &&
> -		    (orig_node->flags & VIS_SERVER) &&
> -		    orig_node->batman_if &&
> -		    orig_node->router) {
> +		if ((!orig_node) || (!orig_node->batman_if) ||
> +		    (!orig_node->router))
> +			continue;
> +		if (!(orig_node->flags & VIS_SERVER))
> +			continue;
> +		/* don't send it if we already received the packet from
> +		 * this node. */
> +		if (recv_list_is_in(&info->recv_list, orig_node->orig))
> +			continue;
>  
> -			/* don't send it if we already received the packet from
> -			 * this node. */
> -			if (recv_list_is_in(&info->recv_list, orig_node->orig))
> -				continue;
> +		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
> +		batman_if = orig_node->batman_if;
> +		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
> +		spin_unlock_irqrestore(&orig_hash_lock, flags);
>  
> -			memcpy(info->packet.target_orig,
> -			       orig_node->orig, ETH_ALEN);
> +		send_raw_packet((unsigned char *)&info->packet,
> +				packet_length, batman_if, dstaddr);
>  
> -			send_raw_packet((unsigned char *) &info->packet,
> -					packet_length,
> -					orig_node->batman_if,
> -					orig_node->router->addr);
> -		}
> +		spin_lock_irqsave(&orig_hash_lock, flags);
> +
>  	}
> +	spin_unlock_irqrestore(&orig_hash_lock, flags);
>  	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
> -	spin_unlock_irqrestore(&orig_hash_lock, flags);
>  }
>  
>  static void unicast_vis_packet(struct vis_info *info, int packet_length)
>  {
>  	struct orig_node *orig_node;
>  	unsigned long flags;
> +	struct batman_if *batman_if;
> +	uint8_t dstaddr[ETH_ALEN];
>  
>  	spin_lock_irqsave(&orig_hash_lock, flags);
>  	orig_node = ((struct orig_node *)
>  		     hash_find(orig_hash, info->packet.target_orig));
>  
> -	if ((orig_node != NULL) &&
> -	    (orig_node->batman_if != NULL) &&
> -	    (orig_node->router != NULL)) {
> -		send_raw_packet((unsigned char *) &info->packet, packet_length,
> -				orig_node->batman_if,
> -				orig_node->router->addr);
> -	}
> +	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
> +		goto out;
> +
> +	/* don't lock while sending the packets ... we therefore
> +	 * copy the required data before sending */
> +	batman_if = orig_node->batman_if;
> +	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
>  	spin_unlock_irqrestore(&orig_hash_lock, flags);
> +
> +	send_raw_packet((unsigned char *)&info->packet,
> +			packet_length, batman_if, dstaddr);
> +	return;
> +
> +out:
> +	spin_unlock_irqrestore(&orig_hash_lock, flags);
>  }
>  
>  /* only send one vis packet. called from send_vis_packets() */
> @@ -503,6 +544,7 @@
>  	unsigned long flags;
>  
>  	spin_lock_irqsave(&vis_hash_lock, flags);
> +
>  	purge_vis_packets();
>  
>  	if (generate_vis_packet() == 0)
> @@ -511,7 +553,11 @@
>  
>  	list_for_each_entry_safe(info, temp, &send_list, send_list) {
>  		list_del_init(&info->send_list);
> +		kref_get(&info->refcount);
> +		spin_unlock_irqrestore(&vis_hash_lock, flags);
>  		send_vis_packet(info);
> +		spin_lock_irqsave(&vis_hash_lock, flags);
> +		kref_put(&info->refcount, free_info);
>  	}
>  	spin_unlock_irqrestore(&vis_hash_lock, flags);
>  	start_vis_timer();
> @@ -544,6 +590,7 @@
>  	my_vis_info->first_seen = jiffies - atomic_read(&vis_interval);
>  	INIT_LIST_HEAD(&my_vis_info->recv_list);
>  	INIT_LIST_HEAD(&my_vis_info->send_list);
> +	kref_init(&my_vis_info->refcount);
>  	my_vis_info->packet.version = COMPAT_VERSION;
>  	my_vis_info->packet.packet_type = BAT_VIS;
>  	my_vis_info->packet.ttl = TTL;
> @@ -557,9 +604,9 @@
>  
>  	if (hash_add(vis_hash, my_vis_info) < 0) {
>  		printk(KERN_ERR
> -			  "batman-adv:Can't add own vis packet into hash\n");
> -		free_info(my_vis_info);	/* not in hash, need to remove it
> -					 * manually. */
> +		       "batman-adv:Can't add own vis packet into hash\n");
> +		/* not in hash, need to remove it manually. */
> +		kref_put(&my_vis_info->refcount, free_info);
>  		goto err;
>  	}
>  
> @@ -573,6 +620,13 @@
>  	return 0;
>  }
>  
> +/* Decrease the reference count on a hash item info */
> +static void free_info_ref(void *data)
> +{
> +	struct vis_info *info = data;
> +	kref_put(&info->refcount, free_info);
> +}
> +
>  /* shutdown vis-server */
>  void vis_quit(void)
>  {
> @@ -584,7 +638,7 @@
>  
>  	spin_lock_irqsave(&vis_hash_lock, flags);
>  	/* properly remove, kill timers ... */
> -	hash_delete(vis_hash, free_info);
> +	hash_delete(vis_hash, free_info_ref);
>  	vis_hash = NULL;
>  	my_vis_info = NULL;
>  	spin_unlock_irqrestore(&vis_hash_lock, flags);
> @@ -594,5 +648,5 @@
>  static void start_vis_timer(void)
>  {
>  	queue_delayed_work(bat_event_workqueue, &vis_timer_wq,
> -			   (atomic_read(&vis_interval) * HZ ) / 1000);
> +			   (atomic_read(&vis_interval) * HZ) / 1000);
>  }
> Index: vis.h
> ===================================================================
> --- vis.h	(revision 1568)
> +++ vis.h	(working copy)
> @@ -29,6 +29,7 @@
>  			    /* list of server-neighbors we received a vis-packet
>  			     * from.  we should not reply to them. */
>  	struct list_head send_list;
> +	struct kref refcount;
>  	/* this packet might be part of the vis send queue. */
>  	struct vis_packet packet;
>  	/* vis_info may follow here*/
> 

[-- Attachment #1.2: 2010-01-31-2030-vis-patch-kernelpanic.log --]
[-- Type: text/plain, Size: 2279 bytes --]

root@OpenWrt:/# batctl vs
[ ] client mode (server disabled)
[x] server mode (server enabled)
root@OpenWrt:/# CPU 0 Unable to handle kernel paging request at virtual address 00000024, epc == 8014d58c, ra == 80410154
Oops[#1]:
Cpu 0
$ 0   : 00000000 10009c00 8070e140 00000006
$ 4   : 8070e140 00000024 00000006 00000001
$ 8   : 00000024 00000000 00000000 00000000
$12   : 00000000 7fd89ed8 00000413 00430000
$16   : 80fef500 8041cdc0 00000024 8070e000
$20   : 0000004c 80d63000 00004305 80c05040
$24   : 00000002 8061c874
$28   : 80290000 80291ca8 00000000 80410154
Hi    : 000000f5
Lo    : 15a32180
epc   : 8014d58c memcmp+0xc/0x38
    Tainted: P
ra    : 80410154 is_my_mac+0x3c/0x78 [batman_adv]
Status: 10009c03    KERNEL EXL IE
Cause : 10800008
BadVA : 00000024
PrId  : 00019064 (MIPS 4KEc)
Modules linked in: ath_ahb ath_hal(P) batman_adv ip6t_REJECT ip6t_LOG ip6t_rt ip6t_hbh ip6t_mh ip6t_ipv6header ip6t_frag ip6t_eui64 ip6t_ah ip6table_raw ip6_queue ip6table_mangle ip6table_filter ip6_tables ebt_redirect ebt_mark ebt_vlan ebt_stp ebt_pkttype ebt_mark_m ebt_limit ebt_among ebt_802_3 ebtable_nat ebtable_filter ebtable_broute ebtables xt_quota xt_pkttype xt_physdev ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables tun ipv6
Process swapper (pid: 0, threadinfo=80290000, task=80292170, tls=00000000)
Stack : 80225540 00000000 80a11c00 8022b330 00000000 00000000 80c05060 80417264
        00000000 8022a728 80000000 806c0052 c02e4000 c02e7000 80f49a40 80c05060
        80c05052 8070e000 80d63000 804124e4 80225540 00000000 80a11c00 00000000
        80c14e40 80fef500 80f49a40 80417bf4 8070e2c0 80c05052 8070e000 80d63000
        80d63000 80f49a40 8070e2c0 806215c4 00000000 80292170 80291db0 000332b6
        ...
Call Trace:
[<8014d58c>] memcmp+0xc/0x38
[<80410154>] is_my_mac+0x3c/0x78 [batman_adv]
[<80417264>] receive_client_update_packet+0x4c/0x154 [batman_adv]
[<804124e4>] recv_vis_packet+0xa4/0xbc [batman_adv]
[<80417bf4>] batman_skb_recv+0x168/0x1dc [batman_adv]
[<806215c4>] ieee80211_saveath+0xb24/0xb80 [ath_ahb]


Code: 08053567  00003821  90430000 <91020000> 00621023  14400006  24c6ffff  00871021  00a74021
Kernel panic - not syncing: Fatal exception in interrupt
Rebooting in 3 seconds..+

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-31 19:37         ` Linus Lüssing
@ 2010-01-31 20:56           ` Andrew Lunn
  0 siblings, 0 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-01-31 20:56 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Sun, Jan 31, 2010 at 08:37:29PM +0100, Linus L??ssing wrote:
> Hi Andrew,
> 
> Yes, ehm it does indeed cause a kernel panic on the server side
> immediately :). See the attachment for the full call trace.

Oh yes, dumb error. I will give you a new patch tomorrow. I might even
try it first this time!

Thanks for testing,

    Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-01-30 16:50       ` Andrew Lunn
  2010-01-31 19:37         ` Linus Lüssing
@ 2010-02-11  9:46         ` Andrew Lunn
  2010-02-11 10:01           ` Andrew Lunn
  1 sibling, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-02-11  9:46 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

Hi Linus

Here is a new version of the patch. I've tested it this time using
five UML machines. It should not immediately opps now. 

Please could you test it with your test setup.

       Thanks
   	  Andrew

Staging: batman-adv: Don't have interrupts disabled while sending.

send_vis_packets() would disable interrupts before calling
dev_queue_xmit() which resulting in a backtrace in local_bh_enable().
Fix this by using kref on the vis_info object so that we can call
send_vis_packets() without holding vis_hash_lock. vis_hash_lock also
used to protect recv_list, so we now need a new lock to protect that
instead of vis_hash_lock.

Also a few checkpatch cleanups.

Reported-by: Linus L\374ssing <linus.luessing@web.de>
Signed-off-by: Andrew Lunn <andrew@lunn.ch>

Index: vis.c


Index: vis.c
===================================================================
--- vis.c	(revision 1568)
+++ vis.c	(working copy)
@@ -30,22 +30,26 @@
 
 struct hashtable_t *vis_hash;
 DEFINE_SPINLOCK(vis_hash_lock);
+static DEFINE_SPINLOCK(recv_list_lock);
 static struct vis_info *my_vis_info;
 static struct list_head send_list;	/* always locked with vis_hash_lock */
 
 static void start_vis_timer(void);
 
 /* free the info */
-static void free_info(void *data)
+static void free_info(struct kref *ref)
 {
-	struct vis_info *info = data;
+	struct vis_info *info = container_of(ref, struct vis_info, refcount);
 	struct recvlist_node *entry, *tmp;
+	unsigned long flags;
 
 	list_del_init(&info->send_list);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry_safe(entry, tmp, &info->recv_list, list) {
 		list_del(&entry->list);
 		kfree(entry);
 	}
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	kfree(info);
 }
 
@@ -147,32 +151,41 @@
 static void recv_list_add(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
+
 	entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC);
 	if (!entry)
 		return;
 
 	memcpy(entry->mac, mac, ETH_ALEN);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_add_tail(&entry->list, recv_list);
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 }
 
 /* returns 1 if this mac is in the recv_list */
 static int recv_list_is_in(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
 
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry(entry, recv_list, list) {
-		if (memcmp(entry->mac, mac, ETH_ALEN) == 0)
+		if (memcmp(entry->mac, mac, ETH_ALEN) == 0) {
+			spin_unlock_irqrestore(&recv_list_lock, flags);
 			return 1;
+		}
 	}
-
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	return 0;
 }
 
 /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old,
- * broken.. ).  vis hash must be locked outside.  is_new is set when the packet
+ * broken.. ).	vis hash must be locked outside.  is_new is set when the packet
  * is newer than old entries in the hash. */
 static struct vis_info *add_packet(struct vis_packet *vis_packet,
-				   int vis_info_len, int *is_new)
+				   int vis_info_len, int *is_new,
+				   int make_broadcast)
 {
 	struct vis_info *info, *old_info;
 	struct vis_info search_elem;
@@ -199,7 +212,7 @@
 		}
 		/* remove old entry */
 		hash_remove(vis_hash, old_info);
-		free_info(old_info);
+		kref_put(&old_info->refcount, free_info);
 	}
 
 	info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC);
@@ -208,6 +221,7 @@
 
 	INIT_LIST_HEAD(&info->send_list);
 	INIT_LIST_HEAD(&info->recv_list);
+	kref_init(&info->refcount);
 	info->first_seen = jiffies;
 	memcpy(&info->packet, vis_packet,
 	       sizeof(struct vis_packet) + vis_info_len);
@@ -215,16 +229,21 @@
 	/* initialize and add new packet. */
 	*is_new = 1;
 
+	/* Make it a broadcast packet, if required */
+	if (make_broadcast)
+		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
+
 	/* repair if entries is longer than packet. */
 	if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len)
-		info->packet.entries = vis_info_len / sizeof(struct vis_info_entry);
+		info->packet.entries = vis_info_len /
+			sizeof(struct vis_info_entry);
 
 	recv_list_add(&info->recv_list, info->packet.sender_orig);
 
 	/* try to add it */
 	if (hash_add(vis_hash, info) < 0) {
 		/* did not work (for some reason) */
-		free_info(info);
+		kref_put(&old_info->refcount, free_info);
 		info = NULL;
 	}
 
@@ -235,19 +254,20 @@
 void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len)
 {
 	struct vis_info *info;
-	int is_new;
+	int is_new, make_broadcast;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
 
+	make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC);
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast);
 	if (info == NULL)
 		goto end;
 
 	/* only if we are server ourselves and packet is newer than the one in
 	 * hash.*/
 	if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) {
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
 		if (list_empty(&info->send_list))
 			list_add_tail(&info->send_list, &send_list);
 	}
@@ -263,24 +283,27 @@
 	int is_new;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
+	int are_target = 0;
 
 	/* clients shall not broadcast. */
 	if (is_bcast(vis_packet->target_orig))
 		return;
 
+	/* Are we the target for this VIS packet? */
+	if (vis_server == VIS_TYPE_SERVER_SYNC	&&
+	    is_my_mac(vis_packet->target_orig))
+		are_target = 1;
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, are_target);
 	if (info == NULL)
 		goto end;
 	/* note that outdated packets will be dropped at this point. */
 
 
 	/* send only if we're the target server or ... */
-	if (vis_server == VIS_TYPE_SERVER_SYNC  &&
-	    is_my_mac(info->packet.target_orig) &&
-	    is_new) {
+	if (are_target && is_new) {
 		info->packet.vis_type = VIS_TYPE_SERVER_SYNC;	/* upgrade! */
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
 		if (list_empty(&info->send_list))
 			list_add_tail(&info->send_list, &send_list);
 
@@ -362,14 +385,17 @@
 	while (hash_iterate(orig_hash, &hashit_global)) {
 		orig_node = hashit_global.bucket->data;
 		if (orig_node->router != NULL
-			&& compare_orig(orig_node->router->addr, orig_node->orig)
+			&& compare_orig(orig_node->router->addr,
+					orig_node->orig)
 			&& orig_node->batman_if
 			&& (orig_node->batman_if->if_active == IF_ACTIVE)
 		    && orig_node->router->tq_avg > 0) {
 
 			/* fill one entry into buffer. */
 			entry = &entry_array[info->packet.entries];
-			memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN);
+			memcpy(entry->src,
+			       orig_node->batman_if->net_dev->dev_addr,
+			       ETH_ALEN);
 			memcpy(entry->dest, orig_node->orig, ETH_ALEN);
 			entry->quality = orig_node->router->tq_avg;
 			info->packet.entries++;
@@ -401,6 +427,8 @@
 	return 0;
 }
 
+/* free old vis packets. Must be called with this vis_hash_lock
+ * held */
 static void purge_vis_packets(void)
 {
 	HASHIT(hashit);
@@ -413,7 +441,7 @@
 		if (time_after(jiffies,
 			       info->first_seen + (VIS_TIMEOUT*HZ)/1000)) {
 			hash_remove_bucket(vis_hash, &hashit);
-			free_info(info);
+			kref_put(&info->refcount, free_info);
 		}
 	}
 }
@@ -423,6 +451,8 @@
 	HASHIT(hashit);
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 
@@ -431,46 +461,57 @@
 		orig_node = hashit.bucket->data;
 
 		/* if it's a vis server and reachable, send it. */
-		if (orig_node &&
-		    (orig_node->flags & VIS_SERVER) &&
-		    orig_node->batman_if &&
-		    orig_node->router) {
+		if ((!orig_node) || (!orig_node->batman_if) ||
+		    (!orig_node->router))
+			continue;
+		if (!(orig_node->flags & VIS_SERVER))
+			continue;
+		/* don't send it if we already received the packet from
+		 * this node. */
+		if (recv_list_is_in(&info->recv_list, orig_node->orig))
+			continue;
 
-			/* don't send it if we already received the packet from
-			 * this node. */
-			if (recv_list_is_in(&info->recv_list, orig_node->orig))
-				continue;
+		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
+		batman_if = orig_node->batman_if;
+		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
+		spin_unlock_irqrestore(&orig_hash_lock, flags);
 
-			memcpy(info->packet.target_orig,
-			       orig_node->orig, ETH_ALEN);
+		send_raw_packet((unsigned char *)&info->packet,
+				packet_length, batman_if, dstaddr);
 
-			send_raw_packet((unsigned char *) &info->packet,
-					packet_length,
-					orig_node->batman_if,
-					orig_node->router->addr);
-		}
+		spin_lock_irqsave(&orig_hash_lock, flags);
+
 	}
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 static void unicast_vis_packet(struct vis_info *info, int packet_length)
 {
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 	orig_node = ((struct orig_node *)
 		     hash_find(orig_hash, info->packet.target_orig));
 
-	if ((orig_node != NULL) &&
-	    (orig_node->batman_if != NULL) &&
-	    (orig_node->router != NULL)) {
-		send_raw_packet((unsigned char *) &info->packet, packet_length,
-				orig_node->batman_if,
-				orig_node->router->addr);
-	}
+	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
+		goto out;
+
+	/* don't lock while sending the packets ... we therefore
+	 * copy the required data before sending */
+	batman_if = orig_node->batman_if;
+	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
 	spin_unlock_irqrestore(&orig_hash_lock, flags);
+
+	send_raw_packet((unsigned char *)&info->packet,
+			packet_length, batman_if, dstaddr);
+	return;
+
+out:
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 /* only send one vis packet. called from send_vis_packets() */
@@ -503,6 +544,7 @@
 	unsigned long flags;
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
+
 	purge_vis_packets();
 
 	if (generate_vis_packet() == 0)
@@ -511,7 +553,11 @@
 
 	list_for_each_entry_safe(info, temp, &send_list, send_list) {
 		list_del_init(&info->send_list);
+		kref_get(&info->refcount);
+		spin_unlock_irqrestore(&vis_hash_lock, flags);
 		send_vis_packet(info);
+		spin_lock_irqsave(&vis_hash_lock, flags);
+		kref_put(&info->refcount, free_info);
 	}
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
 	start_vis_timer();
@@ -544,6 +590,7 @@
 	my_vis_info->first_seen = jiffies - atomic_read(&vis_interval);
 	INIT_LIST_HEAD(&my_vis_info->recv_list);
 	INIT_LIST_HEAD(&my_vis_info->send_list);
+	kref_init(&my_vis_info->refcount);
 	my_vis_info->packet.version = COMPAT_VERSION;
 	my_vis_info->packet.packet_type = BAT_VIS;
 	my_vis_info->packet.ttl = TTL;
@@ -557,9 +604,9 @@
 
 	if (hash_add(vis_hash, my_vis_info) < 0) {
 		printk(KERN_ERR
-			  "batman-adv:Can't add own vis packet into hash\n");
-		free_info(my_vis_info);	/* not in hash, need to remove it
-					 * manually. */
+		       "batman-adv:Can't add own vis packet into hash\n");
+		/* not in hash, need to remove it manually. */
+		kref_put(&my_vis_info->refcount, free_info);
 		goto err;
 	}
 
@@ -573,6 +620,13 @@
 	return 0;
 }
 
+/* Decrease the reference count on a hash item info */
+static void free_info_ref(void *data)
+{
+	struct vis_info *info = data;
+	kref_put(&info->refcount, free_info);
+}
+
 /* shutdown vis-server */
 void vis_quit(void)
 {
@@ -584,7 +638,7 @@
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
 	/* properly remove, kill timers ... */
-	hash_delete(vis_hash, free_info);
+	hash_delete(vis_hash, free_info_ref);
 	vis_hash = NULL;
 	my_vis_info = NULL;
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -594,5 +648,5 @@
 static void start_vis_timer(void)
 {
 	queue_delayed_work(bat_event_workqueue, &vis_timer_wq,
-			   (atomic_read(&vis_interval) * HZ ) / 1000);
+			   (atomic_read(&vis_interval) * HZ) / 1000);
 }
Index: vis.h
===================================================================
--- vis.h	(revision 1568)
+++ vis.h	(working copy)
@@ -29,6 +29,7 @@
 			    /* list of server-neighbors we received a vis-packet
 			     * from.  we should not reply to them. */
 	struct list_head send_list;
+	struct kref refcount;
 	/* this packet might be part of the vis send queue. */
 	struct vis_packet packet;
 	/* vis_info may follow here*/

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-02-11  9:46         ` Andrew Lunn
@ 2010-02-11 10:01           ` Andrew Lunn
  2010-02-19 17:19             ` Linus Lüssing
  2010-02-28 16:34             ` Simon Wunderlich
  0 siblings, 2 replies; 37+ messages in thread
From: Andrew Lunn @ 2010-02-11 10:01 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Thu, Feb 11, 2010 at 10:46:59AM +0100, Andrew Lunn wrote:
> Hi Linus
> 
> Here is a new version of the patch. I've tested it this time using
> five UML machines. It should not immediately opps now. 

Instead is will leak memory and crash after a while...

I will try to find the memory leak.

  Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-02-11 10:01           ` Andrew Lunn
@ 2010-02-19 17:19             ` Linus Lüssing
  2010-02-20 18:04               ` Andrew Lunn
  2010-02-28 16:34             ` Simon Wunderlich
  1 sibling, 1 reply; 37+ messages in thread
From: Linus Lüssing @ 2010-02-19 17:19 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 3065 bytes --]

Hi Andrew,

Sorry, didn't have the time to try your patch any earlier, I'm
right in the middle of my exams :).
Your patch already looks quite good, I couldn't reproduce any
memory leaks or crashes here (tried that with three routers and 1
or 2 vis servers activated, also activating/deactivating them a
lot, no problems with that). And yes, the slow-path warning has
gone with your patch.
However, I'm having some weird output when connecting two routers
over wifi _and_ over ethernet cable. The setup:

Before plugging in the cable:
r1-ath1 <-- wifi --> r2-ath1
------------
root@OpenWrt:~# batctl vd dot
digraph {
        "r1-ath1" -> "r2-ath1" [label="1.32"]
        "r1-ath1" -> "r1-hna" [label="HNA"]
        "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
        subgraph "cluster_r1-ath1" {
                "r1-ath1" [peripheries=2]
        }
        "r2-ath1" -> "r1-ath1" [label="1.11"]
        "r2-ath1" -> "r2-hna" [label="HNA"]
        "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
        subgraph "cluster_r2-ath1" {
                "r2-ath1" [peripheries=2]
        }
}
------------
After plugging in the cable:
r1-ath1 <-- wifi --> r2-ath1 +
r1-eth0.3 <-- cable --> r2-eth0.3
------------
root@OpenWrt:~# batctl vd dot
digraph {
        "r1-ath1" -> "r2-ath1" [label="1.0"]
        "r1-ath1" -> "r2-eth0.3" [label="1.66"]
        "r1-ath1" -> "r1-hna" [label="HNA"]
        "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
        subgraph "cluster_r1-ath1" {
                "r1-ath1" [peripheries=2]
                "r1-eth0.3"
        }
        subgraph "cluster_r1-ath1" {
                "r1-ath1" [peripheries=2]
        }
        "r2-ath1" -> "r1-ath1" [label="1.0"]
        "r2-ath1" -> "r1-eth0.3" [label="1.15"]
        "r2-ath1" -> "r2-hna" [label="HNA"]
        "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
        subgraph "cluster_r2-ath1" {
                "r2-ath1" [peripheries=2]
                "r2-eth0.3"
        }
        subgraph "cluster_r2-ath1" {
                "r2-ath1" [peripheries=2]
        }
}
root@OpenWrt:~# cat /proc/net/batman-adv/vis_data
06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA 5a:2e:1e:1f:64:6b, PRIMARY, SEC 04:22:b0:98:87:de,
06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA 00:22:b0:98:87:f9, HNA 82:31:95:f9:14:6f, SEC 04:22:b0:98:87:fa, PRIMARY,
----------
So the second 'subgraph "cluster_r1-ath1"' is obviously
unnecessary. Also "r1-ath1" -> "r2-eth0.3" looks wrong, should be
"r1-eth0.3" -> "r2-eth0.3" instead (and the same with r2 a few
lines later).

Cheers, Linus

On Thu, Feb 11, 2010 at 11:01:56AM +0100, Andrew Lunn wrote:
> On Thu, Feb 11, 2010 at 10:46:59AM +0100, Andrew Lunn wrote:
> > Hi Linus
> > 
> > Here is a new version of the patch. I've tested it this time using
> > five UML machines. It should not immediately opps now. 
> 
> Instead is will leak memory and crash after a while...
> 
> I will try to find the memory leak.
> 
>   Andrew
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-02-19 17:19             ` Linus Lüssing
@ 2010-02-20 18:04               ` Andrew Lunn
  2010-02-21 13:10                 ` Linus Lüssing
  0 siblings, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-02-20 18:04 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Fri, Feb 19, 2010 at 06:19:05PM +0100, Linus L??ssing wrote:
> Hi Andrew,
> 
> Sorry, didn't have the time to try your patch any earlier, I'm
> right in the middle of my exams :).

Hi Linus

Marek told me. No problems. I remember what its like studying for
exams. However, it is nice to sometimes take a break and do something
totally different. 

> Your patch already looks quite good, I couldn't reproduce any
> memory leaks or crashes here (tried that with three routers and 1
> or 2 vis servers activated, also activating/deactivating them a
> lot, no problems with that). And yes, the slow-path warning has
> gone with your patch.

Great. So we are on the right tracks.

> However, I'm having some weird output when connecting two routers
> over wifi _and_ over ethernet cable. The setup:
> 
> Before plugging in the cable:
> r1-ath1 <-- wifi --> r2-ath1
> ------------
> root@OpenWrt:~# batctl vd dot
> digraph {
>         "r1-ath1" -> "r2-ath1" [label="1.32"]
>         "r1-ath1" -> "r1-hna" [label="HNA"]
>         "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
>         subgraph "cluster_r1-ath1" {
>                 "r1-ath1" [peripheries=2]
>         }
>         "r2-ath1" -> "r1-ath1" [label="1.11"]
>         "r2-ath1" -> "r2-hna" [label="HNA"]
>         "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
>         subgraph "cluster_r2-ath1" {
>                 "r2-ath1" [peripheries=2]
>         }
> }
> ------------
> After plugging in the cable:
> r1-ath1 <-- wifi --> r2-ath1 +
> r1-eth0.3 <-- cable --> r2-eth0.3
> ------------
> root@OpenWrt:~# batctl vd dot
> digraph {
>         "r1-ath1" -> "r2-ath1" [label="1.0"]
>         "r1-ath1" -> "r2-eth0.3" [label="1.66"]
>         "r1-ath1" -> "r1-hna" [label="HNA"]
>         "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
>         subgraph "cluster_r1-ath1" {
>                 "r1-ath1" [peripheries=2]
>                 "r1-eth0.3"
>         }
>         subgraph "cluster_r1-ath1" {
>                 "r1-ath1" [peripheries=2]
>         }
>         "r2-ath1" -> "r1-ath1" [label="1.0"]
>         "r2-ath1" -> "r1-eth0.3" [label="1.15"]
>         "r2-ath1" -> "r2-hna" [label="HNA"]
>         "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
>         subgraph "cluster_r2-ath1" {
>                 "r2-ath1" [peripheries=2]
>                 "r2-eth0.3"
>         }
>         subgraph "cluster_r2-ath1" {
>                 "r2-ath1" [peripheries=2]
>         }
> }
> root@OpenWrt:~# cat /proc/net/batman-adv/vis_data
> 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA 5a:2e:1e:1f:64:6b, PRIMARY, SEC 04:22:b0:98:87:de,
> 06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA 00:22:b0:98:87:f9, HNA 82:31:95:f9:14:6f, SEC 04:22:b0:98:87:fa, PRIMARY,

Actually, this vis_data to does not map to the dot above!  There are
the wrong number of HNA, wrong order etc.

Here is what i think your bat-host file contains:
06:22:b0:98:87:dd r1-ath1
06:22:b0:98:87:f9 r2-ath1
00:22:b0:98:87:dd r1-hna
04:22:b0:98:87:de r1-eth0.3
00:22:b0:98:87:f9 r2-hna
04:22:b0:98:87:fa r2-eth0.3

and this is what i get, assuming i got the MAC->name mapping correct:

digraph {
	"r1-ath1" -> "r2-eth0.3" [label="1.15"]
	"r1-ath1" -> "r1-hna" [label="HNA"]
	"r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
	subgraph "cluster_r1-ath1" {
		"r1-ath1" [peripheries=2]
	}
	subgraph "cluster_r1-ath1" {
		"r1-ath1" [peripheries=2]
		"r1-eth0.3"
	}
	"r2-ath1" -> "r1-ath1" [label="1.0"]
	"r2-ath1" -> "r1-eth0.3" [label="1.15"]
	"r2-ath1" -> "r2-hna" [label="HNA"]
	"r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
	subgraph "cluster_r2-ath1" {
		"r2-ath1" [peripheries=2]
		"r2-eth0.3"
	}
	subgraph "cluster_r2-ath1" {
		"r2-ath1" [peripheries=2]
	}
}

batctl parses top-to-bottom, left-to-right. It does not consolidate
the PRIMARY and the SECONDARY into one cluster. It leaves DOT to do
that. Hence there are two cluster statements for each cluster actually
drawn.

> So the second 'subgraph "cluster_r1-ath1"' is obviously
> unnecessary.

Yes, unnecessary, but makes the batctl code easier.

 Also "r1-ath1" -> "r2-eth0.3" looks wrong, should be
> "r1-eth0.3" -> "r2-eth0.3" instead (and the same with r2 a few
> lines later).

These comments i agree with. A wireless and a wired device should not
be neighbours. We don't have any records which originate from the
secondary MAC address. That is guess is the major problem here.

So, did my/Mareks patch break it, or was it broken before?

First i suggest you go back to just before Simon's patch which
introduced receiving using skbufs:

http://open-mesh.org/changeset/1517

That will tell us if we need to go back further, or our patch broke
it. 

If you need to go back further, i would suggest just before:

http://open-mesh.org/changeset/1510

However, if it is our patch then we can chop the patch into two:

Use Mareks patch:

https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2010-January/002261.html

and 

Index: vis.c
===================================================================
--- vis.c	(revision 1575)
+++ vis.c	(working copy)
@@ -444,10 +444,15 @@
 			memcpy(info->packet.target_orig,
 			       orig_node->orig, ETH_ALEN);
 
+spin_unlock_irqrestore(&orig_hash_lock, flags);
+
 			send_raw_packet((unsigned char *) &info->packet,
 					packet_length,
 					orig_node->batman_if,
 					orig_node->router->addr);
+
+spin_lock_irqsave(&orig_hash_lock, flags);
+
 		}
 	}
 	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);

This adds a race condition, which i hope if O.K. for debugging
purposes, but i hope allows the send to happen without the slowpath
errors. If so, we can test Marek's part of the patch.

I'm on vacation for a week now. I will have Internet access some time,
but not much. 

Have fun debugging.

     Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-02-20 18:04               ` Andrew Lunn
@ 2010-02-21 13:10                 ` Linus Lüssing
  0 siblings, 0 replies; 37+ messages in thread
From: Linus Lüssing @ 2010-02-21 13:10 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 6172 bytes --]

On Sat, Feb 20, 2010 at 07:04:11PM +0100, Andrew Lunn wrote:
> On Fri, Feb 19, 2010 at 06:19:05PM +0100, Linus L??ssing wrote:
> > Hi Andrew,
> > 
> > Sorry, didn't have the time to try your patch any earlier, I'm
> > right in the middle of my exams :).
> 
> Hi Linus
> 
> Marek told me. No problems. I remember what its like studying for
> exams. However, it is nice to sometimes take a break and do something
> totally different. 
> 
> > Your patch already looks quite good, I couldn't reproduce any
> > memory leaks or crashes here (tried that with three routers and 1
> > or 2 vis servers activated, also activating/deactivating them a
> > lot, no problems with that). And yes, the slow-path warning has
> > gone with your patch.
> 
> Great. So we are on the right tracks.
> 
> > However, I'm having some weird output when connecting two routers
> > over wifi _and_ over ethernet cable. The setup:
> > 
> > Before plugging in the cable:
> > r1-ath1 <-- wifi --> r2-ath1
> > ------------
> > root@OpenWrt:~# batctl vd dot
> > digraph {
> >         "r1-ath1" -> "r2-ath1" [label="1.32"]
> >         "r1-ath1" -> "r1-hna" [label="HNA"]
> >         "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
> >         subgraph "cluster_r1-ath1" {
> >                 "r1-ath1" [peripheries=2]
> >         }
> >         "r2-ath1" -> "r1-ath1" [label="1.11"]
> >         "r2-ath1" -> "r2-hna" [label="HNA"]
> >         "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
> >         subgraph "cluster_r2-ath1" {
> >                 "r2-ath1" [peripheries=2]
> >         }
> > }
> > ------------
> > After plugging in the cable:
> > r1-ath1 <-- wifi --> r2-ath1 +
> > r1-eth0.3 <-- cable --> r2-eth0.3
> > ------------
> > root@OpenWrt:~# batctl vd dot
> > digraph {
> >         "r1-ath1" -> "r2-ath1" [label="1.0"]
> >         "r1-ath1" -> "r2-eth0.3" [label="1.66"]
> >         "r1-ath1" -> "r1-hna" [label="HNA"]
> >         "r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
> >         subgraph "cluster_r1-ath1" {
> >                 "r1-ath1" [peripheries=2]
> >                 "r1-eth0.3"
> >         }
> >         subgraph "cluster_r1-ath1" {
> >                 "r1-ath1" [peripheries=2]
> >         }
> >         "r2-ath1" -> "r1-ath1" [label="1.0"]
> >         "r2-ath1" -> "r1-eth0.3" [label="1.15"]
> >         "r2-ath1" -> "r2-hna" [label="HNA"]
> >         "r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
> >         subgraph "cluster_r2-ath1" {
> >                 "r2-ath1" [peripheries=2]
> >                 "r2-eth0.3"
> >         }
> >         subgraph "cluster_r2-ath1" {
> >                 "r2-ath1" [peripheries=2]
> >         }
> > }
> > root@OpenWrt:~# cat /proc/net/batman-adv/vis_data
> > 06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA 5a:2e:1e:1f:64:6b, PRIMARY, SEC 04:22:b0:98:87:de,
> > 06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA 00:22:b0:98:87:f9, HNA 82:31:95:f9:14:6f, SEC 04:22:b0:98:87:fa, PRIMARY,
> 
> Actually, this vis_data to does not map to the dot above!  There are
> the wrong number of HNA, wrong order etc.
Hmm, just noticed, the output also seems to be flapping between
those two from time to time:
------------------
root@OpenWrt:~# cat /proc/net/batman-adv/vis
06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA f6:ae:97:b3:9a:5c, PRIMARY, SEC 04:22:b0:98:87:de,
06:22:b0:98:87:f9,TQ 04:22:b0:98:87:de 251, HNA da:3e:79:2c:d3:3e, HNA 00:22:b0:98:87:f9, PRIMARY, SEC 04:22:b0:98:87:fa,
root@OpenWrt:~# cat /proc/net/batman-adv/vis
06:22:b0:98:87:dd,TQ 04:22:b0:98:87:fa 251, HNA 00:22:b0:98:87:dd, HNA f6:ae:97:b3:9a:5c, PRIMARY, SEC 04:22:b0:98:87:de,
06:22:b0:98:87:f9,TQ 06:22:b0:98:87:dd 255, TQ 04:22:b0:98:87:de 251, HNA da:3e:79:2c:d3:3e, HNA 00:22:b0:98:87:f9, SEC 04:22:b0:98:87:fa, PRIMARY,
------------------

> 
> Here is what i think your bat-host file contains:
> 06:22:b0:98:87:dd r1-ath1
> 06:22:b0:98:87:f9 r2-ath1
> 00:22:b0:98:87:dd r1-hna
> 04:22:b0:98:87:de r1-eth0.3
> 00:22:b0:98:87:f9 r2-hna
> 04:22:b0:98:87:fa r2-eth0.3
> 
> and this is what i get, assuming i got the MAC->name mapping correct:
Yes, correct mapping :).

> 
> digraph {
> 	"r1-ath1" -> "r2-eth0.3" [label="1.15"]
> 	"r1-ath1" -> "r1-hna" [label="HNA"]
> 	"r1-ath1" -> "5a:2e:1e:1f:64:6b" [label="HNA"]
> 	subgraph "cluster_r1-ath1" {
> 		"r1-ath1" [peripheries=2]
> 	}
> 	subgraph "cluster_r1-ath1" {
> 		"r1-ath1" [peripheries=2]
> 		"r1-eth0.3"
> 	}
> 	"r2-ath1" -> "r1-ath1" [label="1.0"]
> 	"r2-ath1" -> "r1-eth0.3" [label="1.15"]
> 	"r2-ath1" -> "r2-hna" [label="HNA"]
> 	"r2-ath1" -> "82:31:95:f9:14:6f" [label="HNA"]
> 	subgraph "cluster_r2-ath1" {
> 		"r2-ath1" [peripheries=2]
> 		"r2-eth0.3"
> 	}
> 	subgraph "cluster_r2-ath1" {
> 		"r2-ath1" [peripheries=2]
> 	}
> }
> 
> batctl parses top-to-bottom, left-to-right. It does not consolidate
> the PRIMARY and the SECONDARY into one cluster. It leaves DOT to do
> that. Hence there are two cluster statements for each cluster actually
> drawn.
> 
> > So the second 'subgraph "cluster_r1-ath1"' is obviously
> > unnecessary.
> 
> Yes, unnecessary, but makes the batctl code easier.
> 
>  Also "r1-ath1" -> "r2-eth0.3" looks wrong, should be
> > "r1-eth0.3" -> "r2-eth0.3" instead (and the same with r2 a few
> > lines later).
> 
> These comments i agree with. A wireless and a wired device should not
> be neighbours. We don't have any records which originate from the
> secondary MAC address. That is guess is the major problem here.
> 
> So, did my/Mareks patch break it, or was it broken before?
> 
> First i suggest you go back to just before Simon's patch which
> introduced receiving using skbufs:
> 
> http://open-mesh.org/changeset/1517
> 
> That will tell us if we need to go back further, or our patch broke
> it. 
> 
> If you need to go back further, i would suggest just before:
> 
> http://open-mesh.org/changeset/1510
Okay, just checked, this got introduced with 1510 already, yes. I
might have a closer look at this next week.

Cheers, Linus

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-02-11 10:01           ` Andrew Lunn
  2010-02-19 17:19             ` Linus Lüssing
@ 2010-02-28 16:34             ` Simon Wunderlich
  2010-03-01  5:59               ` Andrew Lunn
  1 sibling, 1 reply; 37+ messages in thread
From: Simon Wunderlich @ 2010-02-28 16:34 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking


[-- Attachment #1.1: Type: text/plain, Size: 4979 bytes --]

Hello,

i can reproduce the slowpath warning in the trunk within my qemu setup. When i
start a server, some of the clients show this slow path warning.

I've tried Andrews patch, and i can not see any memory leaks - it varies in a 
100k window. However some of the servers crash after some time (30 minutes)
without any special trigger i am aware of. Stack trace is this:

root@OpenWrt:/# ------------[ cut here ]------------
WARNING: at lib/kref.c:43 kref_get+0x18/0x30()
Hardware name:
Modules linked in: via_velocity via_rhine tg3 sis900 r8169 pcnet32 ne2k_pci 8390 e1000 e100 batman_adv 8139too 3c59x nf_nat_tftp nf_conntrack_tftp nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_conntrack_ftp ipt_MASQUERADE iptable_nat nf_nat xt_NOTRACK iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack pppoe pppox libphy ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ppp_async ppp_generic slhc natsemi crc_ccitt ipv6
Pid: 570, comm: bat_events Not tainted 2.6.31.1 #32
Call Trace:
 [<c10dd608>] ? kref_get+0x18/0x30
 [<c101f55f>] ? warn_slowpath_common+0x7f/0xb0
 [<c10dd608>] ? kref_get+0x18/0x30
 [<c101f5a3>] ? warn_slowpath_null+0x13/0x20
 [<c10dd608>] ? kref_get+0x18/0x30
 [<c2ac51a2>] ? proc_vis_read_prim_sec+0x352/0x5a0 [batman_adv]
 [<c2ac4ee0>] ? proc_vis_read_prim_sec+0x90/0x5a0 [batman_adv]
 [<c102d23a>] ? worker_thread+0xca/0x150
 [<c102fd80>] ? autoremove_wake_function+0x0/0x50
 [<c102d170>] ? worker_thread+0x0/0x150
 [<c102fbc3>] ? kthread+0x73/0x90
 [<c102fb50>] ? kthread+0x0/0x90
 [<c1003813>] ? kernel_thread_helper+0x7/0x14
---[ end trace 4f770856ce7e0712 ]---
------------[ cut here ]------------
kernel BUG at mm/slub.c:2929!
invalid opcode: 0000 [#1]
last sysfs file: /sys/kernel/uevent_seqnum
Modules linked in: via_velocity via_rhine tg3 sis900 r8169 pcnet32 ne2k_pci 8390 e1000 e100 batman_adv 8139too 3c59x nf_nat_tftp nf_conntrack_tftp nf_nat_irc nf_conntrack_irc nf_nat_ftp nf_conntrack_ftp ipt_MASQUERADE iptable_nat nf_nat xt_NOTRACK iptable_raw xt_state nf_conntrack_ipv4 nf_defrag_ipv4 nf_conntrack pppoe pppox libphy ipt_REJECT xt_TCPMSS ipt_LOG xt_multiport xt_mac xt_limit iptable_mangle iptable_filter ip_tables xt_tcpudp x_tables ppp_async ppp_generic slhc natsemi crc_ccitt ipv6

Pid: 570, comm: bat_events Tainted: G        W  (2.6.31.1 #32)
EIP: 0060:[<c1064669>] EFLAGS: 00010046 CPU: 0
EIP is at kfree+0x49/0xc0
EAX: 00000000 EBX: c1f5dd94 ECX: 40080000 EDX: c133cba0
ESI: c2ac5540 EDI: c1f5dd80 EBP: 00000216 ESP: c0d06eec
 DS: 007b ES: 007b FS: 0000 GS: 0000 SS: 0068
Process bat_events (pid: 570, ti=c0d06000 task=c18d0000 task.ti=c0d06000)
Stack:
 00000002 c1f5dd94 c1f5dd94 c2ac5540 c1f5dda4 c10dd5cf c1f5dda4 00000000
<0> c2ac538b c0d06f68 c0e0504a c1f5dd84 c1f5dd94 c1f5dd80 c1f5dd80 000022a0
<0> 00000046 c0e05024 c0e05030 00000286 00000086 c1f5dda4 0000003f 00000080
Call Trace:
 [<c2ac5540>] ? vis_init+0x150/0x1b0 [batman_adv]
 [<c10dd5cf>] ? kref_put+0x4f/0x70
 [<c2ac538b>] ? proc_vis_read_prim_sec+0x53b/0x5a0 [batman_adv]
 [<c2ac4ee0>] ? proc_vis_read_prim_sec+0x90/0x5a0 [batman_adv]
 [<c102d23a>] ? worker_thread+0xca/0x150
 [<c102fd80>] ? autoremove_wake_function+0x0/0x50
 [<c102d170>] ? worker_thread+0x0/0x150
 [<c102fbc3>] ? kthread+0x73/0x90
 [<c102fb50>] ? kthread+0x0/0x90
 [<c1003813>] ? kernel_thread_helper+0x7/0x14
Code: 00 40 a1 80 cb 2e c1 c1 ea 0c c1 e2 05 01 c2 8b 0a 89 c8 25 00 80 00 00 66 85 c0 74 05 8b 52 0c 8b 0a 84 c9 78 24 f6 c5 c0 75 09 <0f> 0b 90 8d 74 26 00 eb fe 8b 5c 24 08 89 d0 8b 74 24 0c 8b 7c
EIP: [<c1064669>] kfree+0x49/0xc0 SS:ESP 0068:c0d06eec
---[ end trace 4f770856ce7e0713 ]---

Please note that in this moment, i have not queried the vis_data file, so 
the proc_vis_read_prim_sec call is probably not correctly reported. Furthermore
there are some more stack traces in vfs/mount and then the boxes reboot -
we probably write at some bad memory position.

I've changed Andrews patch a little bit, and could not crash my 9 kvm instances
when running it some hours last night. I've added kref_get() calls when the info 
packets get referenced from the send_list, this reference is removed with 
kref_put() in the send_vis_packets() after sending the packet. The disadvantage 
of this solution is that in a race condition, we might (unnecessarily) send out an 
old packet. No memory leaks as far as i could see after running some hours.

Please give it a try (patch is attached to this e-mail)!

best regards,
	Simon



On Thu, Feb 11, 2010 at 11:01:56AM +0100, Andrew Lunn wrote:
> On Thu, Feb 11, 2010 at 10:46:59AM +0100, Andrew Lunn wrote:
> > Hi Linus
> > 
> > Here is a new version of the patch. I've tested it this time using
> > five UML machines. It should not immediately opps now. 
> 
> Instead is will leak memory and crash after a while...
> 
> I will try to find the memory leak.
> 
>   Andrew
> 

[-- Attachment #1.2: vis_refcount.patch --]
[-- Type: text/x-diff, Size: 13632 bytes --]

taging: batman-adv: Don't have interrupts disabled while sending.

send_vis_packets() would disable interrupts before calling
dev_queue_xmit() which resulting in a backtrace in local_bh_enable().
Fix this by using kref on the vis_info object so that we can call
send_vis_packets() without holding vis_hash_lock. vis_hash_lock also
used to protect recv_list, so we now need a new lock to protect that
instead of vis_hash_lock.

Also a few checkpatch cleanups.

Reported-by: Linus Luessing <linus.luessing@web.de>
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
Signed-off-by: Simon Wunderlich <siwu@hrz.tu-chemnitz.de>
===================================================================
--- batman-adv-kernelland/vis.c	(revision 1578)
+++ batman-adv-kernelland/vis.c	(working copy)
@@ -30,22 +30,26 @@
 
 struct hashtable_t *vis_hash;
 DEFINE_SPINLOCK(vis_hash_lock);
+static DEFINE_SPINLOCK(recv_list_lock);
 static struct vis_info *my_vis_info;
 static struct list_head send_list;	/* always locked with vis_hash_lock */
 
 static void start_vis_timer(void);
 
 /* free the info */
-static void free_info(void *data)
+static void free_info(struct kref *ref)
 {
-	struct vis_info *info = data;
+	struct vis_info *info = container_of(ref, struct vis_info, refcount);
 	struct recvlist_node *entry, *tmp;
+	unsigned long flags;
 
 	list_del_init(&info->send_list);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry_safe(entry, tmp, &info->recv_list, list) {
 		list_del(&entry->list);
 		kfree(entry);
 	}
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	kfree(info);
 }
 
@@ -147,32 +151,41 @@
 static void recv_list_add(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
+
 	entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC);
 	if (!entry)
 		return;
 
 	memcpy(entry->mac, mac, ETH_ALEN);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_add_tail(&entry->list, recv_list);
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 }
 
 /* returns 1 if this mac is in the recv_list */
 static int recv_list_is_in(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
 
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry(entry, recv_list, list) {
-		if (memcmp(entry->mac, mac, ETH_ALEN) == 0)
+		if (memcmp(entry->mac, mac, ETH_ALEN) == 0) {
+			spin_unlock_irqrestore(&recv_list_lock, flags);
 			return 1;
+		}
 	}
-
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	return 0;
 }
 
 /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old,
- * broken.. ).  vis hash must be locked outside.  is_new is set when the packet
+ * broken.. ).	vis hash must be locked outside.  is_new is set when the packet
  * is newer than old entries in the hash. */
 static struct vis_info *add_packet(struct vis_packet *vis_packet,
-				   int vis_info_len, int *is_new)
+				   int vis_info_len, int *is_new,
+				   int make_broadcast)
 {
 	struct vis_info *info, *old_info;
 	struct vis_info search_elem;
@@ -199,13 +212,14 @@
 		}
 		/* remove old entry */
 		hash_remove(vis_hash, old_info);
-		free_info(old_info);
+		kref_put(&old_info->refcount, free_info);
 	}
 
 	info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC);
 	if (info == NULL)
 		return NULL;
 
+	kref_init(&info->refcount);
 	INIT_LIST_HEAD(&info->send_list);
 	INIT_LIST_HEAD(&info->recv_list);
 	info->first_seen = jiffies;
@@ -215,16 +229,21 @@
 	/* initialize and add new packet. */
 	*is_new = 1;
 
+	/* Make it a broadcast packet, if required */
+	if (make_broadcast)
+		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
+
 	/* repair if entries is longer than packet. */
 	if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len)
-		info->packet.entries = vis_info_len / sizeof(struct vis_info_entry);
+		info->packet.entries = vis_info_len /
+			sizeof(struct vis_info_entry);
 
 	recv_list_add(&info->recv_list, info->packet.sender_orig);
 
 	/* try to add it */
 	if (hash_add(vis_hash, info) < 0) {
 		/* did not work (for some reason) */
-		free_info(info);
+		kref_put(&old_info->refcount, free_info);
 		info = NULL;
 	}
 
@@ -235,21 +254,24 @@
 void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len)
 {
 	struct vis_info *info;
-	int is_new;
+	int is_new, make_broadcast;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
 
+	make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC);
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast);
 	if (info == NULL)
 		goto end;
 
 	/* only if we are server ourselves and packet is newer than the one in
 	 * hash.*/
 	if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) {
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-		if (list_empty(&info->send_list))
+		if (list_empty(&info->send_list)) {
+			kref_get(&info->refcount);
 			list_add_tail(&info->send_list, &send_list);
+		}
 	}
 end:
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -263,31 +285,38 @@
 	int is_new;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
+	int are_target = 0;
 
 	/* clients shall not broadcast. */
 	if (is_bcast(vis_packet->target_orig))
 		return;
 
+	/* Are we the target for this VIS packet? */
+	if (vis_server == VIS_TYPE_SERVER_SYNC	&&
+	    is_my_mac(vis_packet->target_orig))
+		are_target = 1;
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, are_target);
 	if (info == NULL)
 		goto end;
 	/* note that outdated packets will be dropped at this point. */
 
 
 	/* send only if we're the target server or ... */
-	if (vis_server == VIS_TYPE_SERVER_SYNC  &&
-	    is_my_mac(info->packet.target_orig) &&
-	    is_new) {
+	if (are_target && is_new) {
 		info->packet.vis_type = VIS_TYPE_SERVER_SYNC;	/* upgrade! */
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-		if (list_empty(&info->send_list))
+		if (list_empty(&info->send_list)) {
+			kref_get(&info->refcount);
 			list_add_tail(&info->send_list, &send_list);
+		}
 
 		/* ... we're not the recipient (and thus need to forward). */
 	} else if (!is_my_mac(info->packet.target_orig)) {
-		if (list_empty(&info->send_list))
+		if (list_empty(&info->send_list)) {
+			kref_get(&info->refcount);
 			list_add_tail(&info->send_list, &send_list);
+		}
 	}
 end:
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -362,14 +391,17 @@
 	while (hash_iterate(orig_hash, &hashit_global)) {
 		orig_node = hashit_global.bucket->data;
 		if (orig_node->router != NULL
-			&& compare_orig(orig_node->router->addr, orig_node->orig)
+			&& compare_orig(orig_node->router->addr,
+					orig_node->orig)
 			&& orig_node->batman_if
 			&& (orig_node->batman_if->if_active == IF_ACTIVE)
 		    && orig_node->router->tq_avg > 0) {
 
 			/* fill one entry into buffer. */
 			entry = &entry_array[info->packet.entries];
-			memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN);
+			memcpy(entry->src,
+			       orig_node->batman_if->net_dev->dev_addr,
+			       ETH_ALEN);
 			memcpy(entry->dest, orig_node->orig, ETH_ALEN);
 			entry->quality = orig_node->router->tq_avg;
 			info->packet.entries++;
@@ -401,6 +433,8 @@
 	return 0;
 }
 
+/* free old vis packets. Must be called with this vis_hash_lock
+ * held */
 static void purge_vis_packets(void)
 {
 	HASHIT(hashit);
@@ -413,7 +447,7 @@
 		if (time_after(jiffies,
 			       info->first_seen + (VIS_TIMEOUT*HZ)/1000)) {
 			hash_remove_bucket(vis_hash, &hashit);
-			free_info(info);
+			kref_put(&info->refcount, free_info);
 		}
 	}
 }
@@ -423,6 +457,8 @@
 	HASHIT(hashit);
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 
@@ -431,46 +467,57 @@
 		orig_node = hashit.bucket->data;
 
 		/* if it's a vis server and reachable, send it. */
-		if (orig_node &&
-		    (orig_node->flags & VIS_SERVER) &&
-		    orig_node->batman_if &&
-		    orig_node->router) {
+		if ((!orig_node) || (!orig_node->batman_if) ||
+		    (!orig_node->router))
+			continue;
+		if (!(orig_node->flags & VIS_SERVER))
+			continue;
+		/* don't send it if we already received the packet from
+		 * this node. */
+		if (recv_list_is_in(&info->recv_list, orig_node->orig))
+			continue;
 
-			/* don't send it if we already received the packet from
-			 * this node. */
-			if (recv_list_is_in(&info->recv_list, orig_node->orig))
-				continue;
+		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
+		batman_if = orig_node->batman_if;
+		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
+		spin_unlock_irqrestore(&orig_hash_lock, flags);
 
-			memcpy(info->packet.target_orig,
-			       orig_node->orig, ETH_ALEN);
+		send_raw_packet((unsigned char *)&info->packet,
+				packet_length, batman_if, dstaddr);
 
-			send_raw_packet((unsigned char *) &info->packet,
-					packet_length,
-					orig_node->batman_if,
-					orig_node->router->addr);
-		}
+		spin_lock_irqsave(&orig_hash_lock, flags);
+
 	}
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 static void unicast_vis_packet(struct vis_info *info, int packet_length)
 {
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 	orig_node = ((struct orig_node *)
 		     hash_find(orig_hash, info->packet.target_orig));
 
-	if ((orig_node != NULL) &&
-	    (orig_node->batman_if != NULL) &&
-	    (orig_node->router != NULL)) {
-		send_raw_packet((unsigned char *) &info->packet, packet_length,
-				orig_node->batman_if,
-				orig_node->router->addr);
-	}
+	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
+		goto out;
+
+	/* don't lock while sending the packets ... we therefore
+	 * copy the required data before sending */
+	batman_if = orig_node->batman_if;
+	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
 	spin_unlock_irqrestore(&orig_hash_lock, flags);
+
+	send_raw_packet((unsigned char *)&info->packet,
+			packet_length, batman_if, dstaddr);
+	return;
+
+out:
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 /* only send one vis packet. called from send_vis_packets() */
@@ -503,15 +550,23 @@
 	unsigned long flags;
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
+
 	purge_vis_packets();
 
-	if (generate_vis_packet() == 0)
+	if (generate_vis_packet() == 0) {
 		/* schedule if generation was successful */
+		kref_get(&my_vis_info->refcount);
 		list_add_tail(&my_vis_info->send_list, &send_list);
+	}
 
 	list_for_each_entry_safe(info, temp, &send_list, send_list) {
+		spin_unlock_irqrestore(&vis_hash_lock, flags);
+
+		send_vis_packet(info);
+
+		spin_lock_irqsave(&vis_hash_lock, flags);
 		list_del_init(&info->send_list);
-		send_vis_packet(info);
+		kref_put(&info->refcount, free_info);
 	}
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
 	start_vis_timer();
@@ -544,6 +599,7 @@
 	my_vis_info->first_seen = jiffies - atomic_read(&vis_interval);
 	INIT_LIST_HEAD(&my_vis_info->recv_list);
 	INIT_LIST_HEAD(&my_vis_info->send_list);
+	kref_init(&my_vis_info->refcount);
 	my_vis_info->packet.version = COMPAT_VERSION;
 	my_vis_info->packet.packet_type = BAT_VIS;
 	my_vis_info->packet.ttl = TTL;
@@ -557,9 +613,9 @@
 
 	if (hash_add(vis_hash, my_vis_info) < 0) {
 		printk(KERN_ERR
-			  "batman-adv:Can't add own vis packet into hash\n");
-		free_info(my_vis_info);	/* not in hash, need to remove it
-					 * manually. */
+		       "batman-adv:Can't add own vis packet into hash\n");
+		/* not in hash, need to remove it manually. */
+		kref_put(&my_vis_info->refcount, free_info);
 		goto err;
 	}
 
@@ -573,6 +629,13 @@
 	return 0;
 }
 
+/* Decrease the reference count on a hash item info */
+static void free_info_ref(void *data)
+{
+	struct vis_info *info = data;
+	kref_put(&info->refcount, free_info);
+}
+
 /* shutdown vis-server */
 void vis_quit(void)
 {
@@ -584,7 +647,7 @@
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
 	/* properly remove, kill timers ... */
-	hash_delete(vis_hash, free_info);
+	hash_delete(vis_hash, free_info_ref);
 	vis_hash = NULL;
 	my_vis_info = NULL;
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -594,5 +657,5 @@
 static void start_vis_timer(void)
 {
 	queue_delayed_work(bat_event_workqueue, &vis_timer_wq,
-			   (atomic_read(&vis_interval) * HZ ) / 1000);
+			   (atomic_read(&vis_interval) * HZ) / 1000);
 }
Index: batman-adv-kernelland/vis.h
===================================================================
--- batman-adv-kernelland/vis.h	(revision 1578)
+++ batman-adv-kernelland/vis.h	(working copy)
@@ -29,6 +29,7 @@
 			    /* list of server-neighbors we received a vis-packet
 			     * from.  we should not reply to them. */
 	struct list_head send_list;
+	struct kref refcount;
 	/* this packet might be part of the vis send queue. */
 	struct vis_packet packet;
 	/* vis_info may follow here*/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-02-28 16:34             ` Simon Wunderlich
@ 2010-03-01  5:59               ` Andrew Lunn
  2010-03-01 16:57                 ` Simon Wunderlich
  0 siblings, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-03-01  5:59 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

Hi Simon

> @@ -503,15 +550,23 @@
>  	unsigned long flags;
>  
>  	spin_lock_irqsave(&vis_hash_lock, flags);
> +
>  	purge_vis_packets();
>  
> -	if (generate_vis_packet() == 0)
> +	if (generate_vis_packet() == 0) {
>  		/* schedule if generation was successful */
> +		kref_get(&my_vis_info->refcount);
>  		list_add_tail(&my_vis_info->send_list, &send_list);
> +	}
>  
>  	list_for_each_entry_safe(info, temp, &send_list, send_list) {
> +		spin_unlock_irqrestore(&vis_hash_lock, flags);
> +
> +		send_vis_packet(info);
> +
> +		spin_lock_irqsave(&vis_hash_lock, flags);
>  		list_del_init(&info->send_list);
> -		send_vis_packet(info);
> +		kref_put(&info->refcount, free_info);
>  	}
>  	spin_unlock_irqrestore(&vis_hash_lock, flags);
>  	start_vis_timer();

Although you say this works, i just looks wrong. It looks
unbalanced. Where is the kref_put for my_vis_info->refcount?  There is
a kref_put inside what looks like a loop, but no kref_get inside the
loop. 

Can you explain this is more detail? Can we make it look correctly
balanced?

	Thanks
		Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-01  5:59               ` Andrew Lunn
@ 2010-03-01 16:57                 ` Simon Wunderlich
  2010-03-02  6:43                   ` Andrew Lunn
  0 siblings, 1 reply; 37+ messages in thread
From: Simon Wunderlich @ 2010-03-01 16:57 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking


[-- Attachment #1.1: Type: text/plain, Size: 1850 bytes --]

Hey Andrew,

all list-adds and list-removes do a kref_get or kref_put respectively,
but that probably was not very clear. As soon as a refence is created 
(by linking into the hash or a list), we should also kref_get() it.

Please find attached a new version of this patch with separate 
send_list_add/remove functions which include the put/get functions. 
I've also added the kref_put/get call to the send_list loop again.
It should look more balanced now. Tested in my 9 qemu setup again,
no memory leaks or crashes but i only did a very quick test 
(15 minutes)

best regards,
	Simon

On Mon, Mar 01, 2010 at 06:59:55AM +0100, Andrew Lunn wrote:
> Hi Simon
> 
> > @@ -503,15 +550,23 @@
> >  	unsigned long flags;
> >  
> >  	spin_lock_irqsave(&vis_hash_lock, flags);
> > +
> >  	purge_vis_packets();
> >  
> > -	if (generate_vis_packet() == 0)
> > +	if (generate_vis_packet() == 0) {
> >  		/* schedule if generation was successful */
> > +		kref_get(&my_vis_info->refcount);
> >  		list_add_tail(&my_vis_info->send_list, &send_list);
> > +	}
> >  
> >  	list_for_each_entry_safe(info, temp, &send_list, send_list) {
> > +		spin_unlock_irqrestore(&vis_hash_lock, flags);
> > +
> > +		send_vis_packet(info);
> > +
> > +		spin_lock_irqsave(&vis_hash_lock, flags);
> >  		list_del_init(&info->send_list);
> > -		send_vis_packet(info);
> > +		kref_put(&info->refcount, free_info);
> >  	}
> >  	spin_unlock_irqrestore(&vis_hash_lock, flags);
> >  	start_vis_timer();
> 
> Although you say this works, i just looks wrong. It looks
> unbalanced. Where is the kref_put for my_vis_info->refcount?  There is
> a kref_put inside what looks like a loop, but no kref_get inside the
> loop. 
> 
> Can you explain this is more detail? Can we make it look correctly
> balanced?
> 
> 	Thanks
> 		Andrew
> 

[-- Attachment #1.2: vis_refcount2.patch --]
[-- Type: text/x-diff, Size: 14283 bytes --]

taging: batman-adv: Don't have interrupts disabled while sending.

send_vis_packets() would disable interrupts before calling
dev_queue_xmit() which resulting in a backtrace in local_bh_enable().
Fix this by using kref on the vis_info object so that we can call
send_vis_packets() without holding vis_hash_lock. vis_hash_lock also
used to protect recv_list, so we now need a new lock to protect that
instead of vis_hash_lock.

Also a few checkpatch cleanups.

Reported-by: Linus Luessing <linus.luessing@web.de>
Signed-off-by: Andrew Lunn <andrew@lunn.ch>
Signed-off-by: Simon Wunderlich <siwu@hrz.tu-chemnitz.de>
Index: a/batman-adv-kernelland/vis.c
===================================================================
--- a/batman-adv-kernelland/vis.c	(revision 1578)
+++ a/batman-adv-kernelland/vis.c	(working copy)
@@ -30,22 +30,26 @@
 
 struct hashtable_t *vis_hash;
 DEFINE_SPINLOCK(vis_hash_lock);
+static DEFINE_SPINLOCK(recv_list_lock);
 static struct vis_info *my_vis_info;
 static struct list_head send_list;	/* always locked with vis_hash_lock */
 
 static void start_vis_timer(void);
 
 /* free the info */
-static void free_info(void *data)
+static void free_info(struct kref *ref)
 {
-	struct vis_info *info = data;
+	struct vis_info *info = container_of(ref, struct vis_info, refcount);
 	struct recvlist_node *entry, *tmp;
+	unsigned long flags;
 
 	list_del_init(&info->send_list);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry_safe(entry, tmp, &info->recv_list, list) {
 		list_del(&entry->list);
 		kfree(entry);
 	}
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	kfree(info);
 }
 
@@ -143,36 +147,65 @@
 	}
 }
 
+/* add the info packet to the send list, if it was not
+ * already linked in. */
+static void send_list_add(struct vis_info *info)
+{
+	if (list_empty(&info->send_list)) {
+		kref_get(&info->refcount);
+		list_add_tail(&info->send_list, &send_list);
+	}
+}
+
+/* delete the info packet from the send list, if it was
+ * linked in. */
+static void send_list_del(struct vis_info *info)
+{
+	if (!list_empty(&info->send_list)) {
+		list_del_init(&info->send_list);
+		kref_put(&info->refcount, free_info);
+	}
+}
+
 /* tries to add one entry to the receive list. */
 static void recv_list_add(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
+
 	entry = kmalloc(sizeof(struct recvlist_node), GFP_ATOMIC);
 	if (!entry)
 		return;
 
 	memcpy(entry->mac, mac, ETH_ALEN);
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_add_tail(&entry->list, recv_list);
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 }
 
 /* returns 1 if this mac is in the recv_list */
 static int recv_list_is_in(struct list_head *recv_list, char *mac)
 {
 	struct recvlist_node *entry;
+	unsigned long flags;
 
+	spin_lock_irqsave(&recv_list_lock, flags);
 	list_for_each_entry(entry, recv_list, list) {
-		if (memcmp(entry->mac, mac, ETH_ALEN) == 0)
+		if (memcmp(entry->mac, mac, ETH_ALEN) == 0) {
+			spin_unlock_irqrestore(&recv_list_lock, flags);
 			return 1;
+		}
 	}
-
+	spin_unlock_irqrestore(&recv_list_lock, flags);
 	return 0;
 }
 
 /* try to add the packet to the vis_hash. return NULL if invalid (e.g. too old,
- * broken.. ).  vis hash must be locked outside.  is_new is set when the packet
+ * broken.. ).	vis hash must be locked outside.  is_new is set when the packet
  * is newer than old entries in the hash. */
 static struct vis_info *add_packet(struct vis_packet *vis_packet,
-				   int vis_info_len, int *is_new)
+				   int vis_info_len, int *is_new,
+				   int make_broadcast)
 {
 	struct vis_info *info, *old_info;
 	struct vis_info search_elem;
@@ -199,13 +232,15 @@
 		}
 		/* remove old entry */
 		hash_remove(vis_hash, old_info);
-		free_info(old_info);
+		send_list_del(old_info);
+		kref_put(&old_info->refcount, free_info);
 	}
 
 	info = kmalloc(sizeof(struct vis_info) + vis_info_len, GFP_ATOMIC);
 	if (info == NULL)
 		return NULL;
 
+	kref_init(&info->refcount);
 	INIT_LIST_HEAD(&info->send_list);
 	INIT_LIST_HEAD(&info->recv_list);
 	info->first_seen = jiffies;
@@ -215,16 +250,21 @@
 	/* initialize and add new packet. */
 	*is_new = 1;
 
+	/* Make it a broadcast packet, if required */
+	if (make_broadcast)
+		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
+
 	/* repair if entries is longer than packet. */
 	if (info->packet.entries * sizeof(struct vis_info_entry) > vis_info_len)
-		info->packet.entries = vis_info_len / sizeof(struct vis_info_entry);
+		info->packet.entries = vis_info_len /
+			sizeof(struct vis_info_entry);
 
 	recv_list_add(&info->recv_list, info->packet.sender_orig);
 
 	/* try to add it */
 	if (hash_add(vis_hash, info) < 0) {
 		/* did not work (for some reason) */
-		free_info(info);
+		kref_put(&old_info->refcount, free_info);
 		info = NULL;
 	}
 
@@ -235,22 +275,21 @@
 void receive_server_sync_packet(struct vis_packet *vis_packet, int vis_info_len)
 {
 	struct vis_info *info;
-	int is_new;
+	int is_new, make_broadcast;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
 
+	make_broadcast = (vis_server == VIS_TYPE_SERVER_SYNC);
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, make_broadcast);
 	if (info == NULL)
 		goto end;
 
 	/* only if we are server ourselves and packet is newer than the one in
 	 * hash.*/
-	if (vis_server == VIS_TYPE_SERVER_SYNC && is_new) {
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-		if (list_empty(&info->send_list))
-			list_add_tail(&info->send_list, &send_list);
-	}
+	if (vis_server == VIS_TYPE_SERVER_SYNC && is_new)
+		send_list_add(info);
 end:
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
 }
@@ -263,31 +302,32 @@
 	int is_new;
 	unsigned long flags;
 	int vis_server = atomic_read(&vis_mode);
+	int are_target = 0;
 
 	/* clients shall not broadcast. */
 	if (is_bcast(vis_packet->target_orig))
 		return;
 
+	/* Are we the target for this VIS packet? */
+	if (vis_server == VIS_TYPE_SERVER_SYNC	&&
+	    is_my_mac(vis_packet->target_orig))
+		are_target = 1;
+
 	spin_lock_irqsave(&vis_hash_lock, flags);
-	info = add_packet(vis_packet, vis_info_len, &is_new);
+	info = add_packet(vis_packet, vis_info_len, &is_new, are_target);
 	if (info == NULL)
 		goto end;
 	/* note that outdated packets will be dropped at this point. */
 
 
 	/* send only if we're the target server or ... */
-	if (vis_server == VIS_TYPE_SERVER_SYNC  &&
-	    is_my_mac(info->packet.target_orig) &&
-	    is_new) {
+	if (are_target && is_new) {
 		info->packet.vis_type = VIS_TYPE_SERVER_SYNC;	/* upgrade! */
-		memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-		if (list_empty(&info->send_list))
-			list_add_tail(&info->send_list, &send_list);
+		send_list_add(info);
 
 		/* ... we're not the recipient (and thus need to forward). */
 	} else if (!is_my_mac(info->packet.target_orig)) {
-		if (list_empty(&info->send_list))
-			list_add_tail(&info->send_list, &send_list);
+		send_list_add(info);
 	}
 end:
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -362,14 +402,17 @@
 	while (hash_iterate(orig_hash, &hashit_global)) {
 		orig_node = hashit_global.bucket->data;
 		if (orig_node->router != NULL
-			&& compare_orig(orig_node->router->addr, orig_node->orig)
+			&& compare_orig(orig_node->router->addr,
+					orig_node->orig)
 			&& orig_node->batman_if
 			&& (orig_node->batman_if->if_active == IF_ACTIVE)
 		    && orig_node->router->tq_avg > 0) {
 
 			/* fill one entry into buffer. */
 			entry = &entry_array[info->packet.entries];
-			memcpy(entry->src, orig_node->batman_if->net_dev->dev_addr, ETH_ALEN);
+			memcpy(entry->src,
+			       orig_node->batman_if->net_dev->dev_addr,
+			       ETH_ALEN);
 			memcpy(entry->dest, orig_node->orig, ETH_ALEN);
 			entry->quality = orig_node->router->tq_avg;
 			info->packet.entries++;
@@ -401,6 +444,8 @@
 	return 0;
 }
 
+/* free old vis packets. Must be called with this vis_hash_lock
+ * held */
 static void purge_vis_packets(void)
 {
 	HASHIT(hashit);
@@ -413,7 +458,8 @@
 		if (time_after(jiffies,
 			       info->first_seen + (VIS_TIMEOUT*HZ)/1000)) {
 			hash_remove_bucket(vis_hash, &hashit);
-			free_info(info);
+			send_list_del(info);
+			kref_put(&info->refcount, free_info);
 		}
 	}
 }
@@ -423,6 +469,8 @@
 	HASHIT(hashit);
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 
@@ -431,46 +479,57 @@
 		orig_node = hashit.bucket->data;
 
 		/* if it's a vis server and reachable, send it. */
-		if (orig_node &&
-		    (orig_node->flags & VIS_SERVER) &&
-		    orig_node->batman_if &&
-		    orig_node->router) {
+		if ((!orig_node) || (!orig_node->batman_if) ||
+		    (!orig_node->router))
+			continue;
+		if (!(orig_node->flags & VIS_SERVER))
+			continue;
+		/* don't send it if we already received the packet from
+		 * this node. */
+		if (recv_list_is_in(&info->recv_list, orig_node->orig))
+			continue;
 
-			/* don't send it if we already received the packet from
-			 * this node. */
-			if (recv_list_is_in(&info->recv_list, orig_node->orig))
-				continue;
+		memcpy(info->packet.target_orig, orig_node->orig, ETH_ALEN);
+		batman_if = orig_node->batman_if;
+		memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
+		spin_unlock_irqrestore(&orig_hash_lock, flags);
 
-			memcpy(info->packet.target_orig,
-			       orig_node->orig, ETH_ALEN);
+		send_raw_packet((unsigned char *)&info->packet,
+				packet_length, batman_if, dstaddr);
 
-			send_raw_packet((unsigned char *) &info->packet,
-					packet_length,
-					orig_node->batman_if,
-					orig_node->router->addr);
-		}
+		spin_lock_irqsave(&orig_hash_lock, flags);
+
 	}
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 	memcpy(info->packet.target_orig, broadcastAddr, ETH_ALEN);
-	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 static void unicast_vis_packet(struct vis_info *info, int packet_length)
 {
 	struct orig_node *orig_node;
 	unsigned long flags;
+	struct batman_if *batman_if;
+	uint8_t dstaddr[ETH_ALEN];
 
 	spin_lock_irqsave(&orig_hash_lock, flags);
 	orig_node = ((struct orig_node *)
 		     hash_find(orig_hash, info->packet.target_orig));
 
-	if ((orig_node != NULL) &&
-	    (orig_node->batman_if != NULL) &&
-	    (orig_node->router != NULL)) {
-		send_raw_packet((unsigned char *) &info->packet, packet_length,
-				orig_node->batman_if,
-				orig_node->router->addr);
-	}
+	if ((!orig_node) || (!orig_node->batman_if) || (!orig_node->router))
+		goto out;
+
+	/* don't lock while sending the packets ... we therefore
+	 * copy the required data before sending */
+	batman_if = orig_node->batman_if;
+	memcpy(dstaddr, orig_node->router->addr, ETH_ALEN);
 	spin_unlock_irqrestore(&orig_hash_lock, flags);
+
+	send_raw_packet((unsigned char *)&info->packet,
+			packet_length, batman_if, dstaddr);
+	return;
+
+out:
+	spin_unlock_irqrestore(&orig_hash_lock, flags);
 }
 
 /* only send one vis packet. called from send_vis_packets() */
@@ -503,15 +562,24 @@
 	unsigned long flags;
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
+
 	purge_vis_packets();
 
-	if (generate_vis_packet() == 0)
+	if (generate_vis_packet() == 0) {
 		/* schedule if generation was successful */
-		list_add_tail(&my_vis_info->send_list, &send_list);
+		send_list_add(my_vis_info);
+	}
 
 	list_for_each_entry_safe(info, temp, &send_list, send_list) {
-		list_del_init(&info->send_list);
+
+		kref_get(&info->refcount);
+		spin_unlock_irqrestore(&vis_hash_lock, flags);
+
 		send_vis_packet(info);
+
+		spin_lock_irqsave(&vis_hash_lock, flags);
+		send_list_del(info);
+		kref_put(&info->refcount, free_info);
 	}
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
 	start_vis_timer();
@@ -544,6 +612,7 @@
 	my_vis_info->first_seen = jiffies - atomic_read(&vis_interval);
 	INIT_LIST_HEAD(&my_vis_info->recv_list);
 	INIT_LIST_HEAD(&my_vis_info->send_list);
+	kref_init(&my_vis_info->refcount);
 	my_vis_info->packet.version = COMPAT_VERSION;
 	my_vis_info->packet.packet_type = BAT_VIS;
 	my_vis_info->packet.ttl = TTL;
@@ -557,9 +626,9 @@
 
 	if (hash_add(vis_hash, my_vis_info) < 0) {
 		printk(KERN_ERR
-			  "batman-adv:Can't add own vis packet into hash\n");
-		free_info(my_vis_info);	/* not in hash, need to remove it
-					 * manually. */
+		       "batman-adv:Can't add own vis packet into hash\n");
+		/* not in hash, need to remove it manually. */
+		kref_put(&my_vis_info->refcount, free_info);
 		goto err;
 	}
 
@@ -573,6 +642,15 @@
 	return 0;
 }
 
+/* Decrease the reference count on a hash item info */
+static void free_info_ref(void *data)
+{
+	struct vis_info *info = data;
+
+	send_list_del(info);
+	kref_put(&info->refcount, free_info);
+}
+
 /* shutdown vis-server */
 void vis_quit(void)
 {
@@ -584,7 +662,7 @@
 
 	spin_lock_irqsave(&vis_hash_lock, flags);
 	/* properly remove, kill timers ... */
-	hash_delete(vis_hash, free_info);
+	hash_delete(vis_hash, free_info_ref);
 	vis_hash = NULL;
 	my_vis_info = NULL;
 	spin_unlock_irqrestore(&vis_hash_lock, flags);
@@ -594,5 +672,5 @@
 static void start_vis_timer(void)
 {
 	queue_delayed_work(bat_event_workqueue, &vis_timer_wq,
-			   (atomic_read(&vis_interval) * HZ ) / 1000);
+			   (atomic_read(&vis_interval) * HZ) / 1000);
 }
Index: a/batman-adv-kernelland/vis.h
===================================================================
--- a/batman-adv-kernelland/vis.h	(revision 1578)
+++ a/batman-adv-kernelland/vis.h	(working copy)
@@ -29,6 +29,7 @@
 			    /* list of server-neighbors we received a vis-packet
 			     * from.  we should not reply to them. */
 	struct list_head send_list;
+	struct kref refcount;
 	/* this packet might be part of the vis send queue. */
 	struct vis_packet packet;
 	/* vis_info may follow here*/

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-01 16:57                 ` Simon Wunderlich
@ 2010-03-02  6:43                   ` Andrew Lunn
  2010-03-02 21:13                     ` Simon Wunderlich
  0 siblings, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-03-02  6:43 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Mon, Mar 01, 2010 at 05:57:06PM +0100, Simon Wunderlich wrote:
> Hey Andrew,
> 
> all list-adds and list-removes do a kref_get or kref_put respectively,
> but that probably was not very clear. As soon as a refence is created 
> (by linking into the hash or a list), we should also kref_get() it.
> 
> Please find attached a new version of this patch with separate 
> send_list_add/remove functions which include the put/get functions. 
> I've also added the kref_put/get call to the send_list loop again.
> It should look more balanced now. Tested in my 9 qemu setup again,
> no memory leaks or crashes but i only did a very quick test 
> (15 minutes)

Hi Simon

This looks better. Did it survive longer testing? 

     Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-02  6:43                   ` Andrew Lunn
@ 2010-03-02 21:13                     ` Simon Wunderlich
  2010-03-02 21:26                       ` elektra
  2010-03-02 21:44                       ` Linus Lüssing
  0 siblings, 2 replies; 37+ messages in thread
From: Simon Wunderlich @ 2010-03-02 21:13 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 1132 bytes --]

Hello Andrew,

no, sorry, and i won't be able to do it this week as i am at the cebit
this whole week (if anyone else is there, please tell me. ;).

I would therefore like to ask you and Linus to test it as well, i will 
perform more tests on sunday.

Thanks,
	Simon

On Tue, Mar 02, 2010 at 07:43:08AM +0100, Andrew Lunn wrote:
> On Mon, Mar 01, 2010 at 05:57:06PM +0100, Simon Wunderlich wrote:
> > Hey Andrew,
> > 
> > all list-adds and list-removes do a kref_get or kref_put respectively,
> > but that probably was not very clear. As soon as a refence is created 
> > (by linking into the hash or a list), we should also kref_get() it.
> > 
> > Please find attached a new version of this patch with separate 
> > send_list_add/remove functions which include the put/get functions. 
> > I've also added the kref_put/get call to the send_list loop again.
> > It should look more balanced now. Tested in my 9 qemu setup again,
> > no memory leaks or crashes but i only did a very quick test 
> > (15 minutes)
> 
> Hi Simon
> 
> This looks better. Did it survive longer testing? 
> 
>      Andrew
> 

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 197 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-02 21:13                     ` Simon Wunderlich
@ 2010-03-02 21:26                       ` elektra
  2010-03-02 21:44                       ` Linus Lüssing
  1 sibling, 0 replies; 37+ messages in thread
From: elektra @ 2010-03-02 21:26 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

Hello Simon -

I'll be in hall 13 at the booth of Atcom E42 to present the Mesh-Potato 
router. (With B.A.T.M.A.N. inside, of course) Will arrive tomorrow afternoon 
and be there until Saturday 18:00.

Cheers,
Elektra


> no, sorry, and i won't be able to do it this week as i am at the cebit
> this whole week (if anyone else is there, please tell me. ;).

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-02 21:13                     ` Simon Wunderlich
  2010-03-02 21:26                       ` elektra
@ 2010-03-02 21:44                       ` Linus Lüssing
  2010-03-04  0:26                         ` Linus Lüssing
  1 sibling, 1 reply; 37+ messages in thread
From: Linus Lüssing @ 2010-03-02 21:44 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 1959 bytes --]

Hi Simon and Andrew,

I already tried both Andrew's intial and Simon's latest patch
yesterday and had both of them running for about 1h+, also with
alternate vis-intervals (1s, 50ms, 30s) but so far I wasn't able
to reproduce any crashes or leaks on two Debian stable VMs
connected with each other. I'll run Andrew's patch version again
this night for some more hours to make sure that I'm able to
reproduce the crash at all. Any other hints for reproducing it
more quickly welcome :).

Cheers, Linus

PS: With how many nodes and what kind of systems could you
reproduce the crash? Were all those nodes connected directly with
each other? How many vis-servers were running?

On Tue, Mar 02, 2010 at 10:13:24PM +0100, Simon Wunderlich wrote:
> Hello Andrew,
> 
> no, sorry, and i won't be able to do it this week as i am at the cebit
> this whole week (if anyone else is there, please tell me. ;).
> 
> I would therefore like to ask you and Linus to test it as well, i will 
> perform more tests on sunday.
> 
> Thanks,
> 	Simon
> 
> On Tue, Mar 02, 2010 at 07:43:08AM +0100, Andrew Lunn wrote:
> > On Mon, Mar 01, 2010 at 05:57:06PM +0100, Simon Wunderlich wrote:
> > > Hey Andrew,
> > > 
> > > all list-adds and list-removes do a kref_get or kref_put respectively,
> > > but that probably was not very clear. As soon as a refence is created 
> > > (by linking into the hash or a list), we should also kref_get() it.
> > > 
> > > Please find attached a new version of this patch with separate 
> > > send_list_add/remove functions which include the put/get functions. 
> > > I've also added the kref_put/get call to the send_list loop again.
> > > It should look more balanced now. Tested in my 9 qemu setup again,
> > > no memory leaks or crashes but i only did a very quick test 
> > > (15 minutes)
> > 
> > Hi Simon
> > 
> > This looks better. Did it survive longer testing? 
> > 
> >      Andrew
> > 



[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-02 21:44                       ` Linus Lüssing
@ 2010-03-04  0:26                         ` Linus Lüssing
  2010-03-04  8:57                           ` Andrew Lunn
  0 siblings, 1 reply; 37+ messages in thread
From: Linus Lüssing @ 2010-03-04  0:26 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

[-- Attachment #1: Type: text/plain, Size: 306 bytes --]

I've been running 3 qemu x86 instances with Debian stable here
(2.6.26 kernel) as well as 3 Atheros ap61 / mips based routers with
OpenWRT (rev. 19468, 2.6.30.10 kernel) for 22 hours. None of them
crashed with Andrew's patch... so, sorry, still can't reproduce the
bug you're talking about.

Cheers, Linus

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-04  0:26                         ` Linus Lüssing
@ 2010-03-04  8:57                           ` Andrew Lunn
  2010-03-04  9:19                             ` Marek Lindner
  0 siblings, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-03-04  8:57 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Thu, Mar 04, 2010 at 01:26:20AM +0100, Linus L??ssing wrote:
> I've been running 3 qemu x86 instances with Debian stable here
> (2.6.26 kernel) as well as 3 Atheros ap61 / mips based routers with
> OpenWRT (rev. 19468, 2.6.30.10 kernel) for 22 hours. None of them
> crashed with Andrew's patch... so, sorry, still can't reproduce the
> bug you're talking about.

If you cannot reproduce it, its probably fixed :-)

Marek, could you commit the patch.

       Thanks
          Andrew


^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-04  8:57                           ` Andrew Lunn
@ 2010-03-04  9:19                             ` Marek Lindner
  2010-03-04  9:49                               ` Andrew Lunn
  0 siblings, 1 reply; 37+ messages in thread
From: Marek Lindner @ 2010-03-04  9:19 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Thursday 04 March 2010 16:57:24 Andrew Lunn wrote:
> On Thu, Mar 04, 2010 at 01:26:20AM +0100, Linus L??ssing wrote:
> > I've been running 3 qemu x86 instances with Debian stable here
> > (2.6.26 kernel) as well as 3 Atheros ap61 / mips based routers with
> > OpenWRT (rev. 19468, 2.6.30.10 kernel) for 22 hours. None of them
> > crashed with Andrew's patch... so, sorry, still can't reproduce the
> > bug you're talking about.
> 
> If you cannot reproduce it, its probably fixed :-)
> 
> Marek, could you commit the patch.

Sure but which one ? I have to admit I lost the overview (there were too many 
patches flying around) ..  :-)

Cheers,
Marek

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-04  9:19                             ` Marek Lindner
@ 2010-03-04  9:49                               ` Andrew Lunn
  2010-03-04 10:00                                 ` Marek Lindner
  0 siblings, 1 reply; 37+ messages in thread
From: Andrew Lunn @ 2010-03-04  9:49 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

> Sure but which one ? I have to admit I lost the overview (there were
> too many patches flying around) ..  :-)

https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2010-March/002354.html

	Andrew

^ permalink raw reply	[flat|nested] 37+ messages in thread

* Re: [B.A.T.M.A.N.] slowpath warning
  2010-03-04  9:49                               ` Andrew Lunn
@ 2010-03-04 10:00                                 ` Marek Lindner
  0 siblings, 0 replies; 37+ messages in thread
From: Marek Lindner @ 2010-03-04 10:00 UTC (permalink / raw)
  To: The list for a Better Approach To Mobile Ad-hoc Networking

On Thursday 04 March 2010 17:49:41 Andrew Lunn wrote:
> > Sure but which one ? I have to admit I lost the overview (there were
> > too many patches flying around) ..  :-)
> 
> https://lists.open-mesh.org/pipermail/b.a.t.m.a.n/2010-March/002354.html

Applied in rev 1579,

Regards,
Marek

^ permalink raw reply	[flat|nested] 37+ messages in thread

end of thread, other threads:[~2010-03-04 10:00 UTC | newest]

Thread overview: 37+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-01-23 17:46 [B.A.T.M.A.N.] bat_events: page allocation failure (batman-adv maint) Linus Lüssing
2010-01-23 18:10 ` Andrew Lunn
2010-01-23 23:30   ` Linus Lüssing
2010-01-24 20:42   ` Linus Lüssing
2010-01-24 21:00     ` Andrew Lunn
2010-01-25  6:46       ` Andrew Lunn
2010-01-25  6:47     ` Andrew Lunn
2010-01-25  8:21       ` Marek Lindner
2010-01-26  1:48       ` Linus Lüssing
2010-01-24  4:24 ` Linus Lüssing
2010-01-26  6:13 ` [B.A.T.M.A.N.] slowpath warning Linus Lüssing
2010-01-26  7:16   ` Marek Lindner
2010-01-27  0:10     ` Linus Lüssing
2010-01-28  0:09       ` Marek Lindner
2010-01-28  6:29         ` Andrew Lunn
2010-01-29  8:25   ` Andrew Lunn
2010-01-29  8:59     ` Marek Lindner
2010-01-30 16:50       ` Andrew Lunn
2010-01-31 19:37         ` Linus Lüssing
2010-01-31 20:56           ` Andrew Lunn
2010-02-11  9:46         ` Andrew Lunn
2010-02-11 10:01           ` Andrew Lunn
2010-02-19 17:19             ` Linus Lüssing
2010-02-20 18:04               ` Andrew Lunn
2010-02-21 13:10                 ` Linus Lüssing
2010-02-28 16:34             ` Simon Wunderlich
2010-03-01  5:59               ` Andrew Lunn
2010-03-01 16:57                 ` Simon Wunderlich
2010-03-02  6:43                   ` Andrew Lunn
2010-03-02 21:13                     ` Simon Wunderlich
2010-03-02 21:26                       ` elektra
2010-03-02 21:44                       ` Linus Lüssing
2010-03-04  0:26                         ` Linus Lüssing
2010-03-04  8:57                           ` Andrew Lunn
2010-03-04  9:19                             ` Marek Lindner
2010-03-04  9:49                               ` Andrew Lunn
2010-03-04 10:00                                 ` Marek Lindner

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).