* [Xenomai] Oops while running "cat /proc/xenomai/stat" @ 2012-10-08 9:32 Stefan Roese 2012-10-08 17:39 ` Gilles Chanteperdrix 0 siblings, 1 reply; 12+ messages in thread From: Stefan Roese @ 2012-10-08 9:32 UTC (permalink / raw) To: xenomai Hi, I'm currently developing an RTDM driver communicating with an FPGA located on the LPB on an MPC5200 PowerPC. This driver already seems to work quite well. But when I run my test application communicating with the device driver, and I try to check the Xenomai stat's, I get a kernel crash: root@generic-powerpc:~# cat /proc/xenomai/stat [ 43.722984] Oops: Kernel access of bad area, sig: 11 [#1] [ 43.728503] mpc5200-simple-platform [ 43.732057] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) [ 43.738107] NIP: c00646a8 LR: c0098b84 CTR: c0098b34 [ 43.743173] REGS: c7025b40 TRAP: 0300 Tainted: G O (3.5.3-00253-g4699145-dirty) [ 43.751776] MSR: 00001032 <ME,IR,DR,RI> CR: 24424488 XER: 20000000 [ 43.758279] DAR: 00000000, DSISR: 22000000 [ 43.762454] TASK = c7b75360[1430] 'cat' THREAD: c7024000 GPR00: 00000000 c7025bf0 c7b75360 c7b1ca0c 00000002 02000007 00000000 00000031 GPR08: c7ab0000 c7b1cc00 00000000 00000000 c0098b34 100a5a74 10017830 10006834 GPR16: 10006770 10006774 c7b1cc00 00000000 00000002 c033d385 c03264ac 00000025 GPR24: c0320000 00000001 00000002 c03b3460 00000000 00000000 c7a6ccc4 c7a6ccb4 [ 43.796697] Call Trace: [ 43.799189] [c7025bf0] [c004c540] 0xc004c540 (unreliable) [ 43.804698] [c7025c20] [c0098b84] 0xc0098b84 [ 43.809051] [c7025c40] [c90e4588] 0xc90e4588 [ 43.813406] [c7025c50] [c90d8114] 0xc90d8114 [ 43.817758] [c7025c60] [c005e85c] 0xc005e85c [ 43.822113] [c7025ca0] [c005aef0] 0xc005aef0 [ 43.826465] [c7025cc0] [c000d664] 0xc000d664 [ 43.830819] [c7025cd0] [c000f710] 0xc000f710 [ 43.835175] --- Exception: 501 at 0xc01b2614 This happens reproducible upon the "cat" command from the shell. Looking at the NIP (c00646a8), the PPC is currently executing xnsynch_flush(). Other proc files from the xenomai directory like "irq" don't cause this crash. And accessing "stat" when this application is not running also works fine: root@generic-powerpc:~# cat /proc/xenomai/stat CPU PID MSW CSW PF STAT %CPU NAME 0 0 0 0 0 00500080 100.0 ROOT 0 0 0 0 0 00000000 0.0 IRQ145: rtcan_mscan 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx 0 0 0 17619 0 00000000 0.0 IRQ512: [timer] Any idea what might go wrong here? Thanks, Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-08 9:32 [Xenomai] Oops while running "cat /proc/xenomai/stat" Stefan Roese @ 2012-10-08 17:39 ` Gilles Chanteperdrix 2012-10-09 6:48 ` Stefan Roese 0 siblings, 1 reply; 12+ messages in thread From: Gilles Chanteperdrix @ 2012-10-08 17:39 UTC (permalink / raw) To: Stefan Roese; +Cc: xenomai On 10/08/2012 11:32 AM, Stefan Roese wrote: > Hi, > > I'm currently developing an RTDM driver communicating with an FPGA > located on the LPB on an MPC5200 PowerPC. This driver already seems > to work quite well. But when I run my test application > communicating with the device driver, and I try to check the > Xenomai stat's, I get a kernel crash: > > root@generic-powerpc:~# cat /proc/xenomai/stat > [ 43.722984] Oops: Kernel access of bad area, sig: 11 [#1] > [ 43.728503] mpc5200-simple-platform > [ 43.732057] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) > [ 43.738107] NIP: c00646a8 LR: c0098b84 CTR: c0098b34 > [ 43.743173] REGS: c7025b40 TRAP: 0300 Tainted: G O (3.5.3-00253-g4699145-dirty) > [ 43.751776] MSR: 00001032 <ME,IR,DR,RI> CR: 24424488 XER: 20000000 > [ 43.758279] DAR: 00000000, DSISR: 22000000 > [ 43.762454] TASK = c7b75360[1430] 'cat' THREAD: c7024000 > GPR00: 00000000 c7025bf0 c7b75360 c7b1ca0c 00000002 02000007 00000000 00000031 > GPR08: c7ab0000 c7b1cc00 00000000 00000000 c0098b34 100a5a74 10017830 10006834 > GPR16: 10006770 10006774 c7b1cc00 00000000 00000002 c033d385 c03264ac 00000025 > GPR24: c0320000 00000001 00000002 c03b3460 00000000 00000000 c7a6ccc4 c7a6ccb4 > [ 43.796697] Call Trace: > [ 43.799189] [c7025bf0] [c004c540] 0xc004c540 (unreliable) > [ 43.804698] [c7025c20] [c0098b84] 0xc0098b84 > [ 43.809051] [c7025c40] [c90e4588] 0xc90e4588 > [ 43.813406] [c7025c50] [c90d8114] 0xc90d8114 > [ 43.817758] [c7025c60] [c005e85c] 0xc005e85c > [ 43.822113] [c7025ca0] [c005aef0] 0xc005aef0 > [ 43.826465] [c7025cc0] [c000d664] 0xc000d664 > [ 43.830819] [c7025cd0] [c000f710] 0xc000f710 > [ 43.835175] --- Exception: 501 at 0xc01b2614 > > This happens reproducible upon the "cat" command from the shell. > > Looking at the NIP (c00646a8), the PPC is currently executing > xnsynch_flush(). > > Other proc files from the xenomai directory like "irq" don't cause > this crash. And accessing "stat" when this application is not > running also works fine: > > root@generic-powerpc:~# cat /proc/xenomai/stat > CPU PID MSW CSW PF STAT %CPU NAME > 0 0 0 0 0 00500080 100.0 ROOT > 0 0 0 0 0 00000000 0.0 IRQ145: rtcan_mscan > 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo > 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx > 0 0 0 17619 0 00000000 0.0 IRQ512: [timer] > > Any idea what might go wrong here? Please enable CONFIG_KALLSYMS so that the backtrace contains readable function names, otherwise, without a disassembly of your kernel, we have no idea what functions the backtrace is referring to. > > Thanks, > Stefan > > _______________________________________________ > Xenomai mailing list > Xenomai@xenomai.org > http://www.xenomai.org/mailman/listinfo/xenomai > -- Gilles. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-08 17:39 ` Gilles Chanteperdrix @ 2012-10-09 6:48 ` Stefan Roese 2012-10-09 9:47 ` Gilles Chanteperdrix 2012-10-09 14:24 ` Philippe Gerum 0 siblings, 2 replies; 12+ messages in thread From: Stefan Roese @ 2012-10-09 6:48 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai Hi Gilles, On 10/08/2012 07:39 PM, Gilles Chanteperdrix wrote: > On 10/08/2012 11:32 AM, Stefan Roese wrote: > >> Hi, >> >> I'm currently developing an RTDM driver communicating with an FPGA >> located on the LPB on an MPC5200 PowerPC. This driver already seems >> to work quite well. But when I run my test application >> communicating with the device driver, and I try to check the >> Xenomai stat's, I get a kernel crash: >> >> root@generic-powerpc:~# cat /proc/xenomai/stat >> [ 43.722984] Oops: Kernel access of bad area, sig: 11 [#1] >> [ 43.728503] mpc5200-simple-platform >> [ 43.732057] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) >> [ 43.738107] NIP: c00646a8 LR: c0098b84 CTR: c0098b34 >> [ 43.743173] REGS: c7025b40 TRAP: 0300 Tainted: G O (3.5.3-00253-g4699145-dirty) >> [ 43.751776] MSR: 00001032 <ME,IR,DR,RI> CR: 24424488 XER: 20000000 >> [ 43.758279] DAR: 00000000, DSISR: 22000000 >> [ 43.762454] TASK = c7b75360[1430] 'cat' THREAD: c7024000 >> GPR00: 00000000 c7025bf0 c7b75360 c7b1ca0c 00000002 02000007 00000000 00000031 >> GPR08: c7ab0000 c7b1cc00 00000000 00000000 c0098b34 100a5a74 10017830 10006834 >> GPR16: 10006770 10006774 c7b1cc00 00000000 00000002 c033d385 c03264ac 00000025 >> GPR24: c0320000 00000001 00000002 c03b3460 00000000 00000000 c7a6ccc4 c7a6ccb4 >> [ 43.796697] Call Trace: >> [ 43.799189] [c7025bf0] [c004c540] 0xc004c540 (unreliable) >> [ 43.804698] [c7025c20] [c0098b84] 0xc0098b84 >> [ 43.809051] [c7025c40] [c90e4588] 0xc90e4588 >> [ 43.813406] [c7025c50] [c90d8114] 0xc90d8114 >> [ 43.817758] [c7025c60] [c005e85c] 0xc005e85c >> [ 43.822113] [c7025ca0] [c005aef0] 0xc005aef0 >> [ 43.826465] [c7025cc0] [c000d664] 0xc000d664 >> [ 43.830819] [c7025cd0] [c000f710] 0xc000f710 >> [ 43.835175] --- Exception: 501 at 0xc01b2614 >> >> This happens reproducible upon the "cat" command from the shell. >> >> Looking at the NIP (c00646a8), the PPC is currently executing >> xnsynch_flush(). >> >> Other proc files from the xenomai directory like "irq" don't cause >> this crash. And accessing "stat" when this application is not >> running also works fine: >> >> root@generic-powerpc:~# cat /proc/xenomai/stat >> CPU PID MSW CSW PF STAT %CPU NAME >> 0 0 0 0 0 00500080 100.0 ROOT >> 0 0 0 0 0 00000000 0.0 IRQ145: rtcan_mscan >> 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo >> 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx >> 0 0 0 17619 0 00000000 0.0 IRQ512: [timer] >> >> Any idea what might go wrong here? > > > Please enable CONFIG_KALLSYMS so that the backtrace contains readable > function names, otherwise, without a disassembly of your kernel, we have > no idea what functions the backtrace is referring to. Yes, sorry my bad. Here the new crash log: root@generic-powerpc:~# cat /proc/xenomai/stat [ 65.215600] Oops: Kernel access of bad area, sig: 11 [#1] [ 65.221118] mpc5200-simple-platform [ 65.224671] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) [ 65.230718] NIP: c0066914 LR: c009adf0 CTR: c009ada0 [ 65.235784] REGS: c716bac0 TRAP: 0300 Tainted: G O (3.5.3-00253-g4699145-dirty) [ 65.244386] MSR: 00001032 <ME,IR,DR,RI> CR: 24000488 XER: 20000000 [ 65.250888] DAR: 00000000, DSISR: 22000000 [ 65.255064] TASK = c705cba0[1400] 'cat' THREAD: c716a000 GPR00: 00000000 c716bb70 c705cba0 c7b3060c 00000002 02000006 00000000 00000030 GPR08: c70f0000 c7b30800 00000000 00000000 c009ada0 100a5a74 10017830 10006834 GPR16: 10006770 10006774 100170f4 00010000 c00733d8 c75e4a60 00000000 c03ee068 GPR24: 00000002 00000001 00000002 c03ec460 00000000 00000000 c7a6ccc4 c7a6ccb4 [ 65.289337] NIP [c0066914] xnsynch_flush+0x64/0x100 [ 65.294328] LR [c009adf0] rtdm_event_signal+0x50/0xe4 [ 65.299474] Call Trace: [ 65.301969] [c716bb70] [00000004] 0x4 (unreliable) [ 65.306864] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 [ 65.312735] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] [ 65.319934] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] [ 65.328461] [c716bbe0] [c0060ac8] xnintr_irq_handler+0x70/0x234 [ 65.334515] [c716bc20] [c005d15c] __ipipe_dispatch_irq+0x148/0x270 [ 65.340832] [c716bc40] [c000d66c] __ipipe_grab_irq+0x38/0x6c [ 65.346615] [c716bc50] [c000f710] __ipipe_ret_from_except+0x0/0xc [ 65.352849] --- Exception: 501 at __ipipe_restore_head+0x84/0xfc [ 65.352849] LR = __vfile_nklock_put+0x30/0x40 [ 65.363775] [c716bd10] [c0069bcc] vfile_stat_next+0x1b0/0x23c (unreliable) [ 65.370795] [c716bd20] [c0072f14] __vfile_nklock_put+0x30/0x40 [ 65.376751] [c716bd30] [c00738d0] vfile_snapshot_open+0x29c/0x31c [ 65.382975] [c716bd70] [c011b1c0] proc_reg_open+0x7c/0x11c [ 65.388576] [c716bd90] [c00d1a94] do_dentry_open.isra.18+0x1f4/0x29c [ 65.395064] [c716bdc0] [c00d2ad8] nameidata_to_filp+0x5c/0xa8 [ 65.400932] [c716bde0] [c00e11c0] do_last.isra.49+0x260/0x78c [ 65.406796] [c716be30] [c00e17ec] path_openat+0xb4/0x3b0 [ 65.412219] [c716be80] [c00e1bdc] do_filp_open+0x30/0x8c [ 65.417641] [c716bf10] [c00d2c20] do_sys_open+0xfc/0x1c8 [ 65.423069] [c716bf40] [c000ee0c] ret_from_syscall+0x0/0x38 [ 65.428757] --- Exception: c01 at 0xff0f47c [ 65.428757] LR = 0xffabb94 [ 65.436118] Instruction dump: [ 65.439139] 833b0918 63200001 573907fe 901b0918 853e0010 7f89f000 419e00ac 3ba00000 [ 65.447061] 48000044 419a0054 81690004 80090000 <900b0000> 81690000 80090004 900b0004 [ 65.455216] Oops: Kernel access of bad area, sig: 11 [#2] [ 65.460723] mpc5200-simple-platform [ 65.464272] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) [ 65.470319] NIP: 00000000 LR: c0060ac8 CTR: 00000000 [ 65.475385] REGS: c716b880 TRAP: 0400 Tainted: G D O (3.5.3-00253-g4699145-dirty) [ 65.483985] MSR: 20001032 <ME,IR,DR,RI> CR: 24000482 XER: 20000000 [ 65.490489] TASK = c705cba0[1400] 'cat' THREAD: c716a000 GPR00: 00000000 c716b930 c705cba0 c7a6ccf0 c7a6ccf0 00000000 c0410c94 00000000 GPR08: c03ecd78 00000000 00000002 00004000 00000000 100a5a74 10017830 10006834 GPR16: 10006770 10006774 100170f4 00010000 c00733d8 c75e4a60 00000000 c03ee068 GPR24: 00000002 c7a6ccf0 00000000 938c6ae2 c041fb40 00000010 c04244b8 c04240b8 [ 65.524729] NIP [00000000] (null) [ 65.528297] LR [c0060ac8] xnintr_irq_handler+0x70/0x234 [ 65.533621] Call Trace: [ 65.536115] [c716b930] [00000001] 0x1 (unreliable) [ 65.541023] [c716b970] [c005d15c] __ipipe_dispatch_irq+0x148/0x270 [ 65.547337] [c716b990] [c000d66c] __ipipe_grab_irq+0x38/0x6c [ 65.553118] [c716b9a0] [c000f710] __ipipe_ret_from_except+0x0/0xc [ 65.559348] --- Exception: 501 at ipipe_unstall_root+0x48/0x5c [ 65.559348] LR = ipipe_unstall_root+0x58/0x5c [ 65.570085] [c716ba70] [c000aa94] die+0x218/0x240 [ 65.574892] [c716baa0] [c0010360] bad_page_fault+0xb4/0xfc [ 65.580492] [c716bab0] [c000f2c4] handle_page_fault+0x7c/0x80 [ 65.586382] --- Exception: 300 at xnsynch_flush+0x64/0x100 [ 65.586382] LR = rtdm_event_signal+0x50/0xe4 [ 65.596674] [c716bb70] [00000004] 0x4 (unreliable) [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] [ 65.623168] [c716bbe0] [c0060ac8] xnintr_irq_handler+0x70/0x234 [ 65.629216] [c716bc20] [c005d15c] __ipipe_dispatch_irq+0x148/0x270 [ 65.635526] [c716bc40] [c000d66c] __ipipe_grab_irq+0x38/0x6c [ 65.641305] [c716bc50] [c000f710] __ipipe_ret_from_except+0x0/0xc [ 65.647541] --- Exception: 501 at __ipipe_restore_head+0x84/0xfc [ 65.647541] LR = __vfile_nklock_put+0x30/0x40 [ 65.658467] [c716bd10] [c0069bcc] vfile_stat_next+0x1b0/0x23c (unreliable) [ 65.665488] [c716bd20] [c0072f14] __vfile_nklock_put+0x30/0x40 [ 65.671443] [c716bd30] [c00738d0] vfile_snapshot_open+0x29c/0x31c [ 65.677667] [c716bd70] [c011b1c0] proc_reg_open+0x7c/0x11c [ 65.683272] [c716bd90] [c00d1a94] do_dentry_open.isra.18+0x1f4/0x29c [ 65.689758] [c716bdc0] [c00d2ad8] nameidata_to_filp+0x5c/0xa8 [ 65.695627] [c716bde0] [c00e11c0] do_last.isra.49+0x260/0x78c [ 65.701493] [c716be30] [c00e17ec] path_openat+0xb4/0x3b0 [ 65.706916] [c716be80] [c00e1bdc] do_filp_open+0x30/0x8c [ 65.712338] [c716bf10] [c00d2c20] do_sys_open+0xfc/0x1c8 [ 65.717764] [c716bf40] [c000ee0c] ret_from_syscall+0x0/0x38 [ 65.723452] --- Exception: c01 at 0xff0f47c [ 65.723452] LR = 0xffabb94 [ 65.730816] Instruction dump: [ 65.733838] XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX [ 65.741753] XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX XXXXXXXX ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-09 6:48 ` Stefan Roese @ 2012-10-09 9:47 ` Gilles Chanteperdrix 2012-10-09 10:18 ` Stefan Roese 2012-10-09 14:24 ` Philippe Gerum 1 sibling, 1 reply; 12+ messages in thread From: Gilles Chanteperdrix @ 2012-10-09 9:47 UTC (permalink / raw) To: Stefan Roese; +Cc: xenomai On 10/09/2012 08:48 AM, Stefan Roese wrote: > root@generic-powerpc:~# cat /proc/xenomai/stat > [ 65.215600] Oops: Kernel access of bad area, sig: 11 [#1] > [ 65.221118] mpc5200-simple-platform > [ 65.224671] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) > [ 65.230718] NIP: c0066914 LR: c009adf0 CTR: c009ada0 > [ 65.235784] REGS: c716bac0 TRAP: 0300 Tainted: G O (3.5.3-00253-g4699145-dirty) > [ 65.244386] MSR: 00001032 <ME,IR,DR,RI> CR: 24000488 XER: 20000000 > [ 65.250888] DAR: 00000000, DSISR: 22000000 > [ 65.255064] TASK = c705cba0[1400] 'cat' THREAD: c716a000 > GPR00: 00000000 c716bb70 c705cba0 c7b3060c 00000002 02000006 00000000 00000030 > GPR08: c70f0000 c7b30800 00000000 00000000 c009ada0 100a5a74 10017830 10006834 > GPR16: 10006770 10006774 100170f4 00010000 c00733d8 c75e4a60 00000000 c03ee068 > GPR24: 00000002 00000001 00000002 c03ec460 00000000 00000000 c7a6ccc4 c7a6ccb4 > [ 65.289337] NIP [c0066914] xnsynch_flush+0x64/0x100 Could you show us the diassembly of xnsynch_flush in the corresponding kernel? -- Gilles. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-09 9:47 ` Gilles Chanteperdrix @ 2012-10-09 10:18 ` Stefan Roese 0 siblings, 0 replies; 12+ messages in thread From: Stefan Roese @ 2012-10-09 10:18 UTC (permalink / raw) To: Gilles Chanteperdrix; +Cc: xenomai On 10/09/2012 11:47 AM, Gilles Chanteperdrix wrote: > On 10/09/2012 08:48 AM, Stefan Roese wrote: >> root@generic-powerpc:~# cat /proc/xenomai/stat >> [ 65.215600] Oops: Kernel access of bad area, sig: 11 [#1] >> [ 65.221118] mpc5200-simple-platform >> [ 65.224671] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) >> [ 65.230718] NIP: c0066914 LR: c009adf0 CTR: c009ada0 >> [ 65.235784] REGS: c716bac0 TRAP: 0300 Tainted: G O (3.5.3-00253-g4699145-dirty) >> [ 65.244386] MSR: 00001032 <ME,IR,DR,RI> CR: 24000488 XER: 20000000 >> [ 65.250888] DAR: 00000000, DSISR: 22000000 >> [ 65.255064] TASK = c705cba0[1400] 'cat' THREAD: c716a000 >> GPR00: 00000000 c716bb70 c705cba0 c7b3060c 00000002 02000006 00000000 00000030 >> GPR08: c70f0000 c7b30800 00000000 00000000 c009ada0 100a5a74 10017830 10006834 >> GPR16: 10006770 10006774 100170f4 00010000 c00733d8 c75e4a60 00000000 c03ee068 >> GPR24: 00000002 00000001 00000002 c03ec460 00000000 00000000 c7a6ccc4 c7a6ccb4 >> [ 65.289337] NIP [c0066914] xnsynch_flush+0x64/0x100 > > Could you show us the diassembly of xnsynch_flush in the corresponding > kernel? Sure. Here you go: Kernel 3.5.3 with ipipe (core-3.5 branch) from git.denx.de: c00668b0 <xnsynch_flush>: * * Rescheduling: never. */ int xnsynch_flush(struct xnsynch *synch, xnflags_t reason) { c00668b0: 94 21 ff d0 stwu r1,-48(r1) c00668b4: 7c 08 02 a6 mflr r0 c00668b8: bf 21 00 14 stmw r25,20(r1) c00668bc: 7c 7f 1b 78 mr r31,r3 c00668c0: 7c 9c 23 78 mr r28,r4 c00668c4: 90 01 00 34 stw r0,52(r1) static inline void hard_local_irq_disable_notrace(void) { #ifdef CONFIG_BOOKE __asm__ __volatile__("wrteei 0": : :"memory"); #else unsigned long msr = mfmsr(); c00668c8: 7c 00 00 a6 mfmsr r0 mtmsr(msr & ~MSR_EE); c00668cc: 54 00 04 5e rlwinm r0,r0,0,17,15 c00668d0: 7c 00 01 24 mtmsr r0 */ static inline int __test_and_set_bit(int nr, volatile unsigned long *addr) { unsigned long mask = BIT_MASK(nr); unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); unsigned long old = *p; c00668d4: 3f 60 c0 3f lis r27,-16321 return qslot->elems; } static inline int emptyq_p(xnqueue_t *qslot) { return qslot->head.next == &qslot->head; c00668d8: 7c 7e 1b 78 mr r30,r3 c00668dc: 3b 7b c4 60 addi r27,r27,-15264 xnlock_get_irqsave(&nklock, s); trace_mark(xn_nucleus, synch_flush, "synch %p reason %lu", synch, reason); status = emptypq_p(&synch->pendq) ? XNSYNCH_DONE : XNSYNCH_RESCHED; c00668e0: 3b 40 00 02 li r26,2 c00668e4: 83 3b 09 18 lwz r25,2328(r27) *p = old | mask; c00668e8: 63 20 00 01 ori r0,r25,1 return (old & mask) != 0; c00668ec: 57 39 07 fe clrlwi r25,r25,31 { unsigned long mask = BIT_MASK(nr); unsigned long *p = ((unsigned long *)addr) + BIT_WORD(nr); unsigned long old = *p; *p = old | mask; c00668f0: 90 1b 09 18 stw r0,2328(r27) c00668f4: 85 3e 00 10 lwzu r9,16(r30) c00668f8: 7f 89 f0 00 cmpw cr7,r9,r30 c00668fc: 41 9e 00 ac beq- cr7,c00669a8 <xnsynch_flush+0xf8> while ((holder = getpq(&synch->pendq)) != NULL) { struct xnthread *sleeper = link2thread(holder, plink); xnthread_set_info(sleeper, reason); sleeper->wchan = NULL; c0066900: 3b a0 00 00 li r29,0 c0066904: 48 00 00 44 b c0066948 <xnsynch_flush+0x98> } static inline xnholder_t *getq(xnqueue_t *qslot) { xnholder_t *holder = getheadq(qslot); if (holder) c0066908: 41 9a 00 54 beq- cr6,c006695c <xnsynch_flush+0xac> head->next = holder; } static inline void dth(xnholder_t *holder) { holder->last->next = holder->next; c006690c: 81 69 00 04 lwz r11,4(r9) c0066910: 80 09 00 00 lwz r0,0(r9) c0066914: 90 0b 00 00 stw r0,0(r11) holder->next->last = holder->last; c0066918: 81 69 00 00 lwz r11,0(r9) c006691c: 80 09 00 04 lwz r0,4(r9) c0066920: 90 0b 00 04 stw r0,4(r11) } static inline void removeq(xnqueue_t *qslot, xnholder_t *holder) { dth(holder); --qslot->elems; c0066924: 81 7f 00 18 lwz r11,24(r31) c0066928: 38 0b ff ff addi r0,r11,-1 c006692c: 90 1f 00 18 stw r0,24(r31) c0066930: 93 a9 00 20 stw r29,32(r9) status = emptypq_p(&synch->pendq) ? XNSYNCH_DONE : XNSYNCH_RESCHED; while ((holder = getpq(&synch->pendq)) != NULL) { struct xnthread *sleeper = link2thread(holder, plink); xnthread_set_info(sleeper, reason); c0066934: 80 09 ff d0 lwz r0,-48(r9) c0066938: 7c 00 e3 78 or r0,r0,r28 c006693c: 90 09 ff d0 stw r0,-48(r9) sleeper->wchan = NULL; xnpod_resume_thread(sleeper, XNPEND); c0066940: 4b ff b2 39 bl c0061b78 <xnpod_resume_thread> c0066944: 81 3f 00 10 lwz r9,16(r31) #endif /* XENO_DEBUG(QUEUES) */ static inline xnholder_t *getheadq(xnqueue_t *qslot) { xnholder_t *holder = qslot->head.next; return holder == &qslot->head ? NULL : holder; c0066948: 7f 9e 48 00 cmpw cr7,r30,r9 } static inline xnholder_t *getq(xnqueue_t *qslot) { xnholder_t *holder = getheadq(qslot); if (holder) c006694c: 2f 09 00 00 cmpwi cr6,r9,0 c0066950: 38 69 fe 0c addi r3,r9,-500 c0066954: 38 80 00 02 li r4,2 #endif /* XENO_DEBUG(QUEUES) */ static inline xnholder_t *getheadq(xnqueue_t *qslot) { xnholder_t *holder = qslot->head.next; return holder == &qslot->head ? NULL : holder; c0066958: 40 9e ff b0 bne+ cr7,c0066908 <xnsynch_flush+0x58> } if (testbits(synch->status, XNSYNCH_CLAIMED)) { c006695c: 80 1f 00 0c lwz r0,12(r31) c0066960: 70 09 00 10 andi. r9,r0,16 c0066964: 41 82 00 14 beq- c0066978 <xnsynch_flush+0xc8> xnsynch_clear_boost(synch, synch->owner); c0066968: 80 9f 00 1c lwz r4,28(r31) c006696c: 7f e3 fb 78 mr r3,r31 status = XNSYNCH_RESCHED; c0066970: 3b 40 00 02 li r26,2 sleeper->wchan = NULL; xnpod_resume_thread(sleeper, XNPEND); } if (testbits(synch->status, XNSYNCH_CLAIMED)) { xnsynch_clear_boost(synch, synch->owner); c0066974: 4b ff f7 05 bl c0066078 <xnsynch_clear_boost> * @nr: bit number to test * @addr: Address to start counting from */ static inline int test_bit(int nr, const volatile unsigned long *addr) { return 1UL & (addr[BIT_WORD(nr)] >> (nr & (BITS_PER_LONG-1))); c0066978: 80 1b 09 18 lwz r0,2328(r27) c006697c: 54 00 07 fe clrlwi r0,r0,31 void __ipipe_restore_head(unsigned long x); static inline void ipipe_restore_head(unsigned long x) { ipipe_check_irqoff(); if ((x ^ test_bit(IPIPE_STALL_FLAG, &__ipipe_head_status)) & 1) c0066980: 7f 80 c8 00 cmpw cr7,r0,r25 c0066984: 41 be 00 0c beq+ cr7,c0066990 <xnsynch_flush+0xe0> __ipipe_restore_head(x); c0066988: 7f 23 cb 78 mr r3,r25 c006698c: 4b ff 64 ad bl c005ce38 <__ipipe_restore_head> xnlock_put_irqrestore(&nklock, s); xnarch_post_graph_if(synch, 0, emptypq_p(&synch->pendq)); return status; } c0066990: 80 01 00 34 lwz r0,52(r1) c0066994: 7f 43 d3 78 mr r3,r26 c0066998: bb 21 00 14 lmw r25,20(r1) c006699c: 38 21 00 30 addi r1,r1,48 c00669a0: 7c 08 03 a6 mtlr r0 c00669a4: 4e 80 00 20 blr xnlock_get_irqsave(&nklock, s); trace_mark(xn_nucleus, synch_flush, "synch %p reason %lu", synch, reason); status = emptypq_p(&synch->pendq) ? XNSYNCH_DONE : XNSYNCH_RESCHED; c00669a8: 3b 40 00 00 li r26,0 c00669ac: 4b ff ff 54 b c0066900 <xnsynch_flush+0x50> c00669b0 <xnsynch_sleep_on>: * xnpod_init_thread), or nanoseconds otherwise. */ xnflags_t xnsynch_sleep_on(struct xnsynch *synch, xnticks_t timeout, xntmode_t timeout_mode) ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-09 6:48 ` Stefan Roese 2012-10-09 9:47 ` Gilles Chanteperdrix @ 2012-10-09 14:24 ` Philippe Gerum 2012-10-09 15:44 ` Stefan Roese 1 sibling, 1 reply; 12+ messages in thread From: Philippe Gerum @ 2012-10-09 14:24 UTC (permalink / raw) To: Stefan Roese; +Cc: xenomai On 10/09/2012 08:48 AM, Stefan Roese wrote: > [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 > [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] > [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] Is any list corruption detected when CONFIG_XENO_OPT_DEBUG_QUEUES is enabled? -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-09 14:24 ` Philippe Gerum @ 2012-10-09 15:44 ` Stefan Roese 2012-10-11 11:56 ` Stefan Roese 0 siblings, 1 reply; 12+ messages in thread From: Stefan Roese @ 2012-10-09 15:44 UTC (permalink / raw) To: Philippe Gerum; +Cc: xenomai On 10/09/2012 04:24 PM, Philippe Gerum wrote: > On 10/09/2012 08:48 AM, Stefan Roese wrote: > >> [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 >> [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] >> [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] > > Is any list corruption detected when CONFIG_XENO_OPT_DEBUG_QUEUES is > enabled? No, I don't see any list corruptions with CONFIG_XENO_OPT_DEBUG_QUEUES enabled. But the crash log is different now. At least part of the "stat" is printed. And the crash happens now in xnintr_irq_handler(). This different crash seems to result from a change of the kernel config (I enabled/disabled some other drivers as well in the meantime). I'm debugging now, how different kernel configurations result in different crash scenarios. Thanks for your help so far. Cheers, Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-09 15:44 ` Stefan Roese @ 2012-10-11 11:56 ` Stefan Roese 2012-10-11 12:40 ` Philippe Gerum 0 siblings, 1 reply; 12+ messages in thread From: Stefan Roese @ 2012-10-11 11:56 UTC (permalink / raw) To: Philippe Gerum; +Cc: xenomai On 10/09/2012 05:44 PM, Stefan Roese wrote: > On 10/09/2012 04:24 PM, Philippe Gerum wrote: >> On 10/09/2012 08:48 AM, Stefan Roese wrote: >> >>> [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 >>> [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] >>> [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] >> >> Is any list corruption detected when CONFIG_XENO_OPT_DEBUG_QUEUES is >> enabled? > > No, I don't see any list corruptions with CONFIG_XENO_OPT_DEBUG_QUEUES > enabled. But the crash log is different now. At least part of the > "stat" is printed. And the crash happens now in xnintr_irq_handler(). > This different crash seems to result from a change of the kernel > config (I enabled/disabled some other drivers as well in the meantime). > I'm debugging now, how different kernel configurations result in different > crash scenarios. I now strapped down my device driver to the absolute minimum. "cat /proc/xenomai/stat" still does crash. But not all the time, and not always with the same output. Very strange is the "0x100100" in the output below. This is included in many of the crash reports. Does this ring a bell? (latest git xenomai-2.6 with latest core-3.5 ipipe on mpc5200) root@generic-powerpc:~# cat /proc/xenomai/stat CPU PID MSW CSW PF STAT %CPU NAME 0 0 0 1458 0 00500080 99.2 ROOT 0 1401 1 1458 0 00300186 0.1 fpga-loop 0 0 0 1456 0 00000000 0.6 IRQ16: rt_fpga 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx 0 0 0 10306 0 00000000 0.1 IRQ512: [timer] root@generic-powerpc:~# cat /proc/xenomai/stat CPU PID MSW CSW PF STAT %CPU NAME 0 0 0 1546 0 00500080 95.9 ROOT 0 1401 1 1546 0 00300186 0.5 fpga-loop 0 0 0 1544 0 00000000 3.4 IRQ16: rt_fpga 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx 0 0 0 10542 0 00000000 0.2 IRQ512: [timer] root@generic-powerpc:~# cat /proc/xenomai/stat CPU PID MSW CSW PF STAT %CPU NAME 0 0 0 1655 0 00500080 95.9 ROOT 0 1401 1 1655 0 00300186 0.5 fpga-loop 0 0 0 1653 0 00000000 3.4 IRQ16: rt_fpga 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx 0 0 0 10833 0 00000000 0.2 IRQ512: [timer] root@generic-powerpc:~# cat /proc/xenomai/stat CPU PID MSW CSW PF STAT %CPU NAME [ 45.161337] Unable to handle kernel paging request for data at address 0x7ffffff7 [ 45.174889] Faulting instruction address: 0xc015ea7c [ 45.180048] Oops: Kernel access of bad area, sig: 11 [#1] [ 45.186022] mpc5200-simple-platform [ 45.189595] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) [ 45.196088] NIP: c015ea7c LR: c00b42a0 CTR: c00b428c [ 45.201171] REGS: c709ddb0 TRAP: 0300 Tainted: G O (3.5.3-00254-g0a88116-dirty) [ 45.210190] MSR: 00009032 <EE,ME,IR,DR,RI> CR: 48004424 XER: 00000000 [ 45.217416] DAR: 7ffffff7, DSISR: 20000000 [ 45.221608] TASK = c70c1740[1409] 'cat' THREAD: c709c000 GPR00: ffffffbf c709de60 c70c1740 aa883045 aa883045 c7b1f408 00000001 00000000 GPR08: 000048dd 00008000 c734cec0 00000001 28004424 100a5a74 10017830 10006834 GPR16: 10006770 10006774 100170f4 00000000 bfc331e0 100a1008 00000000 1007d050 GPR24: bfc65f29 00000027 c7b1f408 00000000 c00b4318 7ffffff7 aa883045 c00b42a0 [ 45.257605] NIP [c015ea7c] kfree+0x40/0x150 [ 45.261896] LR [c00b42a0] vfile_snapshot_free+0x14/0x24 [ 45.267628] Call Trace: [ 45.270153] [c709de60] [00100100] 0x100100 (unreliable) [ 45.275506] [c709de90] [c00b42a0] vfile_snapshot_free+0x14/0x24 [ 45.281965] [c709dea0] [c00b4378] vfile_snapshot_release+0x60/0x88 [ 45.288294] [c709dec0] [c01aca10] proc_reg_release+0xd4/0x170 [ 45.294591] [c709def0] [c0166548] fput+0xbc/0x238 [ 45.299835] [c709df10] [c0162e10] filp_close+0x78/0xa4 [ 45.305092] [c709df30] [c0162ed8] sys_close+0x9c/0xd8 [ 45.310689] [c709df40] [c000ee0c] ret_from_syscall+0x0/0x38 [ 45.316392] --- Exception: c01 at 0xff0fb20 [ 45.316392] LR = 0x1004dc44 [ 45.324149] Instruction dump: [ 45.327195] 7fe802a6 7c7e1b78 90010034 409d00a8 3d20c04a 3fa34000 81294254 57bdc9f4 [ 45.335552] 7c09e82e 7fa9ea14 70098000 4082010c <801d0000> 700b0080 418200e0 3f60c048 [ 45.344089] ---[ end trace a562bf2537c91283 ]--- [ 45.348807] Segmentation fault Thanks, Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-11 11:56 ` Stefan Roese @ 2012-10-11 12:40 ` Philippe Gerum 2012-10-11 12:42 ` Philippe Gerum 2012-10-11 12:53 ` Stefan Roese 0 siblings, 2 replies; 12+ messages in thread From: Philippe Gerum @ 2012-10-11 12:40 UTC (permalink / raw) To: Stefan Roese; +Cc: xenomai On 10/11/2012 01:56 PM, Stefan Roese wrote: > On 10/09/2012 05:44 PM, Stefan Roese wrote: >> On 10/09/2012 04:24 PM, Philippe Gerum wrote: >>> On 10/09/2012 08:48 AM, Stefan Roese wrote: >>> >>>> [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 >>>> [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] >>>> [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] >>> >>> Is any list corruption detected when CONFIG_XENO_OPT_DEBUG_QUEUES is >>> enabled? >> >> No, I don't see any list corruptions with CONFIG_XENO_OPT_DEBUG_QUEUES >> enabled. But the crash log is different now. At least part of the >> "stat" is printed. And the crash happens now in xnintr_irq_handler(). >> This different crash seems to result from a change of the kernel >> config (I enabled/disabled some other drivers as well in the meantime). >> I'm debugging now, how different kernel configurations result in different >> crash scenarios. > > I now strapped down my device driver to the absolute minimum. > "cat /proc/xenomai/stat" still does crash. But not all the time, and not > always with the same output. Very strange is the "0x100100" in the > output below. This is included in many of the crash reports. Does this > ring a bell? #define LIST_POISON1 ((void *) 0x00100100 + POISON_POINTER_DELTA) Somebody might be doing bad things with memory it does not own anymore? -- Philippe. ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-11 12:40 ` Philippe Gerum @ 2012-10-11 12:42 ` Philippe Gerum 2012-10-11 13:07 ` Stefan Roese 2012-10-11 12:53 ` Stefan Roese 1 sibling, 1 reply; 12+ messages in thread From: Philippe Gerum @ 2012-10-11 12:42 UTC (permalink / raw) To: Stefan Roese; +Cc: xenomai On 10/11/2012 02:40 PM, Philippe Gerum wrote: > On 10/11/2012 01:56 PM, Stefan Roese wrote: >> On 10/09/2012 05:44 PM, Stefan Roese wrote: >>> On 10/09/2012 04:24 PM, Philippe Gerum wrote: >>>> On 10/09/2012 08:48 AM, Stefan Roese wrote: >>>> >>>>> [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 >>>>> [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] >>>>> [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] >>>> >>>> Is any list corruption detected when CONFIG_XENO_OPT_DEBUG_QUEUES is >>>> enabled? >>> >>> No, I don't see any list corruptions with CONFIG_XENO_OPT_DEBUG_QUEUES >>> enabled. But the crash log is different now. At least part of the >>> "stat" is printed. And the crash happens now in xnintr_irq_handler(). >>> This different crash seems to result from a change of the kernel >>> config (I enabled/disabled some other drivers as well in the meantime). >>> I'm debugging now, how different kernel configurations result in different >>> crash scenarios. >> >> I now strapped down my device driver to the absolute minimum. >> "cat /proc/xenomai/stat" still does crash. But not all the time, and not >> always with the same output. Very strange is the "0x100100" in the >> output below. This is included in many of the crash reports. Does this >> ring a bell? > > #define LIST_POISON1 ((void *) 0x00100100 + POISON_POINTER_DELTA) > > Somebody might be doing bad things with memory it does not own anymore? > Just to rule this out, could you try raising the kernel thread stack size this way? diff --git a/include/asm-powerpc/system.h b/include/asm-powerpc/system.h index 53fd59d..e8dd57c 100644 --- a/include/asm-powerpc/system.h +++ b/include/asm-powerpc/system.h @@ -31,7 +31,7 @@ #ifdef CONFIG_PPC64 #define XNARCH_THREAD_STACKSZ 8182 #else -#define XNARCH_THREAD_STACKSZ 4096 +#define XNARCH_THREAD_STACKSZ 8192 #endif #define xnarch_stack_size(tcb) ((tcb)->stacksize) -- Philippe. ^ permalink raw reply related [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-11 12:42 ` Philippe Gerum @ 2012-10-11 13:07 ` Stefan Roese 0 siblings, 0 replies; 12+ messages in thread From: Stefan Roese @ 2012-10-11 13:07 UTC (permalink / raw) To: Philippe Gerum; +Cc: xenomai On 10/11/2012 02:42 PM, Philippe Gerum wrote: >>> I now strapped down my device driver to the absolute minimum. >>> "cat /proc/xenomai/stat" still does crash. But not all the time, and not >>> always with the same output. Very strange is the "0x100100" in the >>> output below. This is included in many of the crash reports. Does this >>> ring a bell? >> >> #define LIST_POISON1 ((void *) 0x00100100 + POISON_POINTER_DELTA) >> >> Somebody might be doing bad things with memory it does not own anymore? >> > > Just to rule this out, could you try raising the kernel thread stack > size this way? > > diff --git a/include/asm-powerpc/system.h b/include/asm-powerpc/system.h > index 53fd59d..e8dd57c 100644 > --- a/include/asm-powerpc/system.h > +++ b/include/asm-powerpc/system.h > @@ -31,7 +31,7 @@ > #ifdef CONFIG_PPC64 > #define XNARCH_THREAD_STACKSZ 8182 > #else > -#define XNARCH_THREAD_STACKSZ 4096 > +#define XNARCH_THREAD_STACKSZ 8192 > #endif > > #define xnarch_stack_size(tcb) ((tcb)->stacksize) No, it doesn't fix the problem. But now the 0x100100 doesn't show in the crash dump any more. 3 times without it. Here one log: root@generic-powerpc:~# cat /proc/xenomai/stat CPU PID MSW CSW PF STAT %CPU NAME 0 0 0 1295 0 00500080 95.8 ROOT 0 1402 1 1295 0 00300186 0.6 fpga-loop 0 0 0 1293 0 00000000 3.4 IRQ16: rt_fpga 0 0 0 0 0 00000000 0.0 IRQ151: mpc52xx-lpbfifo 0 0 0 0 0 00000000 0.0 IRQ194: mpc52xx-lpbfifo-rx 0 0 0 12684 0 00000000 0.2 IRQ512: [timer] root@generic-powerpc:~# cat /proc/xenomai/stat [ 51.448640] Unable to handle kernel paging request for data at address 0x81d11f92 [ 51.456435] Faulting instruction address: 0xc008ae48 [ 51.462066] Oops: Kernel access of bad area, sig: 11 [#1] [ 51.467600] mpc5200-simple-platform [ 51.471543] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) [ 51.477621] NIP: c008ae48 LR: c00b37ac CTR: c008ae2c [ 51.483104] REGS: c7081d90 TRAP: 0300 Tainted: G O (3.5.3-00254-g0a88116-dirty) [ 51.492161] MSR: 00009032 <EE,ME,IR,DR,RI> CR: 88422488 XER: 00000000 [ 51.498958] DAR: 81d11f92, DSISR: 20000000 [ 51.503550] TASK = c724f360[1412] 'cat' THREAD: c7080000 GPR00: c00b37ac c7081e40 c724f360 c7a7cb40 81d11f52 c7081e98 00000000 00000001 GPR08: 00000000 c0484e38 c0484dd0 00000000 81d11f52 100a5a74 10017830 10006834 GPR16: 10006770 10006774 81d11f52 00000000 fffff000 0000003e fffff000 00000000 GPR24: 00000000 c726ada8 00000000 c7afae00 bfb66808 00001000 c7081f20 c7a7cb40 [ 51.539159] NIP [c008ae48] vfile_stat_show+0x1c/0x1a8 [ 51.544728] LR [c00b37ac] vfile_snapshot_show+0x2c/0x60 [ 51.550070] Call Trace: [ 51.552576] [c7081e40] [c008afb8] vfile_stat_show+0x18c/0x1a8 (unreliable) [ 51.560020] [c7081e80] [c00b37ac] vfile_snapshot_show+0x2c/0x60 [ 51.566520] [c7081e90] [c01838e0] seq_read+0x3b0/0x5a0 [ 51.571784] [c7081ee0] [c01accb8] proc_reg_read+0x4c/0x70 [ 51.577718] [c7081ef0] [c0165148] vfs_read+0xa8/0x184 [ 51.582888] [c7081f10] [c0165270] sys_read+0x4c/0x8c [ 51.588383] [c7081f40] [c000ee0c] ret_from_syscall+0x0/0x38 [ 51.594086] --- Exception: c01 at 0xff0fb94 [ 51.594086] LR = 0x100064f4 [ 51.601934] Instruction dump: [ 51.604989] 807f05b8 7c0803a6 bbc10008 38210010 4e800020 7c8c2379 9421ffc0 7c0802a6 [ 51.613342] 93e1003c 7c7f1b78 90010044 4182011c <812c0040> 39600000 808c0044 7d202379 [ 51.621877] ---[ end trace b1a0c2afe6b0e729 ]--- [ 51.631010] ------------[ cut here ]------------ [ 51.635846] kernel BUG at mm/slub.c:3474! [ 51.640470] Oops: Exception in kernel mode, sig: 5 [#2] [ 51.645817] mpc5200-simple-platform [ 51.649376] Modules linked in: rt_fpga(O) rt_mpc52xx_lpbfifo(O) [ 51.655441] NIP: c015eb74 LR: c00b42a0 CTR: c00b428c [ 51.660517] REGS: c7081b50 TRAP: 0700 Tainted: G D O (3.5.3-00254-g0a88116-dirty) [ 51.669134] MSR: 00029032 <EE,ME,IR,DR,RI> CR: 24424424 XER: 00000000 [ 51.675921] TASK = c724f360[1412] 'cat' THREAD: c7080000 GPR00: 00000001 c7081c00 c724f360 822c6d2e 822c6d2e c7afae08 00000001 00000000 GPR08: 00004962 00000000 c700fee0 00000000 24424424 100a5a74 10017830 10006834 GPR16: 10006770 10006774 81d11f52 00000000 fffff000 0000003e fffff000 00000000 GPR24: 00000000 c726ada8 c7afae08 00000000 c00b4318 c1d2b8c0 822c6d2e c00b42a0 [ 51.710239] NIP [c015eb74] kfree+0x138/0x150 [ 51.714613] LR [c00b42a0] vfile_snapshot_free+0x14/0x24 [ 51.719944] Call Trace: [ 51.722443] [c7081c00] [c70b9040] 0xc70b9040 (unreliable) [ 51.727966] [c7081c30] [c00b42a0] vfile_snapshot_free+0x14/0x24 [ 51.734018] [c7081c40] [c00b4378] vfile_snapshot_release+0x60/0x88 [ 51.740341] [c7081c60] [c01aca10] proc_reg_release+0xd4/0x170 [ 51.746224] [c7081c90] [c0166548] fput+0xbc/0x238 [ 51.751034] [c7081cb0] [c0162e10] filp_close+0x78/0xa4 [ 51.756290] [c7081cd0] [c001e3c0] put_files_struct+0xdc/0xf8 [ 51.762075] [c7081cf0] [c001e508] do_exit+0x100/0x6c0 [ 51.767240] [c7081d40] [c000aa14] die+0x198/0x240 [ 51.772063] [c7081d70] [c0010360] bad_page_fault+0xb4/0xfc [ 51.777674] [c7081d80] [c000f2c4] handle_page_fault+0x7c/0x80 [ 51.783574] --- Exception: 300 at vfile_stat_show+0x1c/0x1a8 [ 51.783574] LR = vfile_snapshot_show+0x2c/0x60 [ 51.794245] [c7081e40] [c008afb8] vfile_stat_show+0x18c/0x1a8 (unreliable) [ 51.801276] [c7081e80] [c00b37ac] vfile_snapshot_show+0x2c/0x60 [ 51.807336] [c7081e90] [c01838e0] seq_read+0x3b0/0x5a0 [ 51.812592] [c7081ee0] [c01accb8] proc_reg_read+0x4c/0x70 [ 51.818116] [c7081ef0] [c0165148] vfs_read+0xa8/0x184 [ 51.823281] [c7081f10] [c0165270] sys_read+0x4c/0x8c [ 51.828360] [c7081f40] [c000ee0c] ret_from_syscall+0x0/0x38 [ 51.834057] --- Exception: c01 at 0xff0fb94 [ 51.834057] LR = 0x100064f4 [ 51.841519] Instruction dump: [ 51.844547] 7f83e378 7fa4eb78 7fc5f378 7fe6fb78 bb210014 7c0803a6 38210030 4824bc70 [ 51.852483] 801d0000 7009c000 7c000026 54001ffe <0f000000> 7fa3eb78 4bfdb9e5 4bffff80 [ 51.861125] ---[ end trace b1a0c2afe6b0e72a ]--- [ 51.866292] [ 51.868206] Fixing recursive fault but reboot is needed! Any further ideas? Thanks, Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
* Re: [Xenomai] Oops while running "cat /proc/xenomai/stat" 2012-10-11 12:40 ` Philippe Gerum 2012-10-11 12:42 ` Philippe Gerum @ 2012-10-11 12:53 ` Stefan Roese 1 sibling, 0 replies; 12+ messages in thread From: Stefan Roese @ 2012-10-11 12:53 UTC (permalink / raw) To: Philippe Gerum; +Cc: xenomai On 10/11/2012 02:40 PM, Philippe Gerum wrote: > On 10/11/2012 01:56 PM, Stefan Roese wrote: >> On 10/09/2012 05:44 PM, Stefan Roese wrote: >>> On 10/09/2012 04:24 PM, Philippe Gerum wrote: >>>> On 10/09/2012 08:48 AM, Stefan Roese wrote: >>>> >>>>> [ 65.601569] [c716bba0] [c009adf0] rtdm_event_signal+0x50/0xe4 >>>>> [ 65.607440] [c716bbc0] [cb132588] fpga_dma_done_callback+0x18/0x28 [rt_fpga] >>>>> [ 65.614641] [c716bbd0] [cb101114] mpc52xx_lpbfifo_bcom_irq+0x114/0x1c4 [rt_mpc52xx_lpbfifo] >>>> >>>> Is any list corruption detected when CONFIG_XENO_OPT_DEBUG_QUEUES is >>>> enabled? >>> >>> No, I don't see any list corruptions with CONFIG_XENO_OPT_DEBUG_QUEUES >>> enabled. But the crash log is different now. At least part of the >>> "stat" is printed. And the crash happens now in xnintr_irq_handler(). >>> This different crash seems to result from a change of the kernel >>> config (I enabled/disabled some other drivers as well in the meantime). >>> I'm debugging now, how different kernel configurations result in different >>> crash scenarios. >> >> I now strapped down my device driver to the absolute minimum. >> "cat /proc/xenomai/stat" still does crash. But not all the time, and not >> always with the same output. Very strange is the "0x100100" in the >> output below. This is included in many of the crash reports. Does this >> ring a bell? > > #define LIST_POISON1 ((void *) 0x00100100 + POISON_POINTER_DELTA) > > Somebody might be doing bad things with memory it does not own anymore? Yep. That's it. Changing it some other value shows the new changed value. I'm not doing anything with lists though in my code (AFAIR). I'll check the increased stack next and let you know. Thanks, Stefan ^ permalink raw reply [flat|nested] 12+ messages in thread
end of thread, other threads:[~2012-10-11 13:07 UTC | newest] Thread overview: 12+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2012-10-08 9:32 [Xenomai] Oops while running "cat /proc/xenomai/stat" Stefan Roese 2012-10-08 17:39 ` Gilles Chanteperdrix 2012-10-09 6:48 ` Stefan Roese 2012-10-09 9:47 ` Gilles Chanteperdrix 2012-10-09 10:18 ` Stefan Roese 2012-10-09 14:24 ` Philippe Gerum 2012-10-09 15:44 ` Stefan Roese 2012-10-11 11:56 ` Stefan Roese 2012-10-11 12:40 ` Philippe Gerum 2012-10-11 12:42 ` Philippe Gerum 2012-10-11 13:07 ` Stefan Roese 2012-10-11 12:53 ` Stefan Roese
This is an external index of several public inboxes, see mirroring instructions on how to clone and mirror all data and code used by this external index.