From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: (majordomo@vger.kernel.org) by vger.kernel.org via listexpand id S261630AbVFXWh5 (ORCPT ); Fri, 24 Jun 2005 18:37:57 -0400 Received: (majordomo@vger.kernel.org) by vger.kernel.org id S263291AbVFXWf5 (ORCPT ); Fri, 24 Jun 2005 18:35:57 -0400 Received: from unused.mind.net ([69.9.134.98]:9189 "EHLO echo.lysdexia.org") by vger.kernel.org with ESMTP id S261630AbVFXWeC (ORCPT ); Fri, 24 Jun 2005 18:34:02 -0400 Date: Fri, 24 Jun 2005 15:31:54 -0700 (PDT) From: William Weston X-X-Sender: weston@echo.lysdexia.org To: Ingo Molnar cc: "K.R. Foley" , linux-kernel@vger.kernel.org, "Eugeny S. Mints" , Daniel Walker Subject: Re: [patch] Real-Time Preemption, -RT-2.6.12-rc6-V0.7.48-00 In-Reply-To: <20050624070639.GB5941@elte.hu> Message-ID: References: <20050621131249.GB22691@elte.hu> <20050622082450.GA19957@elte.hu> <20050622220007.GA28258@elte.hu> <20050623001023.GC11486@elte.hu> <20050624070639.GB5941@elte.hu> MIME-Version: 1.0 Content-Type: TEXT/PLAIN; charset=US-ASCII Sender: linux-kernel-owner@vger.kernel.org X-Mailing-List: linux-kernel@vger.kernel.org On Fri, 24 Jun 2005, Ingo Molnar wrote: > do you have the NMI watchdog enabled? Find below > serial-logging-earlyprintk-nmi.txt. I had a serial console up, but not NMI watchdog until now. Here's some NMI watchdog traces from both -50-17 and -50-18: ********************************** *** 2.6.12-RT-V0.7.50-17 *** ********************************** NMI watchdog detected lockup on CPU#1 (50000/50000) Pid: 2899, comm: lockupcli EIP: 0073:[<08048392>] CPU: 1 EIP is at 0x8048392 ESP: 007b:bfebb440 EFLAGS: 00003082 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 452a3ff4 ECX: bfebb4ec EDX: 452a3ff4 ESI: bfebb4e4 EDI: bfebb470 EBP: bfebb458 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 139ac000 CR4: 000006d0 NMI show regs on CPU#0: NMI Watchdog detected LOCKUP on CPU1, eip 08048392, registers: Modules linked in: eeprom lm85 i2c_sensor i2c_isa i2c_i801 i2c_dev i2c_core realtime microcode snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc ns83820 CPU: 1 EIP: 0073:[<08048392>] Not tainted VLI EFLAGS: 00003082 (2.6.12-RT-V0.7.50-17) EIP is at 0x8048392 eax: 00000000 ebx: 452a3ff4 ecx: bfebb4ec edx: 452a3ff4 esi: bfebb4e4 edi: bfebb470 ebp: bfebb458 esp: bfebb440 ds: 007b es: 007b ss: 007b preempt: 00010001 Process lockupcli (pid: 2899, threadinfo=d7060000 task=de5126d0) console shuts up ... Pid: 0, comm: swapper EIP: 0060:[] CPU: 0 EIP is at default_idle+0x2a/0x33 EFLAGS: 00000246 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: c0414000 ECX: c13e72e0 EDX: c0414000 ESI: c0414000 EDI: c0443380 EBP: c0443300 DS: 007b ES: 007b CR0: 8005003b CR2: 080dcd9c CR3: 1e0e5000 CR4: 000006d0 [] cpu_idle+0x76/0xaf (4) [] start_kernel+0x195/0x1fb (36) [] unknown_bootoption+0x0/0x1d8 (20) ( IRQ 4-304 |#0): new 3 us maximum-latency wakeup. ( gdmgreeter-2858 |#1): new 4 us maximum-latency wakeup. ( X-2829 |#0): new 10 us maximum-latency wakeup. ( softirq-timer/0-4 |#0): new 11 us maximum-latency wakeup. BUG: soft lockup detected on CPU#1! [] softlockup_tick+0xa3/0xb4 (8) [] smp_apic_timer_interrupt+0xbd/0xc9 (28) [] apic_timer_interrupt+0x1c/0x24 (24) [] smp_call_function+0x92/0x11f (44) [] mce_checkregs+0x0/0xa3 (36) [] up_mutex+0x2b/0x70 (28) [] mce_work_fn+0x33/0x6d (12) [] mce_checkregs+0x0/0xa3 (4) [] worker_thread+0x1e0/0x289 (20) [] mce_work_fn+0x0/0x6d (28) [] default_wake_function+0x0/0x22 (28) [] default_wake_function+0x0/0x22 (32) [] kthread+0x7c/0xcd (16) [] worker_thread+0x0/0x289 (20) [] kthread+0xa3/0xcd (4) [] kthread+0x0/0xcd (28) [] kernel_thread_helper+0x5/0xb (16) NMI show regs on CPU#1: Pid: 21, comm: events/1 EIP: 0060:[] CPU: 1 EIP is at nmi_show_all_regs+0x8d/0x104 EFLAGS: 00000046 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: c044b838 ECX: 00000000 EDX: 00000000 ESI: 00028917 EDI: 00000000 EBP: c14bfeb0 DS: 007b ES: 007b CR0: 8005003b CR2: 451721c0 CR3: 1e0e5000 CR4: 000006d0 [] softlockup_tick+0xa8/0xb4 (24) [] smp_apic_timer_interrupt+0xbd/0xc9 (28) [] apic_timer_interrupt+0x1c/0x24 (24) [] smp_call_function+0x92/0x11f (44) [] mce_checkregs+0x0/0xa3 (36) [] up_mutex+0x2b/0x70 (28) [] mce_work_fn+0x33/0x6d (12) [] mce_checkregs+0x0/0xa3 (4) [] worker_thread+0x1e0/0x289 (20) [] mce_work_fn+0x0/0x6d (28) [] default_wake_function+0x0/0x22 (28) [] default_wake_function+0x0/0x22 (32) [] kthread+0x7c/0xcd (16) [] worker_thread+0x0/0x289 (20) [] kthread+0xa3/0xcd (4) [] kthread+0x0/0xcd (28) [] kernel_thread_helper+0x5/0xb (16) NMI show regs on CPU#0: Pid: 2940, comm: lockupcli EIP: 0073:[<08048392>] CPU: 0 EIP is at 0x8048392 ESP: 007b:bfa21af0 EFLAGS: 00003086 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 452a3ff4 ECX: bfa21b9c EDX: 452a3ff4 ESI: bfa21b94 EDI: bfa21b20 EBP: bfa21b08 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 1397d000 CR4: 000006d0 NMI watchdog detected lockup on CPU#0 (50000/50000) Pid: 2940, comm: lockupcli EIP: 0073:[<08048392>] CPU: 0 EIP is at 0x8048392 ESP: 007b:bfa21af0 EFLAGS: 00003086 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 452a3ff4 ECX: bfa21b9c EDX: 452a3ff4 ESI: bfa21b94 EDI: bfa21b20 EBP: bfa21b08 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 1397d000 CR4: 000006d0 NMI show regs on CPU#1: NMI Watchdog detected LOCKUP on CPU0, eip 08048392, registers: Modules linked in: eeprom lm85 i2c_sensor i2c_isa i2c_i801 i2c_dev i2c_core realtime microcode snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc ns83820 CPU: 0 EIP: 0073:[<08048392>] Not tainted VLI EFLAGS: 00003086 (2.6.12-RT-V0.7.50-17) EIP is at 0x8048392 Pid: 21, comm: events/1 EIP: 0060:[] CPU: 1 eax: 00000000 ebx: 452a3ff4 ecx: bfa21b9c edx: 452a3ff4 EIP is at smp_call_function+0x92/0x11f EFLAGS: 00000297 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 00000001 ECX: 00000000 EDX: 000c08fb ESI: 00000001 EDI: c033fbe4 EBP: 00000000 DS: 007b ES: 007b CR0: 8005003b CR2: 451721c0 CR3: 1e0e5000 CR4: 000006d0 esi: bfa21b94 edi: bfa21b20 ebp: bfa21b08 esp: bfa21af0 []ds: 007b es: 007b ss: 007b preempt: 00010001 mce_checkregs+0x0/0xa3 (28) Process lockupcli (pid: 2940, threadinfo=d3d62000 task=dec5a650) [] up_mutex+0x2b/0x70 (28) []console shuts up ... mce_work_fn+0x33/0x6d (12) [] mce_checkregs+0x0/0xa3 (4) [] worker_thread+0x1e0/0x289 (20) [] mce_work_fn+0x0/0x6d (28) [] default_wake_function+0x0/0x22 (28) [] default_wake_function+0x0/0x22 (32) [] kthread+0x7c/0xcd (16) [] worker_thread+0x0/0x289 (20) [] kthread+0xa3/0xcd (4) [] kthread+0x0/0xcd (28) [] kernel_thread_helper+0x5/0xb (16) ( watchdog/1-19 |#1): new 46581633 us maximum-latency wakeup. BUG: soft lockup detected on CPU#0! [] softlockup_tick+0xa3/0xb4 (8) [] smp_apic_timer_interrupt+0xbd/0xc9 (28) [] apic_timer_interrupt+0x1c/0x24 (24) [] schedule+0x54/0x13a (44) [] kjournald+0x3a2/0x3ab (20) [] autoremove_wake_function+0x0/0x4b (40) [] autoremove_wake_function+0x0/0x4b (32) [] ret_from_fork+0x6/0x14 (20) [] commit_timeout+0x0/0x9 (20) [] kjournald+0x0/0x3ab (16) [] kernel_thread_helper+0x5/0xb (16) NMI show regs on CPU#1: Pid: 13, comm: softirq-timer/1 EIP: 0060:[] CPU: 1 EIP is at _raw_spin_lock+0x15/0x60 EFLAGS: 00000046 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000001 EBX: c0408f04 ECX: 00000000 EDX: 00000000 ESI: c1498000 EDI: 00001c80 EBP: c13f87e0 DS: 007b ES: 007b CR0: 8005003b CR2: 451721c0 CR3: 1e0e5000 CR4: 000006d0 [] run_timer_softirq+0x7d/0x392 (12) [] tcpdiag_rcv+0x21/0x176 (40) [] ksoftirqd+0xf4/0x189 (20) [] ksoftirqd+0x0/0x189 (40) [] kthread+0xa3/0xcd (4) [] kthread+0x0/0xcd (28) [] kernel_thread_helper+0x5/0xb (16) NMI show regs on CPU#0: Pid: 1274, comm: kjournald EIP: 0060:[] CPU: 0 EIP is at nmi_show_all_regs+0x95/0x104 EFLAGS: 00000046 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: c044b838 ECX: 00000000 EDX: 00000000 ESI: 00027d5c EDI: 00000000 EBP: dd6adf1c DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 1e2ef000 CR4: 000006d0 [] softlockup_tick+0xa8/0xb4 (24) [] smp_apic_timer_interrupt+0xbd/0xc9 (28) [] apic_timer_interrupt+0x1c/0x24 (24) [] schedule+0x54/0x13a (44) [] kjournald+0x3a2/0x3ab (20) [] autoremove_wake_function+0x0/0x4b (40) [] autoremove_wake_function+0x0/0x4b (32) [] ret_from_fork+0x6/0x14 (20) [] commit_timeout+0x0/0x9 (20) [] kjournald+0x0/0x3ab (16) [] kernel_thread_helper+0x5/0xb (16) NMI watchdog detected lockup on CPU#0 (50000/50000) Pid: 2941, comm: lockupcli EIP: 0073:[<08048392>] CPU: 0 EIP is at 0x8048392 ESP: 007b:bfb44640 EFLAGS: 00003082 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 452a3ff4 ECX: bfb446ec EDX: 452a3ff4 ESI: bfb446e4 EDI: bfb44670 EBP: bfb44658 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 16741000 CR4: 000006d0 NMI show regs on CPU#1: NMI Watchdog detected LOCKUP on CPU0, eip 08048392, registers: Modules linked in: eeprom lm85 i2c_sensor i2c_isa i2c_i801 i2c_dev i2c_core realtime microcode snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc ns83820 CPU: 0 EIP: 0073:[<08048392>] Not tainted VLI EFLAGS: 00003082 (2.6.12-RT-V0.7.50-17) EIP is at 0x8048392 Pid: 0, comm: swapper EIP: 0060:[] CPU: 1 eax: 00000000 ebx: 452a3ff4 ecx: bfb446ec edx: 452a3ff4 esi: bfb446e4 edi: bfb44670 ebp: bfb44658 esp: bfb44640 EIP is at default_idle+0x2a/0x33 EFLAGS: 00000246 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: def12000 ECX: c13f72e0 EDX: def12000 ESI: def12000 EDI: c0443380 EBP: c0443300 DS: 007b ES: 007b CR0: 8005003b CR2: 080ddcfc CR3: 1e0e5000 CR4: 000006d0 []ds: 007b es: 007b ss: 007b preempt: 00010001 cpu_idle+0x76/0xaf (4) Process lockupcli (pid: 2941, threadinfo=d3d62000 task=d673e130) console shuts up ... <4>hdb: dma_timer_expiry: dma status == 0x64 hdb: DMA interrupt recovery hdb: lost interrupt NMI watchdog detected lockup on CPU#1 (50000/50000) Pid: 2943, comm: lockupcli EIP: 0073:[<08048392>] CPU: 1 EIP is at 0x8048392 ESP: 007b:bfdf1460 EFLAGS: 00003086 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 452a3ff4 ECX: bfdf150c EDX: 452a3ff4 ESI: bfdf1504 EDI: bfdf1490 EBP: bfdf1478 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 18750000 CR4: 000006d0 NMI show regs on CPU#0: NMI Watchdog detected LOCKUP on CPU1, eip 08048392, registers: Modules linked in: eeprom lm85 i2c_sensor i2c_isa i2c_i801 i2c_dev i2c_core realtime microcode snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc ns83820 CPU: 1 EIP: 0073:[<08048392>] Not tainted VLI EFLAGS: 00003086 (2.6.12-RT-V0.7.50-17) EIP is at 0x8048392 Pid: 0, comm: swapper EIP: 0060:[] CPU: 0 eax: 00000000 ebx: 452a3ff4 ecx: bfdf150c edx: 452a3ff4 EIP is at default_idle+0x2a/0x33 EFLAGS: 00000246 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: c0414000 ECX: c13e72e0 EDX: c0414000 ESI: c0414000 EDI: c0443380 EBP: c0443300 DS: 007b ES: 007b CR0: 8005003b CR2: b7f7376e CR3: 1e13e000 CR4: 000006d0 []esi: bfdf1504 edi: bfdf1490 ebp: bfdf1478 esp: bfdf1460 cpu_idle+0x76/0xaf (4) ds: 007b es: 007b ss: 007b preempt: 00010001 []Process lockupcli (pid: 2943, threadinfo=da2c2000 task=de150230) start_kernel+0x195/0x1fb (36) []console shuts up ... unknown_bootoption+0x0/0x1d8 (20) <3>BUG: soft lockup detected on CPU#1! [] softlockup_tick+0xa3/0xb4 (8) [] smp_apic_timer_interrupt+0xbd/0xc9 (28) [] apic_timer_interrupt+0x1c/0x24 (24) [] __cond_resched+0x3f/0x41 (44) [] cond_resched+0x1c/0x25 (16) [] unmap_vmas+0x13e/0x188 (8) [] exit_mmap+0x5d/0x118 (48) [] mmput+0x2c/0x89 (56) [] do_exit+0xcb/0x3dd (16) [] printk+0x17/0x1b (16) [] default_do_nmi+0x0/0x17c (24) [] nmi_watchdog_tick+0x1fe/0x2a8 (24) [] default_do_nmi+0x80/0x17c (52) [] unmap_vma_list+0x1c/0x28 (24) [] sys_munmap+0x45/0x4e (44) [] do_nmi+0x4e/0x5b (28) [] nmi_stack_correct+0x1d/0x22 (20) NMI show regs on CPU#1: Pid: 2943, comm: lockupcli EIP: 0060:[] CPU: 1 EIP is at nmi_show_all_regs+0x95/0x104 EFLAGS: 00003046 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: c044b838 ECX: 00000000 EDX: 00000000 ESI: 00043196 EDI: 00000000 EBP: da2c3e18 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 00447000 CR4: 000006d0 [] softlockup_tick+0xa8/0xb4 (24) [] smp_apic_timer_interrupt+0xbd/0xc9 (28) [] apic_timer_interrupt+0x1c/0x24 (24) [] __cond_resched+0x3f/0x41 (44) [] cond_resched+0x1c/0x25 (16) [] unmap_vmas+0x13e/0x188 (8) [] exit_mmap+0x5d/0x118 (48) [] mmput+0x2c/0x89 (56) [] do_exit+0xcb/0x3dd (16) [] printk+0x17/0x1b (16) [] default_do_nmi+0x0/0x17c (24) [] nmi_watchdog_tick+0x1fe/0x2a8 (24) [] default_do_nmi+0x80/0x17c (52) [] unmap_vma_list+0x1c/0x28 (24) [] sys_munmap+0x45/0x4e (44) [] do_nmi+0x4e/0x5b (28) [] nmi_stack_correct+0x1d/0x22 (20) NMI show regs on CPU#0: Pid: 0, comm: swapper EIP: 0060:[] CPU: 0 EIP is at default_idle+0x2a/0x33 EFLAGS: 00000246 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: c0414000 ECX: c13e72e0 EDX: c0414000 ESI: c0414000 EDI: c0443380 EBP: c0443300 DS: 007b ES: 007b CR0: 8005003b CR2: b7f7376e CR3: 1e13e000 CR4: 000006d0 [] cpu_idle+0x76/0xaf (4) [] start_kernel+0x195/0x1fb (36) [] unknown_bootoption+0x0/0x1d8 (20) NMI watchdog detected lockup on CPU#0 (50000/50000) Pid: 2949, comm: lockupcli EIP: 0073:[<08048392>] CPU: 0 EIP is at 0x8048392 ESP: 007b:bfdc8f00 EFLAGS: 00003086 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: 452a3ff4 ECX: bfdc8fac EDX: 452a3ff4 ESI: bfdc8fa4 EDI: bfdc8f30 EBP: bfdc8f18 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 13c35000 CR4: 000006d0 NMI show regs on CPU#1: NMI Watchdog detected LOCKUP on CPU0, eip 08048392, registers: Modules linked in: eeprom lm85 i2c_sensor i2c_isa i2c_i801 i2c_dev i2c_core realtime microcode snd_intel8x0 snd_ac97_codec snd_pcm_oss snd_mixer_oss snd_pcm snd_timer snd soundcore snd_page_alloc ns83820 CPU: 0 EIP: 0073:[<08048392>] Not tainted VLI EFLAGS: 00003086 (2.6.12-RT-V0.7.50-17) EIP is at 0x8048392 eax: 00000000 ebx: 452a3ff4 ecx: bfdc8fac edx: 452a3ff4 esi: bfdc8fa4 edi: bfdc8f30 ebp: bfdc8f18 esp: bfdc8f00 ds: 007b es: 007b ss: 007b preempt: 00010001 Process lockupcli (pid: 2949, threadinfo=d3c3c000 task=da2ed290) console shuts up ... Pid: 0, comm: swapper EIP: 0060:[] CPU: 1 EIP is at default_idle+0x2a/0x33 EFLAGS: 00000246 Not tainted (2.6.12-RT-V0.7.50-17) EAX: 00000000 EBX: def12000 ECX: c13f72e0 EDX: def12000 ESI: def12000 EDI: c0443380 EBP: c0443300 DS: 007b ES: 007b CR0: 8005003b CR2: 080f3c04 CR3: 1d9e6000 CR4: 000006d0 [] cpu_idle+0x76/0xaf (4) ********************************** *** 2.6.12-RT-V0.7.50-18-debug *** ********************************** NMI watchdog detected lockup on CPU#0 (50000/50000) Pid: 3264, comm: lockupcli EIP: 0073:[<08048392>] CPU: 0 EIP is at 0x8048392 ESP: 007b:bfab1800 EFLAGS: 00003086 Not tainted (2.6.12-RT-V0.7.50-18-debug) EAX: 00000000 EBX: 452a3ff4 ECX: bfab18ac EDX: 452a3ff4 ESI: bfab18a4 EDI: bfab1830 EBP: bfab1818 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 1e10d000 CR4: 000006d0 [] show_regs+0x14b/0x173 (36) [] nmi_watchdog_tick+0x17b/0x2a1 (52) [] default_do_nmi+0x7a/0x17a (96) [] do_nmi+0x6f/0x81 (24) [] nmi_stack_correct+0x1d/0x22 (-323987356) --------------------------- | preempt count: 00010002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [] .... _raw_spin_lock+0x1b/0x85 .....[] .. ( <= nmi_watchdog_tick+0x141/0x2a1) .. [] .... print_traces+0x1b/0x52 .....[] .. ( <= shoce+0xc2/0xd4) ------------------------------ | showing all locks held by: | (lockupcli/3264 [ddcf0cc0, 54]): ------------------------------ NMI show regs on CPU#1: NMI watchdog detected lockup on CPU#0 (50000/50000) Pid: 3031, comm: lockupcli EIP: 0073:[<08048392>] CPU: 0 EIP is at 0x8048392 ESP: 007b:bfdca9e0 EFLAGS: 00003082 Not tainted (2.6.12-RT-V0.7.50-18-debug) EAX: 00000000 EBX: 452a3ff4 ECX: bfdcaa8c EDX: 452a3ff4 ESI: bfdcaa84 EDI: bfdcaa10 EBP: bfdca9f8 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 122bd000 CR4: 000006d0 [] show_regs+0x14b/0x173 (36) [] nmi_watchdog_tick+0x17b/0x2a1 (52) [] default_do_nmi+0x7a/0x17a (96) [] do_nmi+0x6f/0x81 (24) [] nmi_stack_correct+0x1d/0x22 (-308213180) --------------------------- | preempt count: 00010002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [] .... _raw_spin_lock+0x1b/0x85 .....[] .. ( <= nmi_watchdog_tick+0x141/0x2a1) .. [] .... print_traces+0x1b/0x52 .....[<103de5>] .. ( <= show_trace+0xc2/0xd4) ------------------------------ | showing all locks held by: | (lockupcli/3031 [ddeac660, 118]): ------------------------------ NMI show regs on CPU#1: NMI watchdog detected lockup on CPU#1 (50000/50000) Pid: 2937, comm: lockupcli EIP: 0073:[<08048392>] CPU: 1 EIP is at 0x8048392 ESP: 007b:bff205d0 EFLAGS: 00003082 Not tainted (2.6.12-RT-V0.7.50-18-debug) EAX: 00000000 EBX: 452a3ff4 ECX: bff2067c EDX: 452a3ff4 ESI: bff20674 EDI: bff20600 EBP: bff205e8 DS: 007b ES: 007b CR0: 8005003b CR2: 451a5590 CR3: 1382e000 CR4: 000006d0 [] show_regs+0x14b/0x173 (36) [] nmi_watchdog_tick+0x17b/0x2a1 (52) [] default_do_nmi+0x7a/0x17a (96) [] do_nmi+0x6f/0x81 (24) [] nmi_stack_correct+0x1d/0x22 (-373864908) --------------------------- | preempt count: 00010002 ] | 2-level deep critical section nesting: ---------------------------------------- .. [] .... _raw_spin_lock+0x1b/0x85 .....[] .. ( <= nmi_watchdog_tick+0x141/0x2a1) .. [] .... print_traces+0x1b/0x52 .....[] .. ( <= show_trace+0xc2/0xd4) ------------------------------ | showing all locks held by: | (lockupcli/2937 [de4ed980, 54]): ------------------------------ NMI show regs on CPU#0: