* Kernel crash during btrfs benchmark
@ 2010-10-02 20:26 Bart Kus
2010-10-02 23:49 ` Jérôme Poulin
0 siblings, 1 reply; 2+ messages in thread
From: Bart Kus @ 2010-10-02 20:26 UTC (permalink / raw)
To: linux-btrfs
Hello,
On the following system:
Linux jo.bartk.us 2.6.35-gentoo-r9 #1 SMP Sat Oct 2 11:29:36 PDT 2010
x86_64 Intel(R) Core(TM)2 Quad CPU @ 2.40GHz GenuineIntel GNU/Linux
I was running the following command:
dbench -c /usr/share/dbench/client.txt -t 60 64
When everything froze up and console started reporting errors. The
client.txt file can be obtained here:
http://www.bartk.us/dump/client.txt.gz . dbench version 4.00 was being
used. The System.map can be obtained from
http://www.bartk.us/dump/System.map-genkernel-x86_64-2.6.35-gentoo-r9 ,
if that's helpful. The underlying device is an md RAID6 with 11 2TB
HDs, connected via sata_sil24 module (SiI3124 chip), which has its 4
eSATA ports connected to 4 SiI3726 port multipliers. I have pulled a
btrfs-image off the device and it can be obtained here:
http://www.bartk.us/dump/btrfs.image . Here are the details of the md
device:
/dev/md5:
Version : 1.02
Creation Time : Thu Sep 30 10:35:17 2010
Raid Level : raid6
Array Size : 17581629888 (16767.15 GiB 18003.59 GB)
Used Dev Size : 1953514432 (1863.02 GiB 2000.40 GB)
Raid Devices : 11
Total Devices : 11
Persistence : Superblock is persistent
Update Time : Sat Oct 2 13:03:27 2010
State : active, resyncing
Active Devices : 11
Working Devices : 11
Failed Devices : 0
Spare Devices : 0
Chunk Size : 64K
Rebuild Status : 10% complete
Name : jo.bartk.us:5 (local to host jo.bartk.us)
UUID : 5122c697:58113607:3f5bb07e:a8069132
Events : 39
Number Major Minor RaidDevice State
0 8 64 0 active sync /dev/sde
1 8 144 1 active sync /dev/sdj
2 8 0 2 active sync /dev/sda
3 8 16 3 active sync /dev/sdb
4 8 32 4 active sync /dev/sdc
5 8 48 5 active sync /dev/sdd
6 8 80 6 active sync /dev/sdf
7 8 96 7 active sync /dev/sdg
8 8 112 8 active sync /dev/sdh
9 8 128 9 active sync /dev/sdi
10 8 160 10 active sync /dev/sdk
It's resyncing right now due to the crash, but wasn't at time of
failure. There are no partitions involved, /dev/md5 was directly
formatted with btrfs.
Full text of kernel crash follows:
Oct 2 12:07:52 jo kernel: device fsid a34c8b899171b6f6-fa7e695b5cf51590
devid 1 transid 25 /dev/md5
Oct 2 12:16:55 jo kernel: INFO: rcu_sched_state detected stall on CPU 3
(t=1000 jiffies)
Oct 2 12:16:55 jo kernel: sending NMI to all CPUs:
Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 3
Oct 2 12:16:55 jo kernel: CPU 3
Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:16:55 jo kernel:
Oct 2 12:16:55 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff81303c06>]
[<ffffffff81303c06>] 0xffffffff81303c06
Oct 2 12:16:55 jo kernel: RSP: 0018:ffff880001983de0 EFLAGS: 00000803
Oct 2 12:16:55 jo kernel: RAX: 00000000c68de520 RBX: 0000000000000001
RCX: ffff880001980000
Oct 2 12:16:55 jo kernel: RDX: 0000000000249f7d RSI: 0000000000000002
RDI: 0000000000249f7e
Oct 2 12:16:55 jo kernel: RBP: ffff880001983de8 R08: 000000000000001c
R09: 000000000000000a
Oct 2 12:16:55 jo kernel: R10: 0000000000000003 R11: 0000000000000000
R12: ffff88000198e9b0
Oct 2 12:16:55 jo kernel: R13: ffffffff815d3080 R14: ffff88000198e778
R15: ffff880001983f48
Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001980000(0000) knlGS:0000000000000000
Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:16:55 jo kernel: CR2: 00007f18f0b2f000 CR3: 00000000015bf000
CR4: 00000000000006e0
Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:16:55 jo kernel: Process btrfs-worker-4 (pid: 22695,
threadinfo ffff88004bc96000, task ffff88011fe04cb0)
Oct 2 12:16:55 jo kernel: Stack:
Oct 2 12:16:55 jo kernel: ffffffff81303b55 ffff880001983df8
ffffffff81303b91 ffff880001983e18
Oct 2 12:16:55 jo kernel: <0> ffffffff81018645 ffff88000198e778
ffffffff815d3080 ffff880001983e68
Oct 2 12:16:55 jo kernel: <0> ffffffff81073786 ffff880001983e78
ffffffff8106f0e2 000000003b9aca00
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: <IRQ>
Oct 2 12:16:55 jo kernel: [<ffffffff81303b55>] ? 0xffffffff81303b55
Oct 2 12:16:55 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:16:55 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:16:55 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:16:55 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:16:55 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:16:55 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:16:55 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:16:55 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:16:55 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:16:55 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:16:55 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:16:55 jo kernel: <EOI>
Oct 2 12:16:55 jo kernel: [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:16:55 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:16:55 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:16:55 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:16:55 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:16:55 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:16:55 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:16:55 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: Code: 01 00 65 8b 0c 25 60 d3 00 00 48 63 c9
48 8b 0c cd 00 74 60 81 48 6b 94 0a 98 00 00 00 19 f7 e2 48 8d 7a 01 e8
47 ff ff ff c9 c3 <55> 48 89 e5 65 8b 34 25 60 d3 00 00 0f 1f 00 0f ae
e8 0f 31 89
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: <IRQ> [<ffffffff81303b55>] ? 0xffffffff81303b55
Oct 2 12:16:55 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:16:55 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:16:55 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:16:55 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:16:55 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:16:55 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:16:55 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:16:55 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:16:55 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:16:55 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:16:55 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:16:55 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:16:55 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:16:55 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:16:55 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:16:55 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:16:55 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:16:55 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:16:55 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:16:55 jo kernel: [<ffffffff81303c06>] ? 0xffffffff81303c06
Oct 2 12:16:55 jo kernel: <<EOE>> <IRQ> [<ffffffff81303b55>] ?
0xffffffff81303b55
Oct 2 12:16:55 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:16:55 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:16:55 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:16:55 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:16:55 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:16:55 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:16:55 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:16:55 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:16:55 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:16:55 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:16:55 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:16:55 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:16:55 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:16:55 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:16:55 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:16:55 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:16:55 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:16:55 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:16:55 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 2
Oct 2 12:16:55 jo kernel: CPU 2
Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:16:55 jo kernel:
Oct 2 12:16:55 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff812f7e19>]
[<ffffffff812f7e19>] 0xffffffff812f7e19
Oct 2 12:16:55 jo kernel: RSP: 0018:ffff88011f46bbd0 EFLAGS: 00000082
Oct 2 12:16:55 jo kernel: RAX: 0000000000000000 RBX: ffff88011e6ad000
RCX: ffff88011e2f03d0
Oct 2 12:16:55 jo kernel: RDX: 0000000000000001 RSI: ffff88011d350418
RDI: ffff88011e6ad000
Oct 2 12:16:55 jo kernel: RBP: ffff88011f46bbd0 R08: ffff88011fe1ca58
R09: 0000000000000001
Oct 2 12:16:55 jo kernel: R10: ffff88011e083938 R11: ffff88011f46bc90
R12: ffff88011d350418
Oct 2 12:16:55 jo kernel: R13: ffff88011d350418 R14: ffff88011e6ad848
R15: ffff88011ecf21a0
Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001900000(0000) knlGS:0000000000000000
Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:16:55 jo kernel: CR2: 00007f18f0b2f000 CR3: 000000011e50e000
CR4: 00000000000006e0
Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:16:55 jo kernel: Process md5_raid6 (pid: 14616, threadinfo
ffff88011f46a000, task ffff88011d94acf0)
Oct 2 12:16:55 jo kernel: Stack:
Oct 2 12:16:55 jo kernel: ffff88011f46bc40 ffffffff812fab19
ffff88011f46bc40 ffff8800c8be85c0
Oct 2 12:16:55 jo kernel: <0> ffff88011e7d0800 ffff88011e238000
ffff88011e083800 ffff8800d735cd30
Oct 2 12:16:55 jo kernel: <0> ffff88011f46bc50 ffff88011e6ad800
ffff88011e2f1450 ffff88011f46bc58
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: [<ffffffff812fab19>] 0xffffffff812fab19
Oct 2 12:16:55 jo kernel: [<ffffffff812ee214>] 0xffffffff812ee214
Oct 2 12:16:55 jo kernel: [<ffffffff812fdda7>] ? 0xffffffff812fdda7
Oct 2 12:16:55 jo kernel: [<ffffffff813a5a01>] 0xffffffff813a5a01
Oct 2 12:16:55 jo kernel: [<ffffffff8103af84>] ? 0xffffffff8103af84
Oct 2 12:16:55 jo kernel: [<ffffffff812eeb94>] 0xffffffff812eeb94
Oct 2 12:16:55 jo kernel: [<ffffffff812eebc2>] 0xffffffff812eebc2
Oct 2 12:16:55 jo kernel: [<ffffffff812ece41>] 0xffffffff812ece41
Oct 2 12:16:55 jo kernel: [<ffffffffa04468b0>] 0xffffffffa04468b0
Oct 2 12:16:55 jo kernel: [<ffffffffa0448bd0>] 0xffffffffa0448bd0
Oct 2 12:16:55 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
Oct 2 12:16:55 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
Oct 2 12:16:55 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
Oct 2 12:16:55 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: Code: 00 48 89 e5 b8 02 00 00 00 66 83 fa 03
74 09 31 c0 66 83 fa 01 0f 94 c0 48 8b 96 c8 00 00 00 48 85 d2 75 04 0f
0b eb fe 8b 52 10 <85> d2 75 04 0f 0b eb fe 83 f8 02 74 28 8b 46 04 a8
20 74 15 48
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: [<ffffffff812fab19>] 0xffffffff812fab19
Oct 2 12:16:55 jo kernel: [<ffffffff812ee214>] 0xffffffff812ee214
Oct 2 12:16:55 jo kernel: [<ffffffff812fdda7>] ? 0xffffffff812fdda7
Oct 2 12:16:55 jo kernel: [<ffffffff813a5a01>] 0xffffffff813a5a01
Oct 2 12:16:55 jo kernel: [<ffffffff8103af84>] ? 0xffffffff8103af84
Oct 2 12:16:55 jo kernel: [<ffffffff812eeb94>] 0xffffffff812eeb94
Oct 2 12:16:55 jo kernel: [<ffffffff812eebc2>] 0xffffffff812eebc2
Oct 2 12:16:55 jo kernel: [<ffffffff812ece41>] 0xffffffff812ece41
Oct 2 12:16:55 jo kernel: [<ffffffffa04468b0>] 0xffffffffa04468b0
Oct 2 12:16:55 jo kernel: [<ffffffffa0448bd0>] 0xffffffffa0448bd0
Oct 2 12:16:55 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
Oct 2 12:16:55 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
Oct 2 12:16:55 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
Oct 2 12:16:55 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:16:55 jo kernel: [<ffffffff812f7e19>] ? 0xffffffff812f7e19
Oct 2 12:16:55 jo kernel: <<EOE>> [<ffffffff812fab19>] 0xffffffff812fab19
Oct 2 12:16:55 jo kernel: [<ffffffff812ee214>] 0xffffffff812ee214
Oct 2 12:16:55 jo kernel: [<ffffffff812fdda7>] ? 0xffffffff812fdda7
Oct 2 12:16:55 jo kernel: [<ffffffff813a5a01>] 0xffffffff813a5a01
Oct 2 12:16:55 jo kernel: [<ffffffff8103af84>] ? 0xffffffff8103af84
Oct 2 12:16:55 jo kernel: [<ffffffff812eeb94>] 0xffffffff812eeb94
Oct 2 12:16:55 jo kernel: [<ffffffff812eebc2>] 0xffffffff812eebc2
Oct 2 12:16:55 jo kernel: [<ffffffff812ece41>] 0xffffffff812ece41
Oct 2 12:16:55 jo kernel: [<ffffffffa04468b0>] 0xffffffffa04468b0
Oct 2 12:16:55 jo kernel: [<ffffffffa0448bd0>] 0xffffffffa0448bd0
Oct 2 12:16:55 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
Oct 2 12:16:55 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
Oct 2 12:16:55 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
Oct 2 12:16:55 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 0
Oct 2 12:16:55 jo kernel: CPU 0
Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:16:55 jo kernel:
Oct 2 12:16:55 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff812948b0>]
[<ffffffff812948b0>] 0xffffffff812948b0
Oct 2 12:16:55 jo kernel: RSP: 0018:ffff8800df86ba10 EFLAGS: 00010286
Oct 2 12:16:55 jo kernel: RAX: ffffea00038a90d8 RBX: ffff8800981032e0
RCX: 000000000000044c
Oct 2 12:16:55 jo kernel: RDX: ffff8800df86a000 RSI: ffff88010304dbb4
RDI: ffff88010304dbb0
Oct 2 12:16:55 jo kernel: RBP: ffff8800df86ba80 R08: 00000000000003c8
R09: 0000000000000000
Oct 2 12:16:55 jo kernel: R10: ffff88010304d3c8 R11: 6db6db6db6db6db7
R12: 00000000000003cc
Oct 2 12:16:55 jo kernel: R13: 0000000000000c34 R14: ffff8800df86a000
R15: ffffea00038a90d8
Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001800000(0000) knlGS:0000000000000000
Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:16:55 jo kernel: CR2: 0000000000749f44 CR3: 000000011acee000
CR4: 00000000000006f0
Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:16:55 jo kernel: Process btrfs-endio-wri (pid: 23272,
threadinfo ffff8800df86a000, task ffff88011e0a4090)
Oct 2 12:16:55 jo kernel: Stack:
Oct 2 12:16:55 jo kernel: ffff8800df86bac8 ffff8800df86bac0
ffff8800df86bad0 0000000000000000
Oct 2 12:16:55 jo kernel: <0> ffff8800df86ba90 00000000000003c8
00000000000003cc 0000000000000c34
Oct 2 12:16:55 jo kernel: <0> ffff8800981032e0 0000000000000c34
ffff8800981032e0 0000000000000001
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: [<ffffffff8129499b>] 0xffffffff8129499b
Oct 2 12:16:55 jo kernel: [<ffffffff81294e48>] ? 0xffffffff81294e48
Oct 2 12:16:55 jo kernel: [<ffffffff81263a3c>] 0xffffffff81263a3c
Oct 2 12:16:55 jo kernel: [<ffffffff81275acd>] 0xffffffff81275acd
Oct 2 12:16:55 jo kernel: [<ffffffff8147150e>] ? 0xffffffff8147150e
Oct 2 12:16:55 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:16:55 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:16:55 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:16:55 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:16:55 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:16:55 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:16:55 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:16:55 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: Code: db b6 6d 48 c1 fe 03 48 0f af f1 48 b9
00 00 00 00 00 88 ff ff 48 c1 e6 0c 48 01 ce 4c 03 55 b8 48 03 75 c0 48
8b 4d c8 4c 89 d7 <f3> a4 ff 4a 1c 4c 39 f8 74 03 ff 4a 1c 4c 03 65 c8
4c 03 45 c8
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: [<ffffffff8129499b>] 0xffffffff8129499b
Oct 2 12:16:55 jo kernel: [<ffffffff81294e48>] ? 0xffffffff81294e48
Oct 2 12:16:55 jo kernel: [<ffffffff81263a3c>] 0xffffffff81263a3c
Oct 2 12:16:55 jo kernel: [<ffffffff81275acd>] 0xffffffff81275acd
Oct 2 12:16:55 jo kernel: [<ffffffff8147150e>] ? 0xffffffff8147150e
Oct 2 12:16:55 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:16:55 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:16:55 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:16:55 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:16:55 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:16:55 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:16:55 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:16:55 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:16:55 jo kernel: [<ffffffff812948b0>] ? 0xffffffff812948b0
Oct 2 12:16:55 jo kernel: <<EOE>> [<ffffffff8129499b>] 0xffffffff8129499b
Oct 2 12:16:55 jo kernel: [<ffffffff81294e48>] ? 0xffffffff81294e48
Oct 2 12:16:55 jo kernel: [<ffffffff81263a3c>] 0xffffffff81263a3c
Oct 2 12:16:55 jo kernel: [<ffffffff81275acd>] 0xffffffff81275acd
Oct 2 12:16:55 jo kernel: [<ffffffff8147150e>] ? 0xffffffff8147150e
Oct 2 12:16:55 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:16:55 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:16:55 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:16:55 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:16:55 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:16:55 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:16:55 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:16:55 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 1
Oct 2 12:16:55 jo kernel: CPU 1
Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:16:55 jo kernel:
Oct 2 12:16:55 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff81009d68>]
[<ffffffff81009d68>] 0xffffffff81009d68
Oct 2 12:16:55 jo kernel: RSP: 0018:ffff88011fcbbf08 EFLAGS: 00000246
Oct 2 12:16:55 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680
RCX: 0000000000000000
Oct 2 12:16:55 jo kernel: RDX: 0000000000000000 RSI: ffff88011fcbbfd8
RDI: ffffffff81683308
Oct 2 12:16:55 jo kernel: RBP: ffff88011fcbbf08 R08: 0000000000000000
R09: 0000000000000078
Oct 2 12:16:55 jo kernel: R10: ffff88011fe56cf0 R11: 0000000000000003
R12: ffff88011fcbbfd8
Oct 2 12:16:55 jo kernel: R13: 0000000000000000 R14: 0000000000000000
R15: 0000000000000000
Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001880000(0000) knlGS:0000000000000000
Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:16:55 jo kernel: CR2: 00000000007fd2d0 CR3: 00000001072d0000
CR4: 00000000000006e0
Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:16:55 jo kernel: Process swapper (pid: 0, threadinfo
ffff88011fcba000, task ffff88011fcac6a0)
Oct 2 12:16:55 jo kernel: Stack:
Oct 2 12:16:55 jo kernel: ffff88011fcbbf28 ffffffff81001bcf
0000000000000001 0000000000000000
Oct 2 12:16:55 jo kernel: <0> ffff88011fcbbf48 ffffffff8146c288
563dcf676bf8ea76 0000000000000000
Oct 2 12:16:55 jo kernel: <0> 0000000000000000 0000000000000000
0000000000000000 0000000000000000
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:16:55 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:16:55 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86
38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08
31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00
00 48 8b 82
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:16:55 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:16:55 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:16:55 jo kernel: Call Trace:
Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:16:55 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
Oct 2 12:16:55 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:16:55 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:19:15 jo kernel: INFO: rcu_sched_state detected stall on CPU 3
(t=1000 jiffies)
Oct 2 12:19:15 jo kernel: sending NMI to all CPUs:
Oct 2 12:19:15 jo kernel: NMI backtrace for cpu 2
Oct 2 12:19:15 jo kernel: CPU 2
Oct 2 12:19:15 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:19:15 jo kernel:
Oct 2 12:19:15 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:19:15 jo kernel: RIP: 0010:[<ffffffff81009d68>]
[<ffffffff81009d68>] 0xffffffff81009d68
Oct 2 12:19:15 jo kernel: RSP: 0018:ffff88011fccbf08 EFLAGS: 00000246
Oct 2 12:19:15 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680
RCX: 0000000000000000
Oct 2 12:19:28 jo kernel: RDX: 0000000000000000 RSI: ffff88011fccbfd8
RDI: ffffffff81683308
Oct 2 12:19:28 jo kernel: RBP: ffff88011fccbf08 R08: 0000000000000000
R09: 0000000000000001
Oct 2 12:19:28 jo kernel: R10: ffff88011f46bd20 R11: ffff88011e2e3800
R12: ffff88011fccbfd8
Oct 2 12:19:28 jo kernel: R13: 0000000000000000 R14: 0000000000000000
R15: 0000000000000000
Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001900000(0000) knlGS:0000000000000000
Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:19:28 jo kernel: CR2: 00007f18f0b2f000 CR3: 000000011e50e000
CR4: 00000000000006e0
Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:19:28 jo kernel: Process swapper (pid: 0, threadinfo
ffff88011fcca000, task ffff88011fcbecf0)
Oct 2 12:19:28 jo kernel: Stack:
Oct 2 12:19:28 jo kernel: ffff88011fccbf28 ffffffff81001bcf
0000000000000002 0000000000000000
Oct 2 12:19:28 jo kernel: <0> ffff88011fccbf48 ffffffff8146c288
3081aa158d48f824 0000000000000000
Oct 2 12:19:28 jo kernel: <0> 0000000000000000 0000000000000000
0000000000000000 0000000000000000
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:19:28 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:19:28 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86
38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08
31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00
00 48 8b 82
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:19:28 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:19:28 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:19:28 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
Oct 2 12:19:28 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:19:28 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:19:28 jo kernel: NMI backtrace for cpu 3
Oct 2 12:19:28 jo kernel: CPU 3
Oct 2 12:19:28 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:19:28 jo kernel:
Oct 2 12:19:28 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:19:28 jo kernel: RIP: 0010:[<ffffffff81303c24>]
[<ffffffff81303c24>] 0xffffffff81303c24
Oct 2 12:19:28 jo kernel: RSP: 0018:ffff880001983dd8 EFLAGS: 00000046
Oct 2 12:19:28 jo kernel: RAX: 00000000dc35449f RBX: 0000000000000001
RCX: 00000000dc3543a3
Oct 2 12:19:28 jo kernel: RDX: 0000000000000562 RSI: 0000000000000003
RDI: 0000000000249f7e
Oct 2 12:19:28 jo kernel: RBP: ffff880001983dd8 R08: 0000000000000003
R09: 000000000000000a
Oct 2 12:19:28 jo kernel: R10: 0000000000000003 R11: 0000000000000000
R12: ffff88000198e9b0
Oct 2 12:19:28 jo kernel: R13: ffffffff815d3080 R14: ffff88000198e778
R15: ffff880001983f48
Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001980000(0000) knlGS:0000000000000000
Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:19:28 jo kernel: CR2: 00007f18f0b2f000 CR3: 00000000015bf000
CR4: 00000000000006e0
Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:19:28 jo kernel: Process btrfs-worker-4 (pid: 22695,
threadinfo ffff88004bc96000, task ffff88011fe04cb0)
Oct 2 12:19:28 jo kernel: Stack:
Oct 2 12:19:28 jo kernel: ffff880001983de8 ffffffff81303b55
ffff880001983df8 ffffffff81303b91
Oct 2 12:19:28 jo kernel: <0> ffff880001983e18 ffffffff81018645
ffff88000198e778 ffffffff815d3080
Oct 2 12:19:28 jo kernel: <0> ffff880001983e68 ffffffff81073786
ffff880001983e78 ffffffff8106f0e2
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: <IRQ>
Oct 2 12:19:28 jo kernel: [<ffffffff81303b55>] 0xffffffff81303b55
Oct 2 12:19:28 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:19:28 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:19:28 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:19:28 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:19:28 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:19:28 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:19:28 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:19:28 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:19:28 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:19:28 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:19:28 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:19:28 jo kernel: <EOI>
Oct 2 12:19:28 jo kernel: [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:19:28 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:19:28 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:19:28 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:19:28 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:19:28 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:19:28 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:19:28 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:19:28 jo kernel: Code: f7 e2 48 8d 7a 01 e8 47 ff ff ff c9 c3
55 48 89 e5 65 8b 34 25 60 d3 00 00 0f 1f 00 0f ae e8 0f 31 89 c1 0f 1f
00 0f ae e8 0f 31 <89> c0 48 89 c2 48 29 ca 48 39 fa 73 25 f3 90 65 44
8b 04 25 60
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: <IRQ> [<ffffffff81303b55>] 0xffffffff81303b55
Oct 2 12:19:28 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:19:28 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:19:28 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:19:28 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:19:28 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:19:28 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:19:28 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:19:28 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:19:28 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:19:28 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:19:28 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:19:28 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:19:28 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:19:28 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:19:28 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:19:28 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:19:28 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:19:28 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:19:28 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:19:28 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:19:28 jo kernel: [<ffffffff81303c24>] ? 0xffffffff81303c24
Oct 2 12:19:28 jo kernel: <<EOE>> <IRQ> [<ffffffff81303b55>]
0xffffffff81303b55
Oct 2 12:19:28 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:19:28 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:19:28 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:19:28 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:19:28 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:19:28 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:19:28 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:19:28 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:19:28 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:19:28 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:19:28 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:19:28 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:19:28 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:19:28 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:19:28 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:19:28 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:19:28 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:19:28 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:19:28 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:19:28 jo kernel: NMI backtrace for cpu 1
Oct 2 12:19:28 jo kernel: CPU 1
Oct 2 12:19:28 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:19:28 jo kernel:
Oct 2 12:19:28 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:19:28 jo kernel: RIP: 0010:[<ffffffff81295eb7>]
[<ffffffff81295eb7>] 0xffffffff81295eb7
Oct 2 12:19:28 jo kernel: RSP: 0018:ffff8800df86b980 EFLAGS: 00000206
Oct 2 12:19:28 jo kernel: RAX: ffff88007f62adb8 RBX: ffff8800bfea4c18
RCX: ffff88007f62adb8
Oct 2 12:19:28 jo kernel: RDX: ffff88007f62ada8 RSI: 0000000001ff4000
RDI: ffff88007f62ae38
Oct 2 12:19:28 jo kernel: RBP: ffff8800df86b9c0 R08: 0000000000000001
R09: 0000000000000000
Oct 2 12:19:28 jo kernel: R10: ffff8800df86b9e0 R11: 6db6db6db6db6db7
R12: 0000000001ff4000
Oct 2 12:19:28 jo kernel: R13: 0000000001ff4fff R14: 0000000000000004
R15: 0000000000000001
Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001880000(0000) knlGS:0000000000000000
Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:19:28 jo kernel: CR2: 00007f18f0b2f000 CR3: 00000001072d0000
CR4: 00000000000006e0
Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:19:28 jo kernel: Process btrfs-endio-wri (pid: 23272,
threadinfo ffff8800df86a000, task ffff88011e0a4090)
Oct 2 12:19:28 jo kernel: Stack:
Oct 2 12:19:28 jo kernel: 0000000000000000 0000000000000000
0000000000000dee ffff8800454cfd00
Oct 2 12:19:28 jo kernel: <0> ffff8800bfea4c18 0000000000000028
0000000000000dee ffff8800690747f0
Oct 2 12:19:28 jo kernel: <0> ffff8800df86b9f0 ffffffff81295fff
ffff8800454cfd00 ffff8800bfea4c18
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: [<ffffffff81295fff>] 0xffffffff81295fff
Oct 2 12:19:28 jo kernel: [<ffffffff81277803>] 0xffffffff81277803
Oct 2 12:19:28 jo kernel: [<ffffffff812610c0>] 0xffffffff812610c0
Oct 2 12:19:28 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
Oct 2 12:19:28 jo kernel: [<ffffffff81275988>] 0xffffffff81275988
Oct 2 12:19:28 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:19:28 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:19:28 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:19:28 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:19:28 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:19:28 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:19:28 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:19:28 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:19:28 jo kernel: Code: c9 74 17 49 83 79 28 00 74 10 49 8b 11
49 8d 41 10 4c 39 e2 0f 84 b9 00 00 00 48 8b 03 31 d2 31 ff eb 24 48 8d
50 f0 4c 3b 60 f0 <73> 06 48 8b 48 10 eb 0e 4c 3b 60 f8 0f 86 93 00 00
00 48 8b 48
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: [<ffffffff81295fff>] 0xffffffff81295fff
Oct 2 12:19:28 jo kernel: [<ffffffff81277803>] 0xffffffff81277803
Oct 2 12:19:28 jo kernel: [<ffffffff812610c0>] 0xffffffff812610c0
Oct 2 12:19:28 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
Oct 2 12:19:28 jo kernel: [<ffffffff81275988>] 0xffffffff81275988
Oct 2 12:19:28 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:19:28 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:19:28 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:19:28 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:19:28 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:19:28 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:19:28 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:19:28 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:19:28 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:19:28 jo kernel: [<ffffffff81295eb7>] ? 0xffffffff81295eb7
Oct 2 12:19:28 jo kernel: <<EOE>> [<ffffffff81295fff>] 0xffffffff81295fff
Oct 2 12:19:28 jo kernel: [<ffffffff81277803>] 0xffffffff81277803
Oct 2 12:19:28 jo kernel: [<ffffffff812610c0>] 0xffffffff812610c0
Oct 2 12:19:28 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
Oct 2 12:19:28 jo kernel: [<ffffffff81275988>] 0xffffffff81275988
Oct 2 12:19:28 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:19:28 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:19:28 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:19:28 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:19:28 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:19:28 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:19:28 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:19:28 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:19:28 jo kernel: NMI backtrace for cpu 0
Oct 2 12:19:28 jo kernel: CPU 0
Oct 2 12:19:28 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:19:28 jo kernel:
Oct 2 12:19:28 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:19:28 jo kernel: RIP: 0010:[<ffffffff81009d68>]
[<ffffffff81009d68>] 0xffffffff81009d68
Oct 2 12:19:28 jo kernel: RSP: 0018:ffffffff81581f28 EFLAGS: 00000246
Oct 2 12:19:28 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680
RCX: 0000000000000000
Oct 2 12:19:28 jo kernel: RDX: 0000000000000000 RSI: ffffffff81581fd8
RDI: ffffffff81683308
Oct 2 12:19:28 jo kernel: RBP: ffffffff81581f28 R08: 0000000000000000
R09: 0000000000000078
Oct 2 12:19:28 jo kernel: R10: ffff88011fe56cf0 R11: 0000000000000003
R12: ffffffff81581fd8
Oct 2 12:19:28 jo kernel: R13: 0000000000000000 R14: ffffffffffffffff
R15: 0000000000000000
Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001800000(0000) knlGS:0000000000000000
Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:19:28 jo kernel: CR2: 00007f971b171000 CR3: 00000001072d0000
CR4: 00000000000006f0
Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:19:28 jo kernel: Process swapper (pid: 0, threadinfo
ffffffff81580000, task ffffffff815c7020)
Oct 2 12:19:28 jo kernel: Stack:
Oct 2 12:19:28 jo kernel: ffffffff81581f48 ffffffff81001bcf
0000000000000000 6db6db6db6db6db7
Oct 2 12:19:28 jo kernel: <0> ffffffff81581f58 ffffffff8145e61c
ffffffff81581f98 ffffffff8161eccb
Oct 2 12:19:28 jo kernel: <0> ffffffff81581f98 ffffffff81654060
0000000001749dc4 0000000000000000
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:19:28 jo kernel: [<ffffffff8145e61c>] 0xffffffff8145e61c
Oct 2 12:19:28 jo kernel: [<ffffffff8161eccb>] 0xffffffff8161eccb
Oct 2 12:19:28 jo kernel: [<ffffffff8161e293>] 0xffffffff8161e293
Oct 2 12:19:28 jo kernel: [<ffffffff8161e37b>] 0xffffffff8161e37b
Oct 2 12:19:28 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86
38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08
31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00
00 48 8b 82
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:19:28 jo kernel: [<ffffffff8145e61c>] 0xffffffff8145e61c
Oct 2 12:19:28 jo kernel: [<ffffffff8161eccb>] 0xffffffff8161eccb
Oct 2 12:19:28 jo kernel: [<ffffffff8161e293>] 0xffffffff8161e293
Oct 2 12:19:28 jo kernel: [<ffffffff8161e37b>] 0xffffffff8161e37b
Oct 2 12:19:28 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:19:28 jo kernel: Call Trace:
Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:19:28 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
Oct 2 12:19:28 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:19:28 jo kernel: [<ffffffff8145e61c>] 0xffffffff8145e61c
Oct 2 12:19:28 jo kernel: [<ffffffff8161eccb>] 0xffffffff8161eccb
Oct 2 12:19:28 jo kernel: [<ffffffff8161e293>] 0xffffffff8161e293
Oct 2 12:19:28 jo kernel: [<ffffffff8161e37b>] 0xffffffff8161e37b
Oct 2 12:21:32 jo kernel: INFO: rcu_sched_state detected stall on CPU 3
(t=1000 jiffies)
Oct 2 12:21:32 jo kernel: sending NMI to all CPUs:
Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 1
Oct 2 12:21:32 jo kernel: CPU 1
Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:21:32 jo kernel:
Oct 2 12:21:32 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffffa0445184>]
[<ffffffffa0445184>] 0xffffffffa0445184
Oct 2 12:21:32 jo kernel: RSP: 0018:ffff880001883ca0 EFLAGS: 00000286
Oct 2 12:21:32 jo kernel: RAX: ffffffffa0445168 RBX: 0000000000000000
RCX: 0000000000001000
Oct 2 12:21:32 jo kernel: RDX: 0000000000000009 RSI: 0000000000000000
RDI: ffff88006f2e1d40
Oct 2 12:21:32 jo kernel: RBP: ffff880001883cc0 R08: ffff88011e04e4e8
R09: 0000000000000bb8
Oct 2 12:21:32 jo kernel: R10: 0000000000000001 R11: ffffffff81380635
R12: ffff88006f2e1800
Oct 2 12:21:32 jo kernel: R13: ffff88011edbce00 R14: 0000000000000000
R15: 0000000000000000
Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001880000(0000) knlGS:0000000000000000
Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:21:32 jo kernel: CR2: 000000000074d900 CR3: 0000000100c91000
CR4: 00000000000006e0
Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:21:32 jo kernel: Process md5_raid6 (pid: 14616, threadinfo
ffff88011f46a000, task ffff88011d94acf0)
Oct 2 12:21:32 jo kernel: Stack:
Oct 2 12:21:32 jo kernel: ffff88011e2f0ba8 ffff88006f2e1d40
0000000000000000 ffff8800bbc8baa0
Oct 2 12:21:32 jo kernel: <0> ffff880001883cd0 ffffffff810c836a
ffff880001883d00 ffffffff812ed48a
Oct 2 12:21:32 jo kernel: <0> ffff880001883d00 ffff8800bbc8baa0
ffff88006f2e1d40 0000000000010000
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <IRQ>
Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:21:32 jo kernel: [<ffffffff812ed48a>] 0xffffffff812ed48a
Oct 2 12:21:32 jo kernel: [<ffffffff812ed5ff>] 0xffffffff812ed5ff
Oct 2 12:21:32 jo kernel: [<ffffffff812ed822>] 0xffffffff812ed822
Oct 2 12:21:32 jo kernel: [<ffffffff812ee302>] 0xffffffff812ee302
Oct 2 12:21:32 jo kernel: [<ffffffff812ee372>] 0xffffffff812ee372
Oct 2 12:21:32 jo kernel: [<ffffffff813a67b8>] 0xffffffff813a67b8
Oct 2 12:21:32 jo kernel: [<ffffffff813a0cf0>] 0xffffffff813a0cf0
Oct 2 12:21:32 jo kernel: [<ffffffff813a6b99>] 0xffffffff813a6b99
Oct 2 12:21:32 jo kernel: [<ffffffff812f2adb>] 0xffffffff812f2adb
Oct 2 12:21:32 jo kernel: [<ffffffff81035ecd>] 0xffffffff81035ecd
Oct 2 12:21:32 jo kernel: [<ffffffff81019852>] ? 0xffffffff81019852
Oct 2 12:21:32 jo kernel: [<ffffffff8100374c>] 0xffffffff8100374c
Oct 2 12:21:32 jo kernel: [<ffffffff810052df>] 0xffffffff810052df
Oct 2 12:21:32 jo kernel: [<ffffffff81035bf9>] 0xffffffff81035bf9
Oct 2 12:21:32 jo kernel: [<ffffffff810049c3>] 0xffffffff810049c3
Oct 2 12:21:32 jo kernel: [<ffffffff81471513>] 0xffffffff81471513
Oct 2 12:21:32 jo kernel: <EOI>
Oct 2 12:21:32 jo kernel: [<ffffffffa0446075>] ? 0xffffffffa0446075
Oct 2 12:21:32 jo kernel: [<ffffffffa04440c5>] ? 0xffffffffa04440c5
Oct 2 12:21:32 jo kernel: [<ffffffffa0448795>] 0xffffffffa0448795
Oct 2 12:21:32 jo kernel: [<ffffffffa0448ba4>] 0xffffffffa0448ba4
Oct 2 12:21:32 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
Oct 2 12:21:32 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
Oct 2 12:21:32 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
Oct 2 12:21:32 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: Code: ff 48 8d 34 37 48 39 30 72 b6 e9 d7 fe
ff ff 55 48 89 e5 41 55 41 54 53 31 db 48 83 ec 08 4c 8b 67 58 48 8b 57
18 4d 8b 6c 24 20 <41> 8b 44 24 54 eb 16 48 63 cb 48 69 c9 b0 00 00 00
49 8d 4c 0c
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <IRQ> [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:21:32 jo kernel: [<ffffffff812ed48a>] 0xffffffff812ed48a
Oct 2 12:21:32 jo kernel: [<ffffffff812ed5ff>] 0xffffffff812ed5ff
Oct 2 12:21:32 jo kernel: [<ffffffff812ed822>] 0xffffffff812ed822
Oct 2 12:21:32 jo kernel: [<ffffffff812ee302>] 0xffffffff812ee302
Oct 2 12:21:32 jo kernel: [<ffffffff812ee372>] 0xffffffff812ee372
Oct 2 12:21:32 jo kernel: [<ffffffff813a67b8>] 0xffffffff813a67b8
Oct 2 12:21:32 jo kernel: [<ffffffff813a0cf0>] 0xffffffff813a0cf0
Oct 2 12:21:32 jo kernel: [<ffffffff813a6b99>] 0xffffffff813a6b99
Oct 2 12:21:32 jo kernel: [<ffffffff812f2adb>] 0xffffffff812f2adb
Oct 2 12:21:32 jo kernel: [<ffffffff81035ecd>] 0xffffffff81035ecd
Oct 2 12:21:32 jo kernel: [<ffffffff81019852>] ? 0xffffffff81019852
Oct 2 12:21:32 jo kernel: [<ffffffff8100374c>] 0xffffffff8100374c
Oct 2 12:21:32 jo kernel: [<ffffffff810052df>] 0xffffffff810052df
Oct 2 12:21:32 jo kernel: [<ffffffff81035bf9>] 0xffffffff81035bf9
Oct 2 12:21:32 jo kernel: [<ffffffff810049c3>] 0xffffffff810049c3
Oct 2 12:21:32 jo kernel: [<ffffffff81471513>] 0xffffffff81471513
Oct 2 12:21:32 jo kernel: <EOI> [<ffffffffa0446075>] ? 0xffffffffa0446075
Oct 2 12:21:32 jo kernel: [<ffffffffa04440c5>] ? 0xffffffffa04440c5
Oct 2 12:21:32 jo kernel: [<ffffffffa0448795>] 0xffffffffa0448795
Oct 2 12:21:32 jo kernel: [<ffffffffa0448ba4>] 0xffffffffa0448ba4
Oct 2 12:21:32 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
Oct 2 12:21:32 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
Oct 2 12:21:32 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
Oct 2 12:21:32 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:21:32 jo kernel: [<ffffffff81380635>] ? 0xffffffff81380635
Oct 2 12:21:32 jo kernel: [<ffffffffa0445168>] ? 0xffffffffa0445168
Oct 2 12:21:32 jo kernel: [<ffffffffa0445184>] ? 0xffffffffa0445184
Oct 2 12:21:32 jo kernel: <<EOE>> <IRQ> [<ffffffff810c836a>]
0xffffffff810c836a
Oct 2 12:21:32 jo kernel: [<ffffffff812ed48a>] 0xffffffff812ed48a
Oct 2 12:21:32 jo kernel: [<ffffffff812ed5ff>] 0xffffffff812ed5ff
Oct 2 12:21:32 jo kernel: [<ffffffff812ed822>] 0xffffffff812ed822
Oct 2 12:21:32 jo kernel: [<ffffffff812ee302>] 0xffffffff812ee302
Oct 2 12:21:32 jo kernel: [<ffffffff812ee372>] 0xffffffff812ee372
Oct 2 12:21:32 jo kernel: [<ffffffff813a67b8>] 0xffffffff813a67b8
Oct 2 12:21:32 jo kernel: [<ffffffff813a0cf0>] 0xffffffff813a0cf0
Oct 2 12:21:32 jo kernel: [<ffffffff813a6b99>] 0xffffffff813a6b99
Oct 2 12:21:32 jo kernel: [<ffffffff812f2adb>] 0xffffffff812f2adb
Oct 2 12:21:32 jo kernel: [<ffffffff81035ecd>] 0xffffffff81035ecd
Oct 2 12:21:32 jo kernel: [<ffffffff81019852>] ? 0xffffffff81019852
Oct 2 12:21:32 jo kernel: [<ffffffff8100374c>] 0xffffffff8100374c
Oct 2 12:21:32 jo kernel: [<ffffffff810052df>] 0xffffffff810052df
Oct 2 12:21:32 jo kernel: [<ffffffff81035bf9>] 0xffffffff81035bf9
Oct 2 12:21:32 jo kernel: [<ffffffff810049c3>] 0xffffffff810049c3
Oct 2 12:21:32 jo kernel: [<ffffffff81471513>] 0xffffffff81471513
Oct 2 12:21:32 jo kernel: <EOI> [<ffffffffa0446075>] ? 0xffffffffa0446075
Oct 2 12:21:32 jo kernel: [<ffffffffa04440c5>] ? 0xffffffffa04440c5
Oct 2 12:21:32 jo kernel: [<ffffffffa0448795>] 0xffffffffa0448795
Oct 2 12:21:32 jo kernel: [<ffffffffa0448ba4>] 0xffffffffa0448ba4
Oct 2 12:21:32 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
Oct 2 12:21:32 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
Oct 2 12:21:32 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
Oct 2 12:21:32 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 2
Oct 2 12:21:32 jo kernel: CPU 2
Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:21:32 jo kernel:
Oct 2 12:21:32 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffff81009d68>]
[<ffffffff81009d68>] 0xffffffff81009d68
Oct 2 12:21:32 jo kernel: RSP: 0018:ffff88011fccbf08 EFLAGS: 00000246
Oct 2 12:21:32 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680
RCX: 0000000000000000
Oct 2 12:21:32 jo kernel: RDX: 0000000000000000 RSI: ffff88011fccbfd8
RDI: ffffffff81683308
Oct 2 12:21:32 jo kernel: RBP: ffff88011fccbf08 R08: 0000000000000000
R09: 0000000000000000
Oct 2 12:21:32 jo kernel: R10: 0000000000000000 R11: 00000000ffffffff
R12: ffff88011fccbfd8
Oct 2 12:21:32 jo kernel: R13: 0000000000000000 R14: 0000000000000000
R15: 0000000000000000
Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001900000(0000) knlGS:0000000000000000
Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:21:32 jo kernel: CR2: 0000000000408a40 CR3: 0000000100f82000
CR4: 00000000000006e0
Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:21:32 jo kernel: Process swapper (pid: 0, threadinfo
ffff88011fcca000, task ffff88011fcbecf0)
Oct 2 12:21:32 jo kernel: Stack:
Oct 2 12:21:32 jo kernel: ffff88011fccbf28 ffffffff81001bcf
0000000000000002 0000000000000000
Oct 2 12:21:32 jo kernel: <0> ffff88011fccbf48 ffffffff8146c288
3081aa158d48f824 0000000000000000
Oct 2 12:21:32 jo kernel: <0> 0000000000000000 0000000000000000
0000000000000000 0000000000000000
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:21:32 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:21:32 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86
38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08
31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00
00 48 8b 82
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:21:32 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:21:32 jo kernel: Pid: 0, comm: swapper Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:21:32 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
Oct 2 12:21:32 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
Oct 2 12:21:32 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 3
Oct 2 12:21:32 jo kernel: CPU 3
Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:21:32 jo kernel:
Oct 2 12:21:32 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffff81303c06>]
[<ffffffff81303c06>] 0xffffffff81303c06
Oct 2 12:21:32 jo kernel: RSP: 0018:ffff880001983de0 EFLAGS: 00000803
Oct 2 12:21:32 jo kernel: RAX: 00000000c68de520 RBX: 0000000000000001
RCX: ffff880001980000
Oct 2 12:21:32 jo kernel: RDX: 0000000000249f7d RSI: 0000000000000002
RDI: 0000000000249f7e
Oct 2 12:21:32 jo kernel: RBP: ffff880001983de8 R08: 000000000000001c
R09: 000000000000000a
Oct 2 12:21:32 jo kernel: R10: 0000000000000003 R11: 0000000000000000
R12: ffff88000198e9b0
Oct 2 12:21:32 jo kernel: R13: ffffffff815d3080 R14: ffff88000198e778
R15: ffff880001983f48
Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001980000(0000) knlGS:0000000000000000
Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:21:32 jo kernel: CR2: 000000000060f000 CR3: 00000000015bf000
CR4: 00000000000006e0
Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:21:32 jo kernel: Process btrfs-worker-4 (pid: 22695,
threadinfo ffff88004bc96000, task ffff88011fe04cb0)
Oct 2 12:21:32 jo kernel: Stack:
Oct 2 12:21:32 jo kernel: ffffffff81303b55 ffff880001983df8
ffffffff81303b91 ffff880001983e18
Oct 2 12:21:32 jo kernel: <0> ffffffff81018645 ffff88000198e778
ffffffff815d3080 ffff880001983e68
Oct 2 12:21:32 jo kernel: <0> ffffffff81073786 ffff880001983e78
ffffffff8106f0e2 000000003b9aca00
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <IRQ>
Oct 2 12:21:32 jo kernel: [<ffffffff81303b55>] ? 0xffffffff81303b55
Oct 2 12:21:32 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:21:32 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:21:32 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:21:32 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:21:32 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:21:32 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:21:32 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:21:32 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:21:32 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:21:32 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:21:32 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:21:32 jo kernel: <EOI>
Oct 2 12:21:32 jo kernel: [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:21:32 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:21:32 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:21:32 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:21:32 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:21:32 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:21:32 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:21:32 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: Code: 01 00 65 8b 0c 25 60 d3 00 00 48 63 c9
48 8b 0c cd 00 74 60 81 48 6b 94 0a 98 00 00 00 19 f7 e2 48 8d 7a 01 e8
47 ff ff ff c9 c3 <55> 48 89 e5 65 8b 34 25 60 d3 00 00 0f 1f 00 0f ae
e8 0f 31 89
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <IRQ> [<ffffffff81303b55>] ? 0xffffffff81303b55
Oct 2 12:21:32 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:21:32 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:21:32 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:21:32 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:21:32 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:21:32 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:21:32 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:21:32 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:21:32 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:21:32 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:21:32 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:21:32 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:21:32 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:21:32 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:21:32 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:21:32 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:21:32 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:21:32 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:21:32 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:21:32 jo kernel: [<ffffffff81303c06>] ? 0xffffffff81303c06
Oct 2 12:21:32 jo kernel: <<EOE>> <IRQ> [<ffffffff81303b55>] ?
0xffffffff81303b55
Oct 2 12:21:32 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
Oct 2 12:21:32 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
Oct 2 12:21:32 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
Oct 2 12:21:32 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
Oct 2 12:21:32 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
Oct 2 12:21:32 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
Oct 2 12:21:32 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
Oct 2 12:21:32 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
Oct 2 12:21:32 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
Oct 2 12:21:32 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
Oct 2 12:21:32 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
Oct 2 12:21:32 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
Oct 2 12:21:32 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
Oct 2 12:21:32 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
Oct 2 12:21:32 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
Oct 2 12:21:32 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
Oct 2 12:21:32 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
Oct 2 12:21:32 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
Oct 2 12:21:32 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 0
Oct 2 12:21:32 jo kernel: CPU 0
Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw
iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat
nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string
xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark
xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib
i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e
snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd
snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy
e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov
async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0
dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod
scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd usbhid ohci_hcd ssb
uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx
megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss
3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc
scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport
dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx
scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci
libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via
sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530
pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell
pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti
pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class
pcmcia_core pata_ns87415 pata_ns87410 pata_serverworks pata_platform
pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x
pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys
pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
Oct 2 12:21:32 jo kernel:
Oct 2 12:21:32 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted
2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffff8129587a>]
[<ffffffff8129587a>] 0xffffffff8129587a
Oct 2 12:21:32 jo kernel: RSP: 0018:ffff8800df86b970 EFLAGS: 00000286
Oct 2 12:21:32 jo kernel: RAX: ffff88006d6554e8 RBX: ffff8800bfea4c18
RCX: 0000000000000050
Oct 2 12:21:32 jo kernel: RDX: 0000000000001000 RSI: 0000000003b43000
RDI: ffff8800bfea4c3c
Oct 2 12:21:32 jo kernel: RBP: ffff8800df86b990 R08: ffff8800df86b970
R09: ffff8800df86b968
Oct 2 12:21:32 jo kernel: R10: ffff8800df86b960 R11: 6db6db6db6db6db7
R12: ffff8800bd9d81c0
Oct 2 12:21:32 jo kernel: R13: 0000000003b43000 R14: 00000000000008a5
R15: ffff88007b744f40
Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000)
GS:ffff880001800000(0000) knlGS:0000000000000000
Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
Oct 2 12:21:32 jo kernel: CR2: 00007f18f0b2f000 CR3: 0000000100886000
CR4: 00000000000006f0
Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000
DR2: 0000000000000000
Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0
DR7: 0000000000000400
Oct 2 12:21:32 jo kernel: Process btrfs-endio-wri (pid: 23272,
threadinfo ffff8800df86a000, task ffff88011e0a4090)
Oct 2 12:21:32 jo kernel: Stack:
Oct 2 12:21:32 jo kernel: ffff880054b0c000 ffff8800c0744000
ffff8800df86bad0 0000000000000001
Oct 2 12:21:32 jo kernel: <0> ffff8800df86b9a0 ffffffff812773dd
ffff8800df86ba50 ffffffff812610a5
Oct 2 12:21:32 jo kernel: <0> 00000021df86ba28 ffff8801120b9d90
ffff8800df86badc 00000040df86ba28
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: [<ffffffff812773dd>] 0xffffffff812773dd
Oct 2 12:21:32 jo kernel: [<ffffffff812610a5>] 0xffffffff812610a5
Oct 2 12:21:32 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
Oct 2 12:21:32 jo kernel: [<ffffffff81275697>] 0xffffffff81275697
Oct 2 12:21:32 jo kernel: [<ffffffff812956f6>] ? 0xffffffff812956f6
Oct 2 12:21:32 jo kernel: [<ffffffff81275822>] 0xffffffff81275822
Oct 2 12:21:32 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:21:32 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:21:32 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:21:32 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:21:32 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:21:32 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:21:32 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: Code: c0 41 5c 41 5d 41 5e c9 c3 55 48 89 e5
41 55 49 89 f5 41 54 53 48 89 fb 48 83 ec 08 48 8d 7f 24 e8 12 b9 1d 00
48 8b 43 08 eb 16 <4c> 8d 60 a8 4c 3b 68 a8 73 06 48 8b 40 10 eb 06 76
0e 48 8b 40
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: [<ffffffff812773dd>] 0xffffffff812773dd
Oct 2 12:21:32 jo kernel: [<ffffffff812610a5>] 0xffffffff812610a5
Oct 2 12:21:32 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
Oct 2 12:21:32 jo kernel: [<ffffffff81275697>] 0xffffffff81275697
Oct 2 12:21:32 jo kernel: [<ffffffff812956f6>] ? 0xffffffff812956f6
Oct 2 12:21:32 jo kernel: [<ffffffff81275822>] 0xffffffff81275822
Oct 2 12:21:32 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:21:32 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:21:32 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:21:32 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:21:32 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:21:32 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:21:32 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
Oct 2 12:21:32 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted
2.6.35-gentoo-r9 #1
Oct 2 12:21:32 jo kernel: Call Trace:
Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
Oct 2 12:21:32 jo kernel: [<ffffffff8129587a>] ? 0xffffffff8129587a
Oct 2 12:21:32 jo kernel: <<EOE>> [<ffffffff812773dd>] 0xffffffff812773dd
Oct 2 12:21:32 jo kernel: [<ffffffff812610a5>] 0xffffffff812610a5
Oct 2 12:21:32 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
Oct 2 12:21:32 jo kernel: [<ffffffff81275697>] 0xffffffff81275697
Oct 2 12:21:32 jo kernel: [<ffffffff812956f6>] ? 0xffffffff812956f6
Oct 2 12:21:32 jo kernel: [<ffffffff81275822>] 0xffffffff81275822
Oct 2 12:21:32 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
Oct 2 12:21:32 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
Oct 2 12:21:32 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
Oct 2 12:21:32 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
Oct 2 12:21:32 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
Oct 2 12:21:32 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
Oct 2 12:21:32 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
^ permalink raw reply [flat|nested] 2+ messages in thread
* Re: Kernel crash during btrfs benchmark
2010-10-02 20:26 Kernel crash during btrfs benchmark Bart Kus
@ 2010-10-02 23:49 ` Jérôme Poulin
0 siblings, 0 replies; 2+ messages in thread
From: Jérôme Poulin @ 2010-10-02 23:49 UTC (permalink / raw)
To: Bart Kus; +Cc: linux-btrfs
You didn't include kernel debugging informations and probably didn't
include frame pointers in your Kernel making the report useless.
Sent from my mobile device.
On 2010-10-02, at 16:32, Bart Kus <me@bartk.us> wrote:
> Hello,
>
> On the following system:
>
> Linux jo.bartk.us 2.6.35-gentoo-r9 #1 SMP Sat Oct 2 11:29:36 PDT 2010 x86_64 Intel(R) Core(TM)2 Quad CPU @ 2.40GHz GenuineIntel GNU/Linux
>
> I was running the following command:
>
> dbench -c /usr/share/dbench/client.txt -t 60 64
>
> When everything froze up and console started reporting errors. The client.txt file can be obtained here: http://www.bartk.us/dump/client.txt.gz . dbench version 4.00 was being used. The System.map can be obtained from http://www.bartk.us/dump/System.map-genkernel-x86_64-2.6.35-gentoo-r9 , if that's helpful. The underlying device is an md RAID6 with 11 2TB HDs, connected via sata_sil24 module (SiI3124 chip), which has its 4 eSATA ports connected to 4 SiI3726 port multipliers. I have pulled a btrfs-image off the device and it can be obtained here: http://www.bartk.us/dump/btrfs.image . Here are the details of the md device:
>
> /dev/md5:
> Version : 1.02
> Creation Time : Thu Sep 30 10:35:17 2010
> Raid Level : raid6
> Array Size : 17581629888 (16767.15 GiB 18003.59 GB)
> Used Dev Size : 1953514432 (1863.02 GiB 2000.40 GB)
> Raid Devices : 11
> Total Devices : 11
> Persistence : Superblock is persistent
>
> Update Time : Sat Oct 2 13:03:27 2010
> State : active, resyncing
> Active Devices : 11
> Working Devices : 11
> Failed Devices : 0
> Spare Devices : 0
>
> Chunk Size : 64K
>
> Rebuild Status : 10% complete
>
> Name : jo.bartk.us:5 (local to host jo.bartk.us)
> UUID : 5122c697:58113607:3f5bb07e:a8069132
> Events : 39
>
> Number Major Minor RaidDevice State
> 0 8 64 0 active sync /dev/sde
> 1 8 144 1 active sync /dev/sdj
> 2 8 0 2 active sync /dev/sda
> 3 8 16 3 active sync /dev/sdb
> 4 8 32 4 active sync /dev/sdc
> 5 8 48 5 active sync /dev/sdd
> 6 8 80 6 active sync /dev/sdf
> 7 8 96 7 active sync /dev/sdg
> 8 8 112 8 active sync /dev/sdh
> 9 8 128 9 active sync /dev/sdi
> 10 8 160 10 active sync /dev/sdk
>
> It's resyncing right now due to the crash, but wasn't at time of failure. There are no partitions involved, /dev/md5 was directly formatted with btrfs.
>
> Full text of kernel crash follows:
>
> Oct 2 12:07:52 jo kernel: device fsid a34c8b899171b6f6-fa7e695b5cf51590 devid 1 transid 25 /dev/md5
> Oct 2 12:16:55 jo kernel: INFO: rcu_sched_state detected stall on CPU 3 (t=1000 jiffies)
> Oct 2 12:16:55 jo kernel: sending NMI to all CPUs:
> Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 3
> Oct 2 12:16:55 jo kernel: CPU 3
> Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:16:55 jo kernel:
> Oct 2 12:16:55 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff81303c06>] [<ffffffff81303c06>] 0xffffffff81303c06
> Oct 2 12:16:55 jo kernel: RSP: 0018:ffff880001983de0 EFLAGS: 00000803
> Oct 2 12:16:55 jo kernel: RAX: 00000000c68de520 RBX: 0000000000000001 RCX: ffff880001980000
> Oct 2 12:16:55 jo kernel: RDX: 0000000000249f7d RSI: 0000000000000002 RDI: 0000000000249f7e
> Oct 2 12:16:55 jo kernel: RBP: ffff880001983de8 R08: 000000000000001c R09: 000000000000000a
> Oct 2 12:16:55 jo kernel: R10: 0000000000000003 R11: 0000000000000000 R12: ffff88000198e9b0
> Oct 2 12:16:55 jo kernel: R13: ffffffff815d3080 R14: ffff88000198e778 R15: ffff880001983f48
> Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000) GS:ffff880001980000(0000) knlGS:0000000000000000
> Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:16:55 jo kernel: CR2: 00007f18f0b2f000 CR3: 00000000015bf000 CR4: 00000000000006e0
> Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:16:55 jo kernel: Process btrfs-worker-4 (pid: 22695, threadinfo ffff88004bc96000, task ffff88011fe04cb0)
> Oct 2 12:16:55 jo kernel: Stack:
> Oct 2 12:16:55 jo kernel: ffffffff81303b55 ffff880001983df8 ffffffff81303b91 ffff880001983e18
> Oct 2 12:16:55 jo kernel: <0> ffffffff81018645 ffff88000198e778 ffffffff815d3080 ffff880001983e68
> Oct 2 12:16:55 jo kernel: <0> ffffffff81073786 ffff880001983e78 ffffffff8106f0e2 000000003b9aca00
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: <IRQ>
> Oct 2 12:16:55 jo kernel: [<ffffffff81303b55>] ? 0xffffffff81303b55
> Oct 2 12:16:55 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:16:55 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:16:55 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:16:55 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:16:55 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:16:55 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:16:55 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:16:55 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:16:55 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:16:55 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:16:55 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:16:55 jo kernel: <EOI>
> Oct 2 12:16:55 jo kernel: [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:16:55 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:16:55 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:16:55 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:16:55 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:16:55 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:16:55 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:16:55 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: Code: 01 00 65 8b 0c 25 60 d3 00 00 48 63 c9 48 8b 0c cd 00 74 60 81 48 6b 94 0a 98 00 00 00 19 f7 e2 48 8d 7a 01 e8 47 ff ff ff c9 c3 <55> 48 89 e5 65 8b 34 25 60 d3 00 00 0f 1f 00 0f ae e8 0f 31 89
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: <IRQ> [<ffffffff81303b55>] ? 0xffffffff81303b55
> Oct 2 12:16:55 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:16:55 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:16:55 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:16:55 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:16:55 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:16:55 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:16:55 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:16:55 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:16:55 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:16:55 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:16:55 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:16:55 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:16:55 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:16:55 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:16:55 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:16:55 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:16:55 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:16:55 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:16:55 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:16:55 jo kernel: [<ffffffff81303c06>] ? 0xffffffff81303c06
> Oct 2 12:16:55 jo kernel: <<EOE>> <IRQ> [<ffffffff81303b55>] ? 0xffffffff81303b55
> Oct 2 12:16:55 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:16:55 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:16:55 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:16:55 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:16:55 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:16:55 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:16:55 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:16:55 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:16:55 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:16:55 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:16:55 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:16:55 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:16:55 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:16:55 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:16:55 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:16:55 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:16:55 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:16:55 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:16:55 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 2
> Oct 2 12:16:55 jo kernel: CPU 2
> Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:16:55 jo kernel:
> Oct 2 12:16:55 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff812f7e19>] [<ffffffff812f7e19>] 0xffffffff812f7e19
> Oct 2 12:16:55 jo kernel: RSP: 0018:ffff88011f46bbd0 EFLAGS: 00000082
> Oct 2 12:16:55 jo kernel: RAX: 0000000000000000 RBX: ffff88011e6ad000 RCX: ffff88011e2f03d0
> Oct 2 12:16:55 jo kernel: RDX: 0000000000000001 RSI: ffff88011d350418 RDI: ffff88011e6ad000
> Oct 2 12:16:55 jo kernel: RBP: ffff88011f46bbd0 R08: ffff88011fe1ca58 R09: 0000000000000001
> Oct 2 12:16:55 jo kernel: R10: ffff88011e083938 R11: ffff88011f46bc90 R12: ffff88011d350418
> Oct 2 12:16:55 jo kernel: R13: ffff88011d350418 R14: ffff88011e6ad848 R15: ffff88011ecf21a0
> Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000) GS:ffff880001900000(0000) knlGS:0000000000000000
> Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:16:55 jo kernel: CR2: 00007f18f0b2f000 CR3: 000000011e50e000 CR4: 00000000000006e0
> Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:16:55 jo kernel: Process md5_raid6 (pid: 14616, threadinfo ffff88011f46a000, task ffff88011d94acf0)
> Oct 2 12:16:55 jo kernel: Stack:
> Oct 2 12:16:55 jo kernel: ffff88011f46bc40 ffffffff812fab19 ffff88011f46bc40 ffff8800c8be85c0
> Oct 2 12:16:55 jo kernel: <0> ffff88011e7d0800 ffff88011e238000 ffff88011e083800 ffff8800d735cd30
> Oct 2 12:16:55 jo kernel: <0> ffff88011f46bc50 ffff88011e6ad800 ffff88011e2f1450 ffff88011f46bc58
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: [<ffffffff812fab19>] 0xffffffff812fab19
> Oct 2 12:16:55 jo kernel: [<ffffffff812ee214>] 0xffffffff812ee214
> Oct 2 12:16:55 jo kernel: [<ffffffff812fdda7>] ? 0xffffffff812fdda7
> Oct 2 12:16:55 jo kernel: [<ffffffff813a5a01>] 0xffffffff813a5a01
> Oct 2 12:16:55 jo kernel: [<ffffffff8103af84>] ? 0xffffffff8103af84
> Oct 2 12:16:55 jo kernel: [<ffffffff812eeb94>] 0xffffffff812eeb94
> Oct 2 12:16:55 jo kernel: [<ffffffff812eebc2>] 0xffffffff812eebc2
> Oct 2 12:16:55 jo kernel: [<ffffffff812ece41>] 0xffffffff812ece41
> Oct 2 12:16:55 jo kernel: [<ffffffffa04468b0>] 0xffffffffa04468b0
> Oct 2 12:16:55 jo kernel: [<ffffffffa0448bd0>] 0xffffffffa0448bd0
> Oct 2 12:16:55 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
> Oct 2 12:16:55 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
> Oct 2 12:16:55 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
> Oct 2 12:16:55 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: Code: 00 48 89 e5 b8 02 00 00 00 66 83 fa 03 74 09 31 c0 66 83 fa 01 0f 94 c0 48 8b 96 c8 00 00 00 48 85 d2 75 04 0f 0b eb fe 8b 52 10 <85> d2 75 04 0f 0b eb fe 83 f8 02 74 28 8b 46 04 a8 20 74 15 48
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: [<ffffffff812fab19>] 0xffffffff812fab19
> Oct 2 12:16:55 jo kernel: [<ffffffff812ee214>] 0xffffffff812ee214
> Oct 2 12:16:55 jo kernel: [<ffffffff812fdda7>] ? 0xffffffff812fdda7
> Oct 2 12:16:55 jo kernel: [<ffffffff813a5a01>] 0xffffffff813a5a01
> Oct 2 12:16:55 jo kernel: [<ffffffff8103af84>] ? 0xffffffff8103af84
> Oct 2 12:16:55 jo kernel: [<ffffffff812eeb94>] 0xffffffff812eeb94
> Oct 2 12:16:55 jo kernel: [<ffffffff812eebc2>] 0xffffffff812eebc2
> Oct 2 12:16:55 jo kernel: [<ffffffff812ece41>] 0xffffffff812ece41
> Oct 2 12:16:55 jo kernel: [<ffffffffa04468b0>] 0xffffffffa04468b0
> Oct 2 12:16:55 jo kernel: [<ffffffffa0448bd0>] 0xffffffffa0448bd0
> Oct 2 12:16:55 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
> Oct 2 12:16:55 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
> Oct 2 12:16:55 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
> Oct 2 12:16:55 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:16:55 jo kernel: [<ffffffff812f7e19>] ? 0xffffffff812f7e19
> Oct 2 12:16:55 jo kernel: <<EOE>> [<ffffffff812fab19>] 0xffffffff812fab19
> Oct 2 12:16:55 jo kernel: [<ffffffff812ee214>] 0xffffffff812ee214
> Oct 2 12:16:55 jo kernel: [<ffffffff812fdda7>] ? 0xffffffff812fdda7
> Oct 2 12:16:55 jo kernel: [<ffffffff813a5a01>] 0xffffffff813a5a01
> Oct 2 12:16:55 jo kernel: [<ffffffff8103af84>] ? 0xffffffff8103af84
> Oct 2 12:16:55 jo kernel: [<ffffffff812eeb94>] 0xffffffff812eeb94
> Oct 2 12:16:55 jo kernel: [<ffffffff812eebc2>] 0xffffffff812eebc2
> Oct 2 12:16:55 jo kernel: [<ffffffff812ece41>] 0xffffffff812ece41
> Oct 2 12:16:55 jo kernel: [<ffffffffa04468b0>] 0xffffffffa04468b0
> Oct 2 12:16:55 jo kernel: [<ffffffffa0448bd0>] 0xffffffffa0448bd0
> Oct 2 12:16:55 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
> Oct 2 12:16:55 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
> Oct 2 12:16:55 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
> Oct 2 12:16:55 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 0
> Oct 2 12:16:55 jo kernel: CPU 0
> Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:16:55 jo kernel:
> Oct 2 12:16:55 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff812948b0>] [<ffffffff812948b0>] 0xffffffff812948b0
> Oct 2 12:16:55 jo kernel: RSP: 0018:ffff8800df86ba10 EFLAGS: 00010286
> Oct 2 12:16:55 jo kernel: RAX: ffffea00038a90d8 RBX: ffff8800981032e0 RCX: 000000000000044c
> Oct 2 12:16:55 jo kernel: RDX: ffff8800df86a000 RSI: ffff88010304dbb4 RDI: ffff88010304dbb0
> Oct 2 12:16:55 jo kernel: RBP: ffff8800df86ba80 R08: 00000000000003c8 R09: 0000000000000000
> Oct 2 12:16:55 jo kernel: R10: ffff88010304d3c8 R11: 6db6db6db6db6db7 R12: 00000000000003cc
> Oct 2 12:16:55 jo kernel: R13: 0000000000000c34 R14: ffff8800df86a000 R15: ffffea00038a90d8
> Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000) GS:ffff880001800000(0000) knlGS:0000000000000000
> Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:16:55 jo kernel: CR2: 0000000000749f44 CR3: 000000011acee000 CR4: 00000000000006f0
> Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:16:55 jo kernel: Process btrfs-endio-wri (pid: 23272, threadinfo ffff8800df86a000, task ffff88011e0a4090)
> Oct 2 12:16:55 jo kernel: Stack:
> Oct 2 12:16:55 jo kernel: ffff8800df86bac8 ffff8800df86bac0 ffff8800df86bad0 0000000000000000
> Oct 2 12:16:55 jo kernel: <0> ffff8800df86ba90 00000000000003c8 00000000000003cc 0000000000000c34
> Oct 2 12:16:55 jo kernel: <0> ffff8800981032e0 0000000000000c34 ffff8800981032e0 0000000000000001
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: [<ffffffff8129499b>] 0xffffffff8129499b
> Oct 2 12:16:55 jo kernel: [<ffffffff81294e48>] ? 0xffffffff81294e48
> Oct 2 12:16:55 jo kernel: [<ffffffff81263a3c>] 0xffffffff81263a3c
> Oct 2 12:16:55 jo kernel: [<ffffffff81275acd>] 0xffffffff81275acd
> Oct 2 12:16:55 jo kernel: [<ffffffff8147150e>] ? 0xffffffff8147150e
> Oct 2 12:16:55 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:16:55 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:16:55 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:16:55 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:16:55 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:16:55 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:16:55 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:16:55 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: Code: db b6 6d 48 c1 fe 03 48 0f af f1 48 b9 00 00 00 00 00 88 ff ff 48 c1 e6 0c 48 01 ce 4c 03 55 b8 48 03 75 c0 48 8b 4d c8 4c 89 d7 <f3> a4 ff 4a 1c 4c 39 f8 74 03 ff 4a 1c 4c 03 65 c8 4c 03 45 c8
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: [<ffffffff8129499b>] 0xffffffff8129499b
> Oct 2 12:16:55 jo kernel: [<ffffffff81294e48>] ? 0xffffffff81294e48
> Oct 2 12:16:55 jo kernel: [<ffffffff81263a3c>] 0xffffffff81263a3c
> Oct 2 12:16:55 jo kernel: [<ffffffff81275acd>] 0xffffffff81275acd
> Oct 2 12:16:55 jo kernel: [<ffffffff8147150e>] ? 0xffffffff8147150e
> Oct 2 12:16:55 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:16:55 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:16:55 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:16:55 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:16:55 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:16:55 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:16:55 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:16:55 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:16:55 jo kernel: [<ffffffff812948b0>] ? 0xffffffff812948b0
> Oct 2 12:16:55 jo kernel: <<EOE>> [<ffffffff8129499b>] 0xffffffff8129499b
> Oct 2 12:16:55 jo kernel: [<ffffffff81294e48>] ? 0xffffffff81294e48
> Oct 2 12:16:55 jo kernel: [<ffffffff81263a3c>] 0xffffffff81263a3c
> Oct 2 12:16:55 jo kernel: [<ffffffff81275acd>] 0xffffffff81275acd
> Oct 2 12:16:55 jo kernel: [<ffffffff8147150e>] ? 0xffffffff8147150e
> Oct 2 12:16:55 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:16:55 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:16:55 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:16:55 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:16:55 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:16:55 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:16:55 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:16:55 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:16:55 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:16:55 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:16:55 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:16:55 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:16:55 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:16:55 jo kernel: NMI backtrace for cpu 1
> Oct 2 12:16:55 jo kernel: CPU 1
> Oct 2 12:16:55 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:16:55 jo kernel:
> Oct 2 12:16:55 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:16:55 jo kernel: RIP: 0010:[<ffffffff81009d68>] [<ffffffff81009d68>] 0xffffffff81009d68
> Oct 2 12:16:55 jo kernel: RSP: 0018:ffff88011fcbbf08 EFLAGS: 00000246
> Oct 2 12:16:55 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680 RCX: 0000000000000000
> Oct 2 12:16:55 jo kernel: RDX: 0000000000000000 RSI: ffff88011fcbbfd8 RDI: ffffffff81683308
> Oct 2 12:16:55 jo kernel: RBP: ffff88011fcbbf08 R08: 0000000000000000 R09: 0000000000000078
> Oct 2 12:16:55 jo kernel: R10: ffff88011fe56cf0 R11: 0000000000000003 R12: ffff88011fcbbfd8
> Oct 2 12:16:55 jo kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> Oct 2 12:16:55 jo kernel: FS: 0000000000000000(0000) GS:ffff880001880000(0000) knlGS:0000000000000000
> Oct 2 12:16:55 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:16:55 jo kernel: CR2: 00000000007fd2d0 CR3: 00000001072d0000 CR4: 00000000000006e0
> Oct 2 12:16:55 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:16:55 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:16:55 jo kernel: Process swapper (pid: 0, threadinfo ffff88011fcba000, task ffff88011fcac6a0)
> Oct 2 12:16:55 jo kernel: Stack:
> Oct 2 12:16:55 jo kernel: ffff88011fcbbf28 ffffffff81001bcf 0000000000000001 0000000000000000
> Oct 2 12:16:55 jo kernel: <0> ffff88011fcbbf48 ffffffff8146c288 563dcf676bf8ea76 0000000000000000
> Oct 2 12:16:55 jo kernel: <0> 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:16:55 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:16:55 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86 38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00 00 48 8b 82
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:16:55 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:16:55 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:16:55 jo kernel: Call Trace:
> Oct 2 12:16:55 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:16:55 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:16:55 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:16:55 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:16:55 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
> Oct 2 12:16:55 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:16:55 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:19:15 jo kernel: INFO: rcu_sched_state detected stall on CPU 3 (t=1000 jiffies)
> Oct 2 12:19:15 jo kernel: sending NMI to all CPUs:
> Oct 2 12:19:15 jo kernel: NMI backtrace for cpu 2
> Oct 2 12:19:15 jo kernel: CPU 2
> Oct 2 12:19:15 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:19:15 jo kernel:
> Oct 2 12:19:15 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:19:15 jo kernel: RIP: 0010:[<ffffffff81009d68>] [<ffffffff81009d68>] 0xffffffff81009d68
> Oct 2 12:19:15 jo kernel: RSP: 0018:ffff88011fccbf08 EFLAGS: 00000246
> Oct 2 12:19:15 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680 RCX: 0000000000000000
> Oct 2 12:19:28 jo kernel: RDX: 0000000000000000 RSI: ffff88011fccbfd8 RDI: ffffffff81683308
> Oct 2 12:19:28 jo kernel: RBP: ffff88011fccbf08 R08: 0000000000000000 R09: 0000000000000001
> Oct 2 12:19:28 jo kernel: R10: ffff88011f46bd20 R11: ffff88011e2e3800 R12: ffff88011fccbfd8
> Oct 2 12:19:28 jo kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000) GS:ffff880001900000(0000) knlGS:0000000000000000
> Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:19:28 jo kernel: CR2: 00007f18f0b2f000 CR3: 000000011e50e000 CR4: 00000000000006e0
> Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:19:28 jo kernel: Process swapper (pid: 0, threadinfo ffff88011fcca000, task ffff88011fcbecf0)
> Oct 2 12:19:28 jo kernel: Stack:
> Oct 2 12:19:28 jo kernel: ffff88011fccbf28 ffffffff81001bcf 0000000000000002 0000000000000000
> Oct 2 12:19:28 jo kernel: <0> ffff88011fccbf48 ffffffff8146c288 3081aa158d48f824 0000000000000000
> Oct 2 12:19:28 jo kernel: <0> 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:19:28 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:19:28 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86 38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00 00 48 8b 82
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:19:28 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:19:28 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:19:28 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
> Oct 2 12:19:28 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:19:28 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:19:28 jo kernel: NMI backtrace for cpu 3
> Oct 2 12:19:28 jo kernel: CPU 3
> Oct 2 12:19:28 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:19:28 jo kernel:
> Oct 2 12:19:28 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:19:28 jo kernel: RIP: 0010:[<ffffffff81303c24>] [<ffffffff81303c24>] 0xffffffff81303c24
> Oct 2 12:19:28 jo kernel: RSP: 0018:ffff880001983dd8 EFLAGS: 00000046
> Oct 2 12:19:28 jo kernel: RAX: 00000000dc35449f RBX: 0000000000000001 RCX: 00000000dc3543a3
> Oct 2 12:19:28 jo kernel: RDX: 0000000000000562 RSI: 0000000000000003 RDI: 0000000000249f7e
> Oct 2 12:19:28 jo kernel: RBP: ffff880001983dd8 R08: 0000000000000003 R09: 000000000000000a
> Oct 2 12:19:28 jo kernel: R10: 0000000000000003 R11: 0000000000000000 R12: ffff88000198e9b0
> Oct 2 12:19:28 jo kernel: R13: ffffffff815d3080 R14: ffff88000198e778 R15: ffff880001983f48
> Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000) GS:ffff880001980000(0000) knlGS:0000000000000000
> Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:19:28 jo kernel: CR2: 00007f18f0b2f000 CR3: 00000000015bf000 CR4: 00000000000006e0
> Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:19:28 jo kernel: Process btrfs-worker-4 (pid: 22695, threadinfo ffff88004bc96000, task ffff88011fe04cb0)
> Oct 2 12:19:28 jo kernel: Stack:
> Oct 2 12:19:28 jo kernel: ffff880001983de8 ffffffff81303b55 ffff880001983df8 ffffffff81303b91
> Oct 2 12:19:28 jo kernel: <0> ffff880001983e18 ffffffff81018645 ffff88000198e778 ffffffff815d3080
> Oct 2 12:19:28 jo kernel: <0> ffff880001983e68 ffffffff81073786 ffff880001983e78 ffffffff8106f0e2
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: <IRQ>
> Oct 2 12:19:28 jo kernel: [<ffffffff81303b55>] 0xffffffff81303b55
> Oct 2 12:19:28 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:19:28 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:19:28 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:19:28 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:19:28 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:19:28 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:19:28 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:19:28 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:19:28 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:19:28 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:19:28 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:19:28 jo kernel: <EOI>
> Oct 2 12:19:28 jo kernel: [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:19:28 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:19:28 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:19:28 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:19:28 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:19:28 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:19:28 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:19:28 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:19:28 jo kernel: Code: f7 e2 48 8d 7a 01 e8 47 ff ff ff c9 c3 55 48 89 e5 65 8b 34 25 60 d3 00 00 0f 1f 00 0f ae e8 0f 31 89 c1 0f 1f 00 0f ae e8 0f 31 <89> c0 48 89 c2 48 29 ca 48 39 fa 73 25 f3 90 65 44 8b 04 25 60
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: <IRQ> [<ffffffff81303b55>] 0xffffffff81303b55
> Oct 2 12:19:28 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:19:28 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:19:28 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:19:28 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:19:28 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:19:28 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:19:28 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:19:28 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:19:28 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:19:28 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:19:28 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:19:28 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:19:28 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:19:28 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:19:28 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:19:28 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:19:28 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:19:28 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:19:28 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:19:28 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:19:28 jo kernel: [<ffffffff81303c24>] ? 0xffffffff81303c24
> Oct 2 12:19:28 jo kernel: <<EOE>> <IRQ> [<ffffffff81303b55>] 0xffffffff81303b55
> Oct 2 12:19:28 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:19:28 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:19:28 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:19:28 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:19:28 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:19:28 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:19:28 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:19:28 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:19:28 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:19:28 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:19:28 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:19:28 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:19:28 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:19:28 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:19:28 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:19:28 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:19:28 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:19:28 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:19:28 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:19:28 jo kernel: NMI backtrace for cpu 1
> Oct 2 12:19:28 jo kernel: CPU 1
> Oct 2 12:19:28 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:19:28 jo kernel:
> Oct 2 12:19:28 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:19:28 jo kernel: RIP: 0010:[<ffffffff81295eb7>] [<ffffffff81295eb7>] 0xffffffff81295eb7
> Oct 2 12:19:28 jo kernel: RSP: 0018:ffff8800df86b980 EFLAGS: 00000206
> Oct 2 12:19:28 jo kernel: RAX: ffff88007f62adb8 RBX: ffff8800bfea4c18 RCX: ffff88007f62adb8
> Oct 2 12:19:28 jo kernel: RDX: ffff88007f62ada8 RSI: 0000000001ff4000 RDI: ffff88007f62ae38
> Oct 2 12:19:28 jo kernel: RBP: ffff8800df86b9c0 R08: 0000000000000001 R09: 0000000000000000
> Oct 2 12:19:28 jo kernel: R10: ffff8800df86b9e0 R11: 6db6db6db6db6db7 R12: 0000000001ff4000
> Oct 2 12:19:28 jo kernel: R13: 0000000001ff4fff R14: 0000000000000004 R15: 0000000000000001
> Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000) GS:ffff880001880000(0000) knlGS:0000000000000000
> Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:19:28 jo kernel: CR2: 00007f18f0b2f000 CR3: 00000001072d0000 CR4: 00000000000006e0
> Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:19:28 jo kernel: Process btrfs-endio-wri (pid: 23272, threadinfo ffff8800df86a000, task ffff88011e0a4090)
> Oct 2 12:19:28 jo kernel: Stack:
> Oct 2 12:19:28 jo kernel: 0000000000000000 0000000000000000 0000000000000dee ffff8800454cfd00
> Oct 2 12:19:28 jo kernel: <0> ffff8800bfea4c18 0000000000000028 0000000000000dee ffff8800690747f0
> Oct 2 12:19:28 jo kernel: <0> ffff8800df86b9f0 ffffffff81295fff ffff8800454cfd00 ffff8800bfea4c18
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: [<ffffffff81295fff>] 0xffffffff81295fff
> Oct 2 12:19:28 jo kernel: [<ffffffff81277803>] 0xffffffff81277803
> Oct 2 12:19:28 jo kernel: [<ffffffff812610c0>] 0xffffffff812610c0
> Oct 2 12:19:28 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
> Oct 2 12:19:28 jo kernel: [<ffffffff81275988>] 0xffffffff81275988
> Oct 2 12:19:28 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:19:28 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:19:28 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:19:28 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:19:28 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:19:28 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:19:28 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:19:28 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:19:28 jo kernel: Code: c9 74 17 49 83 79 28 00 74 10 49 8b 11 49 8d 41 10 4c 39 e2 0f 84 b9 00 00 00 48 8b 03 31 d2 31 ff eb 24 48 8d 50 f0 4c 3b 60 f0 <73> 06 48 8b 48 10 eb 0e 4c 3b 60 f8 0f 86 93 00 00 00 48 8b 48
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: [<ffffffff81295fff>] 0xffffffff81295fff
> Oct 2 12:19:28 jo kernel: [<ffffffff81277803>] 0xffffffff81277803
> Oct 2 12:19:28 jo kernel: [<ffffffff812610c0>] 0xffffffff812610c0
> Oct 2 12:19:28 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
> Oct 2 12:19:28 jo kernel: [<ffffffff81275988>] 0xffffffff81275988
> Oct 2 12:19:28 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:19:28 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:19:28 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:19:28 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:19:28 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:19:28 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:19:28 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:19:28 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:19:28 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:19:28 jo kernel: [<ffffffff81295eb7>] ? 0xffffffff81295eb7
> Oct 2 12:19:28 jo kernel: <<EOE>> [<ffffffff81295fff>] 0xffffffff81295fff
> Oct 2 12:19:28 jo kernel: [<ffffffff81277803>] 0xffffffff81277803
> Oct 2 12:19:28 jo kernel: [<ffffffff812610c0>] 0xffffffff812610c0
> Oct 2 12:19:28 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
> Oct 2 12:19:28 jo kernel: [<ffffffff81275988>] 0xffffffff81275988
> Oct 2 12:19:28 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:19:28 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:19:28 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:19:28 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:19:28 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:19:28 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:19:28 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:19:28 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:19:28 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:19:28 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:19:28 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:19:28 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:19:28 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:19:28 jo kernel: NMI backtrace for cpu 0
> Oct 2 12:19:28 jo kernel: CPU 0
> Oct 2 12:19:28 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:19:28 jo kernel:
> Oct 2 12:19:28 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:19:28 jo kernel: RIP: 0010:[<ffffffff81009d68>] [<ffffffff81009d68>] 0xffffffff81009d68
> Oct 2 12:19:28 jo kernel: RSP: 0018:ffffffff81581f28 EFLAGS: 00000246
> Oct 2 12:19:28 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680 RCX: 0000000000000000
> Oct 2 12:19:28 jo kernel: RDX: 0000000000000000 RSI: ffffffff81581fd8 RDI: ffffffff81683308
> Oct 2 12:19:28 jo kernel: RBP: ffffffff81581f28 R08: 0000000000000000 R09: 0000000000000078
> Oct 2 12:19:28 jo kernel: R10: ffff88011fe56cf0 R11: 0000000000000003 R12: ffffffff81581fd8
> Oct 2 12:19:28 jo kernel: R13: 0000000000000000 R14: ffffffffffffffff R15: 0000000000000000
> Oct 2 12:19:28 jo kernel: FS: 0000000000000000(0000) GS:ffff880001800000(0000) knlGS:0000000000000000
> Oct 2 12:19:28 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:19:28 jo kernel: CR2: 00007f971b171000 CR3: 00000001072d0000 CR4: 00000000000006f0
> Oct 2 12:19:28 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:19:28 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:19:28 jo kernel: Process swapper (pid: 0, threadinfo ffffffff81580000, task ffffffff815c7020)
> Oct 2 12:19:28 jo kernel: Stack:
> Oct 2 12:19:28 jo kernel: ffffffff81581f48 ffffffff81001bcf 0000000000000000 6db6db6db6db6db7
> Oct 2 12:19:28 jo kernel: <0> ffffffff81581f58 ffffffff8145e61c ffffffff81581f98 ffffffff8161eccb
> Oct 2 12:19:28 jo kernel: <0> ffffffff81581f98 ffffffff81654060 0000000001749dc4 0000000000000000
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:19:28 jo kernel: [<ffffffff8145e61c>] 0xffffffff8145e61c
> Oct 2 12:19:28 jo kernel: [<ffffffff8161eccb>] 0xffffffff8161eccb
> Oct 2 12:19:28 jo kernel: [<ffffffff8161e293>] 0xffffffff8161e293
> Oct 2 12:19:28 jo kernel: [<ffffffff8161e37b>] 0xffffffff8161e37b
> Oct 2 12:19:28 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86 38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00 00 48 8b 82
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:19:28 jo kernel: [<ffffffff8145e61c>] 0xffffffff8145e61c
> Oct 2 12:19:28 jo kernel: [<ffffffff8161eccb>] 0xffffffff8161eccb
> Oct 2 12:19:28 jo kernel: [<ffffffff8161e293>] 0xffffffff8161e293
> Oct 2 12:19:28 jo kernel: [<ffffffff8161e37b>] 0xffffffff8161e37b
> Oct 2 12:19:28 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:19:28 jo kernel: Call Trace:
> Oct 2 12:19:28 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:19:28 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:19:28 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:19:28 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:19:28 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
> Oct 2 12:19:28 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:19:28 jo kernel: [<ffffffff8145e61c>] 0xffffffff8145e61c
> Oct 2 12:19:28 jo kernel: [<ffffffff8161eccb>] 0xffffffff8161eccb
> Oct 2 12:19:28 jo kernel: [<ffffffff8161e293>] 0xffffffff8161e293
> Oct 2 12:19:28 jo kernel: [<ffffffff8161e37b>] 0xffffffff8161e37b
> Oct 2 12:21:32 jo kernel: INFO: rcu_sched_state detected stall on CPU 3 (t=1000 jiffies)
> Oct 2 12:21:32 jo kernel: sending NMI to all CPUs:
> Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 1
> Oct 2 12:21:32 jo kernel: CPU 1
> Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:21:32 jo kernel:
> Oct 2 12:21:32 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffffa0445184>] [<ffffffffa0445184>] 0xffffffffa0445184
> Oct 2 12:21:32 jo kernel: RSP: 0018:ffff880001883ca0 EFLAGS: 00000286
> Oct 2 12:21:32 jo kernel: RAX: ffffffffa0445168 RBX: 0000000000000000 RCX: 0000000000001000
> Oct 2 12:21:32 jo kernel: RDX: 0000000000000009 RSI: 0000000000000000 RDI: ffff88006f2e1d40
> Oct 2 12:21:32 jo kernel: RBP: ffff880001883cc0 R08: ffff88011e04e4e8 R09: 0000000000000bb8
> Oct 2 12:21:32 jo kernel: R10: 0000000000000001 R11: ffffffff81380635 R12: ffff88006f2e1800
> Oct 2 12:21:32 jo kernel: R13: ffff88011edbce00 R14: 0000000000000000 R15: 0000000000000000
> Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000) GS:ffff880001880000(0000) knlGS:0000000000000000
> Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:21:32 jo kernel: CR2: 000000000074d900 CR3: 0000000100c91000 CR4: 00000000000006e0
> Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:21:32 jo kernel: Process md5_raid6 (pid: 14616, threadinfo ffff88011f46a000, task ffff88011d94acf0)
> Oct 2 12:21:32 jo kernel: Stack:
> Oct 2 12:21:32 jo kernel: ffff88011e2f0ba8 ffff88006f2e1d40 0000000000000000 ffff8800bbc8baa0
> Oct 2 12:21:32 jo kernel: <0> ffff880001883cd0 ffffffff810c836a ffff880001883d00 ffffffff812ed48a
> Oct 2 12:21:32 jo kernel: <0> ffff880001883d00 ffff8800bbc8baa0 ffff88006f2e1d40 0000000000010000
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <IRQ>
> Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed48a>] 0xffffffff812ed48a
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed5ff>] 0xffffffff812ed5ff
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed822>] 0xffffffff812ed822
> Oct 2 12:21:32 jo kernel: [<ffffffff812ee302>] 0xffffffff812ee302
> Oct 2 12:21:32 jo kernel: [<ffffffff812ee372>] 0xffffffff812ee372
> Oct 2 12:21:32 jo kernel: [<ffffffff813a67b8>] 0xffffffff813a67b8
> Oct 2 12:21:32 jo kernel: [<ffffffff813a0cf0>] 0xffffffff813a0cf0
> Oct 2 12:21:32 jo kernel: [<ffffffff813a6b99>] 0xffffffff813a6b99
> Oct 2 12:21:32 jo kernel: [<ffffffff812f2adb>] 0xffffffff812f2adb
> Oct 2 12:21:32 jo kernel: [<ffffffff81035ecd>] 0xffffffff81035ecd
> Oct 2 12:21:32 jo kernel: [<ffffffff81019852>] ? 0xffffffff81019852
> Oct 2 12:21:32 jo kernel: [<ffffffff8100374c>] 0xffffffff8100374c
> Oct 2 12:21:32 jo kernel: [<ffffffff810052df>] 0xffffffff810052df
> Oct 2 12:21:32 jo kernel: [<ffffffff81035bf9>] 0xffffffff81035bf9
> Oct 2 12:21:32 jo kernel: [<ffffffff810049c3>] 0xffffffff810049c3
> Oct 2 12:21:32 jo kernel: [<ffffffff81471513>] 0xffffffff81471513
> Oct 2 12:21:32 jo kernel: <EOI>
> Oct 2 12:21:32 jo kernel: [<ffffffffa0446075>] ? 0xffffffffa0446075
> Oct 2 12:21:32 jo kernel: [<ffffffffa04440c5>] ? 0xffffffffa04440c5
> Oct 2 12:21:32 jo kernel: [<ffffffffa0448795>] 0xffffffffa0448795
> Oct 2 12:21:32 jo kernel: [<ffffffffa0448ba4>] 0xffffffffa0448ba4
> Oct 2 12:21:32 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
> Oct 2 12:21:32 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
> Oct 2 12:21:32 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
> Oct 2 12:21:32 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: Code: ff 48 8d 34 37 48 39 30 72 b6 e9 d7 fe ff ff 55 48 89 e5 41 55 41 54 53 31 db 48 83 ec 08 4c 8b 67 58 48 8b 57 18 4d 8b 6c 24 20 <41> 8b 44 24 54 eb 16 48 63 cb 48 69 c9 b0 00 00 00 49 8d 4c 0c
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <IRQ> [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed48a>] 0xffffffff812ed48a
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed5ff>] 0xffffffff812ed5ff
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed822>] 0xffffffff812ed822
> Oct 2 12:21:32 jo kernel: [<ffffffff812ee302>] 0xffffffff812ee302
> Oct 2 12:21:32 jo kernel: [<ffffffff812ee372>] 0xffffffff812ee372
> Oct 2 12:21:32 jo kernel: [<ffffffff813a67b8>] 0xffffffff813a67b8
> Oct 2 12:21:32 jo kernel: [<ffffffff813a0cf0>] 0xffffffff813a0cf0
> Oct 2 12:21:32 jo kernel: [<ffffffff813a6b99>] 0xffffffff813a6b99
> Oct 2 12:21:32 jo kernel: [<ffffffff812f2adb>] 0xffffffff812f2adb
> Oct 2 12:21:32 jo kernel: [<ffffffff81035ecd>] 0xffffffff81035ecd
> Oct 2 12:21:32 jo kernel: [<ffffffff81019852>] ? 0xffffffff81019852
> Oct 2 12:21:32 jo kernel: [<ffffffff8100374c>] 0xffffffff8100374c
> Oct 2 12:21:32 jo kernel: [<ffffffff810052df>] 0xffffffff810052df
> Oct 2 12:21:32 jo kernel: [<ffffffff81035bf9>] 0xffffffff81035bf9
> Oct 2 12:21:32 jo kernel: [<ffffffff810049c3>] 0xffffffff810049c3
> Oct 2 12:21:32 jo kernel: [<ffffffff81471513>] 0xffffffff81471513
> Oct 2 12:21:32 jo kernel: <EOI> [<ffffffffa0446075>] ? 0xffffffffa0446075
> Oct 2 12:21:32 jo kernel: [<ffffffffa04440c5>] ? 0xffffffffa04440c5
> Oct 2 12:21:32 jo kernel: [<ffffffffa0448795>] 0xffffffffa0448795
> Oct 2 12:21:32 jo kernel: [<ffffffffa0448ba4>] 0xffffffffa0448ba4
> Oct 2 12:21:32 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
> Oct 2 12:21:32 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
> Oct 2 12:21:32 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
> Oct 2 12:21:32 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: Pid: 14616, comm: md5_raid6 Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:21:32 jo kernel: [<ffffffff81380635>] ? 0xffffffff81380635
> Oct 2 12:21:32 jo kernel: [<ffffffffa0445168>] ? 0xffffffffa0445168
> Oct 2 12:21:32 jo kernel: [<ffffffffa0445184>] ? 0xffffffffa0445184
> Oct 2 12:21:32 jo kernel: <<EOE>> <IRQ> [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed48a>] 0xffffffff812ed48a
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed5ff>] 0xffffffff812ed5ff
> Oct 2 12:21:32 jo kernel: [<ffffffff812ed822>] 0xffffffff812ed822
> Oct 2 12:21:32 jo kernel: [<ffffffff812ee302>] 0xffffffff812ee302
> Oct 2 12:21:32 jo kernel: [<ffffffff812ee372>] 0xffffffff812ee372
> Oct 2 12:21:32 jo kernel: [<ffffffff813a67b8>] 0xffffffff813a67b8
> Oct 2 12:21:32 jo kernel: [<ffffffff813a0cf0>] 0xffffffff813a0cf0
> Oct 2 12:21:32 jo kernel: [<ffffffff813a6b99>] 0xffffffff813a6b99
> Oct 2 12:21:32 jo kernel: [<ffffffff812f2adb>] 0xffffffff812f2adb
> Oct 2 12:21:32 jo kernel: [<ffffffff81035ecd>] 0xffffffff81035ecd
> Oct 2 12:21:32 jo kernel: [<ffffffff81019852>] ? 0xffffffff81019852
> Oct 2 12:21:32 jo kernel: [<ffffffff8100374c>] 0xffffffff8100374c
> Oct 2 12:21:32 jo kernel: [<ffffffff810052df>] 0xffffffff810052df
> Oct 2 12:21:32 jo kernel: [<ffffffff81035bf9>] 0xffffffff81035bf9
> Oct 2 12:21:32 jo kernel: [<ffffffff810049c3>] 0xffffffff810049c3
> Oct 2 12:21:32 jo kernel: [<ffffffff81471513>] 0xffffffff81471513
> Oct 2 12:21:32 jo kernel: <EOI> [<ffffffffa0446075>] ? 0xffffffffa0446075
> Oct 2 12:21:32 jo kernel: [<ffffffffa04440c5>] ? 0xffffffffa04440c5
> Oct 2 12:21:32 jo kernel: [<ffffffffa0448795>] 0xffffffffa0448795
> Oct 2 12:21:32 jo kernel: [<ffffffffa0448ba4>] 0xffffffffa0448ba4
> Oct 2 12:21:32 jo kernel: [<ffffffff8146f884>] ? 0xffffffff8146f884
> Oct 2 12:21:32 jo kernel: [<ffffffff813cd2c3>] 0xffffffff813cd2c3
> Oct 2 12:21:32 jo kernel: [<ffffffff810443b5>] ? 0xffffffff810443b5
> Oct 2 12:21:32 jo kernel: [<ffffffff813cd1d2>] ? 0xffffffff813cd1d2
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 2
> Oct 2 12:21:32 jo kernel: CPU 2
> Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:21:32 jo kernel:
> Oct 2 12:21:32 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffff81009d68>] [<ffffffff81009d68>] 0xffffffff81009d68
> Oct 2 12:21:32 jo kernel: RSP: 0018:ffff88011fccbf08 EFLAGS: 00000246
> Oct 2 12:21:32 jo kernel: RAX: 0000000000000000 RBX: ffffffff81607680 RCX: 0000000000000000
> Oct 2 12:21:32 jo kernel: RDX: 0000000000000000 RSI: ffff88011fccbfd8 RDI: ffffffff81683308
> Oct 2 12:21:32 jo kernel: RBP: ffff88011fccbf08 R08: 0000000000000000 R09: 0000000000000000
> Oct 2 12:21:32 jo kernel: R10: 0000000000000000 R11: 00000000ffffffff R12: ffff88011fccbfd8
> Oct 2 12:21:32 jo kernel: R13: 0000000000000000 R14: 0000000000000000 R15: 0000000000000000
> Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000) GS:ffff880001900000(0000) knlGS:0000000000000000
> Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:21:32 jo kernel: CR2: 0000000000408a40 CR3: 0000000100f82000 CR4: 00000000000006e0
> Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:21:32 jo kernel: Process swapper (pid: 0, threadinfo ffff88011fcca000, task ffff88011fcbecf0)
> Oct 2 12:21:32 jo kernel: Stack:
> Oct 2 12:21:32 jo kernel: ffff88011fccbf28 ffffffff81001bcf 0000000000000002 0000000000000000
> Oct 2 12:21:32 jo kernel: <0> ffff88011fccbf48 ffffffff8146c288 3081aa158d48f824 0000000000000000
> Oct 2 12:21:32 jo kernel: <0> 0000000000000000 0000000000000000 0000000000000000 0000000000000000
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:21:32 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:21:32 jo kernel: Code: 48 8b 34 25 48 b5 00 00 31 d2 48 8d 86 38 e0 ff ff 48 89 d1 0f 01 c8 0f ae f0 48 8b 86 38 e0 ff ff a8 08 75 08 31 c0 fb 0f 01 c9 <eb> 01 fb c9 c3 55 48 89 e5 65 48 8b 14 25 48 b5 00 00 48 8b 82
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:21:32 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:21:32 jo kernel: Pid: 0, comm: swapper Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:21:32 jo kernel: [<ffffffff81009d68>] ? 0xffffffff81009d68
> Oct 2 12:21:32 jo kernel: <<EOE>> [<ffffffff81001bcf>] 0xffffffff81001bcf
> Oct 2 12:21:32 jo kernel: [<ffffffff8146c288>] 0xffffffff8146c288
> Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 3
> Oct 2 12:21:32 jo kernel: CPU 3
> Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:21:32 jo kernel:
> Oct 2 12:21:32 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffff81303c06>] [<ffffffff81303c06>] 0xffffffff81303c06
> Oct 2 12:21:32 jo kernel: RSP: 0018:ffff880001983de0 EFLAGS: 00000803
> Oct 2 12:21:32 jo kernel: RAX: 00000000c68de520 RBX: 0000000000000001 RCX: ffff880001980000
> Oct 2 12:21:32 jo kernel: RDX: 0000000000249f7d RSI: 0000000000000002 RDI: 0000000000249f7e
> Oct 2 12:21:32 jo kernel: RBP: ffff880001983de8 R08: 000000000000001c R09: 000000000000000a
> Oct 2 12:21:32 jo kernel: R10: 0000000000000003 R11: 0000000000000000 R12: ffff88000198e9b0
> Oct 2 12:21:32 jo kernel: R13: ffffffff815d3080 R14: ffff88000198e778 R15: ffff880001983f48
> Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000) GS:ffff880001980000(0000) knlGS:0000000000000000
> Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:21:32 jo kernel: CR2: 000000000060f000 CR3: 00000000015bf000 CR4: 00000000000006e0
> Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:21:32 jo kernel: Process btrfs-worker-4 (pid: 22695, threadinfo ffff88004bc96000, task ffff88011fe04cb0)
> Oct 2 12:21:32 jo kernel: Stack:
> Oct 2 12:21:32 jo kernel: ffffffff81303b55 ffff880001983df8 ffffffff81303b91 ffff880001983e18
> Oct 2 12:21:32 jo kernel: <0> ffffffff81018645 ffff88000198e778 ffffffff815d3080 ffff880001983e68
> Oct 2 12:21:32 jo kernel: <0> ffffffff81073786 ffff880001983e78 ffffffff8106f0e2 000000003b9aca00
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <IRQ>
> Oct 2 12:21:32 jo kernel: [<ffffffff81303b55>] ? 0xffffffff81303b55
> Oct 2 12:21:32 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:21:32 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:21:32 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:21:32 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:21:32 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:21:32 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:21:32 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:21:32 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:21:32 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:21:32 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:21:32 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:21:32 jo kernel: <EOI>
> Oct 2 12:21:32 jo kernel: [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:21:32 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:21:32 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:21:32 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:21:32 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:21:32 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:21:32 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:21:32 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: Code: 01 00 65 8b 0c 25 60 d3 00 00 48 63 c9 48 8b 0c cd 00 74 60 81 48 6b 94 0a 98 00 00 00 19 f7 e2 48 8d 7a 01 e8 47 ff ff ff c9 c3 <55> 48 89 e5 65 8b 34 25 60 d3 00 00 0f 1f 00 0f ae e8 0f 31 89
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <IRQ> [<ffffffff81303b55>] ? 0xffffffff81303b55
> Oct 2 12:21:32 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:21:32 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:21:32 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:21:32 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:21:32 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:21:32 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:21:32 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:21:32 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:21:32 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:21:32 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:21:32 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:21:32 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:21:32 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:21:32 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:21:32 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:21:32 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:21:32 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:21:32 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:21:32 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: Pid: 22695, comm: btrfs-worker-4 Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:21:32 jo kernel: [<ffffffff81303c06>] ? 0xffffffff81303c06
> Oct 2 12:21:32 jo kernel: <<EOE>> <IRQ> [<ffffffff81303b55>] ? 0xffffffff81303b55
> Oct 2 12:21:32 jo kernel: [<ffffffff81303b91>] 0xffffffff81303b91
> Oct 2 12:21:32 jo kernel: [<ffffffff81018645>] 0xffffffff81018645
> Oct 2 12:21:32 jo kernel: [<ffffffff81073786>] 0xffffffff81073786
> Oct 2 12:21:32 jo kernel: [<ffffffff8106f0e2>] ? 0xffffffff8106f0e2
> Oct 2 12:21:32 jo kernel: [<ffffffff81073adc>] 0xffffffff81073adc
> Oct 2 12:21:32 jo kernel: [<ffffffff8103b064>] 0xffffffff8103b064
> Oct 2 12:21:32 jo kernel: [<ffffffff8104efb5>] 0xffffffff8104efb5
> Oct 2 12:21:32 jo kernel: [<ffffffff81046ba0>] 0xffffffff81046ba0
> Oct 2 12:21:32 jo kernel: [<ffffffff81046dea>] 0xffffffff81046dea
> Oct 2 12:21:32 jo kernel: [<ffffffff81018044>] 0xffffffff81018044
> Oct 2 12:21:32 jo kernel: [<ffffffff81003213>] 0xffffffff81003213
> Oct 2 12:21:32 jo kernel: <EOI> [<ffffffff812eb6a0>] ? 0xffffffff812eb6a0
> Oct 2 12:21:32 jo kernel: [<ffffffff812e5942>] 0xffffffff812e5942
> Oct 2 12:21:32 jo kernel: [<ffffffff81308097>] 0xffffffff81308097
> Oct 2 12:21:32 jo kernel: [<ffffffff812931d2>] ? 0xffffffff812931d2
> Oct 2 12:21:32 jo kernel: [<ffffffff81279494>] 0xffffffff81279494
> Oct 2 12:21:32 jo kernel: [<ffffffff8127640f>] 0xffffffff8127640f
> Oct 2 12:21:32 jo kernel: [<ffffffff8127f048>] 0xffffffff8127f048
> Oct 2 12:21:32 jo kernel: [<ffffffff812773b6>] 0xffffffff812773b6
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: NMI backtrace for cpu 0
> Oct 2 12:21:32 jo kernel: CPU 0
> Oct 2 12:21:32 jo kernel: Modules linked in: tun ipv6 8021q iptable_raw iptable_mangle iptable_filter ipt_MASQUERADE xt_tcpudp iptable_nat nf_nat nf_conntrack_ipv4 nf_defrag_ipv4 ipt_addrtype xt_dscp xt_string xt_multiport xt_hashlimit xt_conntrack xt_DSCP xt_NFQUEUE xt_mark xt_connmark nf_conntrack ip_tables x_tables rtc_cmos rtc_core rtc_lib i2c_i801 i2c_core pcspkr processor snd_hda_codec_via snd_hda_intel atl1e snd_hda_codec thermal thermal_sys snd_hwdep snd_pcm snd_timer snd snd_page_alloc intel_agp button libiscsi scsi_transport_iscsi tg3 libphy e1000 fuse nfs lockd sunrpc jfs raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor xor async_tx raid6_pq raid1 raid0 dm_snapshot dm_crypt dm_mirror dm_region_hash dm_log dm_mod scsi_wait_scan sbp2 ohci1394 ieee1394 sl811_hcd
usbhid ohci_hcd ssb uhci_hcd usb_storage ehci_hcd usbcore aic94xx libsas lpfc qla2xxx megaraid_sas megaraid_mbox megaraid_mm megaraid aacraid sx8 DAC960 cciss 3w_9xxx 3w_xxxx mptsas scsi_transport_sas mptfc scsi_transport_fc scsi_tgt mptspi mptscsih mptbase atp870u dc395x qla1280 imm parport dmx3191d sym53c8xx gdth advansys initio BusLogic arcmsr aic7xxx aic79xx scsi_transport_spi sg pdc_adma sata_inic162x sata_mv ata_piix ahci libahci sata_qstor sata_vsc sata_uli sata_sis sata_sx4 sata_nv sata_via sata_svw sata_sil24 sata_sil sata_promise pata_sl82c105 pata_cs5530 pata_cs5520 pata_via pata_jmicron pata_marvell pata_sis pata_netcell pata_sc1200 pata_pdc202xx_old pata_triflex pata_atiixp pata_opti pata_amd pata_ali pata_it8213 pata_pcmcia pcmcia firmware_class pcmcia_core pata_ns87415 pat
a_ns87410 pata_serverworks pata_platform pata_artop pata_it821x pata_optidma pata_hpt3x2n pata_hpt3x3 pata_hpt37x pata_hpt366 pata_cmd64x pata_efar pata_rz1000 pata_sil680 pata_radisys pata_pdc2027x pata_mpiix libata [last unloaded: microcode]
> Oct 2 12:21:32 jo kernel:
> Oct 2 12:21:32 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted 2.6.35-gentoo-r9 #1 P5Q PRO TURBO/System Product Name
> Oct 2 12:21:32 jo kernel: RIP: 0010:[<ffffffff8129587a>] [<ffffffff8129587a>] 0xffffffff8129587a
> Oct 2 12:21:32 jo kernel: RSP: 0018:ffff8800df86b970 EFLAGS: 00000286
> Oct 2 12:21:32 jo kernel: RAX: ffff88006d6554e8 RBX: ffff8800bfea4c18 RCX: 0000000000000050
> Oct 2 12:21:32 jo kernel: RDX: 0000000000001000 RSI: 0000000003b43000 RDI: ffff8800bfea4c3c
> Oct 2 12:21:32 jo kernel: RBP: ffff8800df86b990 R08: ffff8800df86b970 R09: ffff8800df86b968
> Oct 2 12:21:32 jo kernel: R10: ffff8800df86b960 R11: 6db6db6db6db6db7 R12: ffff8800bd9d81c0
> Oct 2 12:21:32 jo kernel: R13: 0000000003b43000 R14: 00000000000008a5 R15: ffff88007b744f40
> Oct 2 12:21:32 jo kernel: FS: 0000000000000000(0000) GS:ffff880001800000(0000) knlGS:0000000000000000
> Oct 2 12:21:32 jo kernel: CS: 0010 DS: 0000 ES: 0000 CR0: 000000008005003b
> Oct 2 12:21:32 jo kernel: CR2: 00007f18f0b2f000 CR3: 0000000100886000 CR4: 00000000000006f0
> Oct 2 12:21:32 jo kernel: DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
> Oct 2 12:21:32 jo kernel: DR3: 0000000000000000 DR6: 00000000ffff0ff0 DR7: 0000000000000400
> Oct 2 12:21:32 jo kernel: Process btrfs-endio-wri (pid: 23272, threadinfo ffff8800df86a000, task ffff88011e0a4090)
> Oct 2 12:21:32 jo kernel: Stack:
> Oct 2 12:21:32 jo kernel: ffff880054b0c000 ffff8800c0744000 ffff8800df86bad0 0000000000000001
> Oct 2 12:21:32 jo kernel: <0> ffff8800df86b9a0 ffffffff812773dd ffff8800df86ba50 ffffffff812610a5
> Oct 2 12:21:32 jo kernel: <0> 00000021df86ba28 ffff8801120b9d90 ffff8800df86badc 00000040df86ba28
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: [<ffffffff812773dd>] 0xffffffff812773dd
> Oct 2 12:21:32 jo kernel: [<ffffffff812610a5>] 0xffffffff812610a5
> Oct 2 12:21:32 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
> Oct 2 12:21:32 jo kernel: [<ffffffff81275697>] 0xffffffff81275697
> Oct 2 12:21:32 jo kernel: [<ffffffff812956f6>] ? 0xffffffff812956f6
> Oct 2 12:21:32 jo kernel: [<ffffffff81275822>] 0xffffffff81275822
> Oct 2 12:21:32 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:21:32 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:21:32 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:21:32 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:21:32 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:21:32 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:21:32 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: Code: c0 41 5c 41 5d 41 5e c9 c3 55 48 89 e5 41 55 49 89 f5 41 54 53 48 89 fb 48 83 ec 08 48 8d 7f 24 e8 12 b9 1d 00 48 8b 43 08 eb 16 <4c> 8d 60 a8 4c 3b 68 a8 73 06 48 8b 40 10 eb 06 76 0e 48 8b 40
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: [<ffffffff812773dd>] 0xffffffff812773dd
> Oct 2 12:21:32 jo kernel: [<ffffffff812610a5>] 0xffffffff812610a5
> Oct 2 12:21:32 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
> Oct 2 12:21:32 jo kernel: [<ffffffff81275697>] 0xffffffff81275697
> Oct 2 12:21:32 jo kernel: [<ffffffff812956f6>] ? 0xffffffff812956f6
> Oct 2 12:21:32 jo kernel: [<ffffffff81275822>] 0xffffffff81275822
> Oct 2 12:21:32 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:21:32 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:21:32 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:21:32 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:21:32 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:21:32 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:21:32 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
> Oct 2 12:21:32 jo kernel: Pid: 23272, comm: btrfs-endio-wri Not tainted 2.6.35-gentoo-r9 #1
> Oct 2 12:21:32 jo kernel: Call Trace:
> Oct 2 12:21:32 jo kernel: <NMI> [<ffffffff8100a31c>] ? 0xffffffff8100a31c
> Oct 2 12:21:32 jo kernel: [<ffffffff810187b7>] 0xffffffff810187b7
> Oct 2 12:21:32 jo kernel: [<ffffffff8100446e>] 0xffffffff8100446e
> Oct 2 12:21:32 jo kernel: [<ffffffff8147197a>] 0xffffffff8147197a
> Oct 2 12:21:32 jo kernel: [<ffffffff8129587a>] ? 0xffffffff8129587a
> Oct 2 12:21:32 jo kernel: <<EOE>> [<ffffffff812773dd>] 0xffffffff812773dd
> Oct 2 12:21:32 jo kernel: [<ffffffff812610a5>] 0xffffffff812610a5
> Oct 2 12:21:32 jo kernel: [<ffffffff81268296>] 0xffffffff81268296
> Oct 2 12:21:32 jo kernel: [<ffffffff81275697>] 0xffffffff81275697
> Oct 2 12:21:32 jo kernel: [<ffffffff812956f6>] ? 0xffffffff812956f6
> Oct 2 12:21:32 jo kernel: [<ffffffff81275822>] 0xffffffff81275822
> Oct 2 12:21:32 jo kernel: [<ffffffff8127e9ef>] 0xffffffff8127e9ef
> Oct 2 12:21:32 jo kernel: [<ffffffff81286a49>] 0xffffffff81286a49
> Oct 2 12:21:32 jo kernel: [<ffffffff81083406>] ? 0xffffffff81083406
> Oct 2 12:21:32 jo kernel: [<ffffffff81286ac5>] 0xffffffff81286ac5
> Oct 2 12:21:32 jo kernel: [<ffffffff812976c4>] 0xffffffff812976c4
> Oct 2 12:21:32 jo kernel: [<ffffffff810c836a>] 0xffffffff810c836a
> Oct 2 12:21:32 jo kernel: [<ffffffff81279478>] 0xffffffff81279478
> Oct 2 12:21:32 jo kernel: [<ffffffff8103af90>] ? 0xffffffff8103af90
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f757>] 0xffffffff8129f757
> Oct 2 12:21:32 jo kernel: [<ffffffff8129f5d1>] ? 0xffffffff8129f5d1
> Oct 2 12:21:32 jo kernel: [<ffffffff81043fc3>] 0xffffffff81043fc3
> Oct 2 12:21:32 jo kernel: [<ffffffff81003654>] 0xffffffff81003654
> Oct 2 12:21:32 jo kernel: [<ffffffff81043f49>] ? 0xffffffff81043f49
> Oct 2 12:21:32 jo kernel: [<ffffffff81003650>] ? 0xffffffff81003650
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
^ permalink raw reply [flat|nested] 2+ messages in thread
end of thread, other threads:[~2010-10-02 23:49 UTC | newest]
Thread overview: 2+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2010-10-02 20:26 Kernel crash during btrfs benchmark Bart Kus
2010-10-02 23:49 ` Jérôme Poulin
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).