ATA / mvsas issue.

ATA / mvsas issue.

[Caspar Smit]

[-- Attachment #1: Type: text/plain, Size: 824 bytes --]

Hi all,

Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in one
of our test servers.

I created a software raid5 with a hotspare on these 16 disks.

I noticed that the detection of the disks during boottime was very very slow.
Creating the md superblocks went very very slow.

After that all seemed fine (speed of the raid5 init was normal).

I noticed many of these messages in the syslog (also attached):

Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-112'
Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-128'

This seems to correspond to the slow detection of the disks.

Is this an ATA problem, or a problem in the mvsas driver?

I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.

Kind regards,
Caspar Smit

[-- Attachment #2: syslog.txt --]
[-- Type: text/plain, Size: 52915 bytes --]

Mar 31 14:34:12 test kernel: [    2.291587] mvsas 0000:04:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
Mar 31 14:34:12 test kernel: [    2.291662] mvsas 0000:04:00.0: setting latency timer to 64
Mar 31 14:34:12 test kernel: [    2.293329] mvsas 0000:04:00.0: mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps
Mar 31 14:34:12 test kernel: [    8.075221] scsi2 : mvsas
Mar 31 14:34:12 test kernel: [    8.075735] mvsas 0000:0c:00.0: mvsas: driver version 0.8.2
Mar 31 14:34:12 test kernel: [    8.075819] mvsas 0000:0c:00.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
Mar 31 14:34:12 test kernel: [    8.075895] mvsas 0000:0c:00.0: setting latency timer to 64
Mar 31 14:34:12 test kernel: [    8.078089] mvsas 0000:0c:00.0: mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps
Mar 31 14:34:12 test kernel: [   11.652526] ata3.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   11.652600] ata3.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   11.685544] ata3.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   11.685690] scsi 2:0:0:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   13.855222] scsi3 : mvsas
Mar 31 14:34:12 test kernel: [   13.855729] mvsas 0000:0e:00.0: mvsas: driver version 0.8.2
Mar 31 14:34:12 test kernel: [   13.855814] mvsas 0000:0e:00.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
Mar 31 14:34:12 test kernel: [   13.855890] mvsas 0000:0e:00.0: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   13.858013] mvsas 0000:0e:00.0: mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps
Mar 31 14:34:12 test kernel: [   15.483640] ata4.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   15.483714] ata4.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   15.517461] ata4.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   15.517572] scsi 2:0:1:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   17.726157] ata5.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   17.726231] ata5.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   17.759974] ata5.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   17.760081] scsi 3:0:0:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   18.976010] scsi4 : mvsas
Mar 31 14:34:12 test kernel: [   18.976661] drivers/scsi/mvsas/mv_sas.c 1258:set wide port phy map 1
Mar 31 14:34:12 test kernel: [   18.986623] drivers/scsi/mvsas/mv_sas.c 1258:set wide port phy map 2
Mar 31 14:34:12 test kernel: [   19.007597] scsi 4:0:0:0: Direct-Access     SEAGATE  ST9300603SS      0006 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   19.008546] scsi 4:0:1:0: Direct-Access     SEAGATE  ST9300603SS      0004 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   19.165750] async_tx: api initialized (async)
Mar 31 14:34:12 test kernel: [   19.166421] xor: automatically using best checksumming function: generic_sse
Mar 31 14:34:12 test kernel: [   19.183997]    generic_sse:  7668.000 MB/sec
Mar 31 14:34:12 test kernel: [   19.184065] xor: using function: generic_sse (7668.000 MB/sec)
Mar 31 14:34:12 test kernel: [   19.196033] ata6.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   19.196111] ata6.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   19.230337] ata6.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   19.230465] scsi 2:0:2:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   19.252018] raid6: int64x1   1667 MB/s
Mar 31 14:34:12 test kernel: [   19.320009] raid6: int64x2   2235 MB/s
Mar 31 14:34:12 test kernel: [   19.388025] raid6: int64x4   1820 MB/s
Mar 31 14:34:12 test kernel: [   19.456005] raid6: int64x8   1479 MB/s
Mar 31 14:34:12 test kernel: [   19.524007] raid6: sse2x1    3228 MB/s
Mar 31 14:34:12 test kernel: [   19.592006] raid6: sse2x2    5374 MB/s
Mar 31 14:34:12 test kernel: [   19.660003] raid6: sse2x4    6153 MB/s
Mar 31 14:34:12 test kernel: [   19.660070] raid6: using algorithm sse2x4 (6153 MB/s)
Mar 31 14:34:12 test kernel: [   19.668498] md: raid6 personality registered for level 6
Mar 31 14:34:12 test kernel: [   19.668568] md: raid5 personality registered for level 5
Mar 31 14:34:12 test kernel: [   19.668637] md: raid4 personality registered for level 4
Mar 31 14:34:12 test kernel: [   19.673058] md: md0 stopped.
Mar 31 14:34:12 test kernel: [   19.695801] device-mapper: uevent: version 1.0.3
Mar 31 14:34:12 test kernel: [   19.695983] device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
Mar 31 14:34:12 test kernel: [   19.709984] PM: Starting manual resume from disk
Mar 31 14:34:12 test kernel: [   19.710054] PM: Resume from partition 8:5
Mar 31 14:34:12 test kernel: [   19.710056] PM: Checking hibernation image.
Mar 31 14:34:12 test kernel: [   19.710611] PM: Error -22 checking image file
Mar 31 14:34:12 test kernel: [   19.710613] PM: Resume from disk failed.
Mar 31 14:34:12 test kernel: [   19.714540] kjournald starting.  Commit interval 5 seconds
Mar 31 14:34:12 test kernel: [   19.714618] EXT3-fs: mounted filesystem with ordered data mode.
Mar 31 14:34:12 test kernel: [   19.945417] udevd version 125 started
Mar 31 14:34:12 test kernel: [   20.123940] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/PNP0C0C:00/input/input2
Mar 31 14:34:12 test kernel: [   20.124041] ACPI: Power Button [PWRB]
Mar 31 14:34:12 test kernel: [   20.124156] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3
Mar 31 14:34:12 test kernel: [   20.124244] ACPI: Power Button [PWRF]
Mar 31 14:34:12 test kernel: [   20.136262] processor LNXCPU:00: registered as cooling_device0
Mar 31 14:34:12 test kernel: [   20.140714] processor LNXCPU:01: registered as cooling_device1
Mar 31 14:34:12 test kernel: [   20.141616] processor LNXCPU:02: registered as cooling_device2
Mar 31 14:34:12 test kernel: [   20.145725] processor LNXCPU:03: registered as cooling_device3
Mar 31 14:34:12 test kernel: [   20.218289] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
Mar 31 14:34:12 test kernel: [   20.226291] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
Mar 31 14:34:12 test kernel: [   20.393954] input: PC Speaker as /devices/platform/pcspkr/input/input4
Mar 31 14:34:12 test kernel: [   20.520751] EDAC MC: Ver: 2.1.0 Mar 15 2010
Mar 31 14:34:12 test kernel: [   20.549568] EDAC MC0: Giving out device to 'i5400_edac.c' 'I5400': DEV 0000:00:10.0
Mar 31 14:34:12 test kernel: [   20.549677] EDAC PCI0: Giving out device to module 'i5400_edac' controller 'EDAC PCI controller': DEV '0000:00:10.0' (POLLED)
Mar 31 14:34:12 test kernel: [   20.651101] ioatdma: Intel(R) QuickData Technology Driver 4.00
Mar 31 14:34:12 test kernel: [   20.651386]   alloc irq_desc for 33 on node -1
Mar 31 14:34:12 test kernel: [   20.651389]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.651399] ioatdma 0000:00:0f.0: PCI INT A -> GSI 33 (level, low) -> IRQ 33
Mar 31 14:34:12 test kernel: [   20.651551] ioatdma 0000:00:0f.0: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   20.651603]   alloc irq_desc for 82 on node -1
Mar 31 14:34:12 test kernel: [   20.651605]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.651611] ioatdma 0000:00:0f.0: irq 82 for MSI/MSI-X
Mar 31 14:34:12 test kernel: [   20.651613]   alloc irq_desc for 83 on node -1
Mar 31 14:34:12 test kernel: [   20.651615]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.651619] ioatdma 0000:00:0f.0: irq 83 for MSI/MSI-X
Mar 31 14:34:12 test kernel: [   20.651621]   alloc irq_desc for 84 on node -1
Mar 31 14:34:12 test kernel: [   20.651622]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.651626] ioatdma 0000:00:0f.0: irq 84 for MSI/MSI-X
Mar 31 14:34:12 test kernel: [   20.651628]   alloc irq_desc for 85 on node -1
Mar 31 14:34:12 test kernel: [   20.651629]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.651633] ioatdma 0000:00:0f.0: irq 85 for MSI/MSI-X
Mar 31 14:34:12 test kernel: [   20.663218] igb 0000:0f:00.0: DCA enabled
Mar 31 14:34:12 test kernel: [   20.663313] igb 0000:0f:00.1: DCA enabled
Mar 31 14:34:12 test kernel: [   20.663431]   alloc irq_desc for 18 on node -1
Mar 31 14:34:12 test kernel: [   20.663434]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.663442] i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
Mar 31 14:34:12 test kernel: [   20.672367] usbcore: registered new interface driver usbfs
Mar 31 14:34:12 test kernel: [   20.672464] usbcore: registered new interface driver hub
Mar 31 14:34:12 test kernel: [   20.684058] usbcore: registered new device driver usb
Mar 31 14:34:12 test kernel: [   20.697034] intel_rng: FWH not detected
Mar 31 14:34:12 test kernel: [   20.702549] Error: Driver 'pcspkr' is already registered, aborting...
Mar 31 14:34:12 test kernel: [   20.775273] uhci_hcd: USB Universal Host Controller Interface driver
Mar 31 14:34:12 test kernel: [   20.775412]   alloc irq_desc for 20 on node -1
Mar 31 14:34:12 test kernel: [   20.775414]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.775423] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
Mar 31 14:34:12 test kernel: [   20.775503] uhci_hcd 0000:00:1d.0: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   20.775507] uhci_hcd 0000:00:1d.0: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.775602] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 1
Mar 31 14:34:12 test kernel: [   20.775726] uhci_hcd 0000:00:1d.0: irq 20, io base 0x00001800
Mar 31 14:34:12 test kernel: [   20.775844] usb usb1: New USB device found, idVendor=1d6b, idProduct=0001
Mar 31 14:34:12 test kernel: [   20.775917] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Mar 31 14:34:12 test kernel: [   20.776004] usb usb1: Product: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.776073] usb usb1: Manufacturer: Linux 2.6.32-bpo.3-amd64 uhci_hcd
Mar 31 14:34:12 test kernel: [   20.776145] usb usb1: SerialNumber: 0000:00:1d.0
Mar 31 14:34:12 test kernel: [   20.776305] usb usb1: configuration #1 chosen from 1 choice
Mar 31 14:34:12 test kernel: [   20.776406] hub 1-0:1.0: USB hub found
Mar 31 14:34:12 test kernel: [   20.776478] hub 1-0:1.0: 2 ports detected
Mar 31 14:34:12 test kernel: [   20.776595]   alloc irq_desc for 21 on node -1
Mar 31 14:34:12 test kernel: [   20.776597]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.776601] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
Mar 31 14:34:12 test kernel: [   20.776679] uhci_hcd 0000:00:1d.1: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   20.776682] uhci_hcd 0000:00:1d.1: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.776761] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 2
Mar 31 14:34:12 test kernel: [   20.776876] uhci_hcd 0000:00:1d.1: irq 21, io base 0x00001820
Mar 31 14:34:12 test kernel: [   20.776983] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
Mar 31 14:34:12 test kernel: [   20.777088] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Mar 31 14:34:12 test kernel: [   20.777176] usb usb2: Product: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.777245] usb usb2: Manufacturer: Linux 2.6.32-bpo.3-amd64 uhci_hcd
Mar 31 14:34:12 test kernel: [   20.777317] usb usb2: SerialNumber: 0000:00:1d.1
Mar 31 14:34:12 test kernel: [   20.780695] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
Mar 31 14:34:12 test kernel: [   20.780768] Warning! ehci_hcd should always be loaded before uhci_hcd and ohci_hcd, not after
Mar 31 14:34:12 test kernel: [   20.781176] usb usb2: configuration #1 chosen from 1 choice
Mar 31 14:34:12 test kernel: [   20.781276] hub 2-0:1.0: USB hub found
Mar 31 14:34:12 test kernel: [   20.781348] hub 2-0:1.0: 2 ports detected
Mar 31 14:34:12 test kernel: [   20.781465]   alloc irq_desc for 23 on node -1
Mar 31 14:34:12 test kernel: [   20.781467]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.781472] ehci_hcd 0000:00:1d.7: PCI INT D -> GSI 23 (level, low) -> IRQ 23
Mar 31 14:34:12 test kernel: [   20.781615] ehci_hcd 0000:00:1d.7: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   20.781618] ehci_hcd 0000:00:1d.7: EHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.781697] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 3
Mar 31 14:34:12 test kernel: [   20.781811] ehci_hcd 0000:00:1d.7: debug port 1
Mar 31 14:34:12 test kernel: [   20.785774] ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
Mar 31 14:34:12 test kernel: [   20.785797] ehci_hcd 0000:00:1d.7: irq 23, io mem 0xd8b04000
Mar 31 14:34:12 test kernel: [   20.801523] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
Mar 31 14:34:12 test kernel: [   20.801618] usb usb3: New USB device found, idVendor=1d6b, idProduct=0002
Mar 31 14:34:12 test kernel: [   20.801691] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Mar 31 14:34:12 test kernel: [   20.801778] usb usb3: Product: EHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.801847] usb usb3: Manufacturer: Linux 2.6.32-bpo.3-amd64 ehci_hcd
Mar 31 14:34:12 test kernel: [   20.801919] usb usb3: SerialNumber: 0000:00:1d.7
Mar 31 14:34:12 test kernel: [   20.802058] usb usb3: configuration #1 chosen from 1 choice
Mar 31 14:34:12 test kernel: [   20.802159] hub 3-0:1.0: USB hub found
Mar 31 14:34:12 test kernel: [   20.802231] hub 3-0:1.0: 8 ports detected
Mar 31 14:34:12 test kernel: [   20.802368]   alloc irq_desc for 22 on node -1
Mar 31 14:34:12 test kernel: [   20.802371]   alloc kstat_irqs on node -1
Mar 31 14:34:12 test kernel: [   20.802378] uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 22 (level, low) -> IRQ 22
Mar 31 14:34:12 test kernel: [   20.808884] uhci_hcd 0000:00:1d.2: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   20.808887] uhci_hcd 0000:00:1d.2: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.808967] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4
Mar 31 14:34:12 test kernel: [   20.809091] uhci_hcd 0000:00:1d.2: irq 22, io base 0x00001840
Mar 31 14:34:12 test kernel: [   20.809196] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
Mar 31 14:34:12 test kernel: [   20.809269] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Mar 31 14:34:12 test kernel: [   20.809359] usb usb4: Product: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.809430] usb usb4: Manufacturer: Linux 2.6.32-bpo.3-amd64 uhci_hcd
Mar 31 14:34:12 test kernel: [   20.809502] usb usb4: SerialNumber: 0000:00:1d.2
Mar 31 14:34:12 test kernel: [   20.809638] usb usb4: configuration #1 chosen from 1 choice
Mar 31 14:34:12 test kernel: [   20.809735] hub 4-0:1.0: USB hub found
Mar 31 14:34:12 test kernel: [   20.809807] hub 4-0:1.0: 2 ports detected
Mar 31 14:34:12 test kernel: [   20.809939] uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 23 (level, low) -> IRQ 23
Mar 31 14:34:12 test kernel: [   20.810017] uhci_hcd 0000:00:1d.3: setting latency timer to 64
Mar 31 14:34:12 test kernel: [   20.810020] uhci_hcd 0000:00:1d.3: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.810096] uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 5
Mar 31 14:34:12 test kernel: [   20.810207] uhci_hcd 0000:00:1d.3: irq 23, io base 0x00001860
Mar 31 14:34:12 test kernel: [   20.810308] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
Mar 31 14:34:12 test kernel: [   20.810381] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
Mar 31 14:34:12 test kernel: [   20.810470] usb usb5: Product: UHCI Host Controller
Mar 31 14:34:12 test kernel: [   20.810542] usb usb5: Manufacturer: Linux 2.6.32-bpo.3-amd64 uhci_hcd
Mar 31 14:34:12 test kernel: [   20.810614] usb usb5: SerialNumber: 0000:00:1d.3
Mar 31 14:34:12 test kernel: [   20.810748] usb usb5: configuration #1 chosen from 1 choice
Mar 31 14:34:12 test kernel: [   20.810847] hub 5-0:1.0: USB hub found
Mar 31 14:34:12 test kernel: [   20.810924] hub 5-0:1.0: 2 ports detected
Mar 31 14:34:12 test kernel: [   21.443488] ata7.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   21.443566] ata7.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   21.477719] ata7.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   21.477863] scsi 3:0:1:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   22.857434] Adding 4096564k swap on /dev/sda5.  Priority:-1 extents:1 across:4096564k SS
Mar 31 14:34:12 test kernel: [   22.897465] EXT3 FS on sda1, internal journal
Mar 31 14:34:12 test kernel: [   23.206494] ata8.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   23.206592] ata8.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   23.240727] ata8.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   23.240878] scsi 2:0:3:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   23.913420] netconsole: local port 6665
Mar 31 14:34:12 test kernel: [   23.913492] netconsole: local IP 192.168.0.12
Mar 31 14:34:12 test kernel: [   23.913560] netconsole: interface eth0
Mar 31 14:34:12 test kernel: [   23.913626] netconsole: remote port 6666
Mar 31 14:34:12 test kernel: [   23.913694] netconsole: remote IP 192.168.0.10
Mar 31 14:34:12 test kernel: [   23.913762] netconsole: remote ethernet address 00:30:48:88:36:c6
Mar 31 14:34:12 test kernel: [   23.913835] netconsole: device eth0 not up yet, forcing it
Mar 31 14:34:12 test kernel: [   23.925556] ADDRCONF(NETDEV_UP): eth0: link is not ready
Mar 31 14:34:12 test kernel: [   23.928483] igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
Mar 31 14:34:12 test kernel: [   23.929329] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
Mar 31 14:34:12 test kernel: [   23.933005] netconsole: carrier detect appears untrustworthy, waiting 4 seconds
Mar 31 14:34:12 test kernel: [   25.213025] ata9.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   25.213101] ata9.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   25.247420] ata9.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   25.247562] scsi 3:0:2:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   26.932067] ata10.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   26.932144] ata10.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   26.965486] ata10.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   26.965617] scsi 2:0:4:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   28.027130] console [netcon0] enabled
Mar 31 14:34:12 test kernel: [   28.027199] netconsole: network logging started
Mar 31 14:34:12 test kernel: [   28.040037] loop: module loaded
Mar 31 14:34:12 test kernel: [   28.231223] w83627hf: Found W83627HF chip at 0x290
Mar 31 14:34:12 test kernel: [   29.096059] ata11.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   29.096154] ata11.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   29.130152] ata11.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   29.130283] scsi 3:0:3:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   30.605784] ata12.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   30.605863] ata12.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   30.640174] ata12.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   30.640297] scsi 2:0:5:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test kernel: [   32.822666] ata13.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:12 test kernel: [   32.822754] ata13.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:12 test kernel: [   32.857033] ata13.00: configured for UDMA/133
Mar 31 14:34:12 test kernel: [   32.857177] scsi 3:0:4:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:12 test mysqld_safe: Starting mysqld daemon with databases from /var/lib/mysql
Mar 31 14:34:12 test mysqld: 100331 14:34:12 [Note] Plugin 'FEDERATED' is disabled.
Mar 31 14:34:12 test mysqld: 100331 14:34:12  InnoDB: Started; log sequence number 0 43675
Mar 31 14:34:12 test mysqld: 100331 14:34:12 [Note] Event Scheduler: Loaded 0 events
Mar 31 14:34:12 test mysqld: 100331 14:34:12 [Note] /usr/sbin/mysqld: ready for connections.
Mar 31 14:34:12 test mysqld: Version: '5.1.44-3~bpo50+1'  socket: '/var/run/mysqld/mysqld.sock'  port: 3306  (Debian)
Mar 31 14:34:13 test kernel: [   34.480005] eth0: no IPv6 routers present
Mar 31 14:34:13 test /etc/mysql/debian-start[3430]: Upgrading MySQL tables if necessary.
Mar 31 14:34:13 test /etc/mysql/debian-start[3435]: /usr/bin/mysql_upgrade: the '--basedir' option is always ignored
Mar 31 14:34:13 test /etc/mysql/debian-start[3435]: Looking for 'mysql' as: /usr/bin/mysql
Mar 31 14:34:13 test /etc/mysql/debian-start[3435]: Looking for 'mysqlcheck' as: /usr/bin/mysqlcheck
Mar 31 14:34:13 test /etc/mysql/debian-start[3435]: This installation of MySQL is already upgraded to 5.1.44, use --force if you still need to run mysql_upgrade
Mar 31 14:34:13 test /etc/mysql/debian-start[3445]: Checking for insecure root accounts.
Mar 31 14:34:13 test /etc/mysql/debian-start[3456]: WARNING: mysql.user contains 2 root accounts without password!
Mar 31 14:34:13 test /etc/mysql/debian-start[3457]: Triggering myisam-recover for all MyISAM tables
Mar 31 14:34:13 test kernel: [   34.563044] RPC: Registered udp transport module.
Mar 31 14:34:13 test kernel: [   34.563126] RPC: Registered tcp transport module.
Mar 31 14:34:13 test kernel: [   34.563197] RPC: Registered tcp NFSv4.1 backchannel transport module.
Mar 31 14:34:13 test kernel: [   34.588100] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
Mar 31 14:34:13 test kernel: [   34.614150] svc: failed to register lockdv1 RPC service (errno 97).
Mar 31 14:34:13 test kernel: [   34.614801] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
Mar 31 14:34:13 test kernel: [   34.622656] NFSD: starting 90-second grace period
Mar 31 14:34:13 test kernel: [   34.776740] ata14.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:13 test kernel: [   34.776845] ata14.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:13 test smbd[3545]: [2010/03/31 14:34:13,  0] printing/print_cups.c:103(cups_connect) 
Mar 31 14:34:13 test smbd[3545]:   Unable to connect to CUPS server localhost:631 - Connection refused 
Mar 31 14:34:13 test smbd[3547]: [2010/03/31 14:34:13,  0] printing/print_cups.c:103(cups_connect) 
Mar 31 14:34:13 test smbd[3547]:   Unable to connect to CUPS server localhost:631 - Connection refused 
Mar 31 14:34:13 test kernel: [   34.811001] ata14.00: configured for UDMA/133
Mar 31 14:34:13 test kernel: [   34.811134] scsi 2:0:6:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:13 test smbd[3549]: [2010/03/31 14:34:13,  0] smbd/server.c:457(smbd_open_one_socket) 
Mar 31 14:34:13 test smbd[3549]:   smbd_open_once_socket: open_socket_in: Address already in use 
Mar 31 14:34:13 test smbd[3549]: [2010/03/31 14:34:13,  0] smbd/server.c:457(smbd_open_one_socket) 
Mar 31 14:34:13 test smbd[3549]:   smbd_open_once_socket: open_socket_in: Address already in use 
Mar 31 14:34:13 test smartd[3563]: smartd 5.40 2010-02-03 r3060 [x86_64-unknown-linux-gnu] (local build) Copyright (C) 2002-10 by Bruce Allen, http://smartmontools.sourceforge.net  
Mar 31 14:34:13 test smartd[3563]: Opened configuration file /etc/smartd.conf 
Mar 31 14:34:13 test smartd[3563]: Drive: DEVICESCAN, implied '-a' Directive on line 22 of file /etc/smartd.conf 
Mar 31 14:34:13 test smartd[3563]: Configuration file /etc/smartd.conf was parsed, found DEVICESCAN, scanning devices 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda, type changed from 'scsi' to 'sat' 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], opened 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], found in smartd database. 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], can't monitor Current Pending Sector count - no Attribute 197 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], can't monitor Offline Uncorrectable Sector count - no Attribute 198 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], appears to lack SMART Self-Test log; disabling -l selftest (override with -T permissive Directive) 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], appears to lack SMART Error log; disabling -l error (override with -T permissive Directive) 
Mar 31 14:34:13 test smartd[3563]: Device: /dev/sda [SAT], is SMART capable. Adding to "monitor" list. 
Mar 31 14:34:13 test smartd[3563]: Monitoring 1 ATA and 0 SCSI devices 
Mar 31 14:34:13 test smartd[3568]: smartd has fork()ed into background mode. New PID=3568. 
Mar 31 14:34:13 test smartd[3568]: file /var/run/smartd.pid written containing PID 3568 
Mar 31 14:34:14 test rsyncd[3616]: rsyncd version 3.0.7 starting, listening on port 873 
Mar 31 14:34:15 test snmpd[3596]: NET-SNMP version 5.4.1 
Mar 31 14:34:15 test kernel: [   36.519035] ata15.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:15 test kernel: [   36.519135] ata15.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:15 test kernel: [   36.553359] ata15.00: configured for UDMA/133
Mar 31 14:34:15 test kernel: [   36.553490] scsi 3:0:5:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:15 test /usr/sbin/cron[3640]: (CRON) INFO (pidfile fd = 3)
Mar 31 14:34:15 test /usr/sbin/cron[3641]: (CRON) STARTUP (fork ok)
Mar 31 14:34:15 test /usr/sbin/cron[3641]: (CRON) INFO (Running @reboot jobs)
Mar 31 14:34:15 test kernel: [   36.989032] iSCSI Enterprise Target Software - version 1.4.19
Mar 31 14:34:15 test kernel: [   36.989193] iscsi_trgt: Registered io type fileio
Mar 31 14:34:15 test kernel: [   36.989288] iscsi_trgt: Registered io type blockio
Mar 31 14:34:15 test kernel: [   36.989359] iscsi_trgt: Registered io type nullio
Mar 31 14:34:17 test kernel: [   38.608014] ata16.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:17 test kernel: [   38.608100] ata16.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:17 test kernel: [   38.642452] ata16.00: configured for UDMA/133
Mar 31 14:34:17 test kernel: [   38.642578] scsi 2:0:7:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:17 test kernel: [   38.643810] sd 2:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.643942] sd 2:0:0:0: [sdb] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.644024] sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.644105] sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.644327]  sdb:
Mar 31 14:34:17 test kernel: [   38.644439] sd 2:0:1:0: [sdc] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.644628] sd 2:0:1:0: [sdc] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.644704] sd 2:0:1:0: [sdc] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.644727] sd 2:0:1:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.644928]  sdc:
Mar 31 14:34:17 test kernel: [   38.645038] sd 2:0:2:0: [sdd] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.645225] sd 2:0:2:0: [sdd] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.645295] sd 2:0:2:0: [sdd] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.645317] sd 2:0:2:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.645515]  sdd:
Mar 31 14:34:17 test kernel: [   38.645624] sd 2:0:3:0: [sde] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.645815] sd 2:0:3:0: [sde] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.645886] sd 2:0:3:0: [sde] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.645908] sd 2:0:3:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.646108]  sde:
Mar 31 14:34:17 test kernel: [   38.646217] sd 2:0:4:0: [sdf] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.646405] sd 2:0:4:0: [sdf] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.646476] sd 2:0:4:0: [sdf] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.646498] sd 2:0:4:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.646698]  sdf:
Mar 31 14:34:17 test kernel: [   38.646805] sd 2:0:5:0: [sdg] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.646990] sd 2:0:5:0: [sdg] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.647061] sd 2:0:5:0: [sdg] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.647083] sd 2:0:5:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.647278]  sdg:
Mar 31 14:34:17 test kernel: [   38.647380] sd 2:0:6:0: [sdh] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.647568] sd 2:0:6:0: [sdh] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.647639] sd 2:0:6:0: [sdh] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.647666] sd 2:0:6:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.647858]  sdh:
Mar 31 14:34:17 test kernel: [   38.647950] sd 2:0:7:0: [sdi] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:17 test kernel: [   38.649107] sd 2:0:7:0: [sdi] Write Protect is off
Mar 31 14:34:17 test kernel: [   38.649178] sd 2:0:7:0: [sdi] Mode Sense: 00 3a 00 00
Mar 31 14:34:17 test kernel: [   38.649199] sd 2:0:7:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:17 test kernel: [   38.649398]  sdi: unknown partition table
Mar 31 14:34:17 test kernel: [   38.656710] sd 2:0:7:0: [sdi] Attached SCSI disk
Mar 31 14:34:17 test kernel: [   38.691990]  unknown partition table
Mar 31 14:34:17 test kernel: [   38.692286] sd 2:0:6:0: [sdh] Attached SCSI disk
Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-112' 
Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-128' 
Mar 31 14:34:18 test kernel: [   40.016739] ata17.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:18 test kernel: [   40.016825] ata17.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:18 test kernel: [   40.051091] ata17.00: configured for UDMA/133
Mar 31 14:34:18 test kernel: [   40.051223] scsi 3:0:6:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:19 test kernel: [   40.215241]  unknown partition table
Mar 31 14:34:19 test kernel: [   40.215503] sd 2:0:4:0: [sdf] Attached SCSI disk
Mar 31 14:34:19 test ata_id[3800]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-80' 
Mar 31 14:34:19 test ata_id[3814]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-96' 
Mar 31 14:34:19 test kernel: [   40.284985]  unknown partition table
Mar 31 14:34:19 test kernel: [   40.285218] sd 2:0:5:0: [sdg] Attached SCSI disk
Mar 31 14:34:19 test kernel: [   40.297641]  unknown partition table
Mar 31 14:34:19 test kernel: [   40.297874] sd 2:0:2:0: [sdd] Attached SCSI disk
Mar 31 14:34:19 test ata_id[3824]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-48' 
Mar 31 14:34:19 test ata_id[3848]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-64' 
Mar 31 14:34:19 test kernel: [   40.553079]  unknown partition table
Mar 31 14:34:19 test kernel: [   40.553357] sd 2:0:3:0: [sde] Attached SCSI disk
Mar 31 14:34:21 test kernel: [   42.308805]  unknown partition table
Mar 31 14:34:21 test kernel: [   42.309064] sd 2:0:0:0: [sdb] Attached SCSI disk
Mar 31 14:34:21 test ata_id[3874]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-32' 
Mar 31 14:34:21 test kernel: [   42.348732]  unknown partition table
Mar 31 14:34:21 test kernel: [   42.348965] sd 2:0:1:0: [sdc] Attached SCSI disk
Mar 31 14:34:22 test kernel: [   43.709177] ata18.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
Mar 31 14:34:22 test kernel: [   43.709263] ata18.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
Mar 31 14:34:22 test kernel: [   43.743429] ata18.00: configured for UDMA/133
Mar 31 14:34:22 test kernel: [   43.743560] scsi 3:0:7:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
Mar 31 14:34:22 test kernel: [   43.744990] sd 3:0:0:0: [sdj] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.745131] sd 3:0:0:0: [sdj] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.745203] sd 3:0:0:0: [sdj] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.745225] sd 3:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.745437]  sdj:
Mar 31 14:34:22 test kernel: [   43.745552] sd 3:0:1:0: [sdk] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.745739] sd 3:0:1:0: [sdk] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.745810] sd 3:0:1:0: [sdk] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.745831] sd 3:0:1:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.746029]  sdk:
Mar 31 14:34:22 test kernel: [   43.746139] sd 3:0:2:0: [sdl] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.746326] sd 3:0:2:0: [sdl] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.746397] sd 3:0:2:0: [sdl] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.746417] sd 3:0:2:0: [sdl] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.746612]  sdl:
Mar 31 14:34:22 test kernel: [   43.746717] sd 3:0:3:0: [sdm] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.746902] sd 3:0:3:0: [sdm] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.746973] sd 3:0:3:0: [sdm] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.746994] sd 3:0:3:0: [sdm] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.747202]  sdm:
Mar 31 14:34:22 test kernel: [   43.747308] sd 3:0:4:0: [sdn] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.747496] sd 3:0:4:0: [sdn] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.747567] sd 3:0:4:0: [sdn] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.747588] sd 3:0:4:0: [sdn] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.747788]  sdn:
Mar 31 14:34:22 test kernel: [   43.747891] sd 3:0:5:0: [sdo] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.748096] sd 3:0:5:0: [sdo] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.748173] sd 3:0:5:0: [sdo] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.748194] sd 3:0:5:0: [sdo] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.748391]  sdo:
Mar 31 14:34:22 test kernel: [   43.748495] sd 3:0:6:0: [sdp] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.748680] sd 3:0:6:0: [sdp] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.748751] sd 3:0:6:0: [sdp] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.748771] sd 3:0:6:0: [sdp] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.748964]  sdp:
Mar 31 14:34:22 test kernel: [   43.749068] sd 3:0:7:0: [sdq] 976773168 512-byte logical blocks: (500 GB/465 GiB)
Mar 31 14:34:22 test kernel: [   43.756312] sd 3:0:7:0: [sdq] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.756384] sd 3:0:7:0: [sdq] Mode Sense: 00 3a 00 00
Mar 31 14:34:22 test kernel: [   43.756406] sd 3:0:7:0: [sdq] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
Mar 31 14:34:22 test kernel: [   43.756616]  sdq:
Mar 31 14:34:22 test kernel: [   43.756791] sd 4:0:0:0: [sdr] 585937500 512-byte logical blocks: (300 GB/279 GiB)
Mar 31 14:34:22 test kernel: [   43.757528] sd 4:0:1:0: [sds] 585937500 512-byte logical blocks: (300 GB/279 GiB)
Mar 31 14:34:22 test kernel: [   43.758238] sd 4:0:0:0: [sdr] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.758316] sd 4:0:0:0: [sdr] Mode Sense: bb 00 10 08
Mar 31 14:34:22 test kernel: [   43.758895] sd 4:0:1:0: [sds] Write Protect is off
Mar 31 14:34:22 test kernel: [   43.758969] sd 4:0:1:0: [sds] Mode Sense: bb 00 10 08
Mar 31 14:34:22 test kernel: [   43.759536] sd 4:0:0:0: [sdr] Write cache: enabled, read cache: enabled, supports DPO and FUA
Mar 31 14:34:22 test kernel: [   43.760406] sd 4:0:1:0: [sds] Write cache: enabled, read cache: enabled, supports DPO and FUA
Mar 31 14:34:22 test kernel: [   43.762768]  sdr:
Mar 31 14:34:22 test kernel: [   43.763470]  sds: unknown partition table
Mar 31 14:34:22 test kernel: [   43.769036] sd 3:0:7:0: [sdq] Attached SCSI disk
Mar 31 14:34:22 test kernel: [   43.779310]  unknown partition table
Mar 31 14:34:22 test kernel: [   43.779402]  unknown partition table
Mar 31 14:34:22 test kernel: [   43.782542] sd 4:0:0:0: [sdr] Attached SCSI disk
Mar 31 14:34:22 test kernel: [   43.782670] sd 4:0:1:0: [sds] Attached SCSI disk
Mar 31 14:34:22 test kernel: [   43.787659]  unknown partition table
Mar 31 14:34:22 test kernel: [   43.787868] sd 3:0:6:0: [sdp] Attached SCSI disk
Mar 31 14:34:22 test ata_id[3972]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-240' 
Mar 31 14:34:22 test ata_id[3998]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-65-0' 
Mar 31 14:34:24 test ata_id[4039]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-224' 
Mar 31 14:34:24 test kernel: [   45.256504]  unknown partition table
Mar 31 14:34:24 test kernel: [   45.256797] sd 3:0:5:0: [sdo] Attached SCSI disk
Mar 31 14:34:24 test kernel: [   45.323116]  unknown partition table
Mar 31 14:34:24 test kernel: [   45.323356] sd 3:0:2:0: [sdl] Attached SCSI disk
Mar 31 14:34:24 test ata_id[4053]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-176' 
Mar 31 14:34:24 test ata_id[4069]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-192' 
Mar 31 14:34:24 test kernel: [   45.380509]  unknown partition table
Mar 31 14:34:24 test kernel: [   45.380755] sd 3:0:3:0: [sdm] Attached SCSI disk
Mar 31 14:34:24 test kernel: [   45.503057]  unknown partition table
Mar 31 14:34:24 test kernel: [   45.503305] sd 3:0:4:0: [sdn] Attached SCSI disk
Mar 31 14:34:24 test ata_id[4087]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-208' 
Mar 31 14:34:26 test kernel: [   47.542972]  unknown partition table
Mar 31 14:34:26 test kernel: [   47.543237] sd 3:0:1:0: [sdk] Attached SCSI disk
Mar 31 14:34:26 test ata_id[4103]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-160' 
Mar 31 14:34:26 test kernel: [   47.602771]  unknown partition table
Mar 31 14:34:26 test kernel: [   47.603007] sd 3:0:0:0: [sdj] Attached SCSI disk
Mar 31 14:35:01 test /USR/SBIN/CRON[4138]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:35:01 test /USR/SBIN/CRON[4139]: (root) CMD ([ -x /usr/lib/sysstat/sa1 ] && { [ -r "$DEFAULT" ] && . "$DEFAULT" ; [ "$ENABLED" = "true" ] && exec /usr/lib/sysstat/sa1 $SA1_OPTIONS 1 1 ; })
Mar 31 14:36:01 test /USR/SBIN/CRON[4248]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:37:01 test /USR/SBIN/CRON[4261]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:38:01 test /USR/SBIN/CRON[4287]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:39:01 test /USR/SBIN/CRON[4307]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:40:01 test /USR/SBIN/CRON[5227]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:40:26 test kernel: [  407.238173] md: bind<sdb>
Mar 31 14:40:29 test kernel: [  410.903983] md: bind<sdc>
Mar 31 14:40:33 test kernel: [  414.715249] md: bind<sdd>
Mar 31 14:40:37 test kernel: [  418.695204] md: bind<sde>
Mar 31 14:40:41 test kernel: [  422.317381] md: bind<sdf>
Mar 31 14:40:45 test kernel: [  426.094069] md: bind<sdg>
Mar 31 14:40:48 test kernel: [  429.863752] md: bind<sdh>
Mar 31 14:40:52 test kernel: [  433.655351] md: bind<sdi>
Mar 31 14:40:56 test kernel: [  437.433736] md: bind<sdj>
Mar 31 14:41:00 test kernel: [  441.104704] md: bind<sdk>
Mar 31 14:41:01 test /USR/SBIN/CRON[5246]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:41:03 test kernel: [  444.852438] md: bind<sdl>
Mar 31 14:41:07 test kernel: [  448.533076] md: bind<sdm>
Mar 31 14:41:11 test kernel: [  452.217321] md: bind<sdn>
Mar 31 14:41:15 test kernel: [  456.194441] md: bind<sdo>
Mar 31 14:41:18 test kernel: [  459.725240] md: bind<sdp>
Mar 31 14:41:22 test kernel: [  463.284026] md: bind<sdq>
Mar 31 14:41:22 test kernel: [  463.327778] raid5: md0 is not clean -- starting background reconstruction
Mar 31 14:41:22 test kernel: [  463.327871] raid5: device sdp operational as raid disk 14
Mar 31 14:41:22 test kernel: [  463.327941] raid5: device sdo operational as raid disk 13
Mar 31 14:41:22 test kernel: [  463.328018] raid5: device sdn operational as raid disk 12
Mar 31 14:41:22 test kernel: [  463.328090] raid5: device sdm operational as raid disk 11
Mar 31 14:41:22 test kernel: [  463.328164] raid5: device sdl operational as raid disk 10
Mar 31 14:41:22 test kernel: [  463.328235] raid5: device sdk operational as raid disk 9
Mar 31 14:41:22 test kernel: [  463.328305] raid5: device sdj operational as raid disk 8
Mar 31 14:41:22 test kernel: [  463.328375] raid5: device sdi operational as raid disk 7
Mar 31 14:41:22 test kernel: [  463.328446] raid5: device sdh operational as raid disk 6
Mar 31 14:41:22 test kernel: [  463.328516] raid5: device sdg operational as raid disk 5
Mar 31 14:41:22 test kernel: [  463.328587] raid5: device sdf operational as raid disk 4
Mar 31 14:41:22 test kernel: [  463.328661] raid5: device sde operational as raid disk 3
Mar 31 14:41:22 test kernel: [  463.328731] raid5: device sdd operational as raid disk 2
Mar 31 14:41:22 test kernel: [  463.328801] raid5: device sdc operational as raid disk 1
Mar 31 14:41:22 test kernel: [  463.328872] raid5: device sdb operational as raid disk 0
Mar 31 14:41:22 test kernel: [  463.330113] raid5: allocated 15854kB for md0
Mar 31 14:41:22 test kernel: [  463.330206] 14: w=1 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330278] 13: w=2 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330349] 12: w=3 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330420] 11: w=4 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330496] 10: w=5 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330567] 9: w=6 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330638] 8: w=7 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330708] 7: w=8 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330779] 6: w=9 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330851] 5: w=10 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330922] 4: w=11 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.330993] 3: w=12 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.331064] 2: w=13 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.331136] 1: w=14 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.331207] 0: w=15 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
Mar 31 14:41:22 test kernel: [  463.331278] raid5: raid level 5 set md0 active with 15 out of 15 devices, algorithm 2
Mar 31 14:41:22 test kernel: [  463.331367] RAID5 conf printout:
Mar 31 14:41:22 test kernel: [  463.331437]  --- rd:15 wd:15
Mar 31 14:41:22 test kernel: [  463.331503]  disk 0, o:1, dev:sdb
Mar 31 14:41:22 test kernel: [  463.331570]  disk 1, o:1, dev:sdc
Mar 31 14:41:22 test kernel: [  463.331636]  disk 2, o:1, dev:sdd
Mar 31 14:41:22 test kernel: [  463.331703]  disk 3, o:1, dev:sde
Mar 31 14:41:22 test kernel: [  463.331770]  disk 4, o:1, dev:sdf
Mar 31 14:41:22 test kernel: [  463.331837]  disk 5, o:1, dev:sdg
Mar 31 14:41:22 test kernel: [  463.331904]  disk 6, o:1, dev:sdh
Mar 31 14:41:22 test kernel: [  463.331971]  disk 7, o:1, dev:sdi
Mar 31 14:41:22 test kernel: [  463.332052]  disk 8, o:1, dev:sdj
Mar 31 14:41:22 test kernel: [  463.332120]  disk 9, o:1, dev:sdk
Mar 31 14:41:22 test kernel: [  463.332187]  disk 10, o:1, dev:sdl
Mar 31 14:41:22 test kernel: [  463.332254]  disk 11, o:1, dev:sdm
Mar 31 14:41:22 test kernel: [  463.332322]  disk 12, o:1, dev:sdn
Mar 31 14:41:22 test kernel: [  463.332389]  disk 13, o:1, dev:sdo
Mar 31 14:41:22 test kernel: [  463.332459]  disk 14, o:1, dev:sdp
Mar 31 14:41:22 test kernel: [  463.332574] md0: detected capacity change from 0 to 7001507889152
Mar 31 14:41:25 test kernel: [  463.332880]  md0: unknown partition table
Mar 31 14:41:25 test kernel: [  466.918068] md: md0 switched to read-write mode.
Mar 31 14:41:25 test kernel: [  466.918688] md: resync of RAID array md0
Mar 31 14:41:25 test kernel: [  466.918759] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
Mar 31 14:41:25 test kernel: [  466.918838] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
Mar 31 14:41:25 test kernel: [  466.918945] md: using 128k window, over a total of 488386432 blocks.
Mar 31 14:41:30 test sSMTP[5323]: Sent mail for root@supernas.nl (221 2.0.0 ns1.emsolutions.nl closing connection) uid=0 username=root outbytes=845
Mar 31 14:41:30 test mdadm[3607]: NewArray event detected on md device /dev/md0
Mar 31 14:42:01 test /USR/SBIN/CRON[5335]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:43:01 test /USR/SBIN/CRON[5385]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:44:01 test /USR/SBIN/CRON[5484]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:45:01 test /USR/SBIN/CRON[5616]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:45:01 test /USR/SBIN/CRON[5615]: (root) CMD ([ -x /usr/lib/sysstat/sa1 ] && { [ -r "$DEFAULT" ] && . "$DEFAULT" ; [ "$ENABLED" = "true" ] && exec /usr/lib/sysstat/sa1 $SA1_OPTIONS 1 1 ; })
Mar 31 14:46:01 test /USR/SBIN/CRON[5744]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:47:01 test /USR/SBIN/CRON[5877]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:48:01 test /USR/SBIN/CRON[6006]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:49:02 test /USR/SBIN/CRON[6134]: (root) CMD (/etc/webmin/system-status/systeminfo.pl)
Mar 31 14:49:02 test /USR/SBIN/CRON[6135]: (root) CMD (/etc/webmin/status/monitor.pl)
Mar 31 14:49:04 test kernel: [  925.591713] ------------[ cut here ]------------
Mar 31 14:49:04 test kernel: [  925.591804] WARNING: at /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/ata/libata-core.c:5186 ata_qc_issue+0x10a/0x347 [libata]()
Mar 31 14:49:04 test kernel: [  925.591878] Hardware name: X7DWU
Mar 31 14:49:04 test kernel: [  925.591931] Modules linked in: iscsi_trgt crc32c nfsd lockd nfs_acl auth_rpcgss sunrpc exportfs coretemp w83627hf w83793 hwmon_vid loop netconsole configfs container ehci_hcd uhci_hcd snd_pcm snd_timer evdev rng_core i5k_amb i2c_i801 ioatdma i2c_core usbcore snd soundcore snd_page_alloc nls_base i5400_edac edac_core pcspkr shpchp pci_hotplug processor button dm_mirror dm_region_hash dm_log dm_snapshot dm_mod raid456 md_mod async_raid6_recov async_pq raid6_pq async_xor xor async_memcpy async_tx thermal fan thermal_sys mvsas libsas scsi_transport_sas sata_mv e1000e igb dca ext3 jbd mbcache sd_mod crc_t10dif ata_piix libata scsi_mod
Mar 31 14:49:04 test kernel: [  925.595379] Pid: 6170, comm: smartctl Not tainted 2.6.32-bpo.3-amd64 #1
Mar 31 14:49:04 test kernel: [  925.595439] Call Trace:
Mar 31 14:49:04 test kernel: [  925.595498]  [<ffffffffa0034129>] ? ata_qc_issue+0x10a/0x347 [libata]
Mar 31 14:49:04 test kernel: [  925.595562]  [<ffffffffa0034129>] ? ata_qc_issue+0x10a/0x347 [libata]
Mar 31 14:49:04 test kernel: [  925.595626]  [<ffffffff8104dbdc>] ? warn_slowpath_common+0x77/0xa3
Mar 31 14:49:04 test kernel: [  925.595690]  [<ffffffffa0038471>] ? ata_scsi_pass_thru+0x0/0x238 [libata]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa0034129>] ? ata_qc_issue+0x10a/0x347 [libata]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa0038471>] ? ata_scsi_pass_thru+0x0/0x238 [libata]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa00008a5>] ? scsi_done+0x0/0xc [scsi_mod]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa003966a>] ? __ata_scsi_queuecmd+0x185/0x1dc [libata]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa00008a5>] ? scsi_done+0x0/0xc [scsi_mod]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa010ad48>] ? sas_queuecommand+0x93/0x283 [libsas]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa0000b77>] ? scsi_dispatch_cmd+0x1c0/0x23c [scsi_mod]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa000620c>] ? scsi_request_fn+0x3a5/0x506 [scsi_mod]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff81177bcc>] ? __blk_run_queue+0x35/0x66
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8116f940>] ? elv_insert+0xad/0x260
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8117afa0>] ? blk_execute_rq_nowait+0x5d/0x89
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8117b061>] ? blk_execute_rq+0x95/0xd0
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff811770a3>] ? __freed_request+0x26/0x82
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff81177122>] ? freed_request+0x23/0x41
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff81055ee6>] ? capable+0x22/0x41
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8117e1ed>] ? sg_io+0x280/0x3b5
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8117e821>] ? scsi_cmd_ioctl+0x217/0x3f2
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8103a537>] ? activate_task+0x20/0x26
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8104a178>] ? try_to_wake_up+0x249/0x259
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff810114ce>] ? common_interrupt+0xe/0x13
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff81150c21>] ? cap_file_permission+0x0/0x3
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff81051f29>] ? current_fs_time+0x1e/0x24
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffffa0079a7e>] ? sd_ioctl+0x9d/0xcb [sd_mod]
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8117c215>] ? __blkdev_driver_ioctl+0x69/0x7e
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8117ca10>] ? blkdev_ioctl+0x7e6/0x836
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff810c97a2>] ? __do_fault+0x38c/0x3c3
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff810ed1da>] ? do_sync_write+0xce/0x113
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8110fcf2>] ? block_ioctl+0x38/0x3c
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff810f8f02>] ? vfs_ioctl+0x21/0x6c
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff810f9450>] ? do_vfs_ioctl+0x48d/0x4cb
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff8103288f>] ? do_page_fault+0x266/0x282
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff810f94df>] ? sys_ioctl+0x51/0x70
Mar 31 14:49:04 test kernel: [  925.595692]  [<ffffffff81010b42>] ? system_call_fastpath+0x16/0x1b
Mar 31 14:49:04 test kernel: [  925.595692] ---[ end trace 96d81ffd0a33e7d3 ]---

RE: ATA / mvsas issue.

[jack wang]

Hi all,

Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in one
of our test servers.

I created a software raid5 with a hotspare on these 16 disks.

I noticed that the detection of the disks during boottime was very very
slow.
Creating the md superblocks went very very slow.

After that all seemed fine (speed of the raid5 init was normal).

I noticed many of these messages in the syslog (also attached):

Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-112'
Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-128'

This seems to correspond to the slow detection of the disks.

Is this an ATA problem, or a problem in the mvsas driver?

I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.

Kind regards,
Caspar Smit


[Jack] It's normal behavior of SATA disk, SATA disk need more time than SAS 
disk to spin up to response command.

Best Regards

************************************************************************
Jack  Wang   王金浦                 Software Engineer



--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ATA / mvsas issue.

[Caspar Smit]

> Hi all,
>
> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in one
> of our test servers.
>
> I created a software raid5 with a hotspare on these 16 disks.
>
> I noticed that the detection of the disks during boottime was very very
> slow.
> Creating the md superblocks went very very slow.
>
> After that all seemed fine (speed of the raid5 init was normal).
>
> I noticed many of these messages in the syslog (also attached):
>
> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
> '/dev/.tmp-8-112'
> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
> '/dev/.tmp-8-128'
>
> This seems to correspond to the slow detection of the disks.
>
> Is this an ATA problem, or a problem in the mvsas driver?
>
> I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.
>
> Kind regards,
> Caspar Smit
>
>
> [Jack] It's normal behavior of SATA disk, SATA disk need more time than
> SAS
> disk to spin up to response command.

I use loads of other SATA disks which don't have this behavior.

Caspar


>
> Best Regards
>
> ************************************************************************
> Jack  Wang   Íõ½ðÆÖ                 Software Engineer
>
>
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

--
To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

RE: ATA / mvsas issue.

[jack wang]

> Hi all,
>
> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in one
> of our test servers.
>
> I created a software raid5 with a hotspare on these 16 disks.
>
> I noticed that the detection of the disks during boottime was very very
> slow.
> Creating the md superblocks went very very slow.
>
> After that all seemed fine (speed of the raid5 init was normal).
>
> I noticed many of these messages in the syslog (also attached):
>
> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
> '/dev/.tmp-8-112'
> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
> '/dev/.tmp-8-128'
>
> This seems to correspond to the slow detection of the disks.
>
> Is this an ATA problem, or a problem in the mvsas driver?
>
> I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.
>
> Kind regards,
> Caspar Smit
>
>
> [Jack] It's normal behavior of SATA disk, SATA disk need more time than
> SAS
> disk to spin up to response command.

I use loads of other SATA disks which don't have this behavior.

Caspar


[Jack] In my test, some big SATA disks(ie 500G above)have this issues.
Best Regards

************************************************************************
Jack  Wang   

RE: ATA / mvsas issue.

[Caspar Smit]

>
>
>> Hi all,
>>
>> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in
>> one
>> of our test servers.
>>
>> I created a software raid5 with a hotspare on these 16 disks.
>>
>> I noticed that the detection of the disks during boottime was very very
>> slow.
>> Creating the md superblocks went very very slow.
>>
>> After that all seemed fine (speed of the raid5 init was normal).
>>
>> I noticed many of these messages in the syslog (also attached):
>>
>> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
>> '/dev/.tmp-8-112'
>> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
>> '/dev/.tmp-8-128'
>>
>> This seems to correspond to the slow detection of the disks.
>>
>> Is this an ATA problem, or a problem in the mvsas driver?
>>
>> I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.
>>
>> Kind regards,
>> Caspar Smit
>>
>>
>> [Jack] It's normal behavior of SATA disk, SATA disk need more time than
>> SAS
>> disk to spin up to response command.
>
> I use loads of other SATA disks which don't have this behavior.
>
> Caspar
>
>
> [Jack] In my test, some big SATA disks(ie 500G above)have this issues.
> Best Regards

Seagate 500G SATA disks are working fine.

Caspar

>
> ************************************************************************
> Jack  Wang
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

RE: ATA / mvsas issue.

[Caspar Smit]

>
>
>> Hi all,
>>
>> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in
>> one
>> of our test servers.
>>
>> I created a software raid5 with a hotspare on these 16 disks.
>>
>> I noticed that the detection of the disks during boottime was very very
>> slow.
>> Creating the md superblocks went very very slow.
>>
>> After that all seemed fine (speed of the raid5 init was normal).
>>
>> I noticed many of these messages in the syslog (also attached):
>>
>> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
>> '/dev/.tmp-8-112'
>> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
>> '/dev/.tmp-8-128'
>>
>> This seems to correspond to the slow detection of the disks.
>>
>> Is this an ATA problem, or a problem in the mvsas driver?
>>
>> I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.
>>
>> Kind regards,
>> Caspar Smit
>>
>>
>> [Jack] It's normal behavior of SATA disk, SATA disk need more time than
>> SAS
>> disk to spin up to response command.
>
> I use loads of other SATA disks which don't have this behavior.
>
> Caspar
>
>
> [Jack] In my test, some big SATA disks(ie 500G above)have this issues.
> Best Regards

There is definatly something wrong because an:

mkfs -t xfs -f /dev/md0

takes around 15! minutes to complete.

Normally this would take 15 seconds or so.

Caspar Smit


>
> ************************************************************************
> Jack  Wang
>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>

Re: ATA / mvsas issue.

[Stefan *St0fF* Huebner]

Am 01.04.2010 09:55, schrieb Caspar Smit:
>>
>>     
>>> Hi all,
>>>
>>> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in
>>> one
>>> of our test servers.
>>>
>>> I created a software raid5 with a hotspare on these 16 disks.
>>>
>>> I noticed that the detection of the disks during boottime was very very
>>> slow.
>>> Creating the md superblocks went very very slow.
>>>
>>> After that all seemed fine (speed of the raid5 init was normal).
>>>
>>> I noticed many of these messages in the syslog (also attached):
>>>
>>> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
>>> '/dev/.tmp-8-112'
>>> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
>>> '/dev/.tmp-8-128'
>>>
>>> This seems to correspond to the slow detection of the disks.
>>>
>>> Is this an ATA problem, or a problem in the mvsas driver?
>>>
>>> I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.
>>>
>>> Kind regards,
>>> Caspar Smit
>>>
>>>
>>> [Jack] It's normal behavior of SATA disk, SATA disk need more time than
>>> SAS
>>> disk to spin up to response command.
>>>       
>> I use loads of other SATA disks which don't have this behavior.
>>
>> Caspar
>>
>>
>> [Jack] In my test, some big SATA disks(ie 500G above)have this issues.
>> Best Regards
>>     
@Jack: sorry, but that doesn't cause unresponsiveness.  Some 2TB
SATA-Disks need >10s to spin up, but there shouldn't be any notes/logs
like those.  As a matter of fact: these 2,5" drives should spin up in a
matter of 2s.
> Seagate 500G SATA disks are working fine.
>   
Hitachi also.  And I've seen WD5000BEVT work great, too.  8 of a kind in
a QNAP SS-839 (Atom based NAS with a pretty recent linux on it) work fine!

Stefan
> Caspar
>
>   
>> ************************************************************************
>> Jack  Wang
>>
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>>     
> --
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>   

Re: ATA / mvsas issue.

[Joe Eykholt]

Stefan *St0fF* Huebner wrote:
> Am 01.04.2010 09:55, schrieb Caspar Smit:
>>>     
>>>> Hi all,
>>>>
>>>> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives in
>>>> one
>>>> of our test servers.
>>>>
>>>> I created a software raid5 with a hotspare on these 16 disks.
>>>>
>>>> I noticed that the detection of the disks during boottime was very very
>>>> slow.
>>>> Creating the md superblocks went very very slow.
>>>>
>>>> After that all seemed fine (speed of the raid5 init was normal).
>>>>
>>>> I noticed many of these messages in the syslog (also attached):
>>>>
>>>> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
>>>> '/dev/.tmp-8-112'
>>>> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
>>>> '/dev/.tmp-8-128'
>>>>
>>>> This seems to correspond to the slow detection of the disks.
>>>>
>>>> Is this an ATA problem, or a problem in the mvsas driver?
>>>>
>>>> I use the mvsas driver from 2.6.32 with the latest patch from Srinivas.
>>>>
>>>> Kind regards,
>>>> Caspar Smit
>>>>
>>>>
>>>> [Jack] It's normal behavior of SATA disk, SATA disk need more time than
>>>> SAS
>>>> disk to spin up to response command.
>>>>       
>>> I use loads of other SATA disks which don't have this behavior.
>>>
>>> Caspar
>>>
>>>
>>> [Jack] In my test, some big SATA disks(ie 500G above)have this issues.
>>> Best Regards
>>>     
> @Jack: sorry, but that doesn't cause unresponsiveness.  Some 2TB
> SATA-Disks need >10s to spin up, but there shouldn't be any notes/logs
> like those.  As a matter of fact: these 2,5" drives should spin up in a
> matter of 2s.

That seems to match the log that Caspar Smit posted yesterday.  Each drive
took about 2s to probe, and being that there are 16 of them, it took 32s.
It would be good if it could somehow do that in parallel being that this
is the 21st century :-)  Not sure how hard that is ...

I wonder if they spin up during probe or later during attach.

I don't know about the ata_id messages; I just wanted to point out the above.

Maybe ata_id is waiting for spin up but not waiting long enough if there are
so many disks and they spin up sequentially.  Spin up shouldn't be needed for
identity info, I would think.

	Joe

>> Seagate 500G SATA disks are working fine.
>>   
> Hitachi also.  And I've seen WD5000BEVT work great, too.  8 of a kind in
> a QNAP SS-839 (Atom based NAS with a pretty recent linux on it) work fine!
> 
> Stefan
>> Caspar
>>
>>   
>>> ************************************************************************
>>> Jack  Wang

Re: ATA / mvsas issue.

[Caspar Smit]

Thanks for all input but I can now definatly address this as an MVSAS issue.

I put other controllers (AOC-SAT2-MV8, which use the sata_mv kernel
driver) in the system and now all is working fine with the same disks!

Maybe someone can find out what is wrong with the mvsas code that causes
this behavior.

Kind regards,
Caspar Smit



> Stefan *St0fF* Huebner wrote:
>> Am 01.04.2010 09:55, schrieb Caspar Smit:
>>>>
>>>>> Hi all,
>>>>>
>>>>> Today I installed 16 fresh Western Digital WD5000BEVT (2,5") drives
>>>>> in
>>>>> one
>>>>> of our test servers.
>>>>>
>>>>> I created a software raid5 with a hotspare on these 16 disks.
>>>>>
>>>>> I noticed that the detection of the disks during boottime was very
>>>>> very
>>>>> slow.
>>>>> Creating the md superblocks went very very slow.
>>>>>
>>>>> After that all seemed fine (speed of the raid5 init was normal).
>>>>>
>>>>> I noticed many of these messages in the syslog (also attached):
>>>>>
>>>>> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
>>>>> '/dev/.tmp-8-112'
>>>>> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
>>>>> '/dev/.tmp-8-128'
>>>>>
>>>>> This seems to correspond to the slow detection of the disks.
>>>>>
>>>>> Is this an ATA problem, or a problem in the mvsas driver?
>>>>>
>>>>> I use the mvsas driver from 2.6.32 with the latest patch from
>>>>> Srinivas.
>>>>>
>>>>> Kind regards,
>>>>> Caspar Smit
>>>>>
>>>>>
>>>>> [Jack] It's normal behavior of SATA disk, SATA disk need more time
>>>>> than
>>>>> SAS
>>>>> disk to spin up to response command.
>>>>>
>>>> I use loads of other SATA disks which don't have this behavior.
>>>>
>>>> Caspar
>>>>
>>>>
>>>> [Jack] In my test, some big SATA disks(ie 500G above)have this issues.
>>>> Best Regards
>>>>
>> @Jack: sorry, but that doesn't cause unresponsiveness.  Some 2TB
>> SATA-Disks need >10s to spin up, but there shouldn't be any notes/logs
>> like those.  As a matter of fact: these 2,5" drives should spin up in a
>> matter of 2s.
>
> That seems to match the log that Caspar Smit posted yesterday.  Each drive
> took about 2s to probe, and being that there are 16 of them, it took 32s.
> It would be good if it could somehow do that in parallel being that this
> is the 21st century :-)  Not sure how hard that is ...
>
> I wonder if they spin up during probe or later during attach.
>
> I don't know about the ata_id messages; I just wanted to point out the
> above.
>
> Maybe ata_id is waiting for spin up but not waiting long enough if there
> are
> so many disks and they spin up sequentially.  Spin up shouldn't be needed
> for
> identity info, I would think.
>
> 	Joe
>
>>> Seagate 500G SATA disks are working fine.
>>>
>> Hitachi also.  And I've seen WD5000BEVT work great, too.  8 of a kind in
>> a QNAP SS-839 (Atom based NAS with a pretty recent linux on it) work
>> fine!
>>
>> Stefan
>>> Caspar
>>>
>>>
>>>> ************************************************************************
>>>> Jack  Wang
>

Still havind major MVSAS issues.

[Caspar Smit]

Hi Srinivas (and others),

I'm still having troubles with SATA disks using the mvsas driver. Maybe
you read my previous posts about the WD disks.

The problem is easyly recreated:

1) Create a software raidset (i used a 5 disk raid6 as a test)

2) Create an XFS filesystem on it. (mkfs.xfs -f /dev/md0)

I used the 2.6.32 kernel source using your patch.
I get these messages:

[ 5855.816019] drivers/scsi/mvsas/mv_sas.c 1631:mvs_query_task:rc= 5 [
5857.820002] mvsas 0000:04:00.0: Phy7 : No sig fis
[ 5858.024006] drivers/scsi/mvsas/mv_sas.c 1585:mvs_I_T_nexus_reset for
device[4]:rc= 0
[ 5858.024014] ata12: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ
0xb/00/00
[ 5858.028001] ata12.00: device reported invalid CHS sector 0
[ 5858.028001] ata12: status=0x01 { Error }
[ 5858.028001] ata12: error=0x04 { DriveStatusError }
[ 7201.988017] ata18: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ
0xb/00/00
[ 7201.988114] ata18: status=0x01 { Error }
[ 7201.988276] ata18: error=0x04 { DriveStatusError }

Especially the latest 3 messages I got loads more of those on another set
of disks. The creation of an XFS filesystem can take up to 15 minutes in
stead of the normal 15 seconds!

When I use the stock 2.6.32 kernel driver without your patch i get these
errors:

[ 6384.816015]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6384.816020]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6415.804016]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6415.804021]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6445.816017]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6445.816022]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6476.804015]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6476.804020]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5

So there has definatly changed something with the patch.

Further more during the detection of disks during boot I get these
messages for almost any SATA disk:

Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-112'
Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-128'

If you need more info let me know.

Kind regards,
Caspar Smit

Re: Still having major MVSAS issues.

[Jelle de Jong]

Hello everybody,

Caspar Smit wrote, on 09-04-10 17:52:
> I'm still having troubles with SATA disks using the mvsas driver. 
> Maybe you read my previous posts about the WD disks.

I would like to confirm a lot of the issues. I saw the emails from
Caspar Smits passing by the last few days. I got a bunch of the below
storage controllers for testing, and I am not yet able to actually use
them stable.

My major killer issue is that the I am unable to load my Linux kernel
and root file system from the SATA disks attached to the controller.
Grub works fine and starts the initrd, but then mdadm fails because
the hard disk are not found fast enough by the Linux kernel and I see
"HDIO_GET_IDENTITY failed for" messages for all the connected hard drives.

Some system and hardware information.

# cat /proc/version
Linux version 2.6.32-4-amd64 (Debian 2.6.32-11) (maks@debian.org) (gcc
version 4.3.4 (Debian 4.3.4-8) ) #1 SMP Tue Apr 6 08:21:34 UTC 2010

# lspci -nn | grep SCSI
02:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd.
MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)

# modinfo mvsas
filename: /lib/modules/2.6.32-4-amd64/../../scsi/mvsas/mvsas.ko
version: 0.8.2
description: Marvell 88SE6440 SAS/SATA controller driver
srcversion: EE82F304DFF3A7F06086B62

Messages I also got:

[15926.816036]
/build/buildd-linux-2.6_2.6.32-11-i386-4fW2zP/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5

And the HDIO_GET_IDENTITY failed for .. stuff during boot.

I also noticed that some of the attached drives worked fine with:
smartctl -d sat -H /dev/sda but some of the identical drives did not
worked and needed the -T permissive options...

BTW I also have some 04:06.0 SCSI storage controller [0100]: Marvell
Technology Group Ltd. MV88SX6081 8-port SATA II PCI-X Controller
[11ab:6081] (rev 09) hardware and those use the sata_mv module. This
system works stable and no errors during boot. However the SATA
interfaces are very slow here because of the PCI bus limitation of the
hardware.

The MV88SX6081 card also boots very vast to grub compared with the
MV64460/64461/64462 card. Is there something that can be done about
this? I would really like the usability of the MV88SX6081 card but
with the full speed of the SATA links of the PCI-E enabled mvsas cards.

For those intressted I use them with mdadm raid1 with two device
pairs, with lvm2 with striping and ext4 file systems. The root file
system is also on this raid,lvm,ext4 layout.

Thanks in advance,

Kind regards,

Jelle de Jong

RE: Still havind major MVSAS issues.

[Srinivas Naga Venkatasatya Pasagadugula - ERS, HCL Tech]

Hi Caspar,
I will check it once and let you know. But it will take some time to analyze this.  

--Srini.

-----Original Message-----
From: Caspar Smit [mailto:c.smit@truebit.nl] 
Sent: Friday, April 09, 2010 9:22 PM
To: Srinivas Naga Venkatasatya Pasagadugula - ERS, HCL Tech
Cc: linux-scsi@vger.kernel.org
Subject: Still havind major MVSAS issues.

Hi Srinivas (and others),

I'm still having troubles with SATA disks using the mvsas driver. Maybe
you read my previous posts about the WD disks.

The problem is easyly recreated:

1) Create a software raidset (i used a 5 disk raid6 as a test)

2) Create an XFS filesystem on it. (mkfs.xfs -f /dev/md0)

I used the 2.6.32 kernel source using your patch.
I get these messages:

[ 5855.816019] drivers/scsi/mvsas/mv_sas.c 1631:mvs_query_task:rc= 5 [
5857.820002] mvsas 0000:04:00.0: Phy7 : No sig fis
[ 5858.024006] drivers/scsi/mvsas/mv_sas.c 1585:mvs_I_T_nexus_reset for
device[4]:rc= 0
[ 5858.024014] ata12: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ
0xb/00/00
[ 5858.028001] ata12.00: device reported invalid CHS sector 0
[ 5858.028001] ata12: status=0x01 { Error }
[ 5858.028001] ata12: error=0x04 { DriveStatusError }
[ 7201.988017] ata18: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ
0xb/00/00
[ 7201.988114] ata18: status=0x01 { Error }
[ 7201.988276] ata18: error=0x04 { DriveStatusError }

Especially the latest 3 messages I got loads more of those on another set
of disks. The creation of an XFS filesystem can take up to 15 minutes in
stead of the normal 15 seconds!

When I use the stock 2.6.32 kernel driver without your patch i get these
errors:

[ 6384.816015]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6384.816020]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6415.804016]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6415.804021]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6445.816017]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6445.816022]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6476.804015]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6476.804020]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5

So there has definatly changed something with the patch.

Further more during the detection of disks during boot I get these
messages for almost any SATA disk:

Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-112'
Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-128'

If you need more info let me know.

Kind regards,
Caspar Smit

DISCLAIMER:
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. 
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in 
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates. 
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of 
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have 
received this email in error please delete it and notify the sender immediately. Before opening any mail and 
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

Re: Still havind major MVSAS issues.

[Jelle de Jong]

Dear Srinivas Naga Venkatasatya Pasagadugula,

Srinivas Naga Venkatasatya Pasagadugula wrote, on 12-04-10 06:21:
> I will check it once and let you know. But it will take some time to analyze this.

Is there information I can provide to help you speed up the analyses?

Thanks in advance,

Best regards,

Jelle de Jong

RE: Still havind major MVSAS issues.

[Srinivas Naga Venkatasatya Pasagadugula - ERS, HCL Tech]

Hi Caspar/Jelle,

Please provide me the following details.

1. Is this is the problem with only WD SATA drives? (I don't have WD SATA drives to reproduce this issue.)
2. Whether the problem is with direct attached SATA drives or drives connected in expanders also?
3. Could you please provide me the "dmesg" log or "/var/log/messages" log.  
4. How much the capacity of SATA drives connected to controller?
5. Your HBA is having 6440 chipset? 

--Srini.

-----Original Message-----
From: Caspar Smit [mailto:c.smit@truebit.nl] 
Sent: Friday, April 09, 2010 9:22 PM
To: Srinivas Naga Venkatasatya Pasagadugula - ERS, HCL Tech
Cc: linux-scsi@vger.kernel.org
Subject: Still havind major MVSAS issues.

Hi Srinivas (and others),

I'm still having troubles with SATA disks using the mvsas driver. Maybe
you read my previous posts about the WD disks.

The problem is easyly recreated:

1) Create a software raidset (i used a 5 disk raid6 as a test)

2) Create an XFS filesystem on it. (mkfs.xfs -f /dev/md0)

I used the 2.6.32 kernel source using your patch.
I get these messages:

[ 5855.816019] drivers/scsi/mvsas/mv_sas.c 1631:mvs_query_task:rc= 5 [
5857.820002] mvsas 0000:04:00.0: Phy7 : No sig fis
[ 5858.024006] drivers/scsi/mvsas/mv_sas.c 1585:mvs_I_T_nexus_reset for
device[4]:rc= 0
[ 5858.024014] ata12: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ
0xb/00/00
[ 5858.028001] ata12.00: device reported invalid CHS sector 0
[ 5858.028001] ata12: status=0x01 { Error }
[ 5858.028001] ata12: error=0x04 { DriveStatusError }
[ 7201.988017] ata18: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ
0xb/00/00
[ 7201.988114] ata18: status=0x01 { Error }
[ 7201.988276] ata18: error=0x04 { DriveStatusError }

Especially the latest 3 messages I got loads more of those on another set
of disks. The creation of an XFS filesystem can take up to 15 minutes in
stead of the normal 15 seconds!

When I use the stock 2.6.32 kernel driver without your patch i get these
errors:

[ 6384.816015]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6384.816020]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6415.804016]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6415.804021]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6445.816017]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6445.816022]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
[ 6476.804015]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
[ 6476.804020]
/tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5

So there has definatly changed something with the patch.

Further more during the detection of disks during boot I get these
messages for almost any SATA disk:

Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-112'
Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-128'

If you need more info let me know.

Kind regards,
Caspar Smit

DISCLAIMER:
-----------------------------------------------------------------------------------------------------------------------

The contents of this e-mail and any attachment(s) are confidential and intended for the named recipient(s) only. 
It shall not attach any liability on the originator or HCL or its affiliates. Any views or opinions presented in 
this email are solely those of the author and may not necessarily reflect the opinions of HCL or its affiliates. 
Any form of reproduction, dissemination, copying, disclosure, modification, distribution and / or publication of 
this message without the prior written consent of the author of this e-mail is strictly prohibited. If you have 
received this email in error please delete it and notify the sender immediately. Before opening any mail and 
attachments please check them for viruses and defect.

-----------------------------------------------------------------------------------------------------------------------

Re: Still havind major MVSAS issues.

[Jelle de Jong]

Dear Srinivas Naga Venkatastsa Pesagadugala,

Srinivas Naga Venkatasatya Pasagadugula, on 06-05-10 08:01:
> 1. Is this is the problem with only WD SATA drives? (I don't have
> WD SATA drives to reproduce this issue.) 2. Whether the problem is
> with direct attached SATA drives or drives connected in expanders
> also? 3. Could you please provide me the "dmesg" log or
> "/var/log/messages" log. 4. How much the capacity of SATA drives
> connected to controller? 5. Your HBA is having 6440 chipset?

I will check all points next weekend.

I tested it only with WD hard drives. I will attach some Samsung and
Segate drivers.

There where directly attached to the controller. There is are
backplanes in use, but they do not contain any expanders.

dmesg reports will be there this weekend.

All WD drivers where 1 TB, I will test some other drivers sizes this
weekend.

Thank you in advance,

WIth kind regards,

Jelle de Jong

RE: Still havind major MVSAS issues.

[Caspar Smit]

[-- Attachment #1: Type: text/plain, Size: 5690 bytes --]

> Hi Caspar/Jelle,
>
> Please provide me the following details.
>
> 1. Is this is the problem with only WD SATA drives? (I don't have WD
SATA
> drives to reproduce this issue.)

No, I have the same error messages using Seagate 500Gb 2,5" SATA disks
(ST9500420AS) and also with 160Gb 2,5" Intel X-25 SATA SSD's (SSDSA2M160).

> 2. Whether the problem is with direct attached SATA drives or drives
connected in expanders also?

The contoller is connected to a backplane (SuperMicro SAS216A). As fas as
I know there are no expanders involved.

> 3. Could you please provide me the "dmesg" log or "/var/log/messages"
log.

See attachments.

> 4. How much the capacity of SATA drives connected to controller?

See point 1.

> 5. Your HBA is having 6440 chipset?

I have 3x SuperMicro AOC-SASLP-MV8 controllers in one system.

The user guide says it uses the Marvell 88SE6480 chipset.

This is the output of lspci.

#lspci |grep MV

04:00.0 SCSI storage controller: Marvell Technology Group Ltd.
MV64460/64461/64462 System Controller, Revision B (rev 01)
0c:00.0 SCSI storage controller: Marvell Technology Group Ltd.
MV64460/64461/64462 System Controller, Revision B (rev 01)
0e:00.0 SCSI storage controller: Marvell Technology Group Ltd.
MV64460/64461/64462 System Controller, Revision B (rev 01)

If you need anymore info, please let me know.

Thanks for all your efforts.

Kind regards,
Caspar Smit

>
> --Srini.
>
> -----Original Message-----
> From: Caspar Smit [mailto:c.smit@truebit.nl]
> Sent: Friday, April 09, 2010 9:22 PM
> To: Srinivas Naga Venkatasatya Pasagadugula - ERS, HCL Tech
> Cc: linux-scsi@vger.kernel.org
> Subject: Still havind major MVSAS issues.
>
> Hi Srinivas (and others),
>
> I'm still having troubles with SATA disks using the mvsas driver. Maybe
you read my previous posts about the WD disks.
>
> The problem is easyly recreated:
>
> 1) Create a software raidset (i used a 5 disk raid6 as a test)
>
> 2) Create an XFS filesystem on it. (mkfs.xfs -f /dev/md0)
>
> I used the 2.6.32 kernel source using your patch.
> I get these messages:
>
> [ 5855.816019] drivers/scsi/mvsas/mv_sas.c 1631:mvs_query_task:rc= 5 [
5857.820002] mvsas 0000:04:00.0: Phy7 : No sig fis
> [ 5858.024006] drivers/scsi/mvsas/mv_sas.c 1585:mvs_I_T_nexus_reset for
device[4]:rc= 0
> [ 5858.024014] ata12: translated ATA stat/err 0x01/04 to SCSI
SK/ASC/ASCQ
> 0xb/00/00
> [ 5858.028001] ata12.00: device reported invalid CHS sector 0
> [ 5858.028001] ata12: status=0x01 { Error }
> [ 5858.028001] ata12: error=0x04 { DriveStatusError }
> [ 7201.988017] ata18: translated ATA stat/err 0x01/04 to SCSI
SK/ASC/ASCQ
> 0xb/00/00
> [ 7201.988114] ata18: status=0x01 { Error }
> [ 7201.988276] ata18: error=0x04 { DriveStatusError }
>
> Especially the latest 3 messages I got loads more of those on another
set
> of disks. The creation of an XFS filesystem can take up to 15 minutes in
stead of the normal 15 seconds!
>
> When I use the stock 2.6.32 kernel driver without your patch i get these
errors:
>
> [ 6384.816015]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
> [ 6384.816020]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
> [ 6415.804016]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
> [ 6415.804021]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
> [ 6445.816017]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
> [ 6445.816022]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
> [ 6476.804015]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1669:mvs_abort_task:rc= 5
> [ 6476.804020]
> /tmp/buildd/linux-2.6-2.6.32/debian/build/source_amd64_none/drivers/scsi/mvsas/mv_sas.c
1608:mvs_query_task:rc= 5
>
> So there has definatly changed something with the patch.
>
> Further more during the detection of disks during boot I get these
messages for almost any SATA disk:
>
> Mar 31 14:34:17 test ata_id[3752]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-112'
> Mar 31 14:34:17 test ata_id[3765]: main: HDIO_GET_IDENTITY failed for
'/dev/.tmp-8-128'
>
> If you need more info let me know.
>
> Kind regards,
> Caspar Smit
>
> DISCLAIMER:
> -----------------------------------------------------------------------------------------------------------------------
>
> The contents of this e-mail and any attachment(s) are confidential and
intended for the named recipient(s) only.
> It shall not attach any liability on the originator or HCL or its
affiliates. Any views or opinions presented in
> this email are solely those of the author and may not necessarily
reflect
> the opinions of HCL or its affiliates.
> Any form of reproduction, dissemination, copying, disclosure,
> modification, distribution and / or publication of
> this message without the prior written consent of the author of this
e-mail is strictly prohibited. If you have
> received this email in error please delete it and notify the sender
immediately. Before opening any mail and
> attachments please check them for viruses and defect.
>
> -----------------------------------------------------------------------------------------------------------------------
--
> To unsubscribe from this list: send the line "unsubscribe linux-scsi" in
the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>


[-- Attachment #2: dmesg.txt --]
[-- Type: text/plain, Size: 93811 bytes --]

[    0.000000] Initializing cgroup subsys cpuset
[    0.000000] Initializing cgroup subsys cpu
[    0.000000] Linux version 2.6.32-bpo.4-amd64 (Debian 2.6.32-11~bpo50+1) (norbert@tretkowski.de) (gcc version 4.3.2 (Debian 4.3.2-1.1) ) #1 SMP Thu Apr 8 10:20:24 UTC 2010
[    0.000000] Command line: root=/dev/sda1 ro elevator=deadline edd=off 
[    0.000000] KERNEL supported cpus:
[    0.000000]   Intel GenuineIntel
[    0.000000]   AMD AuthenticAMD
[    0.000000]   Centaur CentaurHauls
[    0.000000] BIOS-provided physical RAM map:
[    0.000000]  BIOS-e820: 0000000000000000 - 000000000009b800 (usable)
[    0.000000]  BIOS-e820: 000000000009b800 - 00000000000a0000 (reserved)
[    0.000000]  BIOS-e820: 00000000000e4000 - 0000000000100000 (reserved)
[    0.000000]  BIOS-e820: 0000000000100000 - 00000000cff50000 (usable)
[    0.000000]  BIOS-e820: 00000000cff50000 - 00000000cff61000 (ACPI data)
[    0.000000]  BIOS-e820: 00000000cff61000 - 00000000cff62000 (ACPI NVS)
[    0.000000]  BIOS-e820: 00000000cff62000 - 00000000d0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000e0000000 - 00000000f0000000 (reserved)
[    0.000000]  BIOS-e820: 00000000fec00000 - 00000000fec10000 (reserved)
[    0.000000]  BIOS-e820: 00000000fee00000 - 00000000fee01000 (reserved)
[    0.000000]  BIOS-e820: 00000000ff000000 - 0000000100000000 (reserved)
[    0.000000]  BIOS-e820: 0000000100000000 - 0000000130000000 (usable)
[    0.000000] DMI present.
[    0.000000] Phoenix BIOS detected: BIOS may corrupt low RAM, working around it.
[    0.000000] e820 update range: 0000000000000000 - 0000000000010000 (usable) ==> (reserved)
[    0.000000] last_pfn = 0x130000 max_arch_pfn = 0x400000000
[    0.000000] MTRR default type: uncachable
[    0.000000] MTRR fixed ranges enabled:
[    0.000000]   00000-9FFFF write-back
[    0.000000]   A0000-BFFFF uncachable
[    0.000000]   C0000-CFFFF write-protect
[    0.000000]   D0000-E3FFF uncachable
[    0.000000]   E4000-FFFFF write-protect
[    0.000000] MTRR variable ranges enabled:
[    0.000000]   0 base 00D0000000 mask 3FF0000000 uncachable
[    0.000000]   1 base 00E0000000 mask 3FE0000000 uncachable
[    0.000000]   2 base 0000000000 mask 3F00000000 write-back
[    0.000000]   3 base 0100000000 mask 3FE0000000 write-back
[    0.000000]   4 base 0120000000 mask 3FF0000000 write-back
[    0.000000]   5 base 00CFF80000 mask 3FFFF80000 uncachable
[    0.000000]   6 disabled
[    0.000000]   7 disabled
[    0.000000] x86 PAT enabled: cpu 0, old 0x7040600070406, new 0x7010600070106
[    0.000000] e820 update range: 00000000cff80000 - 0000000100000000 (usable) ==> (reserved)
[    0.000000] last_pfn = 0xcff50 max_arch_pfn = 0x400000000
[    0.000000] initial memory mapped : 0 - 20000000
[    0.000000] init_memory_mapping: 0000000000000000-00000000cff50000
[    0.000000]  0000000000 - 00cfe00000 page 2M
[    0.000000]  00cfe00000 - 00cff50000 page 4k
[    0.000000] kernel direct mapping tables up to cff50000 @ 10000-16000
[    0.000000] init_memory_mapping: 0000000100000000-0000000130000000
[    0.000000]  0100000000 - 0130000000 page 2M
[    0.000000] kernel direct mapping tables up to 130000000 @ 14000-1a000
[    0.000000] RAMDISK: 37d40000 - 37fefbd0
[    0.000000] ACPI: RSDP 00000000000f6d60 00024 (v02 PTLTD )
[    0.000000] ACPI: XSDT 00000000cff59bce 000D4 (v01 PTLTD  ? XSDT   06040000  LTP 00000000)
[    0.000000] ACPI: FACP 00000000cff60300 000F4 (v03 INTEL  STOAKLEY 06040000 PTL  00000003)
[    0.000000] ACPI: DSDT 00000000cff5b790 04AFC (v01  Intel SEABURG  06040000 MSFT 03000001)
[    0.000000] ACPI: FACS 00000000cff61fc0 00040
[    0.000000] ACPI: DMAR 00000000cff603f4 00110 (v01 Intel  OEMDMAR  06040000 LOHR 00000001)
[    0.000000] ACPI: TCPA 00000000cff60504 00032 (v01 Intel  STOAKLEY 06040000 LOHR 0000005A)
[    0.000000] ACPI: APIC 00000000cff60536 00090 (v01 PTLTD  ? APIC   06040000  LTP 00000000)
[    0.000000] ACPI: MCFG 00000000cff605c6 0003C (v01 PTLTD    MCFG   06040000  LTP 00000000)
[    0.000000] ACPI: HPET 00000000cff60602 00038 (v01 PTLTD  HPETTBL  06040000  LTP 00000001)
[    0.000000] ACPI: BOOT 00000000cff6063a 00028 (v01 PTLTD  $SBFTBL$ 06040000  LTP 00000001)
[    0.000000] ACPI: SPCR 00000000cff60662 00050 (v01 PTLTD  $UCRTBL$ 06040000 PTL  00000001)
[    0.000000] ACPI: ERST 00000000cff606b2 00590 (v01 SMCI   ERSTTBL  06040000 SMCI 00000001)
[    0.000000] ACPI: HEST 00000000cff60c42 000A8 (v01 SMCI   HESTTBL  06040000 SMCI 00000001)
[    0.000000] ACPI: BERT 00000000cff60cea 00030 (v01 SMCI   BERTTBL  06040000 SMCI 00000001)
[    0.000000] ACPI: EINJ 00000000cff60d1a 00170 (v01 SMCI   EINJTBL  06040000 SMCI 00000001)
[    0.000000] ACPI: SLIC 00000000cff60e8a 00176 (v01 OEM_ID OEMTABLE 06040000  LTP 00000000)
[    0.000000] ACPI: SSDT 00000000cff5b531 0025F (v01  PmRef  Cpu0Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b48b 000A6 (v01  PmRef  Cpu7Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b3e5 000A6 (v01  PmRef  Cpu6Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b33f 000A6 (v01  PmRef  Cpu5Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b299 000A6 (v01  PmRef  Cpu4Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b1f3 000A6 (v01  PmRef  Cpu3Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b14d 000A6 (v01  PmRef  Cpu2Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff5b0a7 000A6 (v01  PmRef  Cpu1Tst 00003000 INTL 20050228)
[    0.000000] ACPI: SSDT 00000000cff59ca2 01405 (v01  PmRef    CpuPm 00003000 INTL 20050228)
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] No NUMA configuration found
[    0.000000] Faking a node at 0000000000000000-0000000130000000
[    0.000000] Bootmem setup node 0 0000000000000000-0000000130000000
[    0.000000]   NODE_DATA [0000000000015000 - 000000000001cfff]
[    0.000000]   bootmap [000000000001d000 -  0000000000042fff] pages 26
[    0.000000] (8 early reservations) ==> bootmem [0000000000 - 0130000000]
[    0.000000]   #0 [0000000000 - 0000001000]   BIOS data page ==> [0000000000 - 0000001000]
[    0.000000]   #1 [0000006000 - 0000008000]       TRAMPOLINE ==> [0000006000 - 0000008000]
[    0.000000]   #2 [0001000000 - 000166ea94]    TEXT DATA BSS ==> [0001000000 - 000166ea94]
[    0.000000]   #3 [0037d40000 - 0037fefbd0]          RAMDISK ==> [0037d40000 - 0037fefbd0]
[    0.000000]   #4 [000009b800 - 0000100000]    BIOS reserved ==> [000009b800 - 0000100000]
[    0.000000]   #5 [000166f000 - 000166f1ac]              BRK ==> [000166f000 - 000166f1ac]
[    0.000000]   #6 [0000010000 - 0000014000]          PGTABLE ==> [0000010000 - 0000014000]
[    0.000000]   #7 [0000014000 - 0000015000]          PGTABLE ==> [0000014000 - 0000015000]
[    0.000000] found SMP MP-table at [ffff8800000f6d90] f6d90
[    0.000000]  [ffffea0000000000-ffffea00043fffff] PMD -> [ffff880001800000-ffff8800051fffff] on node 0
[    0.000000] Zone PFN ranges:
[    0.000000]   DMA      0x00000010 -> 0x00001000
[    0.000000]   DMA32    0x00001000 -> 0x00100000
[    0.000000]   Normal   0x00100000 -> 0x00130000
[    0.000000] Movable zone start PFN for each node
[    0.000000] early_node_map[3] active PFN ranges
[    0.000000]     0: 0x00000010 -> 0x0000009b
[    0.000000]     0: 0x00000100 -> 0x000cff50
[    0.000000]     0: 0x00100000 -> 0x00130000
[    0.000000] On node 0 totalpages: 1048283
[    0.000000]   DMA zone: 56 pages used for memmap
[    0.000000]   DMA zone: 108 pages reserved
[    0.000000]   DMA zone: 3815 pages, LIFO batch:0
[    0.000000]   DMA32 zone: 14280 pages used for memmap
[    0.000000]   DMA32 zone: 833416 pages, LIFO batch:31
[    0.000000]   Normal zone: 2688 pages used for memmap
[    0.000000]   Normal zone: 193920 pages, LIFO batch:31
[    0.000000] ACPI: PM-Timer IO Port: 0x1008
[    0.000000] ACPI: Local APIC address 0xfee00000
[    0.000000] ACPI: LAPIC (acpi_id[0x00] lapic_id[0x00] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x01] lapic_id[0x01] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x02] lapic_id[0x02] enabled)
[    0.000000] ACPI: LAPIC (acpi_id[0x03] lapic_id[0x03] enabled)
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x00] high edge lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x01] high edge lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x02] high edge lint[0x1])
[    0.000000] ACPI: LAPIC_NMI (acpi_id[0x03] high edge lint[0x1])
[    0.000000] ACPI: IOAPIC (id[0x04] address[0xfec00000] gsi_base[0])
[    0.000000] IOAPIC[0]: apic_id 4, version 32, address 0xfec00000, GSI 0-23
[    0.000000] ACPI: IOAPIC (id[0x05] address[0xfec89000] gsi_base[24])
[    0.000000] IOAPIC[1]: apic_id 5, version 32, address 0xfec89000, GSI 24-47
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 0 global_irq 2 high edge)
[    0.000000] ACPI: INT_SRC_OVR (bus 0 bus_irq 9 global_irq 9 high level)
[    0.000000] ACPI: IRQ0 used by override.
[    0.000000] ACPI: IRQ2 used by override.
[    0.000000] ACPI: IRQ9 used by override.
[    0.000000] Using ACPI (MADT) for SMP configuration information
[    0.000000] ACPI: HPET id: 0x8086a201 base: 0xfed00000
[    0.000000] SMP: Allowing 4 CPUs, 0 hotplug CPUs
[    0.000000] nr_irqs_gsi: 48
[    0.000000] PM: Registered nosave memory: 000000000009b000 - 000000000009c000
[    0.000000] PM: Registered nosave memory: 000000000009c000 - 00000000000a0000
[    0.000000] PM: Registered nosave memory: 00000000000a0000 - 00000000000e4000
[    0.000000] PM: Registered nosave memory: 00000000000e4000 - 0000000000100000
[    0.000000] PM: Registered nosave memory: 00000000cff50000 - 00000000cff61000
[    0.000000] PM: Registered nosave memory: 00000000cff61000 - 00000000cff62000
[    0.000000] PM: Registered nosave memory: 00000000cff62000 - 00000000d0000000
[    0.000000] PM: Registered nosave memory: 00000000d0000000 - 00000000e0000000
[    0.000000] PM: Registered nosave memory: 00000000e0000000 - 00000000f0000000
[    0.000000] PM: Registered nosave memory: 00000000f0000000 - 00000000fec00000
[    0.000000] PM: Registered nosave memory: 00000000fec00000 - 00000000fec10000
[    0.000000] PM: Registered nosave memory: 00000000fec10000 - 00000000fee00000
[    0.000000] PM: Registered nosave memory: 00000000fee00000 - 00000000fee01000
[    0.000000] PM: Registered nosave memory: 00000000fee01000 - 00000000ff000000
[    0.000000] PM: Registered nosave memory: 00000000ff000000 - 0000000100000000
[    0.000000] Allocating PCI resources starting at d0000000 (gap: d0000000:10000000)
[    0.000000] Booting paravirtualized kernel on bare hardware
[    0.000000] NR_CPUS:512 nr_cpumask_bits:512 nr_cpu_ids:4 nr_node_ids:1
[    0.000000] PERCPU: Embedded 29 pages/cpu @ffff880005400000 s89880 r8192 d20712 u524288
[    0.000000] pcpu-alloc: s89880 r8192 d20712 u524288 alloc=1*2097152
[    0.000000] pcpu-alloc: [0] 0 1 2 3 
[    0.000000] Built 1 zonelists in Node order, mobility grouping on.  Total pages: 1031151
[    0.000000] Policy zone: Normal
[    0.000000] Kernel command line: root=/dev/sda1 ro elevator=deadline edd=off 
[    0.000000] PID hash table entries: 4096 (order: 3, 32768 bytes)
[    0.000000] Initializing CPU#0
[    0.000000] xsave/xrstor: enabled xstate_bv 0x3, cntxt size 0x240
[    0.000000] Checking aperture...
[    0.000000] No AGP bridge found
[    0.000000] Calgary: detecting Calgary via BIOS EBDA area
[    0.000000] Calgary: Unable to locate Rio Grande table in EBDA - bailing!
[    0.000000] PCI-DMA: Using software bounce buffering for IO (SWIOTLB)
[    0.000000] Placing 64MB software IO TLB between ffff88000559d000 - ffff88000959d000
[    0.000000] software IO TLB at phys 0x559d000 - 0x959d000
[    0.000000] Memory: 4057552k/4980736k available (3032k kernel code, 787604k absent, 135580k reserved, 1854k data, 580k init)
[    0.000000] SLUB: Genslabs=14, HWalign=64, Order=0-3, MinObjects=0, CPUs=4, Nodes=1
[    0.000000] Hierarchical RCU implementation.
[    0.000000] NR_IRQS:4352 nr_irqs:848
[    0.000000] Extended CMOS year: 2000
[    0.000000] Console: colour VGA+ 80x25
[    0.000000] console [tty0] enabled
[    0.000000] hpet clockevent registered
[    0.000000] HPET: 3 timers in total, 0 timers will be used for per-cpu timer
[    0.000000] Fast TSC calibration using PIT
[    0.000000] Detected 2000.178 MHz processor.
[    0.004006] Calibrating delay loop (skipped), value calculated using timer frequency.. 4000.35 BogoMIPS (lpj=8000712)
[    0.004162] Security Framework initialized
[    0.004235] SELinux:  Disabled at boot.
[    0.004637] Dentry cache hash table entries: 524288 (order: 10, 4194304 bytes)
[    0.009638] Inode-cache hash table entries: 262144 (order: 9, 2097152 bytes)
[    0.010711] Mount-cache hash table entries: 256
[    0.010942] Initializing cgroup subsys ns
[    0.011010] Initializing cgroup subsys cpuacct
[    0.011080] Initializing cgroup subsys devices
[    0.011148] Initializing cgroup subsys freezer
[    0.011216] Initializing cgroup subsys net_cls
[    0.011310] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.011423] CPU: L2 cache: 6144K
[    0.011490] CPU 0/0x0 -> Node 0
[    0.011555] CPU: Physical Processor ID: 0
[    0.011622] CPU: Processor Core ID: 0
[    0.011690] mce: CPU supports 6 MCE banks
[    0.011762] CPU0: Thermal monitoring enabled (TM2)
[    0.011833] using mwait in idle threads.
[    0.012007] Performance Events: Core2 events, Intel PMU driver.
[    0.012170] ... version:                2
[    0.012237] ... bit width:              40
[    0.012303] ... generic registers:      2
[    0.012370] ... value mask:             000000ffffffffff
[    0.012439] ... max period:             000000007fffffff
[    0.012508] ... fixed-purpose events:   3
[    0.012574] ... event mask:             0000000700000003
[    0.016025] ACPI: Core revision 20090903
[    0.027400] DMAR: Host address width 38
[    0.027469] DMAR: DRHD base: 0x000000fe710000 flags: 0x0
[    0.027544] IOMMU fe710000: ver 1:0 cap 900800c2f0462 ecap e01
[    0.027616] DMAR: DRHD base: 0x000000fe712000 flags: 0x0
[    0.027695] IOMMU fe712000: ver 1:0 cap 900800c2f0462 ecap e01
[    0.027766] DMAR: DRHD base: 0x000000fe714000 flags: 0x0
[    0.027840] IOMMU fe714000: ver 1:0 cap 900800c2f0462 ecap e01
[    0.027912] DMAR: DRHD base: 0x000000fe716000 flags: 0x0
[    0.027985] IOMMU fe716000: ver 1:0 cap 900800c2f0462 ecap e01
[    0.028005] DMAR: DRHD base: 0x000000fe719000 flags: 0x0
[    0.028080] IOMMU fe719000: ver 1:0 cap 900800c2f0462 ecap e01
[    0.028151] DMAR: DRHD base: 0x000000fe71a000 flags: 0x0
[    0.028228] IOMMU fe71a000: ver 1:0 cap 4900800c2f0462 ecap e01
[    0.028300] DMAR: DRHD base: 0x000000fe718000 flags: 0x1
[    0.028374] IOMMU fe718000: ver 1:0 cap 900800c2f0462 ecap e01
[    0.028445] DMAR: RMRR base: 0x000000cff6b000 end: 0x000000cff72fff
[    0.028517] DMAR: No ATSR found
[    0.028590] Setting APIC routing to flat
[    0.029038] ..TIMER: vector=0x30 apic1=0 pin1=2 apic2=-1 pin2=-1
[    0.068868] CPU0: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz stepping 0a
[    0.072001] Booting processor 1 APIC 0x1 ip 0x6000
[    0.008000] Initializing CPU#1
[    0.008000] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.008000] CPU: L2 cache: 6144K
[    0.008000] CPU 1/0x1 -> Node 0
[    0.008000] CPU: Physical Processor ID: 0
[    0.008000] CPU: Processor Core ID: 1
[    0.008000] CPU1: Thermal monitoring enabled (TM2)
[    0.156101] CPU1: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz stepping 0a
[    0.156725] checking TSC synchronization [CPU#0 -> CPU#1]: passed.
[    0.160123] Booting processor 2 APIC 0x2 ip 0x6000
[    0.008000] Initializing CPU#2
[    0.008000] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.008000] CPU: L2 cache: 6144K
[    0.008000] CPU 2/0x2 -> Node 0
[    0.008000] CPU: Physical Processor ID: 0
[    0.008000] CPU: Processor Core ID: 2
[    0.008000] CPU2: Thermal monitoring enabled (TM2)
[    0.248103] CPU2: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz stepping 0a
[    0.248726] checking TSC synchronization [CPU#0 -> CPU#2]: passed.
[    0.252078] Booting processor 3 APIC 0x3 ip 0x6000
[    0.008000] Initializing CPU#3
[    0.008000] CPU: L1 I cache: 32K, L1 D cache: 32K
[    0.008000] CPU: L2 cache: 6144K
[    0.008000] CPU 3/0x3 -> Node 0
[    0.008000] CPU: Physical Processor ID: 0
[    0.008000] CPU: Processor Core ID: 3
[    0.008000] CPU3: Thermal monitoring enabled (TM2)
[    0.340057] CPU3: Intel(R) Xeon(R) CPU           E5405  @ 2.00GHz stepping 0a
[    0.340680] checking TSC synchronization [CPU#0 -> CPU#3]: passed.
[    0.344014] Brought up 4 CPUs
[    0.344081] Total of 4 processors activated (16000.79 BogoMIPS).
[    0.348022] CPU0 attaching sched-domain:
[    0.348026]  domain 0: span 0-1 level MC
[    0.348029]   groups: 0 1
[    0.348033]   domain 1: span 0-3 level CPU
[    0.348036]    groups: 0-1 (cpu_power = 2048) 2-3 (cpu_power = 2048)
[    0.348044] CPU1 attaching sched-domain:
[    0.348046]  domain 0: span 0-1 level MC
[    0.348048]   groups: 1 0
[    0.348052]   domain 1: span 0-3 level CPU
[    0.348054]    groups: 0-1 (cpu_power = 2048) 2-3 (cpu_power = 2048)
[    0.348060] CPU2 attaching sched-domain:
[    0.348062]  domain 0: span 2-3 level MC
[    0.348064]   groups: 2 3
[    0.348067]   domain 1: span 0-3 level CPU
[    0.348069]    groups: 2-3 (cpu_power = 2048) 0-1 (cpu_power = 2048)
[    0.348075] CPU3 attaching sched-domain:
[    0.348077]  domain 0: span 2-3 level MC
[    0.348079]   groups: 3 2
[    0.348083]   domain 1: span 0-3 level CPU
[    0.348085]    groups: 2-3 (cpu_power = 2048) 0-1 (cpu_power = 2048)
[    0.348260] devtmpfs: initialized
[    0.352140] regulator: core version 0.5
[    0.352147] NET: Registered protocol family 16
[    0.352317] ACPI: bus type pci registered
[    0.352468] PCI: MCFG configuration 0: base e0000000 segment 0 buses 0 - 16
[    0.352541] PCI: MCFG area at e0000000 reserved in E820
[    0.353429] PCI: Using MMCONFIG at e0000000 - e10fffff
[    0.353498] PCI: Using configuration type 1 for base access
[    0.353960] bio: create slab <bio-0> at 0
[    0.353960] ACPI: EC: Look up EC in DSDT
[    0.356088] ACPI Warning for \_SB_._OSC: Return type mismatch - found Integer, expected Buffer (20090903/nspredef-1006)
[    0.356277] \_SB_:_OSC evaluation returned wrong type
[    0.356278] _OSC request data:1 7 
[    0.358158] ACPI: Interpreter enabled
[    0.358227] ACPI: (supports S0 S1 S4 S5)
[    0.358489] ACPI: Using IOAPIC for interrupt routing
[    0.362674] ACPI: No dock devices found.
[    0.363721] ACPI: PCI Root Bridge [PCI0] (0000:00)
[    0.363863] pci 0000:00:00.0: PME# supported from D0 D3hot D3cold
[    0.363936] pci 0000:00:00.0: PME# disabled
[    0.364051] pci 0000:00:01.0: PME# supported from D0 D3hot D3cold
[    0.364123] pci 0000:00:01.0: PME# disabled
[    0.364240] pci 0000:00:03.0: PME# supported from D0 D3hot D3cold
[    0.364312] pci 0000:00:03.0: PME# disabled
[    0.364427] pci 0000:00:05.0: PME# supported from D0 D3hot D3cold
[    0.364499] pci 0000:00:05.0: PME# disabled
[    0.364613] pci 0000:00:07.0: PME# supported from D0 D3hot D3cold
[    0.364685] pci 0000:00:07.0: PME# disabled
[    0.364800] pci 0000:00:09.0: PME# supported from D0 D3hot D3cold
[    0.364872] pci 0000:00:09.0: PME# disabled
[    0.364968] pci 0000:00:0f.0: reg 10 64bit mmio: [0xfe700000-0xfe703fff]
[    0.365274] pci 0000:00:1d.0: reg 20 io port: [0x1800-0x181f]
[    0.365322] pci 0000:00:1d.1: reg 20 io port: [0x1820-0x183f]
[    0.365369] pci 0000:00:1d.2: reg 20 io port: [0x1840-0x185f]
[    0.365415] pci 0000:00:1d.3: reg 20 io port: [0x1860-0x187f]
[    0.365467] pci 0000:00:1d.7: reg 10 32bit mmio: [0xd8b04000-0xd8b043ff]
[    0.365515] pci 0000:00:1d.7: PME# supported from D0 D3hot D3cold
[    0.365589] pci 0000:00:1d.7: PME# disabled
[    0.365785] pci 0000:00:1f.2: reg 10 io port: [0x18a0-0x18a7]
[    0.365790] pci 0000:00:1f.2: reg 14 io port: [0x1894-0x1897]
[    0.365796] pci 0000:00:1f.2: reg 18 io port: [0x1898-0x189f]
[    0.365802] pci 0000:00:1f.2: reg 1c io port: [0x1890-0x1893]
[    0.365807] pci 0000:00:1f.2: reg 20 io port: [0x1880-0x188f]
[    0.365813] pci 0000:00:1f.2: reg 24 32bit mmio: [0xd8b04400-0xd8b047ff]
[    0.365832] pci 0000:00:1f.2: PME# supported from D3hot
[    0.365903] pci 0000:00:1f.2: PME# disabled
[    0.366012] pci 0000:00:1f.3: reg 20 io port: [0x1100-0x111f]
[    0.366126] pci 0000:02:00.0: PME# supported from D0 D3hot D3cold
[    0.366198] pci 0000:02:00.0: PME# disabled
[    0.366314] pci 0000:02:00.3: PME# supported from D0 D3hot D3cold
[    0.366386] pci 0000:02:00.3: PME# disabled
[    0.366469] pci 0000:02:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.366610] pci 0000:00:03.0: bridge io port: [0x2000-0x2fff]
[    0.366613] pci 0000:00:03.0: bridge 32bit mmio: [0xd8500000-0xd86fffff]
[    0.366666] pci 0000:03:00.0: PME# supported from D0 D3hot D3cold
[    0.366739] pci 0000:03:00.0: PME# disabled
[    0.366819] pci 0000:03:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.366939] pci 0000:02:00.0: bridge io port: [0x2000-0x2fff]
[    0.366943] pci 0000:02:00.0: bridge 32bit mmio: [0xd8600000-0xd86fffff]
[    0.366992] pci 0000:04:00.0: reg 18 io port: [0x2000-0x207f]
[    0.367009] pci 0000:04:00.0: reg 20 64bit mmio: [0xd8600000-0xd860ffff]
[    0.367016] pci 0000:04:00.0: reg 30 32bit mmio pref: [0x000000-0x03ffff]
[    0.367045] pci 0000:04:00.0: supports D1
[    0.367047] pci 0000:04:00.0: PME# supported from D0 D1 D3hot
[    0.367120] pci 0000:04:00.0: PME# disabled
[    0.367208] pci 0000:04:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.367352] pci 0000:03:00.0: bridge io port: [0x2000-0x2fff]
[    0.367356] pci 0000:03:00.0: bridge 32bit mmio: [0xd8600000-0xd86fffff]
[    0.367463] pci 0000:06:00.0: PME# supported from D0 D3hot D3cold
[    0.367537] pci 0000:06:00.0: PME# disabled
[    0.367617] pci 0000:06:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.367753] pci 0000:00:05.0: bridge io port: [0x3000-0x4fff]
[    0.367756] pci 0000:00:05.0: bridge 32bit mmio: [0xd8000000-0xd81fffff]
[    0.367812] pci 0000:07:00.0: PME# supported from D0 D3hot D3cold
[    0.367886] pci 0000:07:00.0: PME# disabled
[    0.367968] pci 0000:07:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.368050] pci 0000:07:01.0: PME# supported from D0 D3hot D3cold
[    0.368124] pci 0000:07:01.0: PME# disabled
[    0.368205] pci 0000:07:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.368332] pci 0000:06:00.0: bridge io port: [0x3000-0x4fff]
[    0.368337] pci 0000:06:00.0: bridge 32bit mmio: [0xd8000000-0xd81fffff]
[    0.368386] pci 0000:08:00.0: reg 10 32bit mmio: [0xd8020000-0xd803ffff]
[    0.368394] pci 0000:08:00.0: reg 14 32bit mmio: [0xd8000000-0xd801ffff]
[    0.368402] pci 0000:08:00.0: reg 18 io port: [0x3000-0x301f]
[    0.368426] pci 0000:08:00.0: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.368458] pci 0000:08:00.0: PME# supported from D0 D3hot D3cold
[    0.368532] pci 0000:08:00.0: PME# disabled
[    0.368654] pci 0000:08:00.1: reg 10 32bit mmio: [0xd8060000-0xd807ffff]
[    0.368662] pci 0000:08:00.1: reg 14 32bit mmio: [0xd8040000-0xd805ffff]
[    0.368669] pci 0000:08:00.1: reg 18 io port: [0x3020-0x303f]
[    0.368694] pci 0000:08:00.1: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.368726] pci 0000:08:00.1: PME# supported from D0 D3hot D3cold
[    0.368800] pci 0000:08:00.1: PME# disabled
[    0.368900] pci 0000:08:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.369068] pci 0000:07:00.0: bridge io port: [0x3000-0x3fff]
[    0.369074] pci 0000:07:00.0: bridge 32bit mmio: [0xd8000000-0xd80fffff]
[    0.369124] pci 0000:09:00.0: reg 10 32bit mmio: [0xd8120000-0xd813ffff]
[    0.369132] pci 0000:09:00.0: reg 14 32bit mmio: [0xd8100000-0xd811ffff]
[    0.369140] pci 0000:09:00.0: reg 18 io port: [0x4000-0x401f]
[    0.369164] pci 0000:09:00.0: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.369196] pci 0000:09:00.0: PME# supported from D0 D3hot
[    0.369269] pci 0000:09:00.0: PME# disabled
[    0.369388] pci 0000:09:00.1: reg 10 32bit mmio: [0xd8160000-0xd817ffff]
[    0.369396] pci 0000:09:00.1: reg 14 32bit mmio: [0xd8140000-0xd815ffff]
[    0.369404] pci 0000:09:00.1: reg 18 io port: [0x4020-0x403f]
[    0.369428] pci 0000:09:00.1: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.369461] pci 0000:09:00.1: PME# supported from D0 D3hot
[    0.369534] pci 0000:09:00.1: PME# disabled
[    0.369634] pci 0000:09:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.369802] pci 0000:07:01.0: bridge io port: [0x4000-0x4fff]
[    0.369807] pci 0000:07:01.0: bridge 32bit mmio: [0xd8100000-0xd81fffff]
[    0.369858] pci 0000:0a:00.0: reg 10 32bit mmio: [0xd8200000-0xd821ffff]
[    0.369892] pci 0000:0a:00.0: PME# supported from D0 D3hot D3cold
[    0.369965] pci 0000:0a:00.0: PME# disabled
[    0.376058] pci 0000:00:07.0: bridge io port: [0x5000-0x6fff]
[    0.376061] pci 0000:00:07.0: bridge 32bit mmio: [0xd8200000-0xd84fffff]
[    0.376127] pci 0000:0b:01.0: PME# supported from D0 D3hot D3cold
[    0.376199] pci 0000:0b:01.0: PME# disabled
[    0.376347] pci 0000:0b:08.0: PME# supported from D0 D3hot D3cold
[    0.376419] pci 0000:0b:08.0: PME# disabled
[    0.376567] pci 0000:0b:09.0: PME# supported from D0 D3hot D3cold
[    0.376640] pci 0000:0b:09.0: PME# disabled
[    0.376763] pci 0000:0a:00.0: bridge io port: [0x5000-0x6fff]
[    0.376767] pci 0000:0a:00.0: bridge 32bit mmio: [0xd8300000-0xd84fffff]
[    0.376819] pci 0000:0c:00.0: reg 18 io port: [0x5000-0x507f]
[    0.376835] pci 0000:0c:00.0: reg 20 64bit mmio: [0xd8300000-0xd830ffff]
[    0.376843] pci 0000:0c:00.0: reg 30 32bit mmio pref: [0x000000-0x03ffff]
[    0.376871] pci 0000:0c:00.0: supports D1
[    0.376873] pci 0000:0c:00.0: PME# supported from D0 D1 D3hot
[    0.376945] pci 0000:0c:00.0: PME# disabled
[    0.377034] pci 0000:0c:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.377189] pci 0000:0b:01.0: bridge io port: [0x5000-0x5fff]
[    0.377193] pci 0000:0b:01.0: bridge 32bit mmio: [0xd8300000-0xd83fffff]
[    0.377296] pci 0000:0e:00.0: reg 18 io port: [0x6000-0x607f]
[    0.377313] pci 0000:0e:00.0: reg 20 64bit mmio: [0xd8400000-0xd840ffff]
[    0.377320] pci 0000:0e:00.0: reg 30 32bit mmio pref: [0x000000-0x03ffff]
[    0.377349] pci 0000:0e:00.0: supports D1
[    0.377350] pci 0000:0e:00.0: PME# supported from D0 D1 D3hot
[    0.377423] pci 0000:0e:00.0: PME# disabled
[    0.377512] pci 0000:0e:00.0: disabling ASPM on pre-1.1 PCIe device.  You can enable it with 'pcie_aspm=force'
[    0.377670] pci 0000:0b:09.0: bridge io port: [0x6000-0x6fff]
[    0.377674] pci 0000:0b:09.0: bridge 32bit mmio: [0xd8400000-0xd84fffff]
[    0.377732] pci 0000:0f:00.0: reg 10 32bit mmio: [0xd8740000-0xd875ffff]
[    0.377739] pci 0000:0f:00.0: reg 14 32bit mmio: [0xd8720000-0xd873ffff]
[    0.377746] pci 0000:0f:00.0: reg 18 io port: [0x7000-0x701f]
[    0.377752] pci 0000:0f:00.0: reg 1c 32bit mmio: [0xd8700000-0xd8703fff]
[    0.377767] pci 0000:0f:00.0: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.377796] pci 0000:0f:00.0: PME# supported from D0 D3hot D3cold
[    0.377869] pci 0000:0f:00.0: PME# disabled
[    0.377982] pci 0000:0f:00.1: reg 10 32bit mmio: [0xd8780000-0xd879ffff]
[    0.377989] pci 0000:0f:00.1: reg 14 32bit mmio: [0xd8760000-0xd877ffff]
[    0.377995] pci 0000:0f:00.1: reg 18 io port: [0x7020-0x703f]
[    0.378002] pci 0000:0f:00.1: reg 1c 32bit mmio: [0xd8704000-0xd8707fff]
[    0.378016] pci 0000:0f:00.1: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.378045] pci 0000:0f:00.1: PME# supported from D0 D3hot D3cold
[    0.378118] pci 0000:0f:00.1: PME# disabled
[    0.384084] pci 0000:00:09.0: bridge io port: [0x7000-0x7fff]
[    0.384087] pci 0000:00:09.0: bridge 32bit mmio: [0xd8700000-0xd87fffff]
[    0.384130] pci 0000:10:01.0: reg 10 32bit mmio pref: [0xd0000000-0xd7ffffff]
[    0.384137] pci 0000:10:01.0: reg 14 io port: [0x8000-0x80ff]
[    0.384144] pci 0000:10:01.0: reg 18 32bit mmio: [0xd8800000-0xd880ffff]
[    0.384165] pci 0000:10:01.0: reg 30 32bit mmio pref: [0x000000-0x01ffff]
[    0.384185] pci 0000:10:01.0: supports D1 D2
[    0.384235] pci 0000:00:1e.0: transparent bridge
[    0.384305] pci 0000:00:1e.0: bridge io port: [0x8000-0x8fff]
[    0.384309] pci 0000:00:1e.0: bridge 32bit mmio: [0xd8800000-0xd88fffff]
[    0.384315] pci 0000:00:1e.0: bridge 64bit mmio pref: [0xd0000000-0xd7ffffff]
[    0.384336] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0._PRT]
[    0.384533] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P1._PRT]
[    0.384602] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P3._PRT]
[    0.384645] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P3.BMF0._PRT]
[    0.384698] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P3.BMF0.BPD0._PRT]
[    0.384775] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P5._PRT]
[    0.384838] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P7._PRT]
[    0.384901] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.P0P9._PRT]
[    0.384964] ACPI: PCI Interrupt Routing Table [\_SB_.PCI0.PCIB._PRT]
[    0.394280] ACPI: PCI Interrupt Link [LNKA] (IRQs 3 4 5 6 7 10 11 14 15) *12
[    0.394982] ACPI: PCI Interrupt Link [LNKB] (IRQs 3 *4 5 6 7 10 11 14 15)
[    0.395636] ACPI: PCI Interrupt Link [LNKC] (IRQs 3 4 5 6 7 10 *11 14 15)
[    0.396281] ACPI: PCI Interrupt Link [LNKD] (IRQs 3 4 5 6 7 10 11 14 15) *0, disabled.
[    0.397042] ACPI: PCI Interrupt Link [LNKE] (IRQs 3 4 5 6 7 *10 11 14 15)
[    0.397695] ACPI: PCI Interrupt Link [LNKF] (IRQs 4 *5 6 7 10 11 14 15)
[    0.398303] ACPI: PCI Interrupt Link [LNKG] (IRQs 3 4 5 6 *7 10 11 14 15)
[    0.398957] ACPI: PCI Interrupt Link [LNKH] (IRQs 4 5 6 7 10 11 14 15) *3
[    0.399642] vgaarb: device added: PCI:0000:10:01.0,decodes=io+mem,owns=io+mem,locks=none
[    0.399731] vgaarb: loaded
[    0.399822] PCI: Using ACPI for IRQ routing
[    0.400192] hpet0: at MMIO 0xfed00000, IRQs 2, 8, 0
[    0.400445] hpet0: 3 comparators, 64-bit 14.318180 MHz counter
[    0.412006] Switching to clocksource tsc
[    0.413433] pnp: PnP ACPI init
[    0.413526] ACPI: bus type pnp registered
[    0.416321] pnp: PnP ACPI: found 10 devices
[    0.416389] ACPI: ACPI bus type pnp unregistered
[    0.416470] system 00:01: ioport range 0x295-0x296 has been reserved
[    0.416542] system 00:01: ioport range 0x4d0-0x4d1 has been reserved
[    0.416614] system 00:01: ioport range 0x800-0x80f has been reserved
[    0.416687] system 00:01: ioport range 0xca0-0xca1 has been reserved
[    0.416759] system 00:01: ioport range 0x1000-0x107f has been reserved
[    0.416832] system 00:01: ioport range 0x1180-0x11bf has been reserved
[    0.416905] system 00:01: ioport range 0xfe00-0xfe00 has been reserved
[    0.416979] system 00:01: iomem range 0xe0000000-0xefffffff has been reserved
[    0.423412] system 00:01: iomem range 0xfee00000-0xfee0ffff could not be reserved
[    0.423501] system 00:01: iomem range 0xfec89000-0xfec89fff could not be reserved
[    0.423589] system 00:01: iomem range 0xfed1c000-0xfed1ffff has been reserved
[    0.423663] system 00:01: iomem range 0xfe000000-0xfe01ffff has been reserved
[    0.423737] system 00:01: iomem range 0xfe600000-0xfe6fffff has been reserved
[    0.423811] system 00:01: iomem range 0xfed20000-0xfed44fff has been reserved
[    0.423894] system 00:01: iomem range 0xfed45000-0xfed8ffff has been reserved
[    0.428842] pci 0000:00:01.0: PCI bridge, secondary bus 0000:01
[    0.428913] pci 0000:00:01.0:   IO window: disabled
[    0.428984] pci 0000:00:01.0:   MEM window: disabled
[    0.429053] pci 0000:00:01.0:   PREFETCH window: disabled
[    0.429127] pci 0000:03:00.0: PCI bridge, secondary bus 0000:04
[    0.429199] pci 0000:03:00.0:   IO window: 0x2000-0x2fff
[    0.429272] pci 0000:03:00.0:   MEM window: 0xd8600000-0xd86fffff
[    0.429345] pci 0000:03:00.0:   PREFETCH window: 0xd8900000-0xd89fffff
[    0.429420] pci 0000:02:00.0: PCI bridge, secondary bus 0000:03
[    0.429492] pci 0000:02:00.0:   IO window: 0x2000-0x2fff
[    0.429564] pci 0000:02:00.0:   MEM window: 0xd8600000-0xd86fffff
[    0.429637] pci 0000:02:00.0:   PREFETCH window: 0xd8900000-0xd89fffff
[    0.429712] pci 0000:02:00.3: PCI bridge, secondary bus 0000:05
[    0.429782] pci 0000:02:00.3:   IO window: disabled
[    0.429853] pci 0000:02:00.3:   MEM window: disabled
[    0.429923] pci 0000:02:00.3:   PREFETCH window: disabled
[    0.429996] pci 0000:00:03.0: PCI bridge, secondary bus 0000:02
[    0.430068] pci 0000:00:03.0:   IO window: 0x2000-0x2fff
[    0.430139] pci 0000:00:03.0:   MEM window: 0xd8500000-0xd86fffff
[    0.430211] pci 0000:00:03.0:   PREFETCH window: 0xd8900000-0xd89fffff
[    0.430288] pci 0000:07:00.0: PCI bridge, secondary bus 0000:08
[    0.430360] pci 0000:07:00.0:   IO window: 0x3000-0x3fff
[    0.430434] pci 0000:07:00.0:   MEM window: 0xd8000000-0xd80fffff
[    0.430508] pci 0000:07:00.0:   PREFETCH window: 0xd8c00000-0xd8cfffff
[    0.430586] pci 0000:07:01.0: PCI bridge, secondary bus 0000:09
[    0.430657] pci 0000:07:01.0:   IO window: 0x4000-0x4fff
[    0.430731] pci 0000:07:01.0:   MEM window: 0xd8100000-0xd81fffff
[    0.430805] pci 0000:07:01.0:   PREFETCH window: 0xd8d00000-0xd8dfffff
[    0.430882] pci 0000:06:00.0: PCI bridge, secondary bus 0000:07
[    0.430954] pci 0000:06:00.0:   IO window: 0x3000-0x4fff
[    0.431028] pci 0000:06:00.0:   MEM window: 0xd8000000-0xd81fffff
[    0.431102] pci 0000:06:00.0:   PREFETCH window: 0xd8c00000-0xd8dfffff
[    0.431179] pci 0000:00:05.0: PCI bridge, secondary bus 0000:06
[    0.431250] pci 0000:00:05.0:   IO window: 0x3000-0x4fff
[    0.431322] pci 0000:00:05.0:   MEM window: 0xd8000000-0xd81fffff
[    0.431394] pci 0000:00:05.0:   PREFETCH window: 0xd8c00000-0xd8dfffff
[    0.431472] pci 0000:0b:08.0: BAR 14: can't allocate mem resource [0xd8500000-0xd84fffff]
[    0.431562] pci 0000:0b:08.0: BAR 13: can't allocate I/O resource [0x7000-0x6fff]
[    0.431650] pci 0000:0b:01.0: PCI bridge, secondary bus 0000:0c
[    0.431722] pci 0000:0b:01.0:   IO window: 0x5000-0x5fff
[    0.431794] pci 0000:0b:01.0:   MEM window: 0xd8300000-0xd83fffff
[    0.431877] pci 0000:0b:01.0:   PREFETCH window: 0xd8e00000-0xd8ffffff
[    0.431952] pci 0000:0b:08.0: PCI bridge, secondary bus 0000:0d
[    0.432023] pci 0000:0b:08.0:   IO window: disabled
[    0.432094] pci 0000:0b:08.0:   MEM window: disabled
[    0.432164] pci 0000:0b:08.0:   PREFETCH window: 0x000000d9000000-0x000000d91fffff
[    0.432256] pci 0000:0b:09.0: PCI bridge, secondary bus 0000:0e
[    0.432328] pci 0000:0b:09.0:   IO window: 0x6000-0x6fff
[    0.432400] pci 0000:0b:09.0:   MEM window: 0xd8400000-0xd84fffff
[    0.432473] pci 0000:0b:09.0:   PREFETCH window: 0xd9200000-0xd93fffff
[    0.432548] pci 0000:0a:00.0: PCI bridge, secondary bus 0000:0b
[    0.432620] pci 0000:0a:00.0:   IO window: 0x5000-0x6fff
[    0.432692] pci 0000:0a:00.0:   MEM window: 0xd8300000-0xd84fffff
[    0.432765] pci 0000:0a:00.0:   PREFETCH window: 0xd8e00000-0xd93fffff
[    0.432840] pci 0000:00:07.0: PCI bridge, secondary bus 0000:0a
[    0.432912] pci 0000:00:07.0:   IO window: 0x5000-0x6fff
[    0.432983] pci 0000:00:07.0:   MEM window: 0xd8200000-0xd84fffff
[    0.433056] pci 0000:00:07.0:   PREFETCH window: 0xd8e00000-0xd93fffff
[    0.433131] pci 0000:00:09.0: PCI bridge, secondary bus 0000:0f
[    0.433203] pci 0000:00:09.0:   IO window: 0x7000-0x7fff
[    0.433274] pci 0000:00:09.0:   MEM window: 0xd8700000-0xd87fffff
[    0.433347] pci 0000:00:09.0:   PREFETCH window: 0xd8a00000-0xd8afffff
[    0.433422] pci 0000:00:1e.0: PCI bridge, secondary bus 0000:10
[    0.433493] pci 0000:00:1e.0:   IO window: 0x8000-0x8fff
[    0.433566] pci 0000:00:1e.0:   MEM window: 0xd8800000-0xd88fffff
[    0.433639] pci 0000:00:1e.0:   PREFETCH window: 0x000000d0000000-0x000000d7ffffff
[    0.433737]   alloc irq_desc for 24 on node -1
[    0.433740]   alloc kstat_irqs on node -1
[    0.433746] pci 0000:00:01.0: PCI INT A -> GSI 24 (level, low) -> IRQ 24
[    0.433820] pci 0000:00:01.0: setting latency timer to 64
[    0.433826]   alloc irq_desc for 26 on node -1
[    0.433828]   alloc kstat_irqs on node -1
[    0.433836] pci 0000:00:03.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
[    0.433909] pci 0000:00:03.0: setting latency timer to 64
[    0.433919] pci 0000:02:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
[    0.433993] pci 0000:02:00.0: setting latency timer to 64
[    0.434000] pci 0000:03:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
[    0.434074] pci 0000:03:00.0: setting latency timer to 64
[    0.434083] pci 0000:02:00.3: setting latency timer to 64
[    0.434088]   alloc irq_desc for 28 on node -1
[    0.434090]   alloc kstat_irqs on node -1
[    0.434095] pci 0000:00:05.0: PCI INT A -> GSI 28 (level, low) -> IRQ 28
[    0.434168] pci 0000:00:05.0: setting latency timer to 64
[    0.434178] pci 0000:06:00.0: setting latency timer to 64
[    0.434188] pci 0000:07:00.0: setting latency timer to 64
[    0.434199] pci 0000:07:01.0: setting latency timer to 64
[    0.434205]   alloc irq_desc for 30 on node -1
[    0.434206]   alloc kstat_irqs on node -1
[    0.434211] pci 0000:00:07.0: PCI INT A -> GSI 30 (level, low) -> IRQ 30
[    0.434284] pci 0000:00:07.0: setting latency timer to 64
[    0.434292] pci 0000:0a:00.0: PCI INT A -> GSI 30 (level, low) -> IRQ 30
[    0.434366] pci 0000:0a:00.0: setting latency timer to 64
[    0.434373]   alloc irq_desc for 37 on node -1
[    0.434375]   alloc kstat_irqs on node -1
[    0.434379] pci 0000:0b:01.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
[    0.434453] pci 0000:0b:01.0: setting latency timer to 64
[    0.434461] pci 0000:0b:08.0: PCI INT A -> GSI 30 (level, low) -> IRQ 30
[    0.434534] pci 0000:0b:08.0: setting latency timer to 64
[    0.434542] pci 0000:0b:09.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
[    0.434616] pci 0000:0b:09.0: setting latency timer to 64
[    0.434625]   alloc irq_desc for 32 on node -1
[    0.434627]   alloc kstat_irqs on node -1
[    0.434631] pci 0000:00:09.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
[    0.434705] pci 0000:00:09.0: setting latency timer to 64
[    0.434711] pci 0000:00:1e.0: setting latency timer to 64
[    0.434715] pci_bus 0000:00: resource 0 io:  [0x00-0xffff]
[    0.434717] pci_bus 0000:00: resource 1 mem: [0x000000-0xffffffffffffffff]
[    0.434720] pci_bus 0000:02: resource 0 io:  [0x2000-0x2fff]
[    0.434722] pci_bus 0000:02: resource 1 mem: [0xd8500000-0xd86fffff]
[    0.434725] pci_bus 0000:02: resource 2 pref mem [0xd8900000-0xd89fffff]
[    0.434727] pci_bus 0000:03: resource 0 io:  [0x2000-0x2fff]
[    0.434730] pci_bus 0000:03: resource 1 mem: [0xd8600000-0xd86fffff]
[    0.434732] pci_bus 0000:03: resource 2 pref mem [0xd8900000-0xd89fffff]
[    0.434735] pci_bus 0000:04: resource 0 io:  [0x2000-0x2fff]
[    0.434737] pci_bus 0000:04: resource 1 mem: [0xd8600000-0xd86fffff]
[    0.434739] pci_bus 0000:04: resource 2 pref mem [0xd8900000-0xd89fffff]
[    0.434742] pci_bus 0000:06: resource 0 io:  [0x3000-0x4fff]
[    0.434744] pci_bus 0000:06: resource 1 mem: [0xd8000000-0xd81fffff]
[    0.434747] pci_bus 0000:06: resource 2 pref mem [0xd8c00000-0xd8dfffff]
[    0.434749] pci_bus 0000:07: resource 0 io:  [0x3000-0x4fff]
[    0.434752] pci_bus 0000:07: resource 1 mem: [0xd8000000-0xd81fffff]
[    0.434754] pci_bus 0000:07: resource 2 pref mem [0xd8c00000-0xd8dfffff]
[    0.434756] pci_bus 0000:08: resource 0 io:  [0x3000-0x3fff]
[    0.434759] pci_bus 0000:08: resource 1 mem: [0xd8000000-0xd80fffff]
[    0.434761] pci_bus 0000:08: resource 2 pref mem [0xd8c00000-0xd8cfffff]
[    0.434763] pci_bus 0000:09: resource 0 io:  [0x4000-0x4fff]
[    0.434766] pci_bus 0000:09: resource 1 mem: [0xd8100000-0xd81fffff]
[    0.434768] pci_bus 0000:09: resource 2 pref mem [0xd8d00000-0xd8dfffff]
[    0.434771] pci_bus 0000:0a: resource 0 io:  [0x5000-0x6fff]
[    0.434773] pci_bus 0000:0a: resource 1 mem: [0xd8200000-0xd84fffff]
[    0.434775] pci_bus 0000:0a: resource 2 pref mem [0xd8e00000-0xd93fffff]
[    0.434778] pci_bus 0000:0b: resource 0 io:  [0x5000-0x6fff]
[    0.434780] pci_bus 0000:0b: resource 1 mem: [0xd8300000-0xd84fffff]
[    0.434782] pci_bus 0000:0b: resource 2 pref mem [0xd8e00000-0xd93fffff]
[    0.434785] pci_bus 0000:0c: resource 0 io:  [0x5000-0x5fff]
[    0.434787] pci_bus 0000:0c: resource 1 mem: [0xd8300000-0xd83fffff]
[    0.434789] pci_bus 0000:0c: resource 2 pref mem [0xd8e00000-0xd8ffffff]
[    0.434792] pci_bus 0000:0d: resource 2 pref mem [0xd9000000-0xd91fffff]
[    0.434794] pci_bus 0000:0e: resource 0 io:  [0x6000-0x6fff]
[    0.434797] pci_bus 0000:0e: resource 1 mem: [0xd8400000-0xd84fffff]
[    0.434799] pci_bus 0000:0e: resource 2 pref mem [0xd9200000-0xd93fffff]
[    0.434802] pci_bus 0000:0f: resource 0 io:  [0x7000-0x7fff]
[    0.434804] pci_bus 0000:0f: resource 1 mem: [0xd8700000-0xd87fffff]
[    0.434806] pci_bus 0000:0f: resource 2 pref mem [0xd8a00000-0xd8afffff]
[    0.434809] pci_bus 0000:10: resource 0 io:  [0x8000-0x8fff]
[    0.434811] pci_bus 0000:10: resource 1 mem: [0xd8800000-0xd88fffff]
[    0.434813] pci_bus 0000:10: resource 2 pref mem [0xd0000000-0xd7ffffff]
[    0.434816] pci_bus 0000:10: resource 3 io:  [0x00-0xffff]
[    0.434818] pci_bus 0000:10: resource 4 mem: [0x000000-0xffffffffffffffff]
[    0.434851] NET: Registered protocol family 2
[    0.435098] IP route cache hash table entries: 131072 (order: 8, 1048576 bytes)
[    0.436245] TCP established hash table entries: 524288 (order: 11, 8388608 bytes)
[    0.439953] TCP bind hash table entries: 65536 (order: 8, 1048576 bytes)
[    0.440533] TCP: Hash tables configured (established 524288 bind 65536)
[    0.440606] TCP reno registered
[    0.440806] NET: Registered protocol family 1
[    0.441011] pci 0000:00:1f.0: rerouting interrupts for [8086:2670]
[    0.441114] pci 0000:0f:00.0: Disabling L0s
[    0.441184] pci 0000:0f:00.1: Disabling L0s
[    0.441256] pci 0000:10:01.0: Boot video device
[    0.441312] Unpacking initramfs...
[    0.511858] Freeing initrd memory: 2750k freed
[    0.512768] Simple Boot Flag at 0x41 set to 0x1
[    0.513282] audit: initializing netlink socket (disabled)
[    0.513363] type=2000 audit(1272981743.511:1): initialized
[    0.513832] HugeTLB registered 2 MB page size, pre-allocated 0 pages
[    0.515362] VFS: Disk quotas dquot_6.5.2
[    0.515483] Dquot-cache hash table entries: 512 (order 0, 4096 bytes)
[    0.515641] msgmni has been set to 7930
[    0.515969] alg: No test for stdrng (krng)
[    0.516086] Block layer SCSI generic (bsg) driver version 0.4 loaded (major 253)
[    0.516175] io scheduler noop registered
[    0.516242] io scheduler anticipatory registered
[    0.516311] io scheduler deadline registered (default)
[    0.516410] io scheduler cfq registered
[    0.516645]   alloc irq_desc for 48 on node -1
[    0.516647]   alloc kstat_irqs on node -1
[    0.516658] pcieport 0000:00:01.0: irq 48 for MSI/MSI-X
[    0.516665] pcieport 0000:00:01.0: setting latency timer to 64
[    0.516756]   alloc irq_desc for 49 on node -1
[    0.516758]   alloc kstat_irqs on node -1
[    0.516763] pcieport 0000:00:03.0: irq 49 for MSI/MSI-X
[    0.516769] pcieport 0000:00:03.0: setting latency timer to 64
[    0.516853]   alloc irq_desc for 50 on node -1
[    0.516855]   alloc kstat_irqs on node -1
[    0.516861] pcieport 0000:00:05.0: irq 50 for MSI/MSI-X
[    0.516866] pcieport 0000:00:05.0: setting latency timer to 64
[    0.516950]   alloc irq_desc for 51 on node -1
[    0.516952]   alloc kstat_irqs on node -1
[    0.516958] pcieport 0000:00:07.0: irq 51 for MSI/MSI-X
[    0.516963] pcieport 0000:00:07.0: setting latency timer to 64
[    0.517055]   alloc irq_desc for 52 on node -1
[    0.517057]   alloc kstat_irqs on node -1
[    0.517062] pcieport 0000:00:09.0: irq 52 for MSI/MSI-X
[    0.517068] pcieport 0000:00:09.0: setting latency timer to 64
[    0.517153] pcieport 0000:02:00.0: setting latency timer to 64
[    0.517242]   alloc irq_desc for 53 on node -1
[    0.517244]   alloc kstat_irqs on node -1
[    0.517251] pcieport 0000:03:00.0: irq 53 for MSI/MSI-X
[    0.517258] pcieport 0000:03:00.0: setting latency timer to 64
[    0.517330] pcieport 0000:06:00.0: setting latency timer to 64
[    0.517425]   alloc irq_desc for 54 on node -1
[    0.517427]   alloc kstat_irqs on node -1
[    0.517435] pcieport 0000:07:00.0: irq 54 for MSI/MSI-X
[    0.517443] pcieport 0000:07:00.0: setting latency timer to 64
[    0.517542]   alloc irq_desc for 55 on node -1
[    0.517543]   alloc kstat_irqs on node -1
[    0.517551] pcieport 0000:07:01.0: irq 55 for MSI/MSI-X
[    0.517559] pcieport 0000:07:01.0: setting latency timer to 64
[    0.517664]   alloc irq_desc for 56 on node -1
[    0.517665]   alloc kstat_irqs on node -1
[    0.517673] pcieport 0000:0a:00.0: irq 56 for MSI/MSI-X
[    0.517680] pcieport 0000:0a:00.0: setting latency timer to 64
[    0.517794]   alloc irq_desc for 57 on node -1
[    0.517796]   alloc kstat_irqs on node -1
[    0.517803] pcieport 0000:0b:01.0: irq 57 for MSI/MSI-X
[    0.517811] pcieport 0000:0b:01.0: setting latency timer to 64
[    0.517934]   alloc irq_desc for 58 on node -1
[    0.517936]   alloc kstat_irqs on node -1
[    0.517944] pcieport 0000:0b:08.0: irq 58 for MSI/MSI-X
[    0.517951] pcieport 0000:0b:08.0: setting latency timer to 64
[    0.518080]   alloc irq_desc for 59 on node -1
[    0.518082]   alloc kstat_irqs on node -1
[    0.518089] pcieport 0000:0b:09.0: irq 59 for MSI/MSI-X
[    0.518096] pcieport 0000:0b:09.0: setting latency timer to 64
[    0.518189] aer 0000:00:01.0:pcie02: AER service couldn't init device: no _OSC support
[    0.518196] aer 0000:00:03.0:pcie02: AER service couldn't init device: no _OSC support
[    0.518202] aer 0000:00:05.0:pcie02: AER service couldn't init device: no _OSC support
[    0.518207] aer 0000:00:07.0:pcie02: AER service couldn't init device: no _OSC support
[    0.518213] aer 0000:00:09.0:pcie02: AER service couldn't init device: no _OSC support
[    0.519727] Linux agpgart interface v0.103
[    0.519852] Serial: 8250/16550 driver, 4 ports, IRQ sharing enabled
[    0.520287] input: Macintosh mouse button emulation as /devices/virtual/input/input0
[    0.520430] PNP: PS/2 Controller [PNP0303:KBC0,PNP0f13:MSE0] at 0x60,0x64 irq 1,12
[    0.772190] serio: i8042 KBD port at 0x60,0x64 irq 1
[    0.772334] mice: PS/2 mouse device common for all mice
[    0.772440] rtc_cmos 00:05: RTC can wake from S4
[    0.772538] rtc_cmos 00:05: rtc core: registered rtc_cmos as rtc0
[    0.772633] rtc0: alarms up to one month, y3k, 114 bytes nvram, hpet irqs
[    0.772715] cpuidle: using governor ladder
[    0.772782] cpuidle: using governor menu
[    0.772852] No iBFT detected.
[    0.773234] TCP cubic registered
[    0.773405] NET: Registered protocol family 10
[    0.773932] lo: Disabled Privacy Extensions
[    0.774272] Mobile IPv6
[    0.774338] NET: Registered protocol family 17
[    0.774480] PM: Resume from disk failed.
[    0.774484] registered taskstats version 1
[    0.775253] rtc_cmos 00:05: setting system clock to 2010-05-04 14:02:24 UTC (1272981744)
[    0.775394] Freeing unused kernel memory: 580k freed
[    0.775650] Write protecting the kernel read-only data: 4156k
[    0.791287] input: AT Translated Set 2 keyboard as /devices/platform/i8042/serio0/input/input1
[    0.801423] SCSI subsystem initialized
[    0.816768] libata version 3.00 loaded.
[    0.820405] ata_piix 0000:00:1f.2: version 2.13
[    0.820420]   alloc irq_desc for 17 on node -1
[    0.820422]   alloc kstat_irqs on node -1
[    0.820430] ata_piix 0000:00:1f.2: PCI INT B -> GSI 17 (level, low) -> IRQ 17
[    0.820506] ata_piix 0000:00:1f.2: MAP [ P0 P2 P1 P3 ]
[    0.975856] ata_piix 0000:00:1f.2: setting latency timer to 64
[    0.975917] scsi0 : ata_piix
[    0.976054] scsi1 : ata_piix
[    0.976152] ata1: SATA max UDMA/133 cmd 0x18a0 ctl 0x1894 bmdma 0x1880 irq 17
[    0.976225] ata2: SATA max UDMA/133 cmd 0x1898 ctl 0x1890 bmdma 0x1888 irq 17
[    1.148205] ata1.00: ATA-7: TS32GSSD25S-M, V090331, max UDMA/133
[    1.148278] ata1.00: 62586880 sectors, multi 1: LBA48 NCQ (depth 0/1)
[    1.156225] ata1.00: configured for UDMA/133
[    1.156384] scsi 0:0:0:0: Direct-Access     ATA      TS32GSSD25S-M    V090 PQ: 0 ANSI: 5
[    1.161331] sd 0:0:0:0: [sda] 62586880 512-byte logical blocks: (32.0 GB/29.8 GiB)
[    1.161457] sd 0:0:0:0: [sda] Write Protect is off
[    1.161539] sd 0:0:0:0: [sda] Mode Sense: 00 3a 00 00
[    1.161561] sd 0:0:0:0: [sda] Write cache: disabled, read cache: enabled, doesn't support DPO or FUA
[    1.161776]  sda: sda1 sda2 < sda5 >
[    1.162829] sd 0:0:0:0: [sda] Attached SCSI disk
[    1.178743] dca service started, version 1.12.1
[    1.181796] Intel(R) Gigabit Ethernet Network Driver - version 1.3.16-k2
[    1.181869] Copyright (c) 2007-2009 Intel Corporation.
[    1.181979] igb 0000:0f:00.0: PCI INT A -> GSI 32 (level, low) -> IRQ 32
[    1.182072] igb 0000:0f:00.0: setting latency timer to 64
[    1.182340]   alloc irq_desc for 60 on node -1
[    1.182342]   alloc kstat_irqs on node -1
[    1.182350] igb 0000:0f:00.0: irq 60 for MSI/MSI-X
[    1.182352]   alloc irq_desc for 61 on node -1
[    1.182354]   alloc kstat_irqs on node -1
[    1.182357] igb 0000:0f:00.0: irq 61 for MSI/MSI-X
[    1.182359]   alloc irq_desc for 62 on node -1
[    1.182361]   alloc kstat_irqs on node -1
[    1.182364] igb 0000:0f:00.0: irq 62 for MSI/MSI-X
[    1.182366]   alloc irq_desc for 63 on node -1
[    1.182368]   alloc kstat_irqs on node -1
[    1.182371] igb 0000:0f:00.0: irq 63 for MSI/MSI-X
[    1.182377]   alloc irq_desc for 64 on node -1
[    1.182379]   alloc kstat_irqs on node -1
[    1.182382] igb 0000:0f:00.0: irq 64 for MSI/MSI-X
[    1.182384]   alloc irq_desc for 65 on node -1
[    1.182385]   alloc kstat_irqs on node -1
[    1.182389] igb 0000:0f:00.0: irq 65 for MSI/MSI-X
[    1.182391]   alloc irq_desc for 66 on node -1
[    1.182392]   alloc kstat_irqs on node -1
[    1.182396] igb 0000:0f:00.0: irq 66 for MSI/MSI-X
[    1.182398]   alloc irq_desc for 67 on node -1
[    1.182399]   alloc kstat_irqs on node -1
[    1.182403] igb 0000:0f:00.0: irq 67 for MSI/MSI-X
[    1.182405]   alloc irq_desc for 68 on node -1
[    1.182407]   alloc kstat_irqs on node -1
[    1.182410] igb 0000:0f:00.0: irq 68 for MSI/MSI-X
[    1.375124] igb 0000:0f:00.0: Intel(R) Gigabit Ethernet Network Connection
[    1.375198] igb 0000:0f:00.0: eth0: (PCIe:2.5Gb/s:Width x4) 00:30:48:b9:46:a6
[    1.375345] igb 0000:0f:00.0: eth0: PBA No: ffffff-0ff
[    1.375414] igb 0000:0f:00.0: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[    1.375520]   alloc irq_desc for 46 on node -1
[    1.375522]   alloc kstat_irqs on node -1
[    1.375528] igb 0000:0f:00.1: PCI INT B -> GSI 46 (level, low) -> IRQ 46
[    1.375617] igb 0000:0f:00.1: setting latency timer to 64
[    1.375855]   alloc irq_desc for 69 on node -1
[    1.375857]   alloc kstat_irqs on node -1
[    1.375861] igb 0000:0f:00.1: irq 69 for MSI/MSI-X
[    1.375864]   alloc irq_desc for 70 on node -1
[    1.375866]   alloc kstat_irqs on node -1
[    1.375869] igb 0000:0f:00.1: irq 70 for MSI/MSI-X
[    1.375871]   alloc irq_desc for 71 on node -1
[    1.375873]   alloc kstat_irqs on node -1
[    1.375876] igb 0000:0f:00.1: irq 71 for MSI/MSI-X
[    1.375878]   alloc irq_desc for 72 on node -1
[    1.375879]   alloc kstat_irqs on node -1
[    1.375883] igb 0000:0f:00.1: irq 72 for MSI/MSI-X
[    1.375885]   alloc irq_desc for 73 on node -1
[    1.375886]   alloc kstat_irqs on node -1
[    1.375889] igb 0000:0f:00.1: irq 73 for MSI/MSI-X
[    1.375891]   alloc irq_desc for 74 on node -1
[    1.375893]   alloc kstat_irqs on node -1
[    1.375896] igb 0000:0f:00.1: irq 74 for MSI/MSI-X
[    1.375898]   alloc irq_desc for 75 on node -1
[    1.375899]   alloc kstat_irqs on node -1
[    1.375903] igb 0000:0f:00.1: irq 75 for MSI/MSI-X
[    1.375905]   alloc irq_desc for 76 on node -1
[    1.375906]   alloc kstat_irqs on node -1
[    1.375909] igb 0000:0f:00.1: irq 76 for MSI/MSI-X
[    1.375911]   alloc irq_desc for 77 on node -1
[    1.375913]   alloc kstat_irqs on node -1
[    1.375916] igb 0000:0f:00.1: irq 77 for MSI/MSI-X
[    1.559131] igb 0000:0f:00.1: Intel(R) Gigabit Ethernet Network Connection
[    1.559205] igb 0000:0f:00.1: eth1: (PCIe:2.5Gb/s:Width x4) 00:30:48:b9:46:a7
[    1.559351] igb 0000:0f:00.1: eth1: PBA No: ffffff-0ff
[    1.559421] igb 0000:0f:00.1: Using MSI-X interrupts. 4 rx queue(s), 4 tx queue(s)
[    1.563815] e1000e: Intel(R) PRO/1000 Network Driver - 1.0.2-k2
[    1.570262] e1000e: Copyright (c) 1999-2008 Intel Corporation.
[    1.570381] e1000e 0000:08:00.0: PCI INT A -> GSI 28 (level, low) -> IRQ 28
[    1.570475] e1000e 0000:08:00.0: setting latency timer to 64
[    1.570594]   alloc irq_desc for 78 on node -1
[    1.570596]   alloc kstat_irqs on node -1
[    1.570610] e1000e 0000:08:00.0: irq 78 for MSI/MSI-X
[    1.751110] 0000:08:00.0: eth2: (PCI Express:2.5GB/s:Width x4) 00:15:17:bb:6f:6c
[    1.751198] 0000:08:00.0: eth2: Intel(R) PRO/1000 Network Connection
[    1.751349] 0000:08:00.0: eth2: MAC: 0, PHY: 4, PBA No: d47316-004
[    1.751447]   alloc irq_desc for 36 on node -1
[    1.751449]   alloc kstat_irqs on node -1
[    1.751454] e1000e 0000:08:00.1: PCI INT B -> GSI 36 (level, low) -> IRQ 36
[    1.751546] e1000e 0000:08:00.1: setting latency timer to 64
[    1.751643]   alloc irq_desc for 79 on node -1
[    1.751645]   alloc kstat_irqs on node -1
[    1.751656] e1000e 0000:08:00.1: irq 79 for MSI/MSI-X
[    1.927118] 0000:08:00.1: eth3: (PCI Express:2.5GB/s:Width x4) 00:15:17:bb:6f:6d
[    1.927206] 0000:08:00.1: eth3: Intel(R) PRO/1000 Network Connection
[    1.927357] 0000:08:00.1: eth3: MAC: 0, PHY: 4, PBA No: d47316-004
[    1.927454] e1000e 0000:09:00.0: PCI INT A -> GSI 36 (level, low) -> IRQ 36
[    1.927545] e1000e 0000:09:00.0: setting latency timer to 64
[    1.927641]   alloc irq_desc for 80 on node -1
[    1.927643]   alloc kstat_irqs on node -1
[    1.927654] e1000e 0000:09:00.0: irq 80 for MSI/MSI-X
[    2.103170] 0000:09:00.0: eth4: (PCI Express:2.5GB/s:Width x4) 00:15:17:bb:6f:6e
[    2.103258] 0000:09:00.0: eth4: Intel(R) PRO/1000 Network Connection
[    2.103409] 0000:09:00.0: eth4: MAC: 0, PHY: 4, PBA No: d47316-004
[    2.103504]   alloc irq_desc for 44 on node -1
[    2.103506]   alloc kstat_irqs on node -1
[    2.103512] e1000e 0000:09:00.1: PCI INT B -> GSI 44 (level, low) -> IRQ 44
[    2.103602] e1000e 0000:09:00.1: setting latency timer to 64
[    2.103699]   alloc irq_desc for 81 on node -1
[    2.103700]   alloc kstat_irqs on node -1
[    2.103711] e1000e 0000:09:00.1: irq 81 for MSI/MSI-X
[    2.279209] 0000:09:00.1: eth5: (PCI Express:2.5GB/s:Width x4) 00:15:17:bb:6f:6f
[    2.279296] 0000:09:00.1: eth5: Intel(R) PRO/1000 Network Connection
[    2.279447] 0000:09:00.1: eth5: MAC: 0, PHY: 4, PBA No: d47316-004
[    2.295993] mvsas 0000:04:00.0: mvsas: driver version 0.8.2
[    2.296074] mvsas 0000:04:00.0: PCI INT A -> GSI 26 (level, low) -> IRQ 26
[    2.296149] mvsas 0000:04:00.0: setting latency timer to 64
[    2.297824] mvsas 0000:04:00.0: mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps
[    8.079220] scsi2 : mvsas
[    8.079776] mvsas 0000:0c:00.0: mvsas: driver version 0.8.2
[    8.079861] mvsas 0000:0c:00.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
[    8.079937] mvsas 0000:0c:00.0: setting latency timer to 64
[    8.081935] mvsas 0000:0c:00.0: mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps
[    8.286910] ata3.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    8.286982] ata3.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    8.318342] ata3.00: configured for UDMA/133
[    8.318476] scsi 2:0:0:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[    8.548479] ata4.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    8.548552] ata4.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    8.560269] ata4.00: configured for UDMA/133
[    8.560377] scsi 2:0:1:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[    8.823591] ata5.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    8.823664] ata5.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    8.835417] ata5.00: configured for UDMA/133
[    8.835525] scsi 2:0:2:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[    9.070146] ata6.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    9.070219] ata6.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    9.081977] ata6.00: configured for UDMA/133
[    9.082085] scsi 2:0:3:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[    9.337820] ata7.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    9.337892] ata7.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    9.349607] ata7.00: configured for UDMA/133
[    9.349713] scsi 2:0:4:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[    9.602546] ata8.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    9.602619] ata8.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    9.614210] ata8.00: configured for UDMA/133
[    9.614316] scsi 2:0:5:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[    9.855620] ata9.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[    9.855693] ata9.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[    9.867278] ata9.00: configured for UDMA/133
[    9.867384] scsi 2:0:6:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   10.111015] ata10.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   10.111087] ata10.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   10.122845] ata10.00: configured for UDMA/133
[   10.122951] scsi 2:0:7:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   10.123231] sd 2:0:0:0: [sdb] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.123307] sd 2:0:1:0: [sdc] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.123347] sd 2:0:1:0: [sdc] Write Protect is off
[   10.123349] sd 2:0:1:0: [sdc] Mode Sense: 00 3a 00 00
[   10.123376] sd 2:0:1:0: [sdc] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.123418] sd 2:0:2:0: [sdd] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.123474] sd 2:0:2:0: [sdd] Write Protect is off
[   10.123476] sd 2:0:2:0: [sdd] Mode Sense: 00 3a 00 00
[   10.123524]  sdc:
[   10.123530] sd 2:0:2:0: [sdd] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.123587] sd 2:0:3:0: [sde] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.123634] sd 2:0:3:0: [sde] Write Protect is off
[   10.123637] sd 2:0:3:0: [sde] Mode Sense: 00 3a 00 00
[   10.123660] sd 2:0:3:0: [sde] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.123694]  sdd:
[   10.123805] sd 2:0:4:0: [sdf] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.123828]  sde:
[   10.123851] sd 2:0:4:0: [sdf] Write Protect is off
[   10.123853] sd 2:0:4:0: [sdf] Mode Sense: 00 3a 00 00
[   10.123881] sd 2:0:4:0: [sdf] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.123886] sd 2:0:5:0: [sdg] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.123923] sd 2:0:5:0: [sdg] Write Protect is off
[   10.123925] sd 2:0:5:0: [sdg] Mode Sense: 00 3a 00 00
[   10.123945] sd 2:0:5:0: [sdg] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.124051]  sdf:
[   10.124109]  sdg:
[   10.124122] sd 2:0:6:0: [sdh] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.124171] sd 2:0:6:0: [sdh] Write Protect is off
[   10.124179] sd 2:0:7:0: [sdi] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   10.124190] sd 2:0:6:0: [sdh] Mode Sense: 00 3a 00 00
[   10.124215] sd 2:0:6:0: [sdh] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.124231] sd 2:0:7:0: [sdi] Write Protect is off
[   10.124234] sd 2:0:7:0: [sdi] Mode Sense: 00 3a 00 00
[   10.124253] sd 2:0:7:0: [sdi] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.124348]  sdh:
[   10.124396]  sdi:
[   10.125530] sd 2:0:0:0: [sdb] Write Protect is off
[   10.125659] sd 2:0:0:0: [sdb] Mode Sense: 00 3a 00 00
[   10.125679] sd 2:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   10.125892]  sdb: sde1
[   10.149361] sd 2:0:3:0: [sde] Attached SCSI disk
[   10.150749]  sdh1
[   10.151047] sd 2:0:6:0: [sdh] Attached SCSI disk
[   10.152554]  sdg1
[   10.152743]  sdf1
[   10.152825] sd 2:0:5:0: [sdg] Attached SCSI disk
[   10.153084] sd 2:0:4:0: [sdf] Attached SCSI disk
[   10.170071]  sdi1
[   10.170340] sd 2:0:7:0: [sdi] Attached SCSI disk
[   10.174730]  sdd1
[   10.175006] sd 2:0:2:0: [sdd] Attached SCSI disk
[   10.179730]  sdb1
[   10.180001] sd 2:0:0:0: [sdb] Attached SCSI disk
[   10.180520]  sdc1
[   10.180802] sd 2:0:1:0: [sdc] Attached SCSI disk
[   13.863221] scsi3 : mvsas
[   13.863719] mvsas 0000:0e:00.0: mvsas: driver version 0.8.2
[   13.863806] mvsas 0000:0e:00.0: PCI INT A -> GSI 37 (level, low) -> IRQ 37
[   13.863882] mvsas 0000:0e:00.0: setting latency timer to 64
[   13.865893] mvsas 0000:0e:00.0: mvsas: PCI-E x4, Bandwidth Usage: 2.5 Gbps
[   14.118335] ata11.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   14.118407] ata11.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   14.120811] ata11.00: configured for UDMA/133
[   14.120923] scsi 3:0:0:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   14.378559] ata12.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   14.378632] ata12.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   14.381043] ata12.00: configured for UDMA/133
[   14.381151] scsi 3:0:1:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   14.634148] ata13.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   14.634221] ata13.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   14.636724] ata13.00: configured for UDMA/133
[   14.636831] scsi 3:0:2:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   14.882425] ata14.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   14.882497] ata14.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   14.885183] ata14.00: configured for UDMA/133
[   14.885290] scsi 3:0:3:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   15.140534] ata15.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   15.140607] ata15.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   15.143051] ata15.00: configured for UDMA/133
[   15.143157] scsi 3:0:4:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   15.393875] ata16.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   15.393948] ata16.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   15.396384] ata16.00: configured for UDMA/133
[   15.396490] scsi 3:0:5:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   15.643306] ata17.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   15.643378] ata17.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   15.645785] ata17.00: configured for UDMA/133
[   15.645891] scsi 3:0:6:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   15.898177] ata18.00: ATA-8: ST9500420AS, 0002SDM1, max UDMA/133
[   15.898250] ata18.00: 976773168 sectors, multi 16: LBA48 NCQ (depth 31/32)
[   15.900763] ata18.00: configured for UDMA/133
[   15.900869] scsi 3:0:7:0: Direct-Access     ATA      ST9500420AS      0002 PQ: 0 ANSI: 5
[   15.901148] sd 3:0:0:0: [sdj] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.901243] sd 3:0:1:0: [sdk] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.901277] sd 3:0:0:0: [sdj] Write Protect is off
[   15.901280] sd 3:0:0:0: [sdj] Mode Sense: 00 3a 00 00
[   15.901301] sd 3:0:0:0: [sdj] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.901356] sd 3:0:2:0: [sdl] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.901404] sd 3:0:2:0: [sdl] Write Protect is off
[   15.901406] sd 3:0:2:0: [sdl] Mode Sense: 00 3a 00 00
[   15.901428] sd 3:0:2:0: [sdl] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.901454]  sdj:
[   15.901557]  sdl:
[   15.901960] sd 3:0:3:0: [sdm] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.901982] sd 3:0:4:0: [sdn] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.902021] sd 3:0:5:0: [sdo] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.902031] sd 3:0:3:0: [sdm] Write Protect is off
[   15.902033] sd 3:0:3:0: [sdm] Mode Sense: 00 3a 00 00
[   15.902037] sd 3:0:4:0: [sdn] Write Protect is off
[   15.902040] sd 3:0:4:0: [sdn] Mode Sense: 00 3a 00 00
[   15.902066] sd 3:0:3:0: [sdm] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.902080] sd 3:0:4:0: [sdn] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.902084] sd 3:0:5:0: [sdo] Write Protect is off
[   15.902086] sd 3:0:5:0: [sdo] Mode Sense: 00 3a 00 00
[   15.902106] sd 3:0:5:0: [sdo] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.902267]  sdm:
[   15.902326]  sdn:
[   15.902330] sd 3:0:6:0: [sdp] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.902389]  sdo:
[   15.902395] sd 3:0:6:0: [sdp] Write Protect is off
[   15.902405] sd 3:0:7:0: [sdq] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   15.902415] sd 3:0:6:0: [sdp] Mode Sense: 00 3a 00 00
[   15.902439] sd 3:0:6:0: [sdp] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.902458] sd 3:0:7:0: [sdq] Write Protect is off
[   15.902460] sd 3:0:7:0: [sdq] Mode Sense: 00 3a 00 00
[   15.902480] sd 3:0:7:0: [sdq] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.902574]  sdp:
[   15.902626]  sdq:
[   15.909750] sd 3:0:1:0: [sdk] Write Protect is off
[   15.909879] sd 3:0:1:0: [sdk] Mode Sense: 00 3a 00 00
[   15.909899] sd 3:0:1:0: [sdk] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   15.910090]  sdk: sdo1
[   15.928430]  sdm1
[   15.928772] sd 3:0:3:0: [sdm] Attached SCSI disk
[   15.928780] sd 3:0:5:0: [sdo] Attached SCSI disk
[   15.929308]  sdq1
[   15.929595] sd 3:0:7:0: [sdq] Attached SCSI disk
[   15.930218]  sdp1
[   15.930362]  sdn1
[   15.930684] sd 3:0:6:0: [sdp] Attached SCSI disk
[   15.930692] sd 3:0:4:0: [sdn] Attached SCSI disk
[   15.956862]  sdj1
[   15.957134] sd 3:0:0:0: [sdj] Attached SCSI disk
[   15.961095]  sdk1
[   15.961369] sd 3:0:1:0: [sdk] Attached SCSI disk
[   15.977514]  sdl1
[   15.977786] sd 3:0:2:0: [sdl] Attached SCSI disk
[   19.567221] scsi4 : mvsas
[   19.567757] drivers/scsi/mvsas/mv_sas.c 1259:set wide port phy map 1
[   19.577726] drivers/scsi/mvsas/mv_sas.c 1259:set wide port phy map 2
[   19.697697] scsi 4:0:0:0: Direct-Access     SEAGATE  ST9300603SS      0006 PQ: 0 ANSI: 5
[   19.698712] scsi 4:0:1:0: Direct-Access     SEAGATE  ST9300603SS      0004 PQ: 0 ANSI: 5
[   19.699766] ata19.00: ATA-7: INTEL SSDSA2M160G2GC, 2CV102G9, max UDMA/133
[   19.699840] ata19.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32)
[   19.700250] ata19.00: configured for UDMA/133
[   19.700363] scsi 4:0:2:0: Direct-Access     ATA      INTEL SSDSA2M160 2CV1 PQ: 0 ANSI: 5
[   19.760427] ata20.00: ATA-8: WDC WD5000BEVT-22A0RT0, 01.01A01, max UDMA/133
[   19.760505] ata20.00: 976773168 sectors, multi 0: LBA48 NCQ (depth 31/32)
[   19.763423] ata20.00: configured for UDMA/133
[   19.763561] scsi 4:0:3:0: Direct-Access     ATA      WDC WD5000BEVT-2 01.0 PQ: 0 ANSI: 5
[   19.764327] ata21.00: ATA-7: INTEL SSDSA2M160G2GC, 2CV102G9, max UDMA/133
[   19.764401] ata21.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32)
[   19.764808] ata21.00: configured for UDMA/133
[   19.764918] scsi 4:0:4:0: Direct-Access     ATA      INTEL SSDSA2M160 2CV1 PQ: 0 ANSI: 5
[   19.765583] ata22.00: ATA-7: INTEL SSDSA2M160G2GC, 2CV102G9, max UDMA/133
[   19.765657] ata22.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32)
[   19.766063] ata22.00: configured for UDMA/133
[   19.766174] scsi 4:0:5:0: Direct-Access     ATA      INTEL SSDSA2M160 2CV1 PQ: 0 ANSI: 5
[   19.766883] ata23.00: ATA-7: INTEL SSDSA2M160G2GC, 2CV102G9, max UDMA/133
[   19.766958] ata23.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32)
[   19.767364] ata23.00: configured for UDMA/133
[   19.767474] scsi 4:0:6:0: Direct-Access     ATA      INTEL SSDSA2M160 2CV1 PQ: 0 ANSI: 5
[   19.768143] ata24.00: ATA-7: INTEL SSDSA2M160G2GC, 2CV102G9, max UDMA/133
[   19.768218] ata24.00: 312581808 sectors, multi 1: LBA48 NCQ (depth 31/32)
[   19.768629] ata24.00: configured for UDMA/133
[   19.768740] scsi 4:0:7:0: Direct-Access     ATA      INTEL SSDSA2M160 2CV1 PQ: 0 ANSI: 5
[   20.075342] md: linear personality registered for level -1
[   20.078172] md: multipath personality registered for level -4
[   20.080678] md: raid0 personality registered for level 0
[   20.084431] md: raid1 personality registered for level 1
[   20.086750] async_tx: api initialized (async)
[   20.088194] xor: automatically using best checksumming function: generic_sse
[   20.105001]    generic_sse:  7667.000 MB/sec
[   20.105069] xor: using function: generic_sse (7667.000 MB/sec)
[   20.173029] raid6: int64x1   1668 MB/s
[   20.241010] raid6: int64x2   2231 MB/s
[   20.309024] raid6: int64x4   1809 MB/s
[   20.377009] raid6: int64x8   1481 MB/s
[   20.445013] raid6: sse2x1    3231 MB/s
[   20.513007] raid6: sse2x2    5378 MB/s
[   20.574152] sd 4:0:2:0: [sdt] 312581808 512-byte logical blocks: (160 GB/149 GiB)
[   20.574266] sd 4:0:3:0: [sdu] 976773168 512-byte logical blocks: (500 GB/465 GiB)
[   20.574302] sd 4:0:2:0: [sdt] Write Protect is off
[   20.574305] sd 4:0:2:0: [sdt] Mode Sense: 00 3a 00 00
[   20.574326] sd 4:0:2:0: [sdt] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   20.574472]  sdt:
[   20.574829] sd 4:0:0:0: [sdr] 585937500 512-byte logical blocks: (300 GB/279 GiB)
[   20.575060]  unknown partition table
[   20.575298] sd 4:0:2:0: [sdt] Attached SCSI disk
[   20.575355] sd 4:0:3:0: [sdu] Write Protect is off
[   20.575357] sd 4:0:3:0: [sdu] Mode Sense: 00 3a 00 00
[   20.575377] sd 4:0:3:0: [sdu] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   20.575494]  sdu:
[   20.575605] sd 4:0:4:0: [sdv] 312581808 512-byte logical blocks: (160 GB/149 GiB)
[   20.575793] sd 4:0:4:0: [sdv] Write Protect is off
[   20.575862] sd 4:0:4:0: [sdv] Mode Sense: 00 3a 00 00
[   20.575883] sd 4:0:4:0: [sdv] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   20.576090]  sdv:
[   20.576337] sd 4:0:0:0: [sdr] Write Protect is off
[   20.576337]  sdv1
[   20.576531] sd 4:0:0:0: [sdr] Mode Sense: bb 00 10 08
[   20.576676] sd 4:0:4:0: [sdv] Attached SCSI disk
[   20.576766] sd 4:0:5:0: [sdw] 312581808 512-byte logical blocks: (160 GB/149 GiB)
[   20.576891] sd 4:0:5:0: [sdw] Write Protect is off
[   20.576961] sd 4:0:5:0: [sdw] Mode Sense: 00 3a 00 00
[   20.576982] sd 4:0:5:0: [sdw] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   20.577185]  sdw: sdw1
[   20.577715] sd 4:0:5:0: [sdw] Attached SCSI disk
[   20.577745] sd 4:0:0:0: [sdr] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   20.578045] sd 4:0:6:0: [sdx] 312581808 512-byte logical blocks: (160 GB/149 GiB)
[   20.578170] sd 4:0:6:0: [sdx] Write Protect is off
[   20.581003] raid6: sse2x4    6124 MB/s
[   20.581005] raid6: using algorithm sse2x4 (6124 MB/s)
[   20.581015] sd 4:0:1:0: [sds] Spinning up disk...
[   20.582713] sd 4:0:7:0: [sdy] 312581808 512-byte logical blocks: (160 GB/149 GiB)
[   20.582809] sd 4:0:7:0: [sdy] Write Protect is off
[   20.582812] sd 4:0:7:0: [sdy] Mode Sense: 00 3a 00 00
[   20.582835] sd 4:0:7:0: [sdy] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   20.582982]  sdy:
[   20.585129] sd 4:0:6:0: [sdx] Mode Sense: 00 3a 00 00
[   20.585146]  sdy1
[   20.585435] sd 4:0:7:0: [sdy] Attached SCSI disk
[   20.587227] sd 4:0:6:0: [sdx] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA
[   20.587435]  sdx:
[   20.587682]  sdr: sdx1
[   20.588226] sd 4:0:6:0: [sdx] Attached SCSI disk
[   20.606932]  unknown partition table
[   20.610770] sd 4:0:0:0: [sdr] Attached SCSI disk
[   20.619931]  unknown partition table
[   20.620160] sd 4:0:3:0: [sdu] Attached SCSI disk
[   21.588016] .ready
[   21.588950] sd 4:0:1:0: [sds] 585937500 512-byte logical blocks: (300 GB/279 GiB)
[   21.590319] sd 4:0:1:0: [sds] Write Protect is off
[   21.590390] sd 4:0:1:0: [sds] Mode Sense: bb 00 10 08
[   21.591604] sd 4:0:1:0: [sds] Write cache: enabled, read cache: enabled, supports DPO and FUA
[   21.594651]  sds: unknown partition table
[   21.616283] sd 4:0:1:0: [sds] Attached SCSI disk
[   21.621228] md: raid6 personality registered for level 6
[   21.621300] md: raid5 personality registered for level 5
[   21.621369] md: raid4 personality registered for level 4
[   21.630219] md: raid10 personality registered for level 10
[   22.235758] md: md1 stopped.
[   22.235847] mdadm[1315]: segfault at 4 ip 00000000004204c5 sp 00007fff8ab36ce0 error 4 in mdadm[400000+4b000]
[   22.294697] device-mapper: uevent: version 1.0.3
[   22.294889] device-mapper: ioctl: 4.15.0-ioctl (2009-04-01) initialised: dm-devel@redhat.com
[   22.309410] PM: Starting manual resume from disk
[   22.309480] PM: Resume from partition 8:5
[   22.309482] PM: Checking hibernation image.
[   22.310028] PM: Error -22 checking image file
[   22.310029] PM: Resume from disk failed.
[   22.313978] kjournald starting.  Commit interval 5 seconds
[   22.313985] EXT3-fs: mounted filesystem with ordered data mode.
[   22.549432] udevd version 125 started
[   22.835671] processor LNXCPU:00: registered as cooling_device0
[   22.836849] processor LNXCPU:01: registered as cooling_device1
[   22.838191] processor LNXCPU:02: registered as cooling_device2
[   22.853750] processor LNXCPU:03: registered as cooling_device3
[   22.892303] input: Power Button as /devices/LNXSYSTM:00/LNXSYBUS:00/PNP0A03:00/PNP0C0C:00/input/input2
[   22.892403] ACPI: Power Button [PWRB]
[   22.892518] input: Power Button as /devices/LNXSYSTM:00/LNXPWRBN:00/input/input3
[   22.892607] ACPI: Power Button [PWRF]
[   23.015348] pci_hotplug: PCI Hot Plug PCI Core version: 0.5
[   23.022556] shpchp: Standard Hot Plug PCI Controller Driver version: 0.4
[   23.656246] EDAC MC: Ver: 2.1.0 Apr  8 2010
[   23.677107] ioatdma: Intel(R) QuickData Technology Driver 4.00
[   23.677245]   alloc irq_desc for 33 on node -1
[   23.677247]   alloc kstat_irqs on node -1
[   23.677256] ioatdma 0000:00:0f.0: PCI INT A -> GSI 33 (level, low) -> IRQ 33
[   23.677403] ioatdma 0000:00:0f.0: setting latency timer to 64
[   23.677451]   alloc irq_desc for 82 on node -1
[   23.677452]   alloc kstat_irqs on node -1
[   23.677459] ioatdma 0000:00:0f.0: irq 82 for MSI/MSI-X
[   23.677461]   alloc irq_desc for 83 on node -1
[   23.677463]   alloc kstat_irqs on node -1
[   23.677466] ioatdma 0000:00:0f.0: irq 83 for MSI/MSI-X
[   23.677469]   alloc irq_desc for 84 on node -1
[   23.677470]   alloc kstat_irqs on node -1
[   23.677473] ioatdma 0000:00:0f.0: irq 84 for MSI/MSI-X
[   23.677475]   alloc irq_desc for 85 on node -1
[   23.677477]   alloc kstat_irqs on node -1
[   23.677481] ioatdma 0000:00:0f.0: irq 85 for MSI/MSI-X
[   23.678119] igb 0000:0f:00.0: DCA enabled
[   23.678248] igb 0000:0f:00.1: DCA enabled
[   23.679359] EDAC MC0: Giving out device to 'i5400_edac.c' 'I5400': DEV 0000:00:10.0
[   23.679475] EDAC PCI0: Giving out device to module 'i5400_edac' controller 'EDAC PCI controller': DEV '0000:00:10.0' (POLLED)
[   23.729065] usbcore: registered new interface driver usbfs
[   23.729163] usbcore: registered new interface driver hub
[   23.733722] usbcore: registered new device driver usb
[   23.737732] ehci_hcd: USB 2.0 'Enhanced' Host Controller (EHCI) Driver
[   23.737843]   alloc irq_desc for 23 on node -1
[   23.737845]   alloc kstat_irqs on node -1
[   23.737854] ehci_hcd 0000:00:1d.7: PCI INT D -> GSI 23 (level, low) -> IRQ 23
[   23.737953] ehci_hcd 0000:00:1d.7: setting latency timer to 64
[   23.737956] ehci_hcd 0000:00:1d.7: EHCI Host Controller
[   23.738050] ehci_hcd 0000:00:1d.7: new USB bus registered, assigned bus number 1
[   23.738164] ehci_hcd 0000:00:1d.7: debug port 1
[   23.742117] ehci_hcd 0000:00:1d.7: cache line size of 32 is not supported
[   23.742386] ehci_hcd 0000:00:1d.7: irq 23, io mem 0xd8b04000
[   23.751300] input: PC Speaker as /devices/platform/pcspkr/input/input4
[   23.760025] ehci_hcd 0000:00:1d.7: USB 2.0 started, EHCI 1.00
[   23.760123] usb usb1: New USB device found, idVendor=1d6b, idProduct=0002
[   23.760196] usb usb1: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   23.760283] usb usb1: Product: EHCI Host Controller
[   23.760352] usb usb1: Manufacturer: Linux 2.6.32-bpo.4-amd64 ehci_hcd
[   23.760424] usb usb1: SerialNumber: 0000:00:1d.7
[   23.760583] usb usb1: configuration #1 chosen from 1 choice
[   23.760682] hub 1-0:1.0: USB hub found
[   23.760755] hub 1-0:1.0: 8 ports detected
[   23.813575] uhci_hcd: USB Universal Host Controller Interface driver
[   23.813722]   alloc irq_desc for 20 on node -1
[   23.813724]   alloc kstat_irqs on node -1
[   23.813734] uhci_hcd 0000:00:1d.0: PCI INT A -> GSI 20 (level, low) -> IRQ 20
[   23.813816] uhci_hcd 0000:00:1d.0: setting latency timer to 64
[   23.813820] uhci_hcd 0000:00:1d.0: UHCI Host Controller
[   23.813906] uhci_hcd 0000:00:1d.0: new USB bus registered, assigned bus number 2
[   23.814033] uhci_hcd 0000:00:1d.0: irq 20, io base 0x00001800
[   23.814149] usb usb2: New USB device found, idVendor=1d6b, idProduct=0001
[   23.814230] usb usb2: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   23.814319] usb usb2: Product: UHCI Host Controller
[   23.814388] usb usb2: Manufacturer: Linux 2.6.32-bpo.4-amd64 uhci_hcd
[   23.814463] usb usb2: SerialNumber: 0000:00:1d.0
[   23.814639] usb usb2: configuration #1 chosen from 1 choice
[   23.814740] hub 2-0:1.0: USB hub found
[   23.814814] hub 2-0:1.0: 2 ports detected
[   23.814933]   alloc irq_desc for 21 on node -1
[   23.814935]   alloc kstat_irqs on node -1
[   23.814939] uhci_hcd 0000:00:1d.1: PCI INT B -> GSI 21 (level, low) -> IRQ 21
[   23.815017] uhci_hcd 0000:00:1d.1: setting latency timer to 64
[   23.815020] uhci_hcd 0000:00:1d.1: UHCI Host Controller
[   23.815098] uhci_hcd 0000:00:1d.1: new USB bus registered, assigned bus number 3
[   23.815212] uhci_hcd 0000:00:1d.1: irq 21, io base 0x00001820
[   23.815313] usb usb3: New USB device found, idVendor=1d6b, idProduct=0001
[   23.821882] usb usb3: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   23.821972] usb usb3: Product: UHCI Host Controller
[   23.822043] usb usb3: Manufacturer: Linux 2.6.32-bpo.4-amd64 uhci_hcd
[   23.822122] usb usb3: SerialNumber: 0000:00:1d.1
[   23.854598] Error: Driver 'pcspkr' is already registered, aborting...
[   23.859782] usb usb3: configuration #1 chosen from 1 choice
[   23.859892] hub 3-0:1.0: USB hub found
[   23.859972] hub 3-0:1.0: 2 ports detected
[   23.860142]   alloc irq_desc for 18 on node -1
[   23.860145]   alloc kstat_irqs on node -1
[   23.860154] i801_smbus 0000:00:1f.3: PCI INT C -> GSI 18 (level, low) -> IRQ 18
[   23.860743]   alloc irq_desc for 22 on node -1
[   23.860745]   alloc kstat_irqs on node -1
[   23.860755] uhci_hcd 0000:00:1d.2: PCI INT C -> GSI 22 (level, low) -> IRQ 22
[   23.860837] uhci_hcd 0000:00:1d.2: setting latency timer to 64
[   23.860840] uhci_hcd 0000:00:1d.2: UHCI Host Controller
[   23.860921] uhci_hcd 0000:00:1d.2: new USB bus registered, assigned bus number 4
[   23.861045] uhci_hcd 0000:00:1d.2: irq 22, io base 0x00001840
[   23.861157] usb usb4: New USB device found, idVendor=1d6b, idProduct=0001
[   23.861231] usb usb4: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   23.861319] usb usb4: Product: UHCI Host Controller
[   23.861388] usb usb4: Manufacturer: Linux 2.6.32-bpo.4-amd64 uhci_hcd
[   23.861461] usb usb4: SerialNumber: 0000:00:1d.2
[   23.861540] intel_rng: FWH not detected
[   23.861687] usb usb4: configuration #1 chosen from 1 choice
[   23.861783] hub 4-0:1.0: USB hub found
[   23.861855] hub 4-0:1.0: 2 ports detected
[   23.861981] uhci_hcd 0000:00:1d.3: PCI INT D -> GSI 23 (level, low) -> IRQ 23
[   23.862058] uhci_hcd 0000:00:1d.3: setting latency timer to 64
[   23.862061] uhci_hcd 0000:00:1d.3: UHCI Host Controller
[   23.862136] uhci_hcd 0000:00:1d.3: new USB bus registered, assigned bus number 5
[   23.862247] uhci_hcd 0000:00:1d.3: irq 23, io base 0x00001860
[   23.862348] usb usb5: New USB device found, idVendor=1d6b, idProduct=0001
[   23.862421] usb usb5: New USB device strings: Mfr=3, Product=2, SerialNumber=1
[   23.862508] usb usb5: Product: UHCI Host Controller
[   23.862577] usb usb5: Manufacturer: Linux 2.6.32-bpo.4-amd64 uhci_hcd
[   23.862649] usb usb5: SerialNumber: 0000:00:1d.3
[   23.862769] usb usb5: configuration #1 chosen from 1 choice
[   23.862866] hub 5-0:1.0: USB hub found
[   23.862937] hub 5-0:1.0: 2 ports detected
[   25.453987] Adding 4096564k swap on /dev/sda5.  Priority:-1 extents:1 across:4096564k SS
[   25.491207] EXT3 FS on sda1, internal journal
[   26.512205] netconsole: local port 6665
[   26.512277] netconsole: local IP 192.168.0.12
[   26.512345] netconsole: interface eth0
[   26.512411] netconsole: remote port 6666
[   26.512478] netconsole: remote IP 192.168.0.10
[   26.512555] netconsole: remote ethernet address 00:30:48:88:36:c6
[   26.512691] netconsole: device eth0 not up yet, forcing it
[   26.524322] ADDRCONF(NETDEV_UP): eth0: link is not ready
[   26.528976] igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[   26.529710] ADDRCONF(NETDEV_CHANGE): eth0: link becomes ready
[   26.532508] netconsole: carrier detect appears untrustworthy, waiting 4 seconds
[   30.669845] console [netcon0] enabled
[   30.669913] netconsole: network logging started
[   30.690160] loop: module loaded
[   30.892186] w83627hf: Found W83627HF chip at 0x290
[   31.217681] md: md1 stopped.
[   31.227758] md: bind<sds>
[   31.227976] md: bind<sdr>
[   31.271964] raid1: raid set md1 active with 2 out of 2 mirrors
[   31.272063] md1: detected capacity change from 0 to 299999887360
[   31.273783]  md1: unknown partition table
[   31.527014] md: md0 stopped.
[   31.528042] md: bind<sdw1>
[   31.528304] md: bind<sdx1>
[   31.528559] md: bind<sdy1>
[   31.528809] md: bind<sdv1>
[   31.573520] raid5: device sdv1 operational as raid disk 0
[   31.573594] raid5: device sdx1 operational as raid disk 2
[   31.573665] raid5: device sdw1 operational as raid disk 1
[   31.574126] raid5: allocated 3230kB for md0
[   31.574233] 0: w=1 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[   31.574304] 2: w=2 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[   31.574375] 1: w=3 pa=0 pr=3 m=1 a=2 r=3 op1=0 op2=0
[   31.574446] raid5: raid level 5 set md0 active with 3 out of 3 devices, algorithm 2
[   31.574534] RAID5 conf printout:
[   31.574600]  --- rd:3 wd:3
[   31.574666]  disk 0, o:1, dev:sdv1
[   31.574734]  disk 1, o:1, dev:sdw1
[   31.574801]  disk 2, o:1, dev:sdx1
[   31.574892] md0: detected capacity change from 0 to 16450322432
[   31.575987]  md0: unknown partition table
[   31.793580] md: md127 stopped.
[   31.794266] md: bind<sdu>
[   31.837644] raid1: raid set md127 active with 1 out of 2 mirrors
[   31.837739] md127: detected capacity change from 0 to 500107771904
[   31.838825]  md127: unknown partition table
[   32.057721] md: md126 stopped.
[   32.062279] md: bind<sdd1>
[   32.062527] md: bind<sdc1>
[   32.062761] md: bind<sdb1>
[   32.063000] md: bind<sdi1>
[   32.063237] md: bind<sdh1>
[   32.063472] md: bind<sdk1>
[   32.063788] md: bind<sdf1>
[   32.064017] md: bind<sdm1>
[   32.064253] md: bind<sdl1>
[   32.064495] md: bind<sdg1>
[   32.064846] md: bind<sdj1>
[   32.065071] md: bind<sdq1>
[   32.065402] md: bind<sdp1>
[   32.065736] md: bind<sdo1>
[   32.065979] md: bind<sdn1>
[   32.066212] md: bind<sde1>
[   32.066299] md: md126 stopped.
[   32.066374] md: unbind<sde1>
[   32.096007] md: export_rdev(sde1)
[   32.096086] md: unbind<sdn1>
[   32.128007] md: export_rdev(sdn1)
[   32.128089] md: unbind<sdo1>
[   32.144007] md: export_rdev(sdo1)
[   32.144083] md: unbind<sdp1>
[   32.172009] md: export_rdev(sdp1)
[   32.172087] md: unbind<sdq1>
[   32.204007] md: export_rdev(sdq1)
[   32.204082] md: unbind<sdj1>
[   32.236006] md: export_rdev(sdj1)
[   32.236089] md: unbind<sdg1>
[   32.268006] md: export_rdev(sdg1)
[   32.268082] md: unbind<sdl1>
[   32.300008] md: export_rdev(sdl1)
[   32.300087] md: unbind<sdm1>
[   32.332007] md: export_rdev(sdm1)
[   32.332083] md: unbind<sdf1>
[   32.364007] md: export_rdev(sdf1)
[   32.364086] md: unbind<sdk1>
[   32.396007] md: export_rdev(sdk1)
[   32.396083] md: unbind<sdh1>
[   32.428007] md: export_rdev(sdh1)
[   32.428086] md: unbind<sdi1>
[   32.460007] md: export_rdev(sdi1)
[   32.460083] md: unbind<sdb1>
[   32.476009] md: export_rdev(sdb1)
[   32.476091] md: unbind<sdc1>
[   32.488008] md: export_rdev(sdc1)
[   32.488084] md: unbind<sdd1>
[   32.504009] md: export_rdev(sdd1)
[   32.568859] md: md126 stopped.
[   32.572489] md: bind<sdd1>
[   32.572728] md: bind<sdc1>
[   32.572967] md: bind<sdb1>
[   32.573199] md: bind<sdi1>
[   32.573446] md: bind<sdh1>
[   32.573682] md: bind<sdk1>
[   32.573990] md: bind<sdf1>
[   32.574231] md: bind<sdm1>
[   32.574471] md: bind<sdl1>
[   32.574708] md: bind<sdg1>
[   32.575047] md: bind<sdj1>
[   32.575282] md: bind<sdq1>
[   32.575602] md: bind<sdp1>
[   32.575934] md: bind<sdo1>
[   32.576266] md: bind<sdn1>
[   32.576491] md: bind<sde1>
[   32.576576] md: md126 stopped.
[   32.576650] md: unbind<sde1>
[   32.608007] md: export_rdev(sde1)
[   32.608083] md: unbind<sdn1>
[   32.640005] md: export_rdev(sdn1)
[   32.640084] md: unbind<sdo1>
[   32.672006] md: export_rdev(sdo1)
[   32.672081] md: unbind<sdp1>
[   32.704006] md: export_rdev(sdp1)
[   32.704084] md: unbind<sdq1>
[   32.736006] md: export_rdev(sdq1)
[   32.736081] md: unbind<sdj1>
[   32.768006] md: export_rdev(sdj1)
[   32.768084] md: unbind<sdg1>
[   32.804009] md: export_rdev(sdg1)
[   32.804087] md: unbind<sdl1>
[   32.824012] md: export_rdev(sdl1)
[   32.824092] md: unbind<sdm1>
[   32.856007] md: export_rdev(sdm1)
[   32.856083] md: unbind<sdf1>
[   32.888007] md: export_rdev(sdf1)
[   32.888086] md: unbind<sdk1>
[   32.904009] md: export_rdev(sdk1)
[   32.904088] md: unbind<sdh1>
[   32.920006] md: export_rdev(sdh1)
[   32.920086] md: unbind<sdi1>
[   32.932007] md: export_rdev(sdi1)
[   32.932083] md: unbind<sdb1>
[   32.960008] md: export_rdev(sdb1)
[   32.960092] md: unbind<sdc1>
[   32.980009] md: export_rdev(sdc1)
[   32.980085] md: unbind<sdd1>
[   33.008009] md: export_rdev(sdd1)
[   36.660506] eth0: no IPv6 routers present
[   39.280196] RPC: Registered udp transport module.
[   39.280278] RPC: Registered tcp transport module.
[   39.280348] RPC: Registered tcp NFSv4.1 backchannel transport module.
[   39.304055] Installing knfsd (copyright (C) 1996 okir@monad.swb.de).
[   39.329513] svc: failed to register lockdv1 RPC service (errno 97).
[   39.330163] NFSD: Using /var/lib/nfs/v4recovery as the NFSv4 state recovery directory
[   39.337003] NFSD: starting 90-second grace period
[   83.579264] iSCSI Enterprise Target Software - version 1.4.20.1
[   83.579427] iscsi_trgt: Registered io type fileio
[   83.579497] iscsi_trgt: Registered io type blockio
[   83.579568] iscsi_trgt: Registered io type nullio
[127107.616125] igb: eth0 NIC Link is Down
[127110.000477] igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
[162440.178712] md: md127 stopped.
[162440.178792] md: unbind<sdu>
[162440.193513] md: export_rdev(sdu)
[162440.193848] md127: detected capacity change from 500107771904 to 0
[163354.981793] md: md2 stopped.
[163355.065936] md: bind<sdd1>
[163355.066192] md: bind<sdc1>
[163355.066443] md: bind<sdb1>
[163355.066696] md: bind<sdi1>
[163355.066948] md: bind<sdh1>
[163355.067200] md: bind<sdk1>
[163355.067451] md: bind<sdf1>
[163355.067790] md: bind<sdm1>
[163355.068089] md: bind<sdl1>
[163355.068360] md: bind<sdg1>
[163355.068609] md: bind<sdj1>
[163355.068933] md: bind<sdq1>
[163355.069268] md: bind<sdp1>
[163355.070109] md: bind<sdo1>
[163355.070456] md: bind<sdn1>
[163355.070698] md: bind<sde1>
[163417.377269] md: md2 stopped.
[163417.377355] md: unbind<sde1>
[163417.400514] md: export_rdev(sde1)
[163417.400602] md: unbind<sdn1>
[163417.436010] md: export_rdev(sdn1)
[163417.436096] md: unbind<sdo1>
[163417.456013] md: export_rdev(sdo1)
[163417.456098] md: unbind<sdp1>
[163417.488011] md: export_rdev(sdp1)
[163417.488093] md: unbind<sdq1>
[163417.536013] md: export_rdev(sdq1)
[163417.536102] md: unbind<sdj1>
[163417.564513] md: export_rdev(sdj1)
[163417.571017] md: unbind<sdg1>
[163417.580509] md: export_rdev(sdg1)
[163417.580595] md: unbind<sdl1>
[163417.596008] md: export_rdev(sdl1)
[163417.596090] md: unbind<sdm1>
[163417.608009] md: export_rdev(sdm1)
[163417.608093] md: unbind<sdf1>
[163417.640007] md: export_rdev(sdf1)
[163417.640087] md: unbind<sdk1>
[163417.672006] md: export_rdev(sdk1)
[163417.672090] md: unbind<sdh1>
[163417.704007] md: export_rdev(sdh1)
[163417.704086] md: unbind<sdi1>
[163417.736007] md: export_rdev(sdi1)
[163417.736090] md: unbind<sdb1>
[163417.768007] md: export_rdev(sdb1)
[163417.768087] md: unbind<sdc1>
[163417.800007] md: export_rdev(sdc1)
[163417.800090] md: unbind<sdd1>
[163417.832006] md: export_rdev(sdd1)
[163418.910422] md: md2 stopped.
[163419.427649] md: bind<sdd1>
[163419.427892] md: bind<sdc1>
[163419.428142] md: bind<sdb1>
[163419.428397] md: bind<sdi1>
[163419.428640] md: bind<sdh1>
[163419.428895] md: bind<sdk1>
[163419.429151] md: bind<sdf1>
[163419.429401] md: bind<sdm1>
[163419.429734] md: bind<sdl1>
[163419.429976] md: bind<sdg1>
[163419.430221] md: bind<sdj1>
[163419.430461] md: bind<sdq1>
[163419.430706] md: bind<sdp1>
[163419.430952] md: bind<sdo1>
[163419.431198] md: bind<sdn1>
[163419.453964] md: bind<sde1>
[163419.454085] md: kicking non-fresh sdn1 from array!
[163419.454159] md: unbind<sdn1>
[163419.476007] md: export_rdev(sdn1)
[163419.476088] md: kicking non-fresh sdj1 from array!
[163419.476160] md: unbind<sdj1>
[163419.508007] md: export_rdev(sdj1)
[163419.552645] raid5: device sde1 operational as raid disk 0
[163419.552718] raid5: device sdo1 operational as raid disk 14
[163419.552790] raid5: device sdp1 operational as raid disk 13
[163419.552861] raid5: device sdq1 operational as raid disk 12
[163419.552933] raid5: device sdg1 operational as raid disk 10
[163419.553010] raid5: device sdl1 operational as raid disk 9
[163419.553081] raid5: device sdm1 operational as raid disk 8
[163419.553152] raid5: device sdf1 operational as raid disk 7
[163419.553223] raid5: device sdk1 operational as raid disk 6
[163419.553295] raid5: device sdh1 operational as raid disk 5
[163419.553366] raid5: device sdi1 operational as raid disk 4
[163419.553437] raid5: device sdb1 operational as raid disk 3
[163419.553509] raid5: device sdc1 operational as raid disk 2
[163419.553580] raid5: device sdd1 operational as raid disk 1
[163419.554787] raid5: allocated 15854kB for md2
[163419.554893] 0: w=1 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.554965] 14: w=2 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555037] 13: w=3 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555109] 12: w=4 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555180] 10: w=5 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555252] 9: w=6 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555324] 8: w=7 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555396] 7: w=8 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555467] 6: w=9 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555538] 5: w=10 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555610] 4: w=11 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555682] 3: w=12 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555754] 2: w=13 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555825] 1: w=14 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
[163419.555901] raid5: raid level 5 set md2 active with 14 out of 15 devices, algorithm 2
[163419.555990] RAID5 conf printout:
[163419.556066]  --- rd:15 wd:14
[163419.556133]  disk 0, o:1, dev:sde1
[163419.556201]  disk 1, o:1, dev:sdd1
[163419.556269]  disk 2, o:1, dev:sdc1
[163419.556336]  disk 3, o:1, dev:sdb1
[163419.556404]  disk 4, o:1, dev:sdi1
[163419.556471]  disk 5, o:1, dev:sdh1
[163419.556539]  disk 6, o:1, dev:sdk1
[163419.556607]  disk 7, o:1, dev:sdf1
[163419.556674]  disk 8, o:1, dev:sdm1
[163419.556742]  disk 9, o:1, dev:sdl1
[163419.556809]  disk 10, o:1, dev:sdg1
[163419.556881]  disk 12, o:1, dev:sdq1
[163419.556948]  disk 13, o:1, dev:sdp1
[163419.557016]  disk 14, o:1, dev:sdo1
[163419.557139] md2: detected capacity change from 0 to 7001471188992
[163419.557397]  md2: unknown partition table
[163613.792018] drivers/scsi/mvsas/mv_sas.c 1631:mvs_query_task:rc= 5
[163615.796001] mvsas 0000:04:00.0: Phy6 : No sig fis
[163616.000007] drivers/scsi/mvsas/mv_sas.c 1585:mvs_I_T_nexus_reset for device[6]:rc= 0
[163616.000015] ata9: translated ATA stat/err 0x01/04 to SCSI SK/ASC/ASCQ 0xb/00/00
[163616.000111] ata9: status=0x01 { Error }
[163616.000274] ata9: error=0x04 { DriveStatusError }

[-- Attachment #3: messages.txt --]
[-- Type: text/plain, Size: 13528 bytes --]

May  5 06:25:18 supernas01 syslogd 1.5.0#5: restart.
May  5 06:43:02 supernas01 -- MARK --
May  5 07:03:02 supernas01 -- MARK --
May  5 07:23:02 supernas01 -- MARK --
May  5 07:43:02 supernas01 -- MARK --
May  5 08:03:02 supernas01 -- MARK --
May  5 08:23:02 supernas01 -- MARK --
May  5 08:43:02 supernas01 -- MARK --
May  5 09:03:02 supernas01 -- MARK --
May  5 09:23:02 supernas01 -- MARK --
May  5 09:43:02 supernas01 -- MARK --
May  5 10:03:02 supernas01 -- MARK --
May  5 10:23:02 supernas01 -- MARK --
May  5 10:43:02 supernas01 -- MARK --
May  5 11:03:02 supernas01 -- MARK --
May  5 11:23:02 supernas01 -- MARK --
May  5 11:43:02 supernas01 -- MARK --
May  5 12:03:02 supernas01 -- MARK --
May  5 12:23:02 supernas01 -- MARK --
May  5 12:43:02 supernas01 -- MARK --
May  5 13:03:02 supernas01 -- MARK --
May  5 13:23:02 supernas01 -- MARK --
May  5 13:43:02 supernas01 -- MARK --
May  5 14:03:02 supernas01 -- MARK --
May  5 14:23:02 supernas01 -- MARK --
May  5 14:43:02 supernas01 -- MARK --
May  5 15:03:02 supernas01 -- MARK --
May  5 15:23:02 supernas01 -- MARK --
May  5 15:43:02 supernas01 -- MARK --
May  5 16:03:02 supernas01 -- MARK --
May  5 16:23:02 supernas01 -- MARK --
May  5 16:43:02 supernas01 -- MARK --
May  5 17:03:02 supernas01 -- MARK --
May  5 17:23:02 supernas01 -- MARK --
May  5 17:43:02 supernas01 -- MARK --
May  5 18:03:02 supernas01 -- MARK --
May  5 18:23:02 supernas01 -- MARK --
May  5 18:43:02 supernas01 -- MARK --
May  5 19:03:02 supernas01 -- MARK --
May  5 19:23:02 supernas01 -- MARK --
May  5 19:43:02 supernas01 -- MARK --
May  5 20:03:02 supernas01 -- MARK --
May  5 20:23:02 supernas01 -- MARK --
May  5 20:43:02 supernas01 -- MARK --
May  5 21:03:02 supernas01 -- MARK --
May  5 21:23:02 supernas01 -- MARK --
May  5 21:43:02 supernas01 -- MARK --
May  5 22:03:02 supernas01 -- MARK --
May  5 22:23:02 supernas01 -- MARK --
May  5 22:43:02 supernas01 -- MARK --
May  5 23:03:02 supernas01 -- MARK --
May  5 23:23:02 supernas01 -- MARK --
May  5 23:43:02 supernas01 -- MARK --
May  6 00:03:02 supernas01 -- MARK --
May  6 00:23:02 supernas01 -- MARK --
May  6 00:43:02 supernas01 -- MARK --
May  6 01:03:02 supernas01 -- MARK --
May  6 01:23:02 supernas01 -- MARK --
May  6 01:43:02 supernas01 -- MARK --
May  6 02:03:02 supernas01 -- MARK --
May  6 02:23:02 supernas01 -- MARK --
May  6 02:43:02 supernas01 -- MARK --
May  6 03:03:02 supernas01 -- MARK --
May  6 03:20:51 supernas01 kernel: [127107.616125] igb: eth0 NIC Link is Down
May  6 03:20:54 supernas01 kernel: [127110.000477] igb: eth0 NIC Link is Up 1000 Mbps Full Duplex, Flow Control: RX/TX
May  6 03:43:02 supernas01 -- MARK --
May  6 04:03:02 supernas01 -- MARK --
May  6 04:23:02 supernas01 -- MARK --
May  6 04:43:02 supernas01 -- MARK --
May  6 05:03:02 supernas01 -- MARK --
May  6 05:23:02 supernas01 -- MARK --
May  6 05:43:02 supernas01 -- MARK --
May  6 06:03:02 supernas01 -- MARK --
May  6 06:23:02 supernas01 -- MARK --
May  6 06:25:16 supernas01 syslogd 1.5.0#5: restart.
May  6 06:43:02 supernas01 -- MARK --
May  6 07:03:02 supernas01 -- MARK --
May  6 07:23:02 supernas01 -- MARK --
May  6 07:43:02 supernas01 -- MARK --
May  6 08:03:02 supernas01 -- MARK --
May  6 08:23:02 supernas01 -- MARK --
May  6 08:43:02 supernas01 -- MARK --
May  6 09:03:02 supernas01 -- MARK --
May  6 09:23:02 supernas01 -- MARK --
May  6 09:43:02 supernas01 -- MARK --
May  6 10:03:02 supernas01 -- MARK --
May  6 10:23:02 supernas01 -- MARK --
May  6 10:43:02 supernas01 -- MARK --
May  6 11:03:02 supernas01 -- MARK --
May  6 11:23:02 supernas01 -- MARK --
May  6 11:43:02 supernas01 -- MARK --
May  6 12:03:02 supernas01 -- MARK --
May  6 12:23:02 supernas01 -- MARK --
May  6 12:43:02 supernas01 -- MARK --
May  6 13:03:02 supernas01 -- MARK --
May  6 13:09:44 supernas01 kernel: [162440.178712] md: md127 stopped.
May  6 13:09:44 supernas01 kernel: [162440.178792] md: unbind<sdu>
May  6 13:09:44 supernas01 kernel: [162440.193513] md: export_rdev(sdu)
May  6 13:09:44 supernas01 kernel: [162440.193848] md127: detected capacity change from 500107771904 to 0
May  6 13:23:02 supernas01 -- MARK --
May  6 13:24:59 supernas01 kernel: [163354.981793] md: md2 stopped.
May  6 13:24:59 supernas01 kernel: [163355.065936] md: bind<sdd1>
May  6 13:24:59 supernas01 kernel: [163355.066192] md: bind<sdc1>
May  6 13:24:59 supernas01 kernel: [163355.066443] md: bind<sdb1>
May  6 13:24:59 supernas01 kernel: [163355.066696] md: bind<sdi1>
May  6 13:24:59 supernas01 kernel: [163355.066948] md: bind<sdh1>
May  6 13:24:59 supernas01 kernel: [163355.067200] md: bind<sdk1>
May  6 13:24:59 supernas01 kernel: [163355.067451] md: bind<sdf1>
May  6 13:24:59 supernas01 kernel: [163355.067790] md: bind<sdm1>
May  6 13:24:59 supernas01 kernel: [163355.068089] md: bind<sdl1>
May  6 13:24:59 supernas01 kernel: [163355.068360] md: bind<sdg1>
May  6 13:24:59 supernas01 kernel: [163355.068609] md: bind<sdj1>
May  6 13:24:59 supernas01 kernel: [163355.068933] md: bind<sdq1>
May  6 13:24:59 supernas01 kernel: [163355.069268] md: bind<sdp1>
May  6 13:24:59 supernas01 kernel: [163355.070109] md: bind<sdo1>
May  6 13:24:59 supernas01 kernel: [163355.070456] md: bind<sdn1>
May  6 13:24:59 supernas01 kernel: [163355.070698] md: bind<sde1>
May  6 13:26:01 supernas01 kernel: [163417.377269] md: md2 stopped.
May  6 13:26:01 supernas01 kernel: [163417.377355] md: unbind<sde1>
May  6 13:26:01 supernas01 kernel: [163417.400514] md: export_rdev(sde1)
May  6 13:26:01 supernas01 kernel: [163417.400602] md: unbind<sdn1>
May  6 13:26:01 supernas01 kernel: [163417.436010] md: export_rdev(sdn1)
May  6 13:26:01 supernas01 kernel: [163417.436096] md: unbind<sdo1>
May  6 13:26:01 supernas01 kernel: [163417.456013] md: export_rdev(sdo1)
May  6 13:26:01 supernas01 kernel: [163417.456098] md: unbind<sdp1>
May  6 13:26:01 supernas01 kernel: [163417.488011] md: export_rdev(sdp1)
May  6 13:26:01 supernas01 kernel: [163417.488093] md: unbind<sdq1>
May  6 13:26:01 supernas01 kernel: [163417.536013] md: export_rdev(sdq1)
May  6 13:26:01 supernas01 kernel: [163417.536102] md: unbind<sdj1>
May  6 13:26:01 supernas01 kernel: [163417.564513] md: export_rdev(sdj1)
May  6 13:26:01 supernas01 kernel: [163417.571017] md: unbind<sdg1>
May  6 13:26:01 supernas01 kernel: [163417.580509] md: export_rdev(sdg1)
May  6 13:26:01 supernas01 kernel: [163417.580595] md: unbind<sdl1>
May  6 13:26:01 supernas01 kernel: [163417.596008] md: export_rdev(sdl1)
May  6 13:26:01 supernas01 kernel: [163417.596090] md: unbind<sdm1>
May  6 13:26:01 supernas01 kernel: [163417.608009] md: export_rdev(sdm1)
May  6 13:26:01 supernas01 kernel: [163417.608093] md: unbind<sdf1>
May  6 13:26:01 supernas01 kernel: [163417.640007] md: export_rdev(sdf1)
May  6 13:26:01 supernas01 kernel: [163417.640087] md: unbind<sdk1>
May  6 13:26:01 supernas01 kernel: [163417.672006] md: export_rdev(sdk1)
May  6 13:26:01 supernas01 kernel: [163417.672090] md: unbind<sdh1>
May  6 13:26:01 supernas01 kernel: [163417.704007] md: export_rdev(sdh1)
May  6 13:26:01 supernas01 kernel: [163417.704086] md: unbind<sdi1>
May  6 13:26:01 supernas01 kernel: [163417.736007] md: export_rdev(sdi1)
May  6 13:26:01 supernas01 kernel: [163417.736090] md: unbind<sdb1>
May  6 13:26:01 supernas01 kernel: [163417.768007] md: export_rdev(sdb1)
May  6 13:26:01 supernas01 kernel: [163417.768087] md: unbind<sdc1>
May  6 13:26:01 supernas01 kernel: [163417.800007] md: export_rdev(sdc1)
May  6 13:26:01 supernas01 kernel: [163417.800090] md: unbind<sdd1>
May  6 13:26:02 supernas01 kernel: [163417.832006] md: export_rdev(sdd1)
May  6 13:26:03 supernas01 kernel: [163418.910422] md: md2 stopped.
May  6 13:26:03 supernas01 kernel: [163419.427649] md: bind<sdd1>
May  6 13:26:03 supernas01 kernel: [163419.427892] md: bind<sdc1>
May  6 13:26:03 supernas01 kernel: [163419.428142] md: bind<sdb1>
May  6 13:26:03 supernas01 kernel: [163419.428397] md: bind<sdi1>
May  6 13:26:03 supernas01 kernel: [163419.428640] md: bind<sdh1>
May  6 13:26:03 supernas01 kernel: [163419.428895] md: bind<sdk1>
May  6 13:26:03 supernas01 kernel: [163419.429151] md: bind<sdf1>
May  6 13:26:03 supernas01 kernel: [163419.429401] md: bind<sdm1>
May  6 13:26:03 supernas01 kernel: [163419.429734] md: bind<sdl1>
May  6 13:26:03 supernas01 kernel: [163419.429976] md: bind<sdg1>
May  6 13:26:03 supernas01 kernel: [163419.430221] md: bind<sdj1>
May  6 13:26:03 supernas01 kernel: [163419.430461] md: bind<sdq1>
May  6 13:26:03 supernas01 kernel: [163419.430706] md: bind<sdp1>
May  6 13:26:03 supernas01 kernel: [163419.430952] md: bind<sdo1>
May  6 13:26:03 supernas01 kernel: [163419.431198] md: bind<sdn1>
May  6 13:26:03 supernas01 kernel: [163419.453964] md: bind<sde1>
May  6 13:26:03 supernas01 kernel: [163419.454085] md: kicking non-fresh sdn1 from array!
May  6 13:26:03 supernas01 kernel: [163419.454159] md: unbind<sdn1>
May  6 13:26:03 supernas01 kernel: [163419.476007] md: export_rdev(sdn1)
May  6 13:26:03 supernas01 kernel: [163419.476088] md: kicking non-fresh sdj1 from array!
May  6 13:26:03 supernas01 kernel: [163419.476160] md: unbind<sdj1>
May  6 13:26:03 supernas01 kernel: [163419.508007] md: export_rdev(sdj1)
May  6 13:26:03 supernas01 kernel: [163419.552645] raid5: device sde1 operational as raid disk 0
May  6 13:26:03 supernas01 kernel: [163419.552718] raid5: device sdo1 operational as raid disk 14
May  6 13:26:03 supernas01 kernel: [163419.552790] raid5: device sdp1 operational as raid disk 13
May  6 13:26:03 supernas01 kernel: [163419.552861] raid5: device sdq1 operational as raid disk 12
May  6 13:26:03 supernas01 kernel: [163419.552933] raid5: device sdg1 operational as raid disk 10
May  6 13:26:03 supernas01 kernel: [163419.553010] raid5: device sdl1 operational as raid disk 9
May  6 13:26:03 supernas01 kernel: [163419.553081] raid5: device sdm1 operational as raid disk 8
May  6 13:26:03 supernas01 kernel: [163419.553152] raid5: device sdf1 operational as raid disk 7
May  6 13:26:03 supernas01 kernel: [163419.553223] raid5: device sdk1 operational as raid disk 6
May  6 13:26:03 supernas01 kernel: [163419.553295] raid5: device sdh1 operational as raid disk 5
May  6 13:26:03 supernas01 kernel: [163419.553366] raid5: device sdi1 operational as raid disk 4
May  6 13:26:03 supernas01 kernel: [163419.553437] raid5: device sdb1 operational as raid disk 3
May  6 13:26:03 supernas01 kernel: [163419.553509] raid5: device sdc1 operational as raid disk 2
May  6 13:26:03 supernas01 kernel: [163419.553580] raid5: device sdd1 operational as raid disk 1
May  6 13:26:03 supernas01 kernel: [163419.554787] raid5: allocated 15854kB for md2
May  6 13:26:03 supernas01 kernel: [163419.554893] 0: w=1 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.554965] 14: w=2 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555037] 13: w=3 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555109] 12: w=4 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555180] 10: w=5 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555252] 9: w=6 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555324] 8: w=7 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555396] 7: w=8 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555467] 6: w=9 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555538] 5: w=10 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555610] 4: w=11 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555682] 3: w=12 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555754] 2: w=13 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555825] 1: w=14 pa=0 pr=15 m=1 a=2 r=15 op1=0 op2=0
May  6 13:26:03 supernas01 kernel: [163419.555990] RAID5 conf printout:
May  6 13:26:03 supernas01 kernel: [163419.556066]  --- rd:15 wd:14
May  6 13:26:03 supernas01 kernel: [163419.556133]  disk 0, o:1, dev:sde1
May  6 13:26:03 supernas01 kernel: [163419.556201]  disk 1, o:1, dev:sdd1
May  6 13:26:03 supernas01 kernel: [163419.556269]  disk 2, o:1, dev:sdc1
May  6 13:26:03 supernas01 kernel: [163419.556336]  disk 3, o:1, dev:sdb1
May  6 13:26:03 supernas01 kernel: [163419.556404]  disk 4, o:1, dev:sdi1
May  6 13:26:03 supernas01 kernel: [163419.556471]  disk 5, o:1, dev:sdh1
May  6 13:26:03 supernas01 kernel: [163419.556539]  disk 6, o:1, dev:sdk1
May  6 13:26:03 supernas01 kernel: [163419.556607]  disk 7, o:1, dev:sdf1
May  6 13:26:03 supernas01 kernel: [163419.556674]  disk 8, o:1, dev:sdm1
May  6 13:26:03 supernas01 kernel: [163419.556742]  disk 9, o:1, dev:sdl1
May  6 13:26:03 supernas01 kernel: [163419.556809]  disk 10, o:1, dev:sdg1
May  6 13:26:03 supernas01 kernel: [163419.556881]  disk 12, o:1, dev:sdq1
May  6 13:26:03 supernas01 kernel: [163419.556948]  disk 13, o:1, dev:sdp1
May  6 13:26:03 supernas01 kernel: [163419.557016]  disk 14, o:1, dev:sdo1
May  6 13:26:03 supernas01 kernel: [163419.557139] md2: detected capacity change from 0 to 7001471188992
May  6 13:26:03 supernas01 kernel: [163419.557397]  md2: unknown partition table
May  6 13:29:20 supernas01 kernel: [163616.000111] ata9: status=0x01 { Error }
May  6 13:29:20 supernas01 kernel: [163616.000274] ata9: error=0x04 { DriveStatusError }
May  6 13:43:02 supernas01 -- MARK --
May  6 14:03:02 supernas01 -- MARK --

[-- Attachment #4: lspci.txt --]
[-- Type: text/plain, Size: 365 bytes --]

04:00.0 SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)
0c:00.0 SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)
0e:00.0 SCSI storage controller: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B (rev 01)

Re: Still havind major MVSAS issues.

[Jelle de Jong]

[-- Attachment #1: Type: text/plain, Size: 1815 bytes --]

Dear Srini,

I spent a few weeks catering information and did some intensive
testing the last few days.

Srinivas Naga Venkatasatya Pasagadugula wrote, on 06-05-10 08:01:
> 1. Is this is the problem with only WD SATA drives? (I don't have WD SATA drives to reproduce this issue.)
> 2. Whether the problem is with direct attached SATA drives or drives connected in expanders also?
> 3. Could you please provide me the "dmesg" log or "/var/log/messages" log.  
> 4. How much the capacity of SATA drives connected to controller?
> 5. Your HBA is having 6440 chipset? 

With a few tricks I managed to boot my OS from the mvsas controller. I
got eleven different sata disks attached from a 4-port mini-sas
backplate without expander to the two Marvel 88SE63xx/64xx mvsas
controllers in my system.

I managed to add five mdadm raid1 arrays without added the actual
active sync device (so one disk for each array) I build my lvm systems
on top of this and did a lot of file transfers for testing.

This worked stable enough, there are HDIO_GET_IDENTITY errors during
boot and operation, but the hard disks seem to be working.

So to debug if issues are related to some brand of harddisk, I started
to add WDC, Hitachi, SAMSUNG, Maxtor and Seagate disks to the
respected raid1 array, the disks are of different sizes (320GB, 500GB,
1TB.)

The sync starts and will fail directly or a while later on failures in
the mvsas driver. I attached an failure example as attachment.

The failures are of grave severity, I have lost complete lvm2 volumes
and raid arrays during testing.

Do you also have a SuperMicro AOC-SASLP-MV8 controller for testing?

I would love to use the controllers for production, but they are
currently unstable. I hope this information helps to solve the mvsas
issues?

With kind regards,

Jelle de Jong

[-- Attachment #2: melany-kernel-debug04.txt --]
[-- Type: text/plain, Size: 20642 bytes --]

#-----------------------------------------------------------------------

2010-06-96 mvsas kernel module issues

#-----------------------------------------------------------------------

# the system seems to be stable without using an active synced raid1 array, on the moment I add a drive to the raid1 array so it can start syncing the mvsas driver is ginging errors locking the hole raid array

# raid1 with /dev/md5 will lockup causing all lvm systems using teh md5 to lockup, system reboot fails at shuting down lvm volumes, SysRq b is needed to reboot.

#-----------------------------------------------------------------------

# lspci -nn | grep SCSI
02:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)
03:00.0 SCSI storage controller [0100]: Marvell Technology Group Ltd. MV64460/64461/64462 System Controller, Revision B [11ab:6485] (rev 01)

#-----------------------------------------------------------------------

Jun  6 13:22:45 melany kernel: [58739.130711] md: bind<sdg1>
Jun  6 13:22:45 melany mdadm[2484]: RebuildStarted event detected on md device /dev/md5
Jun  6 13:22:45 melany kernel: [58739.174013] RAID1 conf printout:
Jun  6 13:22:45 melany kernel: [58739.174016]  --- wd:1 rd:2
Jun  6 13:22:45 melany kernel: [58739.174018]  disk 0, wo:0, o:1, dev:sdb1
Jun  6 13:22:45 melany kernel: [58739.174019]  disk 1, wo:1, o:1, dev:sdg1
Jun  6 13:22:45 melany kernel: [58739.174399] md: recovery of RAID array md5
Jun  6 13:22:45 melany kernel: [58739.174400] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
Jun  6 13:22:45 melany kernel: [58739.174402] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for recovery.
Jun  6 13:22:45 melany kernel: [58739.174405] md: using 128k window, over a total of 312560576 blocks.
Jun  6 13:23:16 melany kernel: [58769.804083] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:23:16 melany kernel: [58769.804087] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:23:46 melany kernel: [58799.816089] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:23:46 melany kernel: [58799.816093] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:24:17 melany kernel: [58830.816071] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:24:17 melany kernel: [58830.816075] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:24:29 melany kernel: [58842.961180] raid1: Disk failure on sdg1, disabling device.
Jun  6 13:24:29 melany kernel: [58842.961181] raid1: Operation continuing on 1 devices.
Jun  6 13:25:00 melany kernel: [58873.816103] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:25:00 melany kernel: [58873.816107] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:25:31 melany kernel: [58904.816073] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:25:31 melany kernel: [58904.816077] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:25:46 melany kernel: [58920.532075] INFO: task md5_resync:2813 blocked for more than 120 seconds.
Jun  6 13:25:46 melany kernel: [58920.532083] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun  6 13:25:46 melany kernel: [58920.532091] md5_resync    D 0000000000000000     0  2813      2 0x00000000
Jun  6 13:25:46 melany kernel: [58920.532094]  ffffffff814471f0 0000000000000046 0000000000000000 ffffffff8104a0b8
Jun  6 13:25:46 melany kernel: [58920.532097]  0000000000000086 0000000000000086 000000000000f8a0 ffff880029657fd8
Jun  6 13:25:46 melany kernel: [58920.532099]  00000000000155c0 00000000000155c0 ffff88006e74b170 ffff88006e74b468
Jun  6 13:25:46 melany kernel: [58920.532102] Call Trace:
Jun  6 13:25:46 melany kernel: [58920.532108]  [<ffffffff8104a0b8>] ? try_to_wake_up+0x249/0x259
Jun  6 13:25:46 melany kernel: [58920.532112]  [<ffffffff8103aa9c>] ? __wake_up_common+0x44/0x72
Jun  6 13:25:46 melany kernel: [58920.532119]  [<ffffffffa01a2424>] ? raise_barrier+0x12c/0x163 [raid1]
Jun  6 13:25:46 melany kernel: [58920.532122]  [<ffffffff8104a0c8>] ? default_wake_function+0x0/0x9
Jun  6 13:25:46 melany kernel: [58920.532124]  [<ffffffff8104a0b8>] ? try_to_wake_up+0x249/0x259
Jun  6 13:25:46 melany kernel: [58920.532127]  [<ffffffffa01a2ed7>] ? sync_request+0x177/0x512 [raid1]
Jun  6 13:25:46 melany kernel: [58920.532131]  [<ffffffffa018b2f1>] ? is_mddev_idle+0xaa/0xfd [md_mod]
Jun  6 13:25:46 melany kernel: [58920.532135]  [<ffffffffa018ba2f>] ? md_do_sync+0x6eb/0xaf6 [md_mod]
Jun  6 13:25:46 melany kernel: [58920.532140]  [<ffffffffa018c664>] ? md_thread+0xf1/0x10f [md_mod]
Jun  6 13:25:46 melany kernel: [58920.532142]  [<ffffffff8103aa9c>] ? __wake_up_common+0x44/0x72
Jun  6 13:25:46 melany kernel: [58920.532145]  [<ffffffffa018c573>] ? md_thread+0x0/0x10f [md_mod]
Jun  6 13:25:46 melany kernel: [58920.532148]  [<ffffffff810644d1>] ? kthread+0x79/0x81
Jun  6 13:25:46 melany kernel: [58920.532151]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
Jun  6 13:25:46 melany kernel: [58920.532153]  [<ffffffff81064458>] ? kthread+0x0/0x81
Jun  6 13:25:46 melany kernel: [58920.532155]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20
Jun  6 13:26:02 melany kernel: [58935.816072] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:26:02 melany kernel: [58935.816075] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:26:02 melany kernel: [58935.816095] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816096] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816099] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 26 41 00 00 80 00
Jun  6 13:26:02 melany kernel: [58935.816104] end_request: I/O error, dev sdb, sector 75329
Jun  6 13:26:02 melany kernel: [58935.816123] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816124] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816126] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 25 41 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816130] end_request: I/O error, dev sdb, sector 75073
Jun  6 13:26:02 melany kernel: [58935.816138] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816139] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816141] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 24 c1 00 00 80 00
Jun  6 13:26:02 melany kernel: [58935.816145] end_request: I/O error, dev sdb, sector 74945
Jun  6 13:26:02 melany kernel: [58935.816152] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816153] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816154] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 23 c1 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816158] end_request: I/O error, dev sdb, sector 74689
Jun  6 13:26:02 melany kernel: [58935.816165] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816166] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816168] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 23 41 00 00 80 00
Jun  6 13:26:02 melany kernel: [58935.816172] end_request: I/O error, dev sdb, sector 74561
Jun  6 13:26:02 melany kernel: [58935.816179] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816180] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816182] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 22 41 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816185] end_request: I/O error, dev sdb, sector 74305
Jun  6 13:26:02 melany kernel: [58935.816192] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816194] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816195] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 21 41 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816199] end_request: I/O error, dev sdb, sector 74049
Jun  6 13:26:02 melany kernel: [58935.816206] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816207] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816209] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 20 41 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816213] end_request: I/O error, dev sdb, sector 73793
Jun  6 13:26:02 melany kernel: [58935.816220] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816221] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816223] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1f 41 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816226] end_request: I/O error, dev sdb, sector 73537
Jun  6 13:26:02 melany kernel: [58935.816234] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816235] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816237] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1d c1 00 01 80 00
Jun  6 13:26:02 melany kernel: [58935.816241] end_request: I/O error, dev sdb, sector 73153
Jun  6 13:26:02 melany kernel: [58935.816248] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816249] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816251] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1c c1 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816255] end_request: I/O error, dev sdb, sector 72897
Jun  6 13:26:02 melany kernel: [58935.816262] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816263] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816265] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1c 41 00 00 80 00
Jun  6 13:26:02 melany kernel: [58935.816269] end_request: I/O error, dev sdb, sector 72769
Jun  6 13:26:02 melany kernel: [58935.816276] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816277] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816279] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1b c1 00 00 80 00
Jun  6 13:26:02 melany kernel: [58935.816282] end_request: I/O error, dev sdb, sector 72641
Jun  6 13:26:02 melany kernel: [58935.816290] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816291] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816292] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1b 41 00 00 80 00
Jun  6 13:26:02 melany kernel: [58935.816296] end_request: I/O error, dev sdb, sector 72513
Jun  6 13:26:02 melany kernel: [58935.816303] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816304] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816306] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 1a 41 00 01 00 00
Jun  6 13:26:02 melany kernel: [58935.816310] end_request: I/O error, dev sdb, sector 72257
Jun  6 13:26:02 melany kernel: [58935.816318] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:26:02 melany kernel: [58935.816319] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:26:02 melany kernel: [58935.816320] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 17 c1 00 02 80 00
Jun  6 13:26:02 melany kernel: [58935.816324] end_request: I/O error, dev sdb, sector 71617
Jun  6 13:26:33 melany kernel: [58966.816067] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:26:33 melany kernel: [58966.816071] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:27:04 melany kernel: [58997.816079] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:27:04 melany kernel: [58997.816083] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:27:35 melany kernel: [59028.816068] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:27:35 melany kernel: [59028.816072] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:27:35 melany kernel: [59028.816083] sd 6:0:1:0: [sdb] Unhandled error code
Jun  6 13:27:35 melany kernel: [59028.816085] sd 6:0:1:0: [sdb] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  6 13:27:35 melany kernel: [59028.816087] sd 6:0:1:0: [sdb] CDB: Read(10): 28 00 00 01 26 c1 00 00 80 00
Jun  6 13:27:35 melany kernel: [59028.816092] end_request: I/O error, dev sdb, sector 75457
Jun  6 13:27:46 melany kernel: [59040.532072] INFO: task md5_raid1:26610 blocked for more than 120 seconds.
Jun  6 13:27:46 melany kernel: [59040.532080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun  6 13:27:46 melany kernel: [59040.532088] md5_raid1     D ffff8800018555c0     0 26610      2 0x00000000
Jun  6 13:27:46 melany kernel: [59040.532091]  ffff88006d248e20 0000000000000046 ffff88006e851240 ffffffff810b406c
Jun  6 13:27:46 melany kernel: [59040.532094]  ffff880032ea4fc0 ffff88006e851240 000000000000f8a0 ffff880023459fd8
Jun  6 13:27:46 melany kernel: [59040.532096]  00000000000155c0 00000000000155c0 ffff88006e74b880 ffff88006e74bb78
Jun  6 13:27:46 melany kernel: [59040.532098] Call Trace:
Jun  6 13:27:46 melany kernel: [59040.532104]  [<ffffffff810b406c>] ? mempool_alloc+0x55/0x106
Jun  6 13:27:46 melany kernel: [59040.532113]  [<ffffffffa018c8a5>] ? md_super_wait+0xae/0xc6 [md_mod]
Jun  6 13:27:46 melany kernel: [59040.532116]  [<ffffffff8106479e>] ? autoremove_wake_function+0x0/0x2e
Jun  6 13:27:46 melany kernel: [59040.532120]  [<ffffffffa018d455>] ? md_update_sb+0x285/0x312 [md_mod]
Jun  6 13:27:46 melany kernel: [59040.532124]  [<ffffffffa018ea10>] ? md_check_recovery+0x1e2/0x4b9 [md_mod]
Jun  6 13:27:46 melany kernel: [59040.532127]  [<ffffffffa01a3ee5>] ? raid1d+0x2f/0xde0 [raid1]
Jun  6 13:27:46 melany kernel: [59040.532130]  [<ffffffff81047e65>] ? finish_task_switch+0x3a/0xa7
Jun  6 13:27:46 melany kernel: [59040.532133]  [<ffffffff812ea1a4>] ? thread_return+0x79/0xe0
Jun  6 13:27:46 melany kernel: [59040.532136]  [<ffffffff812ea544>] ? schedule_timeout+0x1e/0xb8
Jun  6 13:27:46 melany kernel: [59040.532139]  [<ffffffffa018c664>] ? md_thread+0xf1/0x10f [md_mod]
Jun  6 13:27:46 melany kernel: [59040.532141]  [<ffffffff8106479e>] ? autoremove_wake_function+0x0/0x2e
Jun  6 13:27:46 melany kernel: [59040.532145]  [<ffffffffa018c573>] ? md_thread+0x0/0x10f [md_mod]
Jun  6 13:27:46 melany kernel: [59040.532147]  [<ffffffff810644d1>] ? kthread+0x79/0x81
Jun  6 13:27:46 melany kernel: [59040.532150]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
Jun  6 13:27:46 melany kernel: [59040.532151]  [<ffffffff81064458>] ? kthread+0x0/0x81
Jun  6 13:27:46 melany kernel: [59040.532153]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20
Jun  6 13:28:06 melany kernel: [59059.816072] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:28:06 melany kernel: [59059.816076] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  6 13:28:37 melany kernel: [59090.816067] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  6 13:28:37 melany kernel: [59090.816071] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5

#-----------------------------------------------------------------------

# mdadm /dev/md5 --fail /dev/sdg1 --remove /dev/sdg1
mdadm: set /dev/sdg1 faulty in /dev/md5

#-----------------------------------------------------------------------

# lsscsi
[6:0:0:0]    disk    ATA      WDC WD5000AAKS-0 05.0  /dev/sda
[6:0:1:0]    disk    ATA      Hitachi HDT72503 V54O  /dev/sdb
[6:0:2:0]    disk    ATA      ST3500418AS      CC38  /dev/sdc
[6:0:3:0]    disk    ATA      SAMSUNG HD501LJ  CR10  /dev/sdd
[6:0:4:0]    disk    ATA      SAMSUNG HD103UJ  1AA0  /dev/sde
[7:0:0:0]    disk    ATA      WDC WD5000AAKS-6 12.0  /dev/sdf
[7:0:1:0]    disk    ATA      Maxtor 6V320F0   VA11  /dev/sdg
[7:0:2:0]    disk    ATA      ST3500630AS      3.AA  /dev/sdh
[7:0:3:0]    disk    ATA      WDC WD10EARS-00Y 80.0  /dev/sdi
[7:0:4:0]    disk    ATA      SAMSUNG HD501LJ  CR10  /dev/sdj
[7:0:5:0]    disk    ATA      SAMSUNG HD103UJ  1AA0  /dev/sdk

#-----------------------------------------------------------------------

ST3500630AS 3.AA
dd if=/dev/zero of=/dev/sdh bs=1M count=10000
dd if=/dev/sdh of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 134.242 s, 78.1 MB/s
10485760000 bytes (10 GB) copied, 133.4 s, 78.6 MB/s

ST3500418AS CC38
dd if=/dev/zero of=/dev/sdc bs=1M count=10000
dd if=/dev/sdc of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 77.5993 s, 135 MB/s
10485760000 bytes (10 GB) copied, 75.5511 s, 139 MB/s

SAMSUNG HD501LJ  CR10
dd if=/dev/zero of=/dev/sdd bs=1M count=10000
dd if=/dev/sdd of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 128.377 s, 81.7 MB/s
10485760000 bytes (10 GB) copied, 127.639 s, 82.2 MB/s

WDC WD5000AAKS-0 05.0
dd if=/dev/zero of=/dev/sda bs=1M count=10000
dd if=/dev/sda of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 75.7749 s, 138 MB/s
10485760000 bytes (10 GB) copied, 75.4374 s, 139 MB/s

Hitachi HDT72503 V54O
dd if=/dev/zero of=/dev/sdb bs=1M count=10000
dd if=/dev/sdb of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 174.459 s, 60.1 MB/s
10485760000 bytes (10 GB) copied, 136.734 s, 76.7 MB/s

SAMSUNG HD103UJ  1AA0
dd if=/dev/zero of=/dev/sde bs=1M count=10000
dd if=/dev/sde of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 106.147 s, 98.8 MB/s
10485760000 bytes (10 GB) copied, 103.246 s, 102 MB/s

WDC WD5000AAKS-6 12.0
dd if=/dev/zero of=/dev/sdf bs=1M count=10000
dd if=/dev/sdf of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 131.26 s, 79.9 MB/s
10485760000 bytes (10 GB) copied, 131.367 s, 79.8 MB/s

SAMSUNG HD501LJ  CR10
dd if=/dev/zero of=/dev/sdj bs=1M count=10000
dd if=/dev/sdj of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 118.223 s, 88.7 MB/s
10485760000 bytes (10 GB) copied, 117.453 s, 89.3 MB/s

Maxtor 6V320F0   VA11
dd if=/dev/zero of=/dev/sdg bs=1M count=10000
dd if=/dev/sdg of=/dev/null bs=1M count=10000
10485760000 bytes (10 GB) copied, 147.384 s, 71.1 MB/s
10485760000 bytes (10 GB) copied, 146.607 s, 71.5 MB/s

#-----------------------------------------------------------------------

[-- Attachment #3: melany-kernel-debug02.txt --]
[-- Type: text/plain, Size: 32063 bytes --]

Jun  5 16:23:15 melany kernel: [14702.229388] md: md1 stopped.
Jun  5 16:23:15 melany kernel: [14702.229395] md: unbind<sdd>
Jun  5 16:23:15 melany mdadm[2415]: DeviceDisappeared event detected on md device /dev/md1
Jun  5 16:23:15 melany kernel: [14702.240064] md: export_rdev(sdd)
Jun  5 16:23:15 melany kernel: [14702.240147] md1: detected capacity change from 500107771904 to 0
Jun  5 16:28:22 melany kernel: [15009.192715]  sdc: unknown partition table
Jun  5 16:28:25 melany kernel: [15012.484279]  sdc: sdc1
Jun  5 16:28:25 melany ata_id[15369]: main: HDIO_GET_IDENTITY failed for '/dev/.tmp-8-32'
Jun  5 16:29:46 melany kernel: [15092.918641] md: bind<sdd>
Jun  5 16:29:46 melany kernel: [15092.918857] md: bind<sdc>
Jun  5 16:29:46 melany kernel: [15093.043908] raid1: md1 is not clean -- starting background reconstruction
Jun  5 16:29:46 melany kernel: [15093.043911] raid1: raid set md1 active with 2 out of 2 mirrors
Jun  5 16:29:46 melany kernel: [15093.043930] md1: detected capacity change from 0 to 500107771904
Jun  5 16:29:46 melany kernel: [15093.044122]  md1: p1
Jun  5 16:33:54 melany mdadm[2415]: RebuildStarted event detected on md device /dev/md1
Jun  5 16:33:54 melany kernel: [15341.219444] md: resync of RAID array md1
Jun  5 16:33:54 melany kernel: [15341.219448] md: minimum _guaranteed_  speed: 1000 KB/sec/disk.
Jun  5 16:33:54 melany kernel: [15341.219449] md: using maximum available idle IO bandwidth (but not more than 200000 KB/sec) for resync.
Jun  5 16:33:54 melany kernel: [15341.219453] md: using 128k window, over a total of 488386496 blocks.
Jun  5 16:34:27 melany kernel: [15373.816071] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:34:27 melany kernel: [15373.816075] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:34:27 melany kernel: [15373.816089] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:34:27 melany kernel: [15373.816092] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:34:58 melany kernel: [15404.816039] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:34:58 melany kernel: [15404.816043] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:34:58 melany kernel: [15404.816055] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:34:58 melany kernel: [15404.816057] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:35:29 melany kernel: [15435.816074] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:35:29 melany kernel: [15435.816078] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:35:29 melany kernel: [15435.816089] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:35:29 melany kernel: [15435.816092] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:36:00 melany kernel: [15466.816077] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:36:00 melany kernel: [15466.816081] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:36:00 melany kernel: [15466.816093] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:36:00 melany kernel: [15466.816095] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:36:13 melany kernel: [15480.544071] INFO: task md1_resync:15403 blocked for more than 120 seconds.
Jun  5 16:36:13 melany kernel: [15480.544080] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun  5 16:36:13 melany kernel: [15480.544088] md1_resync    D 0000000000000000     0 15403      2 0x00000000
Jun  5 16:36:13 melany kernel: [15480.544091]  ffffffff814471f0 0000000000000046 0000000000000000 ffff880037b3c680
Jun  5 16:36:13 melany kernel: [15480.544094]  ffff880033ab8540 ffffffff8116cfe8 000000000000f8a0 ffff88003448ffd8
Jun  5 16:36:13 melany kernel: [15480.544096]  00000000000155c0 00000000000155c0 ffff88003799f100 ffff88003799f3f8
Jun  5 16:36:13 melany kernel: [15480.544099] Call Trace:
Jun  5 16:36:13 melany kernel: [15480.544105]  [<ffffffff8116cfe8>] ? elv_merged_request+0x30/0x39
Jun  5 16:36:13 melany kernel: [15480.544113]  [<ffffffffa01a2424>] ? raise_barrier+0x12c/0x163 [raid1]
Jun  5 16:36:13 melany kernel: [15480.544116]  [<ffffffff8104a0c8>] ? default_wake_function+0x0/0x9
Jun  5 16:36:13 melany kernel: [15480.544119]  [<ffffffff8104a0b8>] ? try_to_wake_up+0x249/0x259
Jun  5 16:36:13 melany kernel: [15480.544122]  [<ffffffffa01a2ed7>] ? sync_request+0x177/0x512 [raid1]
Jun  5 16:36:13 melany kernel: [15480.544126]  [<ffffffffa018b2f1>] ? is_mddev_idle+0xaa/0xfd [md_mod]
Jun  5 16:36:13 melany kernel: [15480.544130]  [<ffffffffa018ba2f>] ? md_do_sync+0x6eb/0xaf6 [md_mod]
Jun  5 16:36:13 melany kernel: [15480.544135]  [<ffffffffa018c664>] ? md_thread+0xf1/0x10f [md_mod]
Jun  5 16:36:13 melany kernel: [15480.544138]  [<ffffffff8103aa9c>] ? __wake_up_common+0x44/0x72
Jun  5 16:36:13 melany kernel: [15480.544141]  [<ffffffffa018c573>] ? md_thread+0x0/0x10f [md_mod]
Jun  5 16:36:13 melany kernel: [15480.544144]  [<ffffffff810644d1>] ? kthread+0x79/0x81
Jun  5 16:36:13 melany kernel: [15480.544147]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
Jun  5 16:36:13 melany kernel: [15480.544149]  [<ffffffff81064458>] ? kthread+0x0/0x81
Jun  5 16:36:13 melany kernel: [15480.544150]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20
Jun  5 16:36:31 melany kernel: [15497.816089] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:36:31 melany kernel: [15497.816093] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:36:31 melany kernel: [15497.816105] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:36:31 melany kernel: [15497.816107] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:37:02 melany kernel: [15528.816082] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:37:02 melany kernel: [15528.816086] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:37:02 melany kernel: [15528.816098] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:37:02 melany kernel: [15528.816101] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:37:02 melany kernel: [15528.816111] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816112] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816115] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 4a 80 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816120] end_request: I/O error, dev sdd, sector 19072
Jun  5 16:37:02 melany kernel: [15528.816149] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816150] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816152] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 49 80 00 01 00 00
Jun  5 16:37:02 melany kernel: [15528.816156] end_request: I/O error, dev sdd, sector 18816
Jun  5 16:37:02 melany kernel: [15528.816165] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816166] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816168] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 49 00 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816172] end_request: I/O error, dev sdd, sector 18688
Jun  5 16:37:02 melany kernel: [15528.816186] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816187] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816189] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 48 00 00 01 00 00
Jun  5 16:37:02 melany kernel: [15528.816193] end_request: I/O error, dev sdd, sector 18432
Jun  5 16:37:02 melany kernel: [15528.816204] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816205] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816207] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 47 80 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816211] end_request: I/O error, dev sdd, sector 18304
Jun  5 16:37:02 melany kernel: [15528.816220] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816221] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816223] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 46 80 00 01 00 00
Jun  5 16:37:02 melany kernel: [15528.816227] end_request: I/O error, dev sdd, sector 18048
Jun  5 16:37:02 melany kernel: [15528.816239] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816240] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816242] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 45 80 00 01 00 00
Jun  5 16:37:02 melany kernel: [15528.816245] end_request: I/O error, dev sdd, sector 17792
Jun  5 16:37:02 melany kernel: [15528.816257] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816258] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816260] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 45 00 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816264] end_request: I/O error, dev sdd, sector 17664
Jun  5 16:37:02 melany kernel: [15528.816272] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816274] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816275] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 44 80 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816279] end_request: I/O error, dev sdd, sector 17536
Jun  5 16:37:02 melany kernel: [15528.816288] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816289] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816291] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 44 00 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816294] end_request: I/O error, dev sdd, sector 17408
Jun  5 16:37:02 melany kernel: [15528.816303] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816304] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816306] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 43 80 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.816309] end_request: I/O error, dev sdd, sector 17280
Jun  5 16:37:02 melany kernel: [15528.816318] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.816320] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.816321] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 42 80 00 01 00 00
Jun  5 16:37:02 melany kernel: [15528.816325] end_request: I/O error, dev sdd, sector 17024
Jun  5 16:37:02 melany kernel: [15528.817530] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.817531] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.817533] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 41 80 00 01 00 00
Jun  5 16:37:02 melany kernel: [15528.817537] end_request: I/O error, dev sdd, sector 16768
Jun  5 16:37:02 melany kernel: [15528.817549] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.817550] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.817552] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 41 00 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.817556] end_request: I/O error, dev sdd, sector 16640
Jun  5 16:37:02 melany kernel: [15528.817565] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.817566] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.817568] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 40 80 00 00 80 00
Jun  5 16:37:02 melany kernel: [15528.817572] end_request: I/O error, dev sdd, sector 16512
Jun  5 16:37:02 melany kernel: [15528.817581] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.817583] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.817584] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 3e 00 00 02 80 00
Jun  5 16:37:02 melany kernel: [15528.817588] end_request: I/O error, dev sdd, sector 15872
Jun  5 16:37:02 melany kernel: [15528.817614] sd 0:0:0:0: [sda] Unhandled error code
Jun  5 16:37:02 melany kernel: [15528.817615] sd 0:0:0:0: [sda] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:37:02 melany kernel: [15528.817617] sd 0:0:0:0: [sda] CDB: Read(10): 28 00 00 00 00 00 00 00 20 00
Jun  5 16:37:02 melany kernel: [15528.817621] end_request: I/O error, dev sda, sector 0
Jun  5 16:37:02 melany kernel: [15528.817627] Buffer I/O error on device sda, logical block 0
Jun  5 16:37:02 melany kernel: [15528.817634] Buffer I/O error on device sda, logical block 1
Jun  5 16:37:02 melany kernel: [15528.817639] Buffer I/O error on device sda, logical block 2
Jun  5 16:37:02 melany kernel: [15528.817645] Buffer I/O error on device sda, logical block 3
Jun  5 16:37:33 melany kernel: [15559.816070] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:37:33 melany kernel: [15559.816074] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:38:04 melany kernel: [15590.816073] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:38:04 melany kernel: [15590.816077] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:38:35 melany kernel: [15621.816074] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:38:35 melany kernel: [15621.816078] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:39:07 melany kernel: [15653.816067] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:39:07 melany kernel: [15653.816071] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:39:38 melany kernel: [15684.816072] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:39:38 melany kernel: [15684.816076] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:40:09 melany kernel: [15715.816072] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:40:09 melany kernel: [15715.816076] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:40:09 melany kernel: [15715.816089] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:40:09 melany kernel: [15715.816090] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:40:09 melany kernel: [15715.816092] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 4a 80 00 00 08 00
Jun  5 16:40:09 melany kernel: [15715.816098] end_request: I/O error, dev sdd, sector 19072
Jun  5 16:40:09 melany kernel: [15715.816115] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:40:09 melany kernel: [15715.816117] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:40:09 melany kernel: [15715.816118] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 00 00 4b 00 00 02 80 00
Jun  5 16:40:09 melany kernel: [15715.816123] end_request: I/O error, dev sdd, sector 19200
Jun  5 16:40:40 melany kernel: [15746.816066] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:40:40 melany kernel: [15746.816070] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:40:40 melany kernel: [15746.816077] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:40:40 melany kernel: [15746.816080] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:41:11 melany kernel: [15777.817848] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:41:11 melany kernel: [15777.817851] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:41:11 melany kernel: [15777.817857] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:41:11 melany kernel: [15777.817859] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:41:42 melany kernel: [15808.816072] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:41:42 melany kernel: [15808.816076] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:41:42 melany kernel: [15808.816083] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:41:42 melany kernel: [15808.816085] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:41:42 melany kernel: [15808.816098] sd 0:0:3:0: [sdd] Unhandled error code
Jun  5 16:41:42 melany kernel: [15808.816099] sd 0:0:3:0: [sdd] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:41:42 melany kernel: [15808.816101] sd 0:0:3:0: [sdd] CDB: Read(10): 28 00 3a 38 5f 80 00 00 08 00
Jun  5 16:41:42 melany kernel: [15808.816106] end_request: I/O error, dev sdd, sector 976772992
Jun  5 16:42:13 melany kernel: [15839.816079] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:42:13 melany kernel: [15839.816083] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:42:13 melany kernel: [15840.544069] INFO: task md1_raid1:15386 blocked for more than 120 seconds.
Jun  5 16:42:13 melany kernel: [15840.544078] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun  5 16:42:13 melany kernel: [15840.544086] md1_raid1     D ffff8800018555c0     0 15386      2 0x00000000
Jun  5 16:42:13 melany kernel: [15840.544089]  ffff88003799a350 0000000000000046 ffff880037a9b928 ffff88006d33fd00
Jun  5 16:42:13 melany kernel: [15840.544092]  0000000000000000 ffff880037b3b4e0 000000000000f8a0 ffff88001e0a9fd8
Jun  5 16:42:13 melany kernel: [15840.544094]  00000000000155c0 00000000000155c0 ffff88003799cdb0 ffff88003799d0a8
Jun  5 16:42:13 melany kernel: [15840.544097] Call Trace:
Jun  5 16:42:13 melany kernel: [15840.544102]  [<ffffffff8116dd4e>] ? elv_insert+0x190/0x260
Jun  5 16:42:13 melany kernel: [15840.544106]  [<ffffffff81175fad>] ? __generic_unplug_device+0x12/0x2c
Jun  5 16:42:13 melany kernel: [15840.544108]  [<ffffffff81176ac1>] ? __make_request+0x3d2/0x428
Jun  5 16:42:13 melany kernel: [15840.544111]  [<ffffffff812ea544>] ? schedule_timeout+0x1e/0xb8
Jun  5 16:42:13 melany kernel: [15840.544114]  [<ffffffff811751fd>] ? generic_make_request+0x28c/0x2d7
Jun  5 16:42:13 melany kernel: [15840.544116]  [<ffffffff812ea419>] ? wait_for_common+0xde/0x14f
Jun  5 16:42:13 melany kernel: [15840.544120]  [<ffffffff8104a0c8>] ? default_wake_function+0x0/0x9
Jun  5 16:42:13 melany kernel: [15840.544122]  [<ffffffff8117530e>] ? submit_bio+0xc6/0xcd
Jun  5 16:42:13 melany kernel: [15840.544130]  [<ffffffffa018c93e>] ? sync_page_io+0x81/0xa6 [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544134]  [<ffffffffa01a43f7>] ? raid1d+0x541/0xde0 [raid1]
Jun  5 16:42:13 melany kernel: [15840.544137]  [<ffffffff81047e65>] ? finish_task_switch+0x3a/0xa7
Jun  5 16:42:13 melany kernel: [15840.544139]  [<ffffffff812ea1a4>] ? thread_return+0x79/0xe0
Jun  5 16:42:13 melany kernel: [15840.544141]  [<ffffffff8104021a>] ? check_preempt_wakeup+0x1dd/0x268
Jun  5 16:42:13 melany kernel: [15840.544144]  [<ffffffff812ea544>] ? schedule_timeout+0x1e/0xb8
Jun  5 16:42:13 melany kernel: [15840.544148]  [<ffffffffa018c664>] ? md_thread+0xf1/0x10f [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544150]  [<ffffffff8106479e>] ? autoremove_wake_function+0x0/0x2e
Jun  5 16:42:13 melany kernel: [15840.544154]  [<ffffffffa018c573>] ? md_thread+0x0/0x10f [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544156]  [<ffffffff810644d1>] ? kthread+0x79/0x81
Jun  5 16:42:13 melany kernel: [15840.544159]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
Jun  5 16:42:13 melany kernel: [15840.544161]  [<ffffffff81064458>] ? kthread+0x0/0x81
Jun  5 16:42:13 melany kernel: [15840.544162]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20
Jun  5 16:42:13 melany kernel: [15840.544164] INFO: task md1_resync:15403 blocked for more than 120 seconds.
Jun  5 16:42:13 melany kernel: [15840.544170] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun  5 16:42:13 melany kernel: [15840.544177] md1_resync    D 0000000000000000     0 15403      2 0x00000000
Jun  5 16:42:13 melany kernel: [15840.544180]  ffffffff814471f0 0000000000000046 0000000000000000 ffff880037b3c680
Jun  5 16:42:13 melany kernel: [15840.544182]  ffff880033ab8540 ffffffff8116cfe8 000000000000f8a0 ffff88003448ffd8
Jun  5 16:42:13 melany kernel: [15840.544184]  00000000000155c0 00000000000155c0 ffff88003799f100 ffff88003799f3f8
Jun  5 16:42:13 melany kernel: [15840.544187] Call Trace:
Jun  5 16:42:13 melany kernel: [15840.544189]  [<ffffffff8116cfe8>] ? elv_merged_request+0x30/0x39
Jun  5 16:42:13 melany kernel: [15840.544192]  [<ffffffffa01a2424>] ? raise_barrier+0x12c/0x163 [raid1]
Jun  5 16:42:13 melany kernel: [15840.544195]  [<ffffffff8104a0c8>] ? default_wake_function+0x0/0x9
Jun  5 16:42:13 melany kernel: [15840.544197]  [<ffffffff8104a0b8>] ? try_to_wake_up+0x249/0x259
Jun  5 16:42:13 melany kernel: [15840.544200]  [<ffffffffa01a2ed7>] ? sync_request+0x177/0x512 [raid1]
Jun  5 16:42:13 melany kernel: [15840.544204]  [<ffffffffa018b2f1>] ? is_mddev_idle+0xaa/0xfd [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544207]  [<ffffffffa018ba2f>] ? md_do_sync+0x6eb/0xaf6 [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544212]  [<ffffffffa018c664>] ? md_thread+0xf1/0x10f [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544214]  [<ffffffff8103aa9c>] ? __wake_up_common+0x44/0x72
Jun  5 16:42:13 melany kernel: [15840.544218]  [<ffffffffa018c573>] ? md_thread+0x0/0x10f [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544220]  [<ffffffff810644d1>] ? kthread+0x79/0x81
Jun  5 16:42:13 melany kernel: [15840.544222]  [<ffffffff81011baa>] ? child_rip+0xa/0x20
Jun  5 16:42:13 melany kernel: [15840.544224]  [<ffffffff81064458>] ? kthread+0x0/0x81
Jun  5 16:42:13 melany kernel: [15840.544225]  [<ffffffff81011ba0>] ? child_rip+0x0/0x20
Jun  5 16:42:13 melany kernel: [15840.544227] INFO: task pvremove:15408 blocked for more than 120 seconds.
Jun  5 16:42:13 melany kernel: [15840.544232] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
Jun  5 16:42:13 melany kernel: [15840.544239] pvremove      D 0000000000000000     0 15408   9453 0x00020000
Jun  5 16:42:13 melany kernel: [15840.544242]  ffff88006fb73880 0000000000000086 0000000000000000 ffffffff8104a0b8
Jun  5 16:42:13 melany kernel: [15840.544244]  ffff88006f803080 0000000000000092 000000000000f8a0 ffff88002b10bfd8
Jun  5 16:42:13 melany kernel: [15840.544246]  00000000000155c0 00000000000155c0 ffff88003799a350 ffff88003799a648
Jun  5 16:42:13 melany kernel: [15840.544248] Call Trace:
Jun  5 16:42:13 melany kernel: [15840.544251]  [<ffffffff8104a0b8>] ? try_to_wake_up+0x249/0x259
Jun  5 16:42:13 melany kernel: [15840.544253]  [<ffffffff8103aa9c>] ? __wake_up_common+0x44/0x72
Jun  5 16:42:13 melany kernel: [15840.544256]  [<ffffffffa01a24f5>] ? wait_barrier+0x9a/0xd7 [raid1]
Jun  5 16:42:13 melany kernel: [15840.544258]  [<ffffffff8104a0c8>] ? default_wake_function+0x0/0x9
Jun  5 16:42:13 melany kernel: [15840.544260]  [<ffffffff810646bf>] ? bit_waitqueue+0x10/0xa0
Jun  5 16:42:13 melany kernel: [15840.544263]  [<ffffffffa01a505a>] ? make_request+0x70/0x70e [raid1]
Jun  5 16:42:13 melany kernel: [15840.544265]  [<ffffffff810e2491>] ? get_partial_node+0x15/0x79
Jun  5 16:42:13 melany kernel: [15840.544268]  [<ffffffff810e4785>] ? __slab_alloc+0xd3/0x39b
Jun  5 16:42:13 melany kernel: [15840.544270]  [<ffffffff810b406c>] ? mempool_alloc+0x55/0x106
Jun  5 16:42:13 melany kernel: [15840.544274]  [<ffffffffa018a9f2>] ? md_make_request+0xb6/0xf1 [md_mod]
Jun  5 16:42:13 melany kernel: [15840.544277]  [<ffffffff811751fd>] ? generic_make_request+0x28c/0x2d7
Jun  5 16:42:13 melany kernel: [15840.544279]  [<ffffffff8117530e>] ? submit_bio+0xc6/0xcd
Jun  5 16:42:13 melany kernel: [15840.544282]  [<ffffffff81110810>] ? dio_bio_submit+0x52/0x66
Jun  5 16:42:13 melany kernel: [15840.544285]  [<ffffffff81111336>] ? __blockdev_direct_IO+0x851/0xa60
Jun  5 16:42:13 melany kernel: [15840.544288]  [<ffffffff81186133>] ? kobject_get+0x12/0x17
Jun  5 16:42:13 melany kernel: [15840.544290]  [<ffffffff8110f8af>] ? blkdev_direct_IO+0x45/0x4a
Jun  5 16:42:13 melany kernel: [15840.544293]  [<ffffffff8110ec1b>] ? blkdev_get_blocks+0x0/0x8b
Jun  5 16:42:13 melany kernel: [15840.544296]  [<ffffffff810b398f>] ? generic_file_aio_read+0xf6/0x527
Jun  5 16:42:13 melany kernel: [15840.544298]  [<ffffffff81182e74>] ? compat_blkdev_ioctl+0x458/0x162c
Jun  5 16:42:13 melany kernel: [15840.544300]  [<ffffffff8110ff7e>] ? __blkdev_get+0x298/0x344
Jun  5 16:42:13 melany kernel: [15840.544303]  [<ffffffff81110031>] ? blkdev_open+0x0/0x96
Jun  5 16:42:13 melany kernel: [15840.544305]  [<ffffffff81110098>] ? blkdev_open+0x67/0x96
Jun  5 16:42:13 melany kernel: [15840.544307]  [<ffffffff810ec649>] ? do_sync_read+0xce/0x113
Jun  5 16:42:13 melany kernel: [15840.544310]  [<ffffffff8106479e>] ? autoremove_wake_function+0x0/0x2e
Jun  5 16:42:13 melany kernel: [15840.544312]  [<ffffffff810efb03>] ? vfs_getattr+0x56/0x61
Jun  5 16:42:13 melany kernel: [15840.544314]  [<ffffffff810ed092>] ? vfs_read+0xa6/0xff
Jun  5 16:42:13 melany kernel: [15840.544317]  [<ffffffff810ed1a7>] ? sys_read+0x45/0x6e
Jun  5 16:42:13 melany kernel: [15840.544319]  [<ffffffff810374ff>] ? cstar_dispatch+0x7/0x2e
Jun  5 16:42:44 melany kernel: [15870.804393] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:42:44 melany kernel: [15870.804397] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:43:15 melany mdadm[2415]: Rebuild99 event detected on md device /dev/md1
Jun  5 16:43:15 melany kernel: [15901.804466] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:43:15 melany kernel: [15901.804470] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
Jun  5 16:43:15 melany kernel: [15901.804477] sd 0:0:2:0: [sdc] Unhandled error code
Jun  5 16:43:15 melany kernel: [15901.804478] sd 0:0:2:0: [sdc] Result: hostbyte=DID_OK driverbyte=DRIVER_TIMEOUT
Jun  5 16:43:15 melany kernel: [15901.804481] sd 0:0:2:0: [sdc] CDB: Read(10): 28 00 00 00 4a 80 00 00 08 00
Jun  5 16:43:15 melany kernel: [15901.804486] end_request: I/O error, dev sdc, sector 19072
Jun  5 16:43:15 melany kernel: [15901.804506] raid1: Disk failure on sdd, disabling device.
Jun  5 16:43:15 melany kernel: [15901.804507] raid1: Operation continuing on 1 devices.
Jun  5 16:43:15 melany kernel: [15901.804518] raid1: sdd: unrecoverable I/O read error for block 19072
Jun  5 16:43:15 melany kernel: [15901.804556] md: md1: resync done.
Jun  5 16:43:15 melany mdadm[2415]: Fail event detected on md device /dev/md1, component device /dev/sdd
Jun  5 16:43:46 melany kernel: [15932.804362] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
Jun  5 16:43:46 melany kernel: [15932.804367] /build/buildd-linux-2.6_2.6.32-9-i386-0BnCTJ/linux-2.6-2.6.32/debian/build/source_i386_none/drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5

Re: Still havind major MVSAS issues.

[Konstantinos Skarlatos]

[-- Attachment #1: Type: text/plain, Size: 2890 bytes --]

  Hello all,

I have also got severe problems with mvsas, but have managed to at least 
make it usable. My config is a AOC-SASLP-MV8 with an HP SAS expander, 
with WD and Seagate sata disks connected to the expander using a norco 
4020 case that does not have any on board expanders. Kernel version is 
2.6.33 with latest srinivas patch. I also dont raid, the 13 disks have 
their own filesystem.

My experiences are:
The only filesystem that works is JFS. XFS and btrfs crash the 
controller making all the disks unreadable and that can only be solved 
by rebooting. (mkfs succeeds, crashes happen only after mounting or when 
fscking)
JFS works ok as long as i access it via samba or make copies and moves 
with cp and mv. If I try file operations with Thunar file manager, the 
controller crashes. Smartctl has not caused any problems so far, and 
works ok.

Attached are some kernel logs captured from those crashes.

Kind regards,
Konstantinos Skarlatos





On 6/6/2010 3:13 μμ, Jelle de Jong wrote:
> Dear Srini,
>
> I spent a few weeks catering information and did some intensive
> testing the last few days.
>
> Srinivas Naga Venkatasatya Pasagadugula wrote, on 06-05-10 08:01:
>> 1. Is this is the problem with only WD SATA drives? (I don't have WD SATA drives to reproduce this issue.)
>> 2. Whether the problem is with direct attached SATA drives or drives connected in expanders also?
>> 3. Could you please provide me the "dmesg" log or "/var/log/messages" log.
>> 4. How much the capacity of SATA drives connected to controller?
>> 5. Your HBA is having 6440 chipset?
> With a few tricks I managed to boot my OS from the mvsas controller. I
> got eleven different sata disks attached from a 4-port mini-sas
> backplate without expander to the two Marvel 88SE63xx/64xx mvsas
> controllers in my system.
>
> I managed to add five mdadm raid1 arrays without added the actual
> active sync device (so one disk for each array) I build my lvm systems
> on top of this and did a lot of file transfers for testing.
>
> This worked stable enough, there are HDIO_GET_IDENTITY errors during
> boot and operation, but the hard disks seem to be working.
>
> So to debug if issues are related to some brand of harddisk, I started
> to add WDC, Hitachi, SAMSUNG, Maxtor and Seagate disks to the
> respected raid1 array, the disks are of different sizes (320GB, 500GB,
> 1TB.)
>
> The sync starts and will fail directly or a while later on failures in
> the mvsas driver. I attached an failure example as attachment.
>
> The failures are of grave severity, I have lost complete lvm2 volumes
> and raid arrays during testing.
>
> Do you also have a SuperMicro AOC-SASLP-MV8 controller for testing?
>
> I would love to use the controllers for production, but they are
> currently unstable. I hope this information helps to solve the mvsas
> issues?
>
> With kind regards,
>
> Jelle de Jong


[-- Attachment #2: mvsas errors.txt --]
[-- Type: text/plain, Size: 8079 bytes --]

------------[ cut here ]------------
WARNING: at drivers/ata/libata-core.c:5186 ata_qc_issue+0x31f/0x330 [libata]()
Hardware name:
Modules linked in: ipv6 hwmon_vid jfs cpufreq_powersave fan cpufreq_ondemand edac_core powernow_k8 firewire_ohci psmouse firewire_core freq_table serio_raw pcspkr k8temp thermal crc_itu_t evdev edac_mce_amd skge processor button i2c_nforce2 sg forcedeth i2c_core fuse rtc_cmos rtc_core rtc_lib ext2 mbcache dm_crypt dm_mod ses enclosure sd_mod usb_storage ohci_hcd mvsas libsas sata_sil ehci_hcd scsi_transport_sas sata_nv usbcore pata_amd sata_via ata_generic pata_via pata_acpi libata scsi_mod
Pid: 3308, comm: smartctl Not tainted 2.6.33-ARCH #1
Call Trace:
 [<ffffffff810528c8>] warn_slowpath_common+0x78/0xb0
 [<ffffffff8105290f>] warn_slowpath_null+0xf/0x20
 [<ffffffffa002c14f>] ata_qc_issue+0x31f/0x330 [libata]
 [<ffffffffa0006fae>] ? scsi_init_sgtable+0x4e/0x90 [scsi_mod]
 [<ffffffffa0033cd0>] ? ata_scsi_pass_thru+0x0/0x2f0 [libata]
 [<ffffffffa00310c6>] ata_scsi_translate+0xa6/0x180 [libata]
 [<ffffffffa0000b10>] ? scsi_done+0x0/0x20 [scsi_mod]
 [<ffffffffa0000b10>] ? scsi_done+0x0/0x20 [scsi_mod]
 [<ffffffffa0034369>] ata_sas_queuecmd+0x139/0x2b0 [libata]
 [<ffffffffa00f3098>] sas_queuecommand+0x98/0x300 [libsas]
 [<ffffffffa0000c25>] scsi_dispatch_cmd+0xf5/0x230 [scsi_mod]
 [<ffffffffa0006ba2>] scsi_request_fn+0x322/0x3e0 [scsi_mod]
 [<ffffffff811b72bd>] __generic_unplug_device+0x2d/0x40
 [<ffffffff811bcbf8>] blk_execute_rq_nowait+0x68/0xb0
 [<ffffffff811bccc1>] blk_execute_rq+0x81/0xf0
 [<ffffffff811b4d0b>] ? blk_rq_bio_prep+0x2b/0xd0
 [<ffffffff811bc866>] ? blk_rq_map_kern+0xd6/0x150
 [<ffffffffa0007ee7>] scsi_execute+0xf7/0x160 [scsi_mod]
 [<ffffffffa0033167>] ata_cmd_ioctl+0x177/0x320 [libata]
 [<ffffffffa0033467>] ata_sas_scsi_ioctl+0x157/0x2b0 [libata]
 [<ffffffffa00f25f7>] sas_ioctl+0x47/0x50 [libsas]
 [<ffffffffa0002225>] scsi_ioctl+0xd5/0x390 [scsi_mod]
 [<ffffffffa0134d3e>] sd_ioctl+0xce/0xe0 [sd_mod]
 [<ffffffff811be35f>] __blkdev_driver_ioctl+0x8f/0xb0
 [<ffffffff811be82e>] blkdev_ioctl+0x22e/0x820
 [<ffffffff8114fdf7>] block_ioctl+0x37/0x40
 [<ffffffff81131ac8>] vfs_ioctl+0x38/0xd0
 [<ffffffff81131c70>] do_vfs_ioctl+0x80/0x560
 [<ffffffff811cff46>] ? __up_read+0xa6/0xd0
 [<ffffffff81077c29>] ? up_read+0x9/0x10
 [<ffffffff811321d1>] sys_ioctl+0x81/0xa0
 [<ffffffff8100a002>] system_call_fastpath+0x16/0x1b
---[ end trace 115ad6bf347654e7 ]---
 sdm: sdm1
 sdn: sdn1
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
INFO: task smbd:3348 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd          D ffff88000180f948     0  3348   2949 0x00000000
 ffff88002dfe38b8 0000000000000086 0000000000000000 ffffffffa000692b
 000000013d124bd0 0000000000011250 ffff88003caff938 ffff88003cf36690
 000000010061522b ffff88002dfe3fd8 ffff88002dfe2000 ffff88002dfe2000
Call Trace:
 [<ffffffffa000692b>] ? scsi_request_fn+0xab/0x3e0 [scsi_mod]
 [<ffffffff810dbd00>] ? sync_page+0x0/0x50
 [<ffffffff8135970e>] io_schedule+0x6e/0xb0
 [<ffffffff810dbd3d>] sync_page+0x3d/0x50
 [<ffffffff81359d32>] __wait_on_bit_lock+0x52/0xb0
 [<ffffffff810dbce2>] __lock_page+0x62/0x70
 [<ffffffff81073090>] ? wake_bit_function+0x0/0x40
 [<ffffffff810dc3d9>] do_read_cache_page+0x159/0x180
 [<ffffffffa02e11e0>] ? metapage_readpage+0x0/0x180 [jfs]
 [<ffffffff810dc434>] read_cache_page_async+0x14/0x20
 [<ffffffff810dc449>] read_cache_page+0x9/0x20
 [<ffffffffa02e1d85>] __get_metapage+0x95/0x5a0 [jfs]
 [<ffffffffa02d4fb5>] diRead+0x155/0x200 [jfs]
 [<ffffffffa02c8d08>] jfs_iget+0x38/0x160 [jfs]
 [<ffffffffa02cb461>] jfs_lookup+0x71/0x140 [jfs]
 [<ffffffff81110000>] ? calculate_sizes+0x220/0x4a0
 [<ffffffff81359d53>] ? __wait_on_bit_lock+0x73/0xb0
 [<ffffffff8135a45d>] ? __mutex_lock_slowpath+0x26d/0x370
 [<ffffffff8112bafb>] do_lookup+0x1db/0x270
 [<ffffffff8112e127>] link_path_walk+0x6b7/0xf10
 [<ffffffff810e1b28>] ? free_hot_page+0x28/0x90
 [<ffffffff8112eb1c>] path_walk+0x5c/0xc0
 [<ffffffff8112ecb3>] do_path_lookup+0x53/0xa0
 [<ffffffff8112f8f2>] user_path_at+0x52/0xa0
 [<ffffffff8115f01e>] ? locks_free_lock+0x3e/0x60
 [<ffffffff8115fb74>] ? fcntl_setlk+0x64/0x350
 [<ffffffff811260a7>] vfs_fstatat+0x37/0x70
 [<ffffffff81126206>] vfs_stat+0x16/0x20
 [<ffffffff8112622f>] sys_newstat+0x1f/0x50
 [<ffffffff81131370>] ? sys_fcntl+0x160/0x5d0
 [<ffffffff8100a002>] system_call_fastpath+0x16/0x1b
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 30 b4 cb ff 00 04 00 00
end_request: I/O error, dev sdc, sector 817155071
sd 10:0:12:0: [sdm] Unhandled error code
sd 10:0:12:0: [sdm] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:12:0: [sdm] CDB: cdb[0]=0x2a: 2a 00 6e 04 66 c8 00 04 00 00
end_request: I/O error, dev sdm, sector 1845782216
sd 10:0:2:0: [sdd] Unhandled error code
sd 10:0:2:0: [sdd] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:2:0: [sdd] CDB: cdb[0]=0x2a: 2a 00 ae 97 38 ff 00 00 08 00
end_request: I/O error, dev sdd, sector 2929146111
sd 10:0:2:0: [sdd] Unhandled error code
sd 10:0:2:0: [sdd] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:2:0: [sdd] CDB: cdb[0]=0x28: 28 00 68 6a 4d 57 00 00 40 00
end_request: I/O error, dev sdd, sector 1751797079
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5

Re: Still havind major MVSAS issues.

[Konstantinos Skarlatos]

[-- Attachment #1: Type: text/plain, Size: 2113 bytes --]

  Here is one more log with crash information. All of my disks are 
1.5TB, Seagates and WD. The expander did not affect the crashes, i had 
the same issues with direct attached drives.


On 6/6/2010 3:13 μμ, Jelle de Jong wrote:
> Dear Srini,
>
> I spent a few weeks catering information and did some intensive
> testing the last few days.
>
> Srinivas Naga Venkatasatya Pasagadugula wrote, on 06-05-10 08:01:
>> 1. Is this is the problem with only WD SATA drives? (I don't have WD SATA drives to reproduce this issue.)
>> 2. Whether the problem is with direct attached SATA drives or drives connected in expanders also?
>> 3. Could you please provide me the "dmesg" log or "/var/log/messages" log.
>> 4. How much the capacity of SATA drives connected to controller?
>> 5. Your HBA is having 6440 chipset?
> With a few tricks I managed to boot my OS from the mvsas controller. I
> got eleven different sata disks attached from a 4-port mini-sas
> backplate without expander to the two Marvel 88SE63xx/64xx mvsas
> controllers in my system.
>
> I managed to add five mdadm raid1 arrays without added the actual
> active sync device (so one disk for each array) I build my lvm systems
> on top of this and did a lot of file transfers for testing.
>
> This worked stable enough, there are HDIO_GET_IDENTITY errors during
> boot and operation, but the hard disks seem to be working.
>
> So to debug if issues are related to some brand of harddisk, I started
> to add WDC, Hitachi, SAMSUNG, Maxtor and Seagate disks to the
> respected raid1 array, the disks are of different sizes (320GB, 500GB,
> 1TB.)
>
> The sync starts and will fail directly or a while later on failures in
> the mvsas driver. I attached an failure example as attachment.
>
> The failures are of grave severity, I have lost complete lvm2 volumes
> and raid arrays during testing.
>
> Do you also have a SuperMicro AOC-SASLP-MV8 controller for testing?
>
> I would love to use the controllers for production, but they are
> currently unstable. I hope this information helps to solve the mvsas
> issues?
>
> With kind regards,
>
> Jelle de Jong


[-- Attachment #2: mvsas errors2.txt --]
[-- Type: text/plain, Size: 22148 bytes --]


drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 1c b2 86 6f 00 03 00 00
end_request: I/O error, dev sdc, sector 481461871
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 03 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442630912
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 ff 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442629888
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 fb 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442628864
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 f7 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442627840
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 f3 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442626816
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 61 ef 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442625792
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 00 00 c8 c8 00 00 08 00
end_request: I/O error, dev sdf, sector 51400
metapage_write_end_io: I/O error
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 00 00 db 90 00 00 58 00
end_request: I/O error, dev sdf, sector 56208
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
metapage_write_end_io: I/O error
INFO: task filezilla:18572 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla     D ffffffff810dc2d0     0 18572   2063 0x00000000
 ffff8800175b3768 0000000000000082 00000000000530d7 0000000000000008
 0000000000800020 ffff88003ca52cb0 ffff8800175b3798 ffffffff811b6fa4
 ffff8800175b3740 ffff8800175b3fd8 ffff8800175b2000 ffff8800175b2000
Call Trace:
 [<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
 [<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff810dc2d0>] ? sync_page+0x0/0x50
 [<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
 [<ffffffff810dc30d>] sync_page+0x3d/0x50
 [<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
 [<ffffffff810dc2b2>] __lock_page+0x62/0x70
 [<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
 [<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
 [<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
 [<ffffffff810dca04>] read_cache_page_async+0x14/0x20
 [<ffffffff810dca19>] read_cache_page+0x9/0x20
 [<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
 [<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
 [<ffffffffa02f87eb>] ? xtLookup+0x18b/0x1a0 [jfs]
 [<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
 [<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
 [<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
 [<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
 [<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
 [<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
 [<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
 [<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
 [<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
 [<ffffffff810581f2>] ? current_fs_time+0x22/0x30
 [<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
 [<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
 [<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
 [<ffffffff81122002>] do_sync_write+0xd2/0x110
 [<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
 [<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
 [<ffffffff81122b48>] vfs_write+0xb8/0x1a0
 [<ffffffff81122d0c>] sys_write+0x4c/0x80
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task filezilla:18574 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla     D ffffffff810dc2d0     0 18574   2063 0x00000000
 ffff88003c967768 0000000000000082 0000000000045ca7 0000000000000008
 0000000000800020 ffff88003ca52cb0 ffff88003c967798 ffffffff811b6fa4
 ffff88003c967740 ffff88003c967fd8 ffff88003c966000 ffff88003c966000
Call Trace:
 [<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
 [<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff810dc2d0>] ? sync_page+0x0/0x50
 [<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
 [<ffffffff810dc30d>] sync_page+0x3d/0x50
 [<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
 [<ffffffff810dc2b2>] __lock_page+0x62/0x70
 [<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
 [<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
 [<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
 [<ffffffff810dca04>] read_cache_page_async+0x14/0x20
 [<ffffffff810dca19>] read_cache_page+0x9/0x20
 [<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
 [<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
 [<ffffffffa02f86ed>] ? xtLookup+0x8d/0x1a0 [jfs]
 [<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
 [<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
 [<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
 [<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
 [<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
 [<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
 [<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
 [<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
 [<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
 [<ffffffff810581f2>] ? current_fs_time+0x22/0x30
 [<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
 [<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
 [<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
 [<ffffffff81122002>] do_sync_write+0xd2/0x110
 [<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
 [<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
 [<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
 [<ffffffff81122b48>] vfs_write+0xb8/0x1a0
 [<ffffffff81122d0c>] sys_write+0x4c/0x80
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task smbd:18658 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd          D ffff88000190f948     0 18658  16782 0x00000000
 ffff8800328cd608 0000000000000086 0000000000000000 ffffffff812772b2
 ffff8800328cd5a8 ffffffffa000694a ffff880000000000 ffff88003c921830
 00000001061958d0 ffff8800328cdfd8 ffff8800328cc000 ffff8800328cc000
Call Trace:
 [<ffffffff812772b2>] ? put_device+0x12/0x20
 [<ffffffffa000694a>] ? scsi_request_fn+0xaa/0x4d0 [scsi_mod]
 [<ffffffff810dc2d0>] ? sync_page+0x0/0x50
 [<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
 [<ffffffff810dc30d>] sync_page+0x3d/0x50
 [<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
 [<ffffffff810dc2b2>] __lock_page+0x62/0x70
 [<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
 [<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
 [<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
 [<ffffffff810dca04>] read_cache_page_async+0x14/0x20
 [<ffffffff810dca19>] read_cache_page+0x9/0x20
 [<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
 [<ffffffffa0309851>] dtSearch+0x781/0xb00 [jfs]
 [<ffffffffa02f7503>] jfs_lookup+0x103/0x150 [jfs]
 [<ffffffff81137648>] ? d_alloc+0x158/0x1c0
 [<ffffffff8112c64d>] __lookup_hash+0xed/0x150
 [<ffffffff8112cb95>] lookup_one_len+0x75/0xb0
 [<ffffffffa03a114d>] vfsub_lookup_one_len+0x1d/0x50 [aufs]
 [<ffffffffa03a7ae2>] au_lkup_one+0x32/0xf0 [aufs]
 [<ffffffff810f9619>] ? handle_mm_fault+0x6d9/0xa30
 [<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
 [<ffffffff8135d0c5>] ? __down_write_nested+0xd5/0xe0
 [<ffffffffa03a526f>] au_wh_test+0x1f/0xd0 [aufs]
 [<ffffffff8135bbc1>] ? mutex_lock+0x11/0x30
 [<ffffffffa03a8286>] au_lkup_dentry+0x336/0x530 [aufs]
 [<ffffffffa03af3cf>] aufs_lookup+0xef/0x230 [aufs]
 [<ffffffff8112c4cb>] do_lookup+0x1db/0x270
 [<ffffffff8112eb59>] link_path_walk+0x679/0xe50
 [<ffffffff8112f502>] path_walk+0x62/0xe0
 [<ffffffff8112f6c3>] do_path_lookup+0x53/0xa0
 [<ffffffff8112f735>] kern_path+0x25/0x50
 [<ffffffff811cb30b>] ? cpumask_any_but+0x2b/0x40
 [<ffffffff810fdbb0>] ? unmap_region+0x170/0x1a0
 [<ffffffff81151629>] lookup_bdev+0x39/0xb0
 [<ffffffff8112e26e>] ? getname+0x1ce/0x230
 [<ffffffff81174b03>] sys_quotactl+0xb3/0x3d0
 [<ffffffff81077df9>] ? up_write+0x9/0x10
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:6:0: [sdh] Unhandled error code
sd 10:0:6:0: [sdh] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:6:0: [sdh] CDB: cdb[0]=0x28: 28 00 27 63 5d 6f 00 00 08 00
end_request: I/O error, dev sdh, sector 660823407
metapage_read_end_io: I/O error
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
INFO: task filezilla:18572 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla     D ffffffff810dc2d0     0 18572   2063 0x00000000
 ffff8800175b3768 0000000000000082 00000000000530d7 0000000000000008
 0000000000800020 ffff88003ca52cb0 ffff8800175b3798 ffffffff811b6fa4
 ffff8800175b3740 ffff8800175b3fd8 ffff8800175b2000 ffff8800175b2000
Call Trace:
 [<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
 [<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff810dc2d0>] ? sync_page+0x0/0x50
 [<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
 [<ffffffff810dc30d>] sync_page+0x3d/0x50
 [<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
 [<ffffffff810dc2b2>] __lock_page+0x62/0x70
 [<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
 [<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
 [<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
 [<ffffffff810dca04>] read_cache_page_async+0x14/0x20
 [<ffffffff810dca19>] read_cache_page+0x9/0x20
 [<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
 [<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
 [<ffffffffa02f87eb>] ? xtLookup+0x18b/0x1a0 [jfs]
 [<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
 [<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
 [<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
 [<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
 [<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
 [<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
 [<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
 [<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
 [<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
 [<ffffffff810581f2>] ? current_fs_time+0x22/0x30
 [<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
 [<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
 [<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
 [<ffffffff81122002>] do_sync_write+0xd2/0x110
 [<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
 [<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
 [<ffffffff81122b48>] vfs_write+0xb8/0x1a0
 [<ffffffff81122d0c>] sys_write+0x4c/0x80
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task filezilla:18574 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
filezilla     D ffffffff810dc2d0     0 18574   2063 0x00000000
 ffff88003c967768 0000000000000082 0000000000045ca7 0000000000000008
 0000000000800020 ffff88003ca52cb0 ffff88003c967798 ffffffff811b6fa4
 ffff88003c967740 ffff88003c967fd8 ffff88003c966000 ffff88003c966000
Call Trace:
 [<ffffffff811b6fa4>] ? generic_make_request+0x184/0x4f0
 [<ffffffff8107d739>] ? ktime_get_ts+0xa9/0xe0
 [<ffffffff810dc2d0>] ? sync_page+0x0/0x50
 [<ffffffff8135ad5e>] io_schedule+0x6e/0xb0
 [<ffffffff810dc30d>] sync_page+0x3d/0x50
 [<ffffffff8135b382>] __wait_on_bit_lock+0x52/0xb0
 [<ffffffff810dc2b2>] __lock_page+0x62/0x70
 [<ffffffff810732b0>] ? wake_bit_function+0x0/0x40
 [<ffffffff810dc9a9>] do_read_cache_page+0x159/0x180
 [<ffffffffa030d210>] ? metapage_readpage+0x0/0x180 [jfs]
 [<ffffffff810dca04>] read_cache_page_async+0x14/0x20
 [<ffffffff810dca19>] read_cache_page+0x9/0x20
 [<ffffffffa030ddb5>] __get_metapage+0x95/0x5a0 [jfs]
 [<ffffffff8135d1b4>] ? __down_read+0xd4/0xd6
 [<ffffffffa02f86ed>] ? xtLookup+0x8d/0x1a0 [jfs]
 [<ffffffffa0304227>] dbAlloc+0x147/0x480 [jfs]
 [<ffffffffa030cb92>] extAlloc+0x162/0x4d0 [jfs]
 [<ffffffff8111d740>] ? mem_cgroup_cache_charge+0x140/0x1e0
 [<ffffffffa02f4b21>] jfs_get_block+0x1c1/0x220 [jfs]
 [<ffffffff8114dd0a>] nobh_write_begin+0x1ea/0x4b0
 [<ffffffff8114ae16>] ? __set_page_dirty+0x76/0xd0
 [<ffffffffa02f461e>] jfs_write_begin+0x1e/0x20 [jfs]
 [<ffffffffa02f4960>] ? jfs_get_block+0x0/0x220 [jfs]
 [<ffffffff810db70d>] generic_file_buffered_write+0x10d/0x280
 [<ffffffff810581f2>] ? current_fs_time+0x22/0x30
 [<ffffffff810dd658>] __generic_file_aio_write+0x238/0x450
 [<ffffffff8135baad>] ? __mutex_lock_slowpath+0x26d/0x370
 [<ffffffff810dd8d4>] generic_file_aio_write+0x64/0xd0
 [<ffffffff81122002>] do_sync_write+0xd2/0x110
 [<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
 [<ffffffff8135dc8e>] ? common_interrupt+0xe/0x13
 [<ffffffff8119fcb1>] ? security_file_permission+0x11/0x20
 [<ffffffff81122b48>] vfs_write+0xb8/0x1a0
 [<ffffffff81122d0c>] sys_write+0x4c/0x80
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task smbd:18659 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd          D ffff88000180f948     0 18659  16782 0x00000000
 ffff8800328e9b18 0000000000000086 0000000000000000 ffff8800328e9db8
 0000000020000000 0000000000000000 0000000000000000 ffff88003c9209f0
 000000010619a86e ffff8800328e9fd8 ffff8800328e8000 ffff8800328e8000
Call Trace:
 [<ffffffff8135d085>] __down_write_nested+0x95/0xe0
 [<ffffffff8135d0db>] __down_write+0xb/0x10
 [<ffffffff8135c6d9>] down_write+0x9/0x10
 [<ffffffffa03a74c7>] di_write_lock+0x27/0x50 [aufs]
 [<ffffffffa03983f6>] aufs_read_lock+0x106/0x110 [aufs]
 [<ffffffff81077e09>] ? up_read+0x9/0x10
 [<ffffffffa03afdd3>] ? aufs_permission+0x313/0x3d0 [aufs]
 [<ffffffffa03a8ae1>] aufs_d_revalidate+0x31/0x450 [aufs]
 [<ffffffff8112eab2>] link_path_walk+0x5d2/0xe50
 [<ffffffff8112f502>] path_walk+0x62/0xe0
 [<ffffffff8112f6c3>] do_path_lookup+0x53/0xa0
 [<ffffffff81130302>] user_path_at+0x52/0xa0
 [<ffffffff8113db14>] ? mntput_no_expire+0x24/0x100
 [<ffffffff811d1276>] ? __up_read+0xa6/0xd0
 [<ffffffff81077e09>] ? up_read+0x9/0x10
 [<ffffffff81126927>] vfs_fstatat+0x37/0x70
 [<ffffffff8113db14>] ? mntput_no_expire+0x24/0x100
 [<ffffffff81126a86>] vfs_stat+0x16/0x20
 [<ffffffff81126aaf>] sys_newstat+0x1f/0x50
 [<ffffffff8112c0ec>] ? path_put+0x2c/0x40
 [<ffffffff81120ee1>] ? sys_chdir+0x51/0x80
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
INFO: task smbd:18660 blocked for more than 120 seconds.
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
smbd          D ffff88000190f948     0 18660  16782 0x00000000
 ffff88003a767d18 0000000000000086 0000000000000000 ffff88003a767d08
 000001c73a767cb8 000001c8000001c8 000000083a767dd8 ffff88003c922d90
 000000010619fccf ffff88003a767fd8 ffff88003a766000 ffff88003a766000
Call Trace:
 [<ffffffff8135d175>] __down_read+0x95/0xd6
 [<ffffffff8135c6e9>] down_read+0x9/0x10
 [<ffffffffa03a76b6>] di_read_lock+0x26/0x90 [aufs]
 [<ffffffffa03af5f2>] aufs_getattr+0xe2/0x4a0 [aufs]
 [<ffffffff8113030d>] ? user_path_at+0x5d/0xa0
 [<ffffffff811268bc>] vfs_getattr+0x4c/0x80
 [<ffffffff81126948>] vfs_fstatat+0x58/0x70
 [<ffffffff811269c9>] vfs_lstat+0x19/0x20
 [<ffffffff811269ef>] sys_newlstat+0x1f/0x50
 [<ffffffff81009fc2>] system_call_fastpath+0x16/0x1b
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 12 a0 00 04 00 00
end_request: I/O error, dev sdf, sector 442634912
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 0e a0 00 04 00 00
end_request: I/O error, dev sdf, sector 442633888
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 0b 00 00 03 a0 00
end_request: I/O error, dev sdf, sector 442632960
sd 10:0:4:0: [sdf] Unhandled error code
sd 10:0:4:0: [sdf] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:4:0: [sdf] CDB: cdb[0]=0x2a: 2a 00 1a 62 07 00 00 04 00 00
end_request: I/O error, dev sdf, sector 442631936
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 1c b2 87 6f 00 00 08 00
end_request: I/O error, dev sdc, sector 481462127
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1669:mvs_abort_task:rc= 5
drivers/scsi/mvsas/mv_sas.c 1608:mvs_query_task:rc= 5
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 00 04 5c a7 00 00 08 00
end_request: I/O error, dev sdc, sector 285863
metapage_read_end_io: I/O error
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x28: 28 00 00 05 30 d7 00 00 08 00
end_request: I/O error, dev sdc, sector 340183
metapage_read_end_io: I/O error
sd 10:0:1:0: [sdc] Unhandled error code
sd 10:0:1:0: [sdc] Result: hostbyte=0x00 driverbyte=0x06
sd 10:0:1:0: [sdc] CDB: cdb[0]=0x2a: 2a 00 ae a6 3d 67 00 00 08 00
end_request: I/O error, dev sdc, sector 2930130279