* I got file system corruption with XFS
@ 2018-12-11 20:32 Luciano ES
2018-12-11 20:57 ` Eric Sandeen
` (5 more replies)
0 siblings, 6 replies; 18+ messages in thread
From: Luciano ES @ 2018-12-11 20:32 UTC (permalink / raw)
To: linux-xfs
I needed to restore something from my backups, an external hard disk
that is kept separately, always disconnected until I really need it.
But the file system refused to be mounted: "structure needs cleaning,"
it said.
I googled and didn't find much hope about it. I followed what little
advice I found: I ran xfs_repair and it didn't work. So I ran it
again with -L and it worked, but the software itself warned me that
some files could not be recovered. I'll never know which ones.
I always liked XFS and thought those dreaded days of file system
corruption and lost files were far behind. So my only question is:
Why does that happen? The disk is not even used 99% of the time.
How does an XFS file system go belly up just like that?
TIA
--
Luciano ES
>>
^ permalink raw reply [flat|nested] 18+ messages in thread* Re: I got file system corruption with XFS 2018-12-11 20:32 I got file system corruption with XFS Luciano ES @ 2018-12-11 20:57 ` Eric Sandeen 2018-12-11 22:54 ` Luciano ES 2018-12-11 21:27 ` Dave Chinner ` (4 subsequent siblings) 5 siblings, 1 reply; 18+ messages in thread From: Eric Sandeen @ 2018-12-11 20:57 UTC (permalink / raw) To: Luciano ES, linux-xfs On 12/11/18 2:32 PM, Luciano ES wrote: > I needed to restore something from my backups, an external hard disk > that is kept separately, always disconnected until I really need it. > > But the file system refused to be mounted: "structure needs cleaning," > it said. > > I googled and didn't find much hope about it. I followed what little > advice I found: I ran xfs_repair and it didn't work. So I ran it > again with -L and it worked, but the software itself warned me that > some files could not be recovered. I'll never know which ones. > > I always liked XFS and thought those dreaded days of file system > corruption and lost files were far behind. So my only question is: > Why does that happen? The disk is not even used 99% of the time. > How does an XFS file system go belly up just like that? There is no way for us to know. You didn't provide nearly enough information to even hazard a guess. But ok fine, I'll hazard a wild guess anyway: your external drive had a corrupt log because the enclosure didn't honor a cache flush issued by the filesystem after some previous mount. -Eric ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: I got file system corruption with XFS 2018-12-11 20:57 ` Eric Sandeen @ 2018-12-11 22:54 ` Luciano ES 2018-12-12 1:01 ` Eric Sandeen 0 siblings, 1 reply; 18+ messages in thread From: Luciano ES @ 2018-12-11 22:54 UTC (permalink / raw) To: linux-xfs On Tue, 11 Dec 2018 14:57:45 -0600, Eric Sandeen wrote: > On 12/11/18 2:32 PM, Luciano ES wrote: > > I needed to restore something from my backups, an external hard > > disk that is kept separately, always disconnected until I really > > need it. > > > > But the file system refused to be mounted: "structure needs > > cleaning," it said. > > > > I googled and didn't find much hope about it. I followed what > > little advice I found: I ran xfs_repair and it didn't work. So I > > ran it again with -L and it worked, but the software itself warned > > me that some files could not be recovered. I'll never know which > > ones. > > > > I always liked XFS and thought those dreaded days of file system > > corruption and lost files were far behind. So my only question is: > > Why does that happen? The disk is not even used 99% of the time. > > How does an XFS file system go belly up just like that? > > There is no way for us to know. You didn't provide nearly enough > information to even hazard a guess. > > But ok fine, I'll hazard a wild guess anyway: your external drive had > a corrupt log because the enclosure didn't honor a cache flush issued > by the filesystem after some previous mount. > > -Eric ************************** I understand you don't have much to work with, but I can't tell you more than I have. It happened several days ago and I didn't write anything down. As far as I can remember, there wasn't really much to note. "Structure needs cleaning" was pretty much all I was ever told. There must have been more when I ran xfs_repair, but it looked incomprehensible to me and I thought I shouldn't bother anyone else about it. Your theory about the enclosure sounds good. Do you think it is so flawed that issuing a 'sync' command manually before umounting wouldn't have made any difference? -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: I got file system corruption with XFS 2018-12-11 22:54 ` Luciano ES @ 2018-12-12 1:01 ` Eric Sandeen 0 siblings, 0 replies; 18+ messages in thread From: Eric Sandeen @ 2018-12-12 1:01 UTC (permalink / raw) To: Luciano ES, linux-xfs On 12/11/18 4:54 PM, Luciano ES wrote: > On Tue, 11 Dec 2018 14:57:45 -0600, Eric Sandeen wrote: > >> On 12/11/18 2:32 PM, Luciano ES wrote: >>> I needed to restore something from my backups, an external hard >>> disk that is kept separately, always disconnected until I really >>> need it. >>> >>> But the file system refused to be mounted: "structure needs >>> cleaning," it said. >>> >>> I googled and didn't find much hope about it. I followed what >>> little advice I found: I ran xfs_repair and it didn't work. So I >>> ran it again with -L and it worked, but the software itself warned >>> me that some files could not be recovered. I'll never know which >>> ones. >>> >>> I always liked XFS and thought those dreaded days of file system >>> corruption and lost files were far behind. So my only question is: >>> Why does that happen? The disk is not even used 99% of the time. >>> How does an XFS file system go belly up just like that? >> >> There is no way for us to know. You didn't provide nearly enough >> information to even hazard a guess. >> >> But ok fine, I'll hazard a wild guess anyway: your external drive had >> a corrupt log because the enclosure didn't honor a cache flush issued >> by the filesystem after some previous mount. >> >> -Eric > > > ************************** > > I understand you don't have much to work with, but I can't tell you > more than I have. That's ok. It just means I can't help you more than I have. ;) > It happened several days ago and I didn't write > anything down. As far as I can remember, there wasn't really much > to note. "Structure needs cleaning" was pretty much all I was ever > told. There must have been more when I ran xfs_repair, but it looked > incomprehensible to me and I thought I shouldn't bother anyone else > about it. > > Your theory about the enclosure sounds good. Do you think it is so > flawed that issuing a 'sync' command manually before umounting > wouldn't have made any difference? If the enclosure ignores cache flush commands, then sync wouldn't have helped. Sync /shoudn't/ be necessary before unmount tho. You may have been in more jeopardy if you pulled out the USB plug, or lost power. But that's all just a guess. -Eric ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: I got file system corruption with XFS 2018-12-11 20:32 I got file system corruption with XFS Luciano ES 2018-12-11 20:57 ` Eric Sandeen @ 2018-12-11 21:27 ` Dave Chinner 2019-03-31 22:49 ` File system corruption in two hard disks Dave Chinner ` (3 subsequent siblings) 5 siblings, 0 replies; 18+ messages in thread From: Dave Chinner @ 2018-12-11 21:27 UTC (permalink / raw) To: Luciano ES; +Cc: linux-xfs On Tue, Dec 11, 2018 at 06:32:03PM -0200, Luciano ES wrote: > I needed to restore something from my backups, an external hard disk > that is kept separately, always disconnected until I really need it. > > But the file system refused to be mounted: "structure needs cleaning," > it said. > > I googled and didn't find much hope about it. I followed what little > advice I found: I ran xfs_repair and it didn't work. So I ran it > again with -L and it worked, but the software itself warned me that > some files could not be recovered. I'll never know which ones. > > I always liked XFS and thought those dreaded days of file system > corruption and lost files were far behind. So my only question is: > Why does that happen? The disk is not even used 99% of the time. > How does an XFS file system go belly up just like that? What was the actual error message? http://xfs.org/index.php/XFS_FAQ#Q:_What_information_should_I_include_when_reporting_a_problem.3F Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2018-12-11 20:32 I got file system corruption with XFS Luciano ES 2018-12-11 20:57 ` Eric Sandeen 2018-12-11 21:27 ` Dave Chinner @ 2019-03-31 22:49 ` Dave Chinner 2019-04-01 21:13 ` Luciano ES 2019-08-03 0:53 ` XFS file system corruption, refuses to mount Luciano ES ` (2 subsequent siblings) 5 siblings, 1 reply; 18+ messages in thread From: Dave Chinner @ 2019-03-31 22:49 UTC (permalink / raw) To: Luciano ES; +Cc: linux-xfs On Sun, Mar 31, 2019 at 04:41:47PM -0300, Luciano ES wrote: > I have two hard disks that have been kept in storage for a little > more than one year. I can read neither of them. > > DISK 1: I tried to open it with cryptsetup. It wasn't recognized > as a LUKS partition, which is weird because I always use LUKS. > > But it pops up as a mountable file system on the pcmanfm file manager. > Clicking it to mount it as is gives me this error message: > > Error mounting /dev/sdc1 at /media/ext/sm640: Command-line > `mount -t "xfs" -o "uhelper=udisks2,nodev,nosuid" "/dev/sdc1" "/media/ext/sm640"' > exited with non-zero exit status 32: mount: > mount /dev/sdc1 on /media/ext/sm640 failed: Structure needs cleaning And the errors that XFS emitted in dmesg are... ? Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2019-03-31 22:49 ` File system corruption in two hard disks Dave Chinner @ 2019-04-01 21:13 ` Luciano ES 2019-04-01 21:32 ` Dave Chinner 0 siblings, 1 reply; 18+ messages in thread From: Luciano ES @ 2019-04-01 21:13 UTC (permalink / raw) To: linux-xfs On Mon, 1 Apr 2019 09:49:18 +1100, Dave Chinner wrote: > On Sun, Mar 31, 2019 at 04:41:47PM -0300, Luciano ES wrote: > > I have two hard disks that have been kept in storage for a little > > more than one year. I can read neither of them. > > > > DISK 1: I tried to open it with cryptsetup. It wasn't recognized > > as a LUKS partition, which is weird because I always use LUKS. > > > > But it pops up as a mountable file system on the pcmanfm file > > manager. Clicking it to mount it as is gives me this error message: > > > > Error mounting /dev/sdc1 at /media/ext/sm640: Command-line > > `mount -t "xfs" -o "uhelper=udisks2,nodev,nosuid" "/dev/sdc1" > > "/media/ext/sm640"' exited with non-zero exit status 32: mount: > > mount /dev/sdc1 on /media/ext/sm640 failed: Structure needs > > cleaning > > And the errors that XFS emitted in dmesg are... ? > > Cheers, > > Dave. ************************** Thank you for your reply. Sorry I couldn't get back to you earlier. Here is the dmesg output. I tried mounting them then running 'xfs_repair -n' on each one of them. DISK 1: [58267.353026] usb 2-1.2.4: new high-speed USB device number 17 using ehci-pci [58267.470031] usb 2-1.2.4: New USB device found, idVendor=13fd, idProduct=0840 [58267.470033] usb 2-1.2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [58267.470033] usb 2-1.2.4: Product: External [58267.470034] usb 2-1.2.4: Manufacturer: Generic [58267.470035] usb 2-1.2.4: SerialNumber: [elided] [58267.470472] usb-storage 2-1.2.4:1.0: USB Mass Storage device detected [58267.470676] scsi host4: usb-storage 2-1.2.4:1.0 [58268.481677] scsi 4:0:0:0: Direct-Access Generic External 1.14 PQ: 0 ANSI: 4 [58268.482369] sd 4:0:0:0: Attached scsi generic sg2 type 0 [58268.482918] sd 4:0:0:0: [sdc] 1250263727 512-byte logical blocks: (640 GB/596 GiB) [58268.483543] sd 4:0:0:0: [sdc] Write Protect is off [58268.483544] sd 4:0:0:0: [sdc] Mode Sense: 23 00 00 00 [58268.484183] sd 4:0:0:0: [sdc] No Caching mode page found [58268.484186] sd 4:0:0:0: [sdc] Assuming drive cache: write through [58268.515922] sdc: sdc1 [58268.518417] sd 4:0:0:0: [sdc] Attached SCSI disk [58268.723169] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [58268.723171] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] [58268.723173] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information [58268.723175] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00 [58269.531556] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [58269.531559] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] [58269.531561] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information [58269.531563] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 b0 00 00 [58271.587320] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58271.587323] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58271.587324] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58271.587326] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 00 00 00 f0 00 [58271.587327] blk_update_request: critical medium error, dev sdc, sector 2560 [58273.860713] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58273.860716] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58273.860717] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58273.860719] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 50 00 00 02 00 [58273.860720] blk_update_request: critical medium error, dev sdc, sector 2640 [58273.860724] Buffer I/O error on dev sdc1, logical block 296, async page read [58276.091606] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58276.091609] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58276.091610] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58276.091612] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 54 00 00 04 00 [58276.091613] blk_update_request: critical medium error, dev sdc, sector 2644 [58276.091616] Buffer I/O error on dev sdc1, logical block 298, async page read [58276.091618] Buffer I/O error on dev sdc1, logical block 299, async page read [58278.368375] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58278.368377] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58278.368378] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58278.368380] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 00 00 00 f0 00 [58278.368381] blk_update_request: critical medium error, dev sdc, sector 2560 [58280.608664] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58280.608667] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58280.608669] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58280.608671] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 50 00 00 02 00 [58280.608672] blk_update_request: critical medium error, dev sdc, sector 2640 [58280.608676] Buffer I/O error on dev sdc1, logical block 296, async page read [58282.839557] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58282.839560] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58282.839561] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58282.839563] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 54 00 00 04 00 [58282.839565] blk_update_request: critical medium error, dev sdc, sector 2644 [58282.839568] Buffer I/O error on dev sdc1, logical block 298, async page read [58282.839570] Buffer I/O error on dev sdc1, logical block 299, async page read [58285.094179] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58285.094181] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58285.094183] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58285.094185] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 00 00 00 f0 00 [58285.094186] blk_update_request: critical medium error, dev sdc, sector 2560 [58287.323337] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58287.323340] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58287.323341] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58287.323344] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 50 00 00 02 00 [58287.323345] blk_update_request: critical medium error, dev sdc, sector 2640 [58287.323349] Buffer I/O error on dev sdc1, logical block 296, async page read [58289.587476] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58289.587479] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58289.587480] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58289.587482] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 54 00 00 04 00 [58289.587483] blk_update_request: critical medium error, dev sdc, sector 2644 [58289.587487] Buffer I/O error on dev sdc1, logical block 298, async page read [58289.587489] Buffer I/O error on dev sdc1, logical block 299, async page read [58289.788582] XFS (sdc1): Offline file system operation in progress! [58289.788614] XFS (sdc1): Metadata corruption detected at xfs_sb_read_verify+0x111/0x140 [xfs], xfs_sb block 0xffffffffffffffff [58289.788617] XFS (sdc1): Unmount and run xfs_repair [58289.788618] XFS (sdc1): First 64 bytes of corrupted metadata buffer: [58289.788620] ffff9b34d9cd3000: 58 46 53 42 00 00 10 00 00 00 00 00 09 50 af 55 XFSB.........P.U [58289.788621] ffff9b34d9cd3010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [58289.788622] ffff9b34d9cd3020: 53 df 87 aa a0 e0 42 30 ba 5f 36 4c c5 42 be e7 S.....B0._6L.B.. [58289.788623] ffff9b34d9cd3030: 00 00 00 00 08 00 00 05 ff ff ff ff ff ff ff ff ................ [58289.788628] XFS (sdc1): SB validate failed with error -117. [58290.764466] XFS (sdc1): Offline file system operation in progress! [58290.764499] XFS (sdc1): Metadata corruption detected at xfs_sb_read_verify+0x111/0x140 [xfs], xfs_sb block 0xffffffffffffffff [58290.764502] XFS (sdc1): Unmount and run xfs_repair [58290.764502] XFS (sdc1): First 64 bytes of corrupted metadata buffer: [58290.764504] ffff9b348d3fa000: 58 46 53 42 00 00 10 00 00 00 00 00 09 50 af 55 XFSB.........P.U [58290.764505] ffff9b348d3fa010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [58290.764506] ffff9b348d3fa020: 53 df 87 aa a0 e0 42 30 ba 5f 36 4c c5 42 be e7 S.....B0._6L.B.. [58290.764507] ffff9b348d3fa030: 00 00 00 00 08 00 00 05 ff ff ff ff ff ff ff ff ................ [58290.764511] XFS (sdc1): SB validate failed with error -117. [58316.492438] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58316.492440] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58316.492441] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58316.492443] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 00 00 00 f0 00 [58316.492445] blk_update_request: critical medium error, dev sdc, sector 2560 [58318.699428] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58318.699431] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58318.699432] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58318.699434] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 50 00 00 02 00 [58318.699435] blk_update_request: critical medium error, dev sdc, sector 2640 [58318.699438] Buffer I/O error on dev sdc1, logical block 296, async page read [58320.952446] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58320.952449] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58320.952450] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58320.952452] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 54 00 00 04 00 [58320.952453] blk_update_request: critical medium error, dev sdc, sector 2644 [58320.952457] Buffer I/O error on dev sdc1, logical block 298, async page read [58320.952459] Buffer I/O error on dev sdc1, logical block 299, async page read [58321.152815] XFS (sdc1): Offline file system operation in progress! [58321.152849] XFS (sdc1): Metadata corruption detected at xfs_sb_read_verify+0x111/0x140 [xfs], xfs_sb block 0xffffffffffffffff [58321.152852] XFS (sdc1): Unmount and run xfs_repair [58321.152853] XFS (sdc1): First 64 bytes of corrupted metadata buffer: [58321.152855] ffff9b34acf07000: 58 46 53 42 00 00 10 00 00 00 00 00 09 50 af 55 XFSB.........P.U [58321.152856] ffff9b34acf07010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [58321.152857] ffff9b34acf07020: 53 df 87 aa a0 e0 42 30 ba 5f 36 4c c5 42 be e7 S.....B0._6L.B.. [58321.152859] ffff9b34acf07030: 00 00 00 00 08 00 00 05 ff ff ff ff ff ff ff ff ................ [58321.152864] XFS (sdc1): SB validate failed with error -117. [58332.196671] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [58332.196674] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] [58332.196675] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error [58332.196677] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 09 e0 00 00 f0 00 [58332.196678] blk_update_request: critical medium error, dev sdc, sector 2528 [58341.573221] XFS (sdc1): Offline file system operation in progress! [58341.573252] XFS (sdc1): Metadata corruption detected at xfs_sb_read_verify+0x111/0x140 [xfs], xfs_sb block 0xffffffffffffffff [58341.573255] XFS (sdc1): Unmount and run xfs_repair [58341.573256] XFS (sdc1): First 64 bytes of corrupted metadata buffer: [58341.573257] ffff9b34c7b05000: 58 46 53 42 00 00 10 00 00 00 00 00 09 50 af 55 XFSB.........P.U [58341.573258] ffff9b34c7b05010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [58341.573259] ffff9b34c7b05020: 53 df 87 aa a0 e0 42 30 ba 5f 36 4c c5 42 be e7 S.....B0._6L.B.. [58341.573260] ffff9b34c7b05030: 00 00 00 00 08 00 00 05 ff ff ff ff ff ff ff ff ................ [58341.573286] XFS (sdc1): SB validate failed with error -117. [58345.707124] usb 2-1.2.4: USB disconnect, device number 17 DISK 2: [57949.398186] usb 2-1.2.4: new high-speed USB device number 16 using ehci-pci [57949.519158] usb 2-1.2.4: New USB device found, idVendor=13fd, idProduct=0840 [57949.519160] usb 2-1.2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 [57949.519161] usb 2-1.2.4: Product: External [57949.519161] usb 2-1.2.4: Manufacturer: Generic [57949.519162] usb 2-1.2.4: SerialNumber: [elided] [57949.519441] usb-storage 2-1.2.4:1.0: USB Mass Storage device detected [57949.519883] scsi host4: usb-storage 2-1.2.4:1.0 [57950.522787] scsi 4:0:0:0: Direct-Access Generic External 1.14 PQ: 0 ANSI: 4 [57950.523080] sd 4:0:0:0: Attached scsi generic sg2 type 0 [57950.523790] sd 4:0:0:0: [sdc] 234441647 512-byte logical blocks: (120 GB/112 GiB) [57950.524526] sd 4:0:0:0: [sdc] Write Protect is off [57950.524528] sd 4:0:0:0: [sdc] Mode Sense: 23 00 00 00 [57950.525248] sd 4:0:0:0: [sdc] No Caching mode page found [57950.525251] sd 4:0:0:0: [sdc] Assuming drive cache: write through [57950.573656] sdc: sdc1 [57950.573659] sdc: p1 size 234439600 extends beyond EOD, enabling native capacity [57950.578529] sdc: sdc1 [57950.578530] sdc: p1 size 234439600 extends beyond EOD, truncated [57950.580963] sd 4:0:0:0: [sdc] Attached SCSI disk [57950.913655] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [57950.913658] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] [57950.913659] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information [57950.913661] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00 [57951.045034] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [57951.045037] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] [57951.045038] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information [57951.045040] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 b0 00 00 [57997.645037] attempt to access beyond end of device [57997.645039] dm-4: rw=16, want=234435504, limit=234435503 [57997.645041] XFS (dm-4): last sector read failed [58040.866246] attempt to access beyond end of device [58040.866249] dm-4: rw=16, want=234435504, limit=234435503 [58040.866251] XFS (dm-4): last sector read failed [58046.339021] attempt to access beyond end of device [58046.339023] dm-4: rw=16, want=234435504, limit=234435503 [58046.339025] XFS (dm-4): last sector read failed [58190.058091] usb 2-1.2.4: USB disconnect, device number 16 (Note: I ran disk2 before disk 1 and inverted the order of the outputs here just to keep it consistent with my initial report. It probably doesn't matter.) -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2019-04-01 21:13 ` Luciano ES @ 2019-04-01 21:32 ` Dave Chinner 2019-04-02 16:23 ` Luciano ES 0 siblings, 1 reply; 18+ messages in thread From: Dave Chinner @ 2019-04-01 21:32 UTC (permalink / raw) To: Luciano ES; +Cc: linux-xfs On Mon, Apr 01, 2019 at 06:13:11PM -0300, Luciano ES wrote: > On Mon, 1 Apr 2019 09:49:18 +1100, Dave Chinner wrote: > > > On Sun, Mar 31, 2019 at 04:41:47PM -0300, Luciano ES wrote: > > > I have two hard disks that have been kept in storage for a little > > > more than one year. I can read neither of them. > > > > > > DISK 1: I tried to open it with cryptsetup. It wasn't recognized > > > as a LUKS partition, which is weird because I always use LUKS. > > > > > > But it pops up as a mountable file system on the pcmanfm file > > > manager. Clicking it to mount it as is gives me this error message: > > > > > > Error mounting /dev/sdc1 at /media/ext/sm640: Command-line > > > `mount -t "xfs" -o "uhelper=udisks2,nodev,nosuid" "/dev/sdc1" > > > "/media/ext/sm640"' exited with non-zero exit status 32: mount: > > > mount /dev/sdc1 on /media/ext/sm640 failed: Structure needs > > > cleaning > > > > And the errors that XFS emitted in dmesg are... ? > > > > Cheers, > > > > Dave. > > > ************************** > Thank you for your reply. Sorry I couldn't get back to you earlier. > > Here is the dmesg output. I tried mounting them then running 'xfs_repair -n' > on each one of them. > > DISK 1: > [58267.353026] usb 2-1.2.4: new high-speed USB device number 17 using ehci-pci > [58267.470031] usb 2-1.2.4: New USB device found, idVendor=13fd, idProduct=0840 > [58267.470033] usb 2-1.2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 > [58267.470033] usb 2-1.2.4: Product: External > [58267.470034] usb 2-1.2.4: Manufacturer: Generic > [58267.470035] usb 2-1.2.4: SerialNumber: [elided] > [58267.470472] usb-storage 2-1.2.4:1.0: USB Mass Storage device detected > [58267.470676] scsi host4: usb-storage 2-1.2.4:1.0 > [58268.481677] scsi 4:0:0:0: Direct-Access Generic External 1.14 PQ: 0 ANSI: 4 > [58268.482369] sd 4:0:0:0: Attached scsi generic sg2 type 0 > [58268.482918] sd 4:0:0:0: [sdc] 1250263727 512-byte logical blocks: (640 GB/596 GiB) > [58268.483543] sd 4:0:0:0: [sdc] Write Protect is off > [58268.483544] sd 4:0:0:0: [sdc] Mode Sense: 23 00 00 00 > [58268.484183] sd 4:0:0:0: [sdc] No Caching mode page found > [58268.484186] sd 4:0:0:0: [sdc] Assuming drive cache: write through > [58268.515922] sdc: sdc1 > [58268.518417] sd 4:0:0:0: [sdc] Attached SCSI disk > [58268.723169] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE > [58268.723171] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] > [58268.723173] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information > [58268.723175] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00 > [58269.531556] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE > [58269.531559] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] > [58269.531561] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information > [58269.531563] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 b0 00 00 > [58271.587320] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE > [58271.587323] sd 4:0:0:0: [sdc] tag#0 Sense Key : Medium Error [current] > [58271.587324] sd 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error > [58271.587326] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a 00 00 00 f0 00 > [58271.587327] blk_update_request: critical medium error, dev sdc, sector 2560 You're getting read errors from the disk itself i.e. the data on the storage medium has degraded and no longer contains the same information that was written to it. > [58289.587483] blk_update_request: critical medium error, dev sdc, sector 2644 > [58289.587487] Buffer I/O error on dev sdc1, logical block 298, async page read > [58289.587489] Buffer I/O error on dev sdc1, logical block 299, async page read > [58289.788582] XFS (sdc1): Offline file system operation in progress! Indicative of single bit errors in the superblock i.e. the "in progress flag is set, which is only set during mkfs and is zero at all other times. You could probably clear that field using xfs_db, but it's likely there are lots of other problems with the disk... > DISK 2: > [57949.398186] usb 2-1.2.4: new high-speed USB device number 16 using ehci-pci > [57949.519158] usb 2-1.2.4: New USB device found, idVendor=13fd, idProduct=0840 > [57949.519160] usb 2-1.2.4: New USB device strings: Mfr=1, Product=2, SerialNumber=3 > [57949.519161] usb 2-1.2.4: Product: External > [57949.519161] usb 2-1.2.4: Manufacturer: Generic > [57949.519162] usb 2-1.2.4: SerialNumber: [elided] > [57949.519441] usb-storage 2-1.2.4:1.0: USB Mass Storage device detected > [57949.519883] scsi host4: usb-storage 2-1.2.4:1.0 > [57950.522787] scsi 4:0:0:0: Direct-Access Generic External 1.14 PQ: 0 ANSI: 4 > [57950.523080] sd 4:0:0:0: Attached scsi generic sg2 type 0 > [57950.523790] sd 4:0:0:0: [sdc] 234441647 512-byte logical blocks: (120 GB/112 GiB) > [57950.524526] sd 4:0:0:0: [sdc] Write Protect is off > [57950.524528] sd 4:0:0:0: [sdc] Mode Sense: 23 00 00 00 > [57950.525248] sd 4:0:0:0: [sdc] No Caching mode page found > [57950.525251] sd 4:0:0:0: [sdc] Assuming drive cache: write through > [57950.573656] sdc: sdc1 > [57950.573659] sdc: p1 size 234439600 extends beyond EOD, enabling native capacity > [57950.578529] sdc: sdc1 > [57950.578530] sdc: p1 size 234439600 extends beyond EOD, truncated The partition table indicates that partition 1 extends past the end of the disk. > [57950.580963] sd 4:0:0:0: [sdc] Attached SCSI disk > [57950.913655] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE > [57950.913658] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] > [57950.913659] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information > [57950.913661] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 00 00 00 e5 00 > [57951.045034] sd 4:0:0:0: [sdc] tag#0 FAILED Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE > [57951.045037] sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] [descriptor] > [57951.045038] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No additional sense information > [57951.045040] sd 4:0:0:0: [sdc] tag#0 CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 b0 00 00 > [57997.645037] attempt to access beyond end of device > [57997.645039] dm-4: rw=16, want=234435504, limit=234435503 > [57997.645041] XFS (dm-4): last sector read failed > [58040.866246] attempt to access beyond end of device And XFs is trying to validate that it can read the last block of the filesystem, which it can't because the partition is beyond the end of the device. At this point, I suspect your problems are the USB enclosure your are using. Plug the disks directly into a SATA port and see if that makes the IO errors go away. This looks like hardware problems, not XFS issues... Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2019-04-01 21:32 ` Dave Chinner @ 2019-04-02 16:23 ` Luciano ES 2019-04-02 16:42 ` Eric Sandeen ` (2 more replies) 0 siblings, 3 replies; 18+ messages in thread From: Luciano ES @ 2019-04-02 16:23 UTC (permalink / raw) To: XFS mailing list On Tue, 2 Apr 2019 08:32:26 +1100, Dave Chinner wrote: > On Mon, Apr 01, 2019 at 06:13:11PM -0300, Luciano ES wrote: > > On Mon, 1 Apr 2019 09:49:18 +1100, Dave Chinner wrote: > > > > > On Sun, Mar 31, 2019 at 04:41:47PM -0300, Luciano ES wrote: > > > > I have two hard disks that have been kept in storage for a > > > > little more than one year. I can read neither of them. > > > > > > > > DISK 1: I tried to open it with cryptsetup. It wasn't > > > > recognized as a LUKS partition, which is weird because I always > > > > use LUKS. > > > > > > > > But it pops up as a mountable file system on the pcmanfm file > > > > manager. Clicking it to mount it as is gives me this error > > > > message: > > > > > > > > Error mounting /dev/sdc1 at /media/ext/sm640: Command-line > > > > `mount -t "xfs" -o "uhelper=udisks2,nodev,nosuid" "/dev/sdc1" > > > > "/media/ext/sm640"' exited with non-zero exit status 32: mount: > > > > mount /dev/sdc1 on /media/ext/sm640 failed: Structure needs > > > > cleaning > > > > > > And the errors that XFS emitted in dmesg are... ? > > > > > > Cheers, > > > > > > Dave. > > > > > > ************************** > > Thank you for your reply. Sorry I couldn't get back to you earlier. > > > > Here is the dmesg output. I tried mounting them then running > > 'xfs_repair -n' on each one of them. > > > > DISK 1: > > [58267.353026] usb 2-1.2.4: new high-speed USB device number 17 > > using ehci-pci [58267.470031] usb 2-1.2.4: New USB device found, > > idVendor=13fd, idProduct=0840 [58267.470033] usb 2-1.2.4: New USB > > device strings: Mfr=1, Product=2, SerialNumber=3 [58267.470033] usb > > 2-1.2.4: Product: External [58267.470034] usb 2-1.2.4: > > Manufacturer: Generic [58267.470035] usb 2-1.2.4: SerialNumber: > > [elided] [58267.470472] usb-storage 2-1.2.4:1.0: USB Mass Storage > > device detected [58267.470676] scsi host4: usb-storage 2-1.2.4:1.0 > > [58268.481677] scsi 4:0:0:0: Direct-Access Generic > > External 1.14 PQ: 0 ANSI: 4 [58268.482369] sd 4:0:0:0: > > Attached scsi generic sg2 type 0 [58268.482918] sd 4:0:0:0: [sdc] > > 1250263727 512-byte logical blocks: (640 GB/596 GiB) [58268.483543] > > sd 4:0:0:0: [sdc] Write Protect is off [58268.483544] sd 4:0:0:0: > > [sdc] Mode Sense: 23 00 00 00 [58268.484183] sd 4:0:0:0: [sdc] No > > Caching mode page found [58268.484186] sd 4:0:0:0: [sdc] Assuming > > drive cache: write through [58268.515922] sdc: sdc1 > > [58268.518417] sd 4:0:0:0: [sdc] Attached SCSI disk > > [58268.723169] sd 4:0:0:0: [sdc] tag#0 FAILED Result: > > hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [58268.723171] sd > > 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] > > [descriptor] [58268.723173] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No > > additional sense information [58268.723175] sd 4:0:0:0: [sdc] tag#0 > > CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 > > 00 00 00 e5 00 [58269.531556] sd 4:0:0:0: [sdc] tag#0 FAILED > > Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [58269.531559] > > sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] > > [descriptor] [58269.531561] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No > > additional sense information [58269.531563] sd 4:0:0:0: [sdc] tag#0 > > CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 > > b0 00 00 [58271.587320] sd 4:0:0:0: [sdc] tag#0 FAILED Result: > > hostbyte=DID_OK driverbyte=DRIVER_SENSE [58271.587323] sd 4:0:0:0: > > [sdc] tag#0 Sense Key : Medium Error [current] [58271.587324] sd > > 4:0:0:0: [sdc] tag#0 Add. Sense: Unrecovered read error > > [58271.587326] sd 4:0:0:0: [sdc] tag#0 CDB: Read(10) 28 00 00 00 0a > > 00 00 00 f0 00 [58271.587327] blk_update_request: critical medium > > error, dev sdc, sector 2560 > > You're getting read errors from the disk itself i.e. the data > on the storage medium has degraded and no longer contains the same > information that was written to it. > > > [58289.587483] blk_update_request: critical medium error, dev sdc, > > sector 2644 [58289.587487] Buffer I/O error on dev sdc1, logical > > block 298, async page read [58289.587489] Buffer I/O error on dev > > sdc1, logical block 299, async page read [58289.788582] XFS (sdc1): > > Offline file system operation in progress! > > Indicative of single bit errors in the superblock i.e. the "in > progress flag is set, which is only set during mkfs and is zero at > all other times. You could probably clear that field using xfs_db, > but it's likely there are lots of other problems with the disk... > > > DISK 2: > > [57949.398186] usb 2-1.2.4: new high-speed USB device number 16 > > using ehci-pci [57949.519158] usb 2-1.2.4: New USB device found, > > idVendor=13fd, idProduct=0840 [57949.519160] usb 2-1.2.4: New USB > > device strings: Mfr=1, Product=2, SerialNumber=3 [57949.519161] usb > > 2-1.2.4: Product: External [57949.519161] usb 2-1.2.4: > > Manufacturer: Generic [57949.519162] usb 2-1.2.4: SerialNumber: > > [elided] [57949.519441] usb-storage 2-1.2.4:1.0: USB Mass Storage > > device detected [57949.519883] scsi host4: usb-storage 2-1.2.4:1.0 > > [57950.522787] scsi 4:0:0:0: Direct-Access Generic > > External 1.14 PQ: 0 ANSI: 4 [57950.523080] sd 4:0:0:0: > > Attached scsi generic sg2 type 0 [57950.523790] sd 4:0:0:0: [sdc] > > 234441647 512-byte logical blocks: (120 GB/112 GiB) [57950.524526] > > sd 4:0:0:0: [sdc] Write Protect is off [57950.524528] sd 4:0:0:0: > > [sdc] Mode Sense: 23 00 00 00 [57950.525248] sd 4:0:0:0: [sdc] No > > Caching mode page found [57950.525251] sd 4:0:0:0: [sdc] Assuming > > drive cache: write through [57950.573656] sdc: sdc1 > > [57950.573659] sdc: p1 size 234439600 extends beyond EOD, enabling > > native capacity [57950.578529] sdc: sdc1 > > [57950.578530] sdc: p1 size 234439600 extends beyond EOD, > > truncated > > The partition table indicates that partition 1 extends past the end > of the disk. > > > [57950.580963] sd 4:0:0:0: [sdc] Attached SCSI disk > > [57950.913655] sd 4:0:0:0: [sdc] tag#0 FAILED Result: > > hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [57950.913658] sd > > 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] > > [descriptor] [57950.913659] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No > > additional sense information [57950.913661] sd 4:0:0:0: [sdc] tag#0 > > CDB: ATA command pass through(16) 85 06 20 00 00 00 00 00 00 00 00 > > 00 00 00 e5 00 [57951.045034] sd 4:0:0:0: [sdc] tag#0 FAILED > > Result: hostbyte=DID_ERROR driverbyte=DRIVER_SENSE [57951.045037] > > sd 4:0:0:0: [sdc] tag#0 Sense Key : Hardware Error [current] > > [descriptor] [57951.045038] sd 4:0:0:0: [sdc] tag#0 Add. Sense: No > > additional sense information [57951.045040] sd 4:0:0:0: [sdc] tag#0 > > CDB: ATA command pass through(12)/Blank a1 06 20 da 00 00 4f c2 00 > > b0 00 00 [57997.645037] attempt to access beyond end of device > > [57997.645039] dm-4: rw=16, want=234435504, limit=234435503 > > [57997.645041] XFS (dm-4): last sector read failed [58040.866246] > > attempt to access beyond end of device > > And XFs is trying to validate that it can read the last block of the > filesystem, which it can't because the partition is beyond the end > of the device. > > At this point, I suspect your problems are the USB enclosure your > are using. Plug the disks directly into a SATA port and see if that > makes the IO errors go away. This looks like hardware problems, not > XFS issues... > > Cheers, > > Dave. ************************** Hi. Thank you for your kind attention again. I tested the disks directly connected to the SATA ports this time. One of them worked flawlessly. Damn! It's the third USB enclosure that fails on me in one year! The other disk failed again. Here is some dmesg: [ 1.243231] sd 1:0:0:0: [sdb] 1250263728 512-byte logical blocks: (640 GB/596 GiB) [ 1.243274] sd 1:0:0:0: [sdb] Write Protect is off [ 1.243276] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00 [ 1.243299] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA [ 1.270253] sdb: sdb1 [ 1.271133] sd 1:0:0:0: [sdb] Attached SCSI disk [ 1.272908] sda: sda1 sda2 sda3 sda4 sda5 sda6 [ 1.273680] sd 0:0:0:0: [sda] Attached SCSI disk elided: lots of stuff about probing USB ports... [ 3.777831] ata2.00: exception Emask 0x0 SAct 0x1000 SErr 0x0 action 0x0 [ 3.777882] ata2.00: irq_stat 0x40000008 [ 3.777926] ata2.00: failed command: READ FPDMA QUEUED [ 3.777974] ata2.00: cmd 60/00:60:00:0a:00/01:00:00:00:00/40 tag 12 ncq dma 131072 in res 41/40:00:50:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 3.778038] ata2.00: status: { DRDY ERR } [ 3.778080] ata2.00: error: { UNC } [ 3.790298] ata2.00: configured for UDMA/133 [ 3.790312] sd 1:0:0:0: [sdb] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 3.790315] sd 1:0:0:0: [sdb] tag#12 Sense Key : Medium Error [current] [ 3.790318] sd 1:0:0:0: [sdb] tag#12 Add. Sense: Unrecovered read error - auto reallocate failed [ 3.790321] sd 1:0:0:0: [sdb] tag#12 CDB: Read(10) 28 00 00 00 0a 00 00 01 00 00 [ 3.790323] blk_update_request: I/O error, dev sdb, sector 2640 [ 3.790401] ata2: EH complete [ 6.033996] ata2.00: exception Emask 0x0 SAct 0x3c000000 SErr 0x0 action 0x0 [ 6.034060] ata2.00: irq_stat 0x40000008 [ 6.034115] ata2.00: failed command: READ FPDMA QUEUED [ 6.034175] ata2.00: cmd 60/02:d0:50:0a:00/00:00:00:00:00/40 tag 26 ncq dma 1024 in res 41/40:00:50:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 6.034287] ata2.00: status: { DRDY ERR } [ 6.034341] ata2.00: error: { UNC } [ 6.046572] ata2.00: configured for UDMA/133 [ 6.046584] sd 1:0:0:0: [sdb] tag#26 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 6.046586] sd 1:0:0:0: [sdb] tag#26 Sense Key : Medium Error [current] [ 6.046588] sd 1:0:0:0: [sdb] tag#26 Add. Sense: Unrecovered read error - auto reallocate failed [ 6.046591] sd 1:0:0:0: [sdb] tag#26 CDB: Read(10) 28 00 00 00 0a 50 00 00 02 00 [ 6.046593] blk_update_request: I/O error, dev sdb, sector 2640 [ 6.046653] Buffer I/O error on dev sdb1, logical block 296, async page read [ 6.046723] ata2: EH complete [ 8.286163] ata2.00: exception Emask 0x0 SAct 0x1c000 SErr 0x0 action 0x0 [ 8.286226] ata2.00: irq_stat 0x40000008 [ 8.286282] ata2.00: failed command: READ FPDMA QUEUED [ 8.286342] ata2.00: cmd 60/02:70:56:0a:00/00:00:00:00:00/40 tag 14 ncq dma 1024 in res 41/40:00:56:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 8.286454] ata2.00: status: { DRDY ERR } [ 8.286508] ata2.00: error: { UNC } [ 8.298738] ata2.00: configured for UDMA/133 [ 8.298749] sd 1:0:0:0: [sdb] tag#14 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 8.298752] sd 1:0:0:0: [sdb] tag#14 Sense Key : Medium Error [current] [ 8.298754] sd 1:0:0:0: [sdb] tag#14 Add. Sense: Unrecovered read error - auto reallocate failed [ 8.298757] sd 1:0:0:0: [sdb] tag#14 CDB: Read(10) 28 00 00 00 0a 56 00 00 02 00 [ 8.298758] blk_update_request: I/O error, dev sdb, sector 2646 [ 8.298818] Buffer I/O error on dev sdb1, logical block 299, async page read [ 8.298889] ata2: EH complete elided: lots of initialization/ACPI stuff... [ 15.630752] ata2.00: exception Emask 0x0 SAct 0x8000000 SErr 0x0 action 0x0 [ 15.630803] ata2.00: irq_stat 0x40000008 [ 15.630830] ata2.00: failed command: READ FPDMA QUEUED [ 15.630865] ata2.00: cmd 60/00:d8:00:0a:00/01:00:00:00:00/40 tag 27 ncq dma 131072 in res 41/40:00:50:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 15.630951] ata2.00: status: { DRDY ERR } [ 15.630976] ata2.00: error: { UNC } [ 15.643146] ata2.00: configured for UDMA/133 [ 15.643162] sd 1:0:0:0: [sdb] tag#27 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 15.643165] sd 1:0:0:0: [sdb] tag#27 Sense Key : Medium Error [current] [ 15.643167] sd 1:0:0:0: [sdb] tag#27 Add. Sense: Unrecovered read error - auto reallocate failed [ 15.643170] sd 1:0:0:0: [sdb] tag#27 CDB: Read(10) 28 00 00 00 0a 00 00 01 00 00 [ 15.643172] blk_update_request: I/O error, dev sdb, sector 2640 [ 15.643229] ata2: EH complete [ 17.882935] ata2.00: exception Emask 0x0 SAct 0x3c00 SErr 0x0 action 0x0 [ 17.882984] ata2.00: irq_stat 0x40000008 [ 17.883011] ata2.00: failed command: READ FPDMA QUEUED [ 17.883045] ata2.00: cmd 60/02:50:50:0a:00/00:00:00:00:00/40 tag 10 ncq dma 1024 in res 41/40:00:50:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 17.883130] ata2.00: status: { DRDY ERR } [ 17.883155] ata2.00: error: { UNC } [ 17.895264] ata2.00: configured for UDMA/133 [ 17.895276] sd 1:0:0:0: [sdb] tag#10 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 17.895279] sd 1:0:0:0: [sdb] tag#10 Sense Key : Medium Error [current] [ 17.895281] sd 1:0:0:0: [sdb] tag#10 Add. Sense: Unrecovered read error - auto reallocate failed [ 17.895284] sd 1:0:0:0: [sdb] tag#10 CDB: Read(10) 28 00 00 00 0a 50 00 00 02 00 [ 17.895286] blk_update_request: I/O error, dev sdb, sector 2640 [ 17.895328] Buffer I/O error on dev sdb1, logical block 296, async page read [ 17.895377] ata2: EH complete [ 20.127110] ata2.00: exception Emask 0x0 SAct 0x60000001 SErr 0x0 action 0x0 [ 20.127161] ata2.00: irq_stat 0x40000008 [ 20.127188] ata2.00: failed command: READ FPDMA QUEUED [ 20.127223] ata2.00: cmd 60/02:e8:56:0a:00/00:00:00:00:00/40 tag 29 ncq dma 1024 in res 41/40:00:56:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 20.127307] ata2.00: status: { DRDY ERR } [ 20.127332] ata2.00: error: { UNC } [ 20.139450] ata2.00: configured for UDMA/133 [ 20.139468] sd 1:0:0:0: [sdb] tag#29 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 20.139470] sd 1:0:0:0: [sdb] tag#29 Sense Key : Medium Error [current] [ 20.139473] sd 1:0:0:0: [sdb] tag#29 Add. Sense: Unrecovered read error - auto reallocate failed [ 20.139475] sd 1:0:0:0: [sdb] tag#29 CDB: Read(10) 28 00 00 00 0a 56 00 00 02 00 [ 20.139477] blk_update_request: I/O error, dev sdb, sector 2646 [ 20.139517] Buffer I/O error on dev sdb1, logical block 299, async page read [ 20.139564] ata2: EH complete elided: more initialization, sound, other partitions being mounted cleanly etc... [ 149.808922] XFS (sdb1): Offline file system operation in progress! [ 149.808973] XFS (sdb1): Metadata corruption detected at xfs_sb_read_verify+0x111/0x140 [xfs], xfs_sb block 0xffffffffffffffff [ 149.808978] XFS (sdb1): Unmount and run xfs_repair [ 149.808980] XFS (sdb1): First 64 bytes of corrupted metadata buffer: [ 149.808983] ffff9b459bf9a000: 58 46 53 42 00 00 10 00 00 00 00 00 09 50 af 55 XFSB.........P.U [ 149.808985] ffff9b459bf9a010: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ [ 149.808987] ffff9b459bf9a020: 53 df 87 aa a0 e0 42 30 ba 5f 36 4c c5 42 be e7 S.....B0._6L.B.. [ 149.808989] ffff9b459bf9a030: 00 00 00 00 08 00 00 05 ff ff ff ff ff ff ff ff ................ [ 149.809036] XFS (sdb1): SB validate failed with error -117. [ 234.394294] ata2.00: exception Emask 0x0 SAct 0x400 SErr 0x0 action 0x0 [ 234.394299] ata2.00: irq_stat 0x40000008 [ 234.394303] ata2.00: failed command: READ FPDMA QUEUED [ 234.394308] ata2.00: cmd 60/00:50:00:08:00/04:00:00:00:00/40 tag 10 ncq dma 524288 in res 41/40:00:50:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> [ 234.394311] ata2.00: status: { DRDY ERR } [ 234.394313] ata2.00: error: { UNC } [ 234.406484] ata2.00: configured for UDMA/133 [ 234.406502] sd 1:0:0:0: [sdb] tag#10 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE [ 234.406505] sd 1:0:0:0: [sdb] tag#10 Sense Key : Medium Error [current] [ 234.406507] sd 1:0:0:0: [sdb] tag#10 Add. Sense: Unrecovered read error - auto reallocate failed [ 234.406510] sd 1:0:0:0: [sdb] tag#10 CDB: Read(10) 28 00 00 00 08 00 00 04 00 00 [ 234.406512] blk_update_request: I/O error, dev sdb, sector 2640 [ 234.406538] ata2: EH complete Also: # xfs_repair -n /dev/sdb1 Phase 1 - find and verify superblock... superblock read failed, offset 0, size 524288, ag 0, rval -1 fatal error -- Input/output error I didn't have time to investigate more so I didn't even try smartctl on it. But looks like that disk is dead, doesn't it? :-( -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2019-04-02 16:23 ` Luciano ES @ 2019-04-02 16:42 ` Eric Sandeen 2019-04-02 18:54 ` Chris Murphy 2019-04-02 21:37 ` Dave Chinner 2 siblings, 0 replies; 18+ messages in thread From: Eric Sandeen @ 2019-04-02 16:42 UTC (permalink / raw) To: Luciano ES, XFS mailing list On 4/2/19 11:23 AM, Luciano ES wrote: > The other disk failed again. Here is some dmesg: > > [ 1.243231] sd 1:0:0:0: [sdb] 1250263728 512-byte logical blocks: (640 GB/596 GiB) > [ 1.243274] sd 1:0:0:0: [sdb] Write Protect is off > [ 1.243276] sd 1:0:0:0: [sdb] Mode Sense: 00 3a 00 00 > [ 1.243299] sd 1:0:0:0: [sdb] Write cache: enabled, read cache: enabled, doesn't support DPO or FUA > [ 1.270253] sdb: sdb1 > [ 1.271133] sd 1:0:0:0: [sdb] Attached SCSI disk > [ 1.272908] sda: sda1 sda2 sda3 sda4 sda5 sda6 > [ 1.273680] sd 0:0:0:0: [sda] Attached SCSI disk > elided: lots of stuff about probing USB ports... > [ 3.777831] ata2.00: exception Emask 0x0 SAct 0x1000 SErr 0x0 action 0x0 > [ 3.777882] ata2.00: irq_stat 0x40000008 > [ 3.777926] ata2.00: failed command: READ FPDMA QUEUED > [ 3.777974] ata2.00: cmd 60/00:60:00:0a:00/01:00:00:00:00/40 tag 12 ncq dma 131072 in > res 41/40:00:50:0a:00/00:00:00:00:00/40 Emask 0x409 (media error) <F> > [ 3.778038] ata2.00: status: { DRDY ERR } > [ 3.778080] ata2.00: error: { UNC } > [ 3.790298] ata2.00: configured for UDMA/133 > [ 3.790312] sd 1:0:0:0: [sdb] tag#12 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE This continues to look like a hardware/drive failure, not an xfs problem. ... > Also: > # xfs_repair -n /dev/sdb1 > Phase 1 - find and verify superblock... > superblock read failed, offset 0, size 524288, ag 0, rval -1 > fatal error -- Input/output error as does this. > > I didn't have time to investigate more so I didn't even try smartctl on it. > But looks like that disk is dead, doesn't it? yep! > :-( > > ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2019-04-02 16:23 ` Luciano ES 2019-04-02 16:42 ` Eric Sandeen @ 2019-04-02 18:54 ` Chris Murphy 2019-04-02 21:37 ` Dave Chinner 2 siblings, 0 replies; 18+ messages in thread From: Chris Murphy @ 2019-04-02 18:54 UTC (permalink / raw) To: Luciano ES; +Cc: XFS mailing list On Tue, Apr 2, 2019 at 10:24 AM Luciano ES <lucmove@gmail.com> wrote: > [ 3.790321] sd 1:0:0:0: [sdb] tag#12 CDB: Read(10) 28 00 00 00 0a 00 00 01 00 00 > [ 3.790323] blk_update_request: I/O error, dev sdb, sector 2640 Common bad sector error, includes the LBA for the sector. There's a scant chance it's recoverable if the drive supports configurable SCT ERC, and just happens to have a low timeout value (common on NAS and enterprise drives). You can check it with: # smartctl -l scterc /dev/sdb # cat /sys/block/sdb/device/timeout These are two different things. The first is internal to the drive (firmware). The second is the kernel's command queue timer for that block device. If the SCT ERC value is something short like 70 deciseconds, you can try disabling it. # smartctl -l scterc,0,0 /dev/sdb And then increase the kernel command timer to something ridiculous like 180 seconds. # echo 180 > /sys/block/sdb/device/timeout Try your repair again. xfs_repair might appear to hang. My guess is it fails again right away. But there's some chance giving the drive more time to recover that sector, and it might just do it. Thing is, if there's no problem with the contents on that bad sector, it won't likely be overwritten, and it only gets "repaired" by an overwrite. Once the xfs_repair completes and if successful, you'll want to mount the file system rw, make some trivial change like touching a file, then unmount. A reboot will reset all of these values, and you'll quickly learn if this is fixed. If not...well cross that bridge later depending on what results you get. > [ 8.298754] sd 1:0:0:0: [sdb] tag#14 Add. Sense: Unrecovered read error - auto reallocate failed > [ 8.298757] sd 1:0:0:0: [sdb] tag#14 CDB: Read(10) 28 00 00 00 0a 56 00 00 02 00 > [ 8.298758] blk_update_request: I/O error, dev sdb, sector 2646 2640 and 2646 are likely the same 4096 physical sector; they get different values because of 512 byte sector emulation. What do you get for # blockdev --getss --getpbsz > I didn't have time to investigate more so I didn't even try smartctl on it. > But looks like that disk is dead, doesn't it? > :-( Uncertain. Some number of bad sectors are considered acceptable by the manufacturer if they remap. Well, yours went bad before the remap so I'd complain if the drive is under warranty. But that's separate from recovery... # smartctl -x /dev/sdb -- Chris Murphy ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: File system corruption in two hard disks 2019-04-02 16:23 ` Luciano ES 2019-04-02 16:42 ` Eric Sandeen 2019-04-02 18:54 ` Chris Murphy @ 2019-04-02 21:37 ` Dave Chinner 2 siblings, 0 replies; 18+ messages in thread From: Dave Chinner @ 2019-04-02 21:37 UTC (permalink / raw) To: Luciano ES; +Cc: XFS mailing list On Tue, Apr 02, 2019 at 01:23:57PM -0300, Luciano ES wrote: > On Tue, 2 Apr 2019 08:32:26 +1100, Dave Chinner wrote: > > At this point, I suspect your problems are the USB enclosure your > > are using. Plug the disks directly into a SATA port and see if that > > makes the IO errors go away. This looks like hardware problems, not > > XFS issues... > > ************************** > > Hi. Thank you for your kind attention again. > > I tested the disks directly connected to the SATA ports this time. > One of them worked flawlessly. That's good! > Damn! It's the third USB enclosure that fails on me in one year! That's about par for the course, I think. I simply don't trust USB storage enclosures at all... > The other disk failed again. Here is some dmesg: .... > [ 3.790315] sd 1:0:0:0: [sdb] tag#12 Sense Key : Medium Error [current] > [ 3.790318] sd 1:0:0:0: [sdb] tag#12 Add. Sense: Unrecovered read error - auto reallocate failed Basically, this is saying the sector on the disk is bad, and there is no spare sectors on the disk for recovery of that sector. i.e the sector is now permanently dead. The drive is essentially unusable at this point - it will only continue to get worse and lose more data. AFAICT, the only way you'll get anything off this drive is via ddrescue to image it and then perform data recovery on the decayed corpse it extracts.... Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* XFS file system corruption, refuses to mount 2018-12-11 20:32 I got file system corruption with XFS Luciano ES ` (2 preceding siblings ...) 2019-03-31 22:49 ` File system corruption in two hard disks Dave Chinner @ 2019-08-03 0:53 ` Luciano ES 2019-08-03 1:11 ` Darrick J. Wong 2019-08-15 20:32 ` XFS file system " Luciano ES 5 siblings, 0 replies; 18+ messages in thread From: Luciano ES @ 2019-08-03 0:53 UTC (permalink / raw) To: linux-xfs I've had this internal disk running for a long time. I had to disconnect it from the SATA and power plugs for two days. Now it won't mount. mount: wrong fs type, bad option, bad superblock on /dev/mapper/cab3, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. I get this in dmesg: [ 30.301450] XFS (dm-1): Mounting V5 Filesystem [ 30.426206] XFS (dm-1): Corruption warning: Metadata has LSN (16:367696) ahead of current LSN (16:367520). Please unmount and run xfs_repair (>= v4.3) to resolve. [ 30.426209] XFS (dm-1): log mount/recovery failed: error -22 [ 30.426310] XFS (dm-1): log mount failed Note that the entire disk is encrypted with cryptsetup/LUKS, which is working fine. Wrong passwords fail. The right password opens it. But then it refuses to mount. This has been happening a lot to me with XFS file systems. Why is this happening? Is there something I can do to recover the data? -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: XFS file system corruption, refuses to mount 2018-12-11 20:32 I got file system corruption with XFS Luciano ES ` (3 preceding siblings ...) 2019-08-03 0:53 ` XFS file system corruption, refuses to mount Luciano ES @ 2019-08-03 1:11 ` Darrick J. Wong 2019-08-03 1:53 ` Luciano ES 2019-08-03 8:05 ` Luciano ES 2019-08-15 20:32 ` XFS file system " Luciano ES 5 siblings, 2 replies; 18+ messages in thread From: Darrick J. Wong @ 2019-08-03 1:11 UTC (permalink / raw) To: Luciano ES; +Cc: linux-xfs On Fri, Aug 02, 2019 at 09:53:56PM -0300, Luciano ES wrote: > I've had this internal disk running for a long time. I had to > disconnect it from the SATA and power plugs for two days. > Now it won't mount. > > mount: wrong fs type, bad option, bad superblock on /dev/mapper/cab3, > missing codepage or helper program, or other error > In some cases useful info is found in syslog - try > dmesg | tail or so. > > I get this in dmesg: > > [ 30.301450] XFS (dm-1): Mounting V5 Filesystem > [ 30.426206] XFS (dm-1): Corruption warning: Metadata has LSN > (16:367696) ahead of current LSN (16:367520). Please unmount and run > xfs_repair (>= v4.3) to resolve. Hm, I think this means the superblock LSN is behind the log LSN, which could mean that... software is buggy? The disk didn't flush its cache before it was unplugged? Something else? What kernel & xfsprogs? And how did you disconnect it from the power plugs? > [ 30.426209] XFS (dm-1): log mount/recovery failed: error -22 > [ 30.426310] XFS (dm-1): log mount failed > > Note that the entire disk is encrypted with cryptsetup/LUKS, > which is working fine. Wrong passwords fail. The right password > opens it. But then it refuses to mount. > > This has been happening a lot to me with XFS file systems. > Why is this happening? > > Is there something I can do to recover the data? Try xfs_repair -n to see what it would do if you ran repair? --D > -- > Luciano ES > >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: XFS file system corruption, refuses to mount 2019-08-03 1:11 ` Darrick J. Wong @ 2019-08-03 1:53 ` Luciano ES 2019-08-03 5:35 ` Dave Chinner 2019-08-03 8:05 ` Luciano ES 1 sibling, 1 reply; 18+ messages in thread From: Luciano ES @ 2019-08-03 1:53 UTC (permalink / raw) To: XFS mailing list On Fri, 2 Aug 2019 18:11:06 -0700, Darrick J. Wong wrote: > On Fri, Aug 02, 2019 at 09:53:56PM -0300, Luciano ES wrote: > > I've had this internal disk running for a long time. I had to > > disconnect it from the SATA and power plugs for two days. > > Now it won't mount. > > > > mount: wrong fs type, bad option, bad superblock > > on /dev/mapper/cab3, missing codepage or helper program, or other > > error In some cases useful info is found in syslog - try > > dmesg | tail or so. > > > > I get this in dmesg: > > > > [ 30.301450] XFS (dm-1): Mounting V5 Filesystem > > [ 30.426206] XFS (dm-1): Corruption warning: Metadata has LSN > > (16:367696) ahead of current LSN (16:367520). Please unmount and run > > xfs_repair (>= v4.3) to resolve. > > Hm, I think this means the superblock LSN is behind the log LSN, which > could mean that... software is buggy? The disk didn't flush its cache > before it was unplugged? Something else? > > What kernel & xfsprogs? Debian 4.9.0-3-amd64, xfsprogs 4.9.0. > And how did you disconnect it from the power plugs? I shut down the machine, opened the box's cover and disconnected the data and power cables. I used them on the CD/DVD drive, which I never use but this time I had to. The hard disk drive remained quiet in its bay. Then I shut down the machine and reconnected the cables to the hard disk and this problem came up. I also tried another cable and another SATA port, to no avail. > > [ 30.426209] XFS (dm-1): log mount/recovery failed: error -22 > > [ 30.426310] XFS (dm-1): log mount failed > > > > Note that the entire disk is encrypted with cryptsetup/LUKS, > > which is working fine. Wrong passwords fail. The right password > > opens it. But then it refuses to mount. > > > > This has been happening a lot to me with XFS file systems. > > Why is this happening? > > > > Is there something I can do to recover the data? > > Try xfs_repair -n to see what it would do if you ran repair? I tried and got this output: Phase 1 - find and verify superblock... bad primary superblock - bad magic number !!! attempting to find secondary superblock... and it's been printing an endless stream of dots for a very long time. I'm about to go to bed and let this running overnight. It looks like it has a long way to go. -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: XFS file system corruption, refuses to mount 2019-08-03 1:53 ` Luciano ES @ 2019-08-03 5:35 ` Dave Chinner 0 siblings, 0 replies; 18+ messages in thread From: Dave Chinner @ 2019-08-03 5:35 UTC (permalink / raw) To: Luciano ES; +Cc: XFS mailing list On Fri, Aug 02, 2019 at 10:53:20PM -0300, Luciano ES wrote: > On Fri, 2 Aug 2019 18:11:06 -0700, Darrick J. Wong wrote: > > > On Fri, Aug 02, 2019 at 09:53:56PM -0300, Luciano ES wrote: > > > I've had this internal disk running for a long time. I had to > > > disconnect it from the SATA and power plugs for two days. > > > Now it won't mount. > > > > > > mount: wrong fs type, bad option, bad superblock > > > on /dev/mapper/cab3, missing codepage or helper program, or other > > > error In some cases useful info is found in syslog - try > > > dmesg | tail or so. > > > > > > I get this in dmesg: > > > > > > [ 30.301450] XFS (dm-1): Mounting V5 Filesystem > > > [ 30.426206] XFS (dm-1): Corruption warning: Metadata has LSN > > > (16:367696) ahead of current LSN (16:367520). Please unmount and run > > > xfs_repair (>= v4.3) to resolve. > > > > Hm, I think this means the superblock LSN is behind the log LSN, which > > could mean that... software is buggy? The disk didn't flush its cache > > before it was unplugged? Something else? Given the difference in LSNs is only 176 sectors, it seems very likely that the drive isn't honoring device flushes and so the log writes that moved the tail haven't hit the disk before the metadata which was issued after the log write completed... What is the drive you are using (brand, model number age, etc)? What is the output from demsg when the device is first discovered on boot? > > What kernel & xfsprogs? > > Debian 4.9.0-3-amd64, xfsprogs 4.9.0. > > > > And how did you disconnect it from the power plugs? > > I shut down the machine, opened the box's cover and disconnected the > data and power cables. I used them on the CD/DVD drive, which I never > use but this time I had to. The hard disk drive remained quiet in its > bay. Then I shut down the machine and reconnected the cables to the > hard disk and this problem came up. I also tried another cable and > another SATA port, to no avail. > > > > > [ 30.426209] XFS (dm-1): log mount/recovery failed: error -22 > > > [ 30.426310] XFS (dm-1): log mount failed > > > > > > Note that the entire disk is encrypted with cryptsetup/LUKS, > > > which is working fine. Wrong passwords fail. The right password > > > opens it. But then it refuses to mount. > > > > > > This has been happening a lot to me with XFS file systems. > > > Why is this happening? > > > > > > Is there something I can do to recover the data? > > > > Try xfs_repair -n to see what it would do if you ran repair? > > I tried and got this output: > > > Phase 1 - find and verify superblock... > bad primary superblock - bad magic number !!! > > attempting to find secondary superblock... Unlock the encrypted device first? What does blkid tell you about that device? Cheers, Dave. -- Dave Chinner david@fromorbit.com ^ permalink raw reply [flat|nested] 18+ messages in thread
* Re: XFS file system corruption, refuses to mount 2019-08-03 1:11 ` Darrick J. Wong 2019-08-03 1:53 ` Luciano ES @ 2019-08-03 8:05 ` Luciano ES 1 sibling, 0 replies; 18+ messages in thread From: Luciano ES @ 2019-08-03 8:05 UTC (permalink / raw) To: XFS mailing list On Fri, 2 Aug 2019 18:11:06 -0700, Darrick J. Wong wrote: > > Is there something I can do to recover the data? > > Try xfs_repair -n to see what it would do if you ran repair? I tried it. It took a very long time. Phase 1 - find and verify superblock... bad primary superblock - bad magic number !!! attempting to find secondary superblock... ...................................... (a million dots) Sorry, could not find valid secondary superblock Exiting now. I tested the file system (unlocked) with the fsck, lsblk, blkid and file commands, all of which confirm that it is an XFS file system. It just won't mount. Is there any recovery procedure? -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
* XFS file system refuses to mount 2018-12-11 20:32 I got file system corruption with XFS Luciano ES ` (4 preceding siblings ...) 2019-08-03 1:11 ` Darrick J. Wong @ 2019-08-15 20:32 ` Luciano ES 5 siblings, 0 replies; 18+ messages in thread From: Luciano ES @ 2019-08-15 20:32 UTC (permalink / raw) To: linux-xfs If this sounds familiar, it's because I've been here before about this same problem. New events are at the end of this message. I've had this internal disk running for a long time. I had to disconnect it from the SATA and power plugs for two days. Now it won't mount. mount: wrong fs type, bad option, bad superblock on /dev/mapper/cab3, missing codepage or helper program, or other error In some cases useful info is found in syslog - try dmesg | tail or so. I get this in dmesg: [ 30.301450] XFS (dm-1): Mounting V5 Filesystem [ 30.426206] XFS (dm-1): Corruption warning: Metadata has LSN (16:367696) ahead of current LSN (16:367520). Please unmount and run xfs_repair (>= v4.3) to resolve. [ 30.426209] XFS (dm-1): log mount/recovery failed: error -22 [ 30.426310] XFS (dm-1): log mount failed Note that the entire disk is encrypted with cryptsetup/LUKS, which is working fine. Wrong passwords fail. The right password opens it. But then the partition refuses to mount. I've tried xfs_repair -n on it and got this: Phase 1 - find and verify superblock... bad primary superblock - bad magic number !!! attempting to find secondary superblock... It prints an endless stream of dots for a very long time then it says no file system has been found. Is there something I can do to recover the data? The new event is that I am cloning the partition to another one with dd right now as I type, and I would like to know how I can use that to fix the partition so it's mountable or if I can at least scan it and rescue any files. Thank you in advance. -- Luciano ES >> ^ permalink raw reply [flat|nested] 18+ messages in thread
end of thread, other threads:[~2019-08-15 20:32 UTC | newest] Thread overview: 18+ messages (download: mbox.gz / follow: Atom feed) -- links below jump to the message on this page -- 2018-12-11 20:32 I got file system corruption with XFS Luciano ES 2018-12-11 20:57 ` Eric Sandeen 2018-12-11 22:54 ` Luciano ES 2018-12-12 1:01 ` Eric Sandeen 2018-12-11 21:27 ` Dave Chinner 2019-03-31 22:49 ` File system corruption in two hard disks Dave Chinner 2019-04-01 21:13 ` Luciano ES 2019-04-01 21:32 ` Dave Chinner 2019-04-02 16:23 ` Luciano ES 2019-04-02 16:42 ` Eric Sandeen 2019-04-02 18:54 ` Chris Murphy 2019-04-02 21:37 ` Dave Chinner 2019-08-03 0:53 ` XFS file system corruption, refuses to mount Luciano ES 2019-08-03 1:11 ` Darrick J. Wong 2019-08-03 1:53 ` Luciano ES 2019-08-03 5:35 ` Dave Chinner 2019-08-03 8:05 ` Luciano ES 2019-08-15 20:32 ` XFS file system " Luciano ES
This is a public inbox, see mirroring instructions for how to clone and mirror all data and code used for this inbox; as well as URLs for NNTP newsgroup(s).