Re: Rebuilding 24TB Raid5 array (was btrfs corruption: parent transid verify failed + open_ctree failed)

[Marc MERLIN <marc@merlins.org>]

On Wed, Jun 08, 2022 at 06:46:53PM -0400, Josef Bacik wrote:
> Ok I've added some stuff to fix the device extents.  Go ahead and run
> with --repair and lets see how that goes.  After that finishes run
> again without --repair so we can see what's still broken, I imagine
> I'll have to clean some other stuff up.  Thanks,

Can't determine the filetype for inode 69105, assume it is a normal file
Can't get file type for inode 69105, using FILE as fallback
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 69105
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 69135
reset isize for dir 69136 root 164823
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 69136
Trying to rebuild inode:69252
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 69252
Trying to rebuild inode:74108
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 74108
Trying to rebuild inode:74132
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 74132
Trying to rebuild inode:74193
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 74193
Trying to rebuild inode:74838
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 74838
Trying to rebuild inode:76221
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 76221
Trying to rebuild inode:76328
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 76328
Trying to rebuild inode:76329
Moving file f to 'lost+found' dir since it has no valid backref
Fixed the nlink of inode 76329
found 43825717248 bytes used, error(s) found
total csum bytes: 0
total tree bytes: 10846208
total fs tree bytes: 6848512
total extent tree bytes: 1146880
btree space waste bytes: 3705949
file data blocks allocated: 67228672000
 referenced 67225485312
gargamel:/var/local/src/btrfs-progs-josefbacik# mount -o ro,recovery /dev/mapper/dshelf1 /mnt/mnt
mount: /mnt/mnt: wrong fs type, bad option, bad superblock on /dev/mapper/dshelf1, missing codepage or helper program, or other error.
gargamel:/var/local/src/btrfs-progs-josefbacik# dmtail
[3890613.672704] BTRFS info (device dm-1): flagging fs with big metadata feature
[3890613.694891] BTRFS warning (device dm-1): 'recovery' is deprecated, use 'rescue=usebackuproot' instead
[3890613.759915] BTRFS info (device dm-1): trying to use backup root at mount time
[3890613.782884] BTRFS info (device dm-1): disk space caching is enabled
[3890613.802960] BTRFS info (device dm-1): has skinny extents
[3890613.826455] BTRFS error (device dm-1): super_num_devices 1 mismatch with num_devices 0 found here
[3890613.855092] BTRFS error (device dm-1): failed to read chunk tree: -22
[3890613.876716] BTRFS error (device dm-1): open_ctree failed

btrfs check ends with:
root 164624 inode 73099 errors 1400, nbytes wrong, some csum missing
root 164624 inode 73100 errors 1400, nbytes wrong, some csum missing
        unresolved ref dir 791 index 0 namelen 25 name f filetype 1 errors 6, no dir index, no inode ref
        unresolved ref dir 3676 index 0 namelen 62 name f filetype 1 errors 6, no dir index, no inode ref
root 164629 inode 3752 errors 200, dir isize wrong
        unresolved ref dir 73103 index 540 namelen 4 name f filetype mismatch
root 164629 inode 3965 errors 200, dir isize wrong
        unresolved ref dir 4179 index 0 namelen 10 name f filetype 1 errors 6, no dir index, no inode ref
root 164629 inode 4549 errors 200, dir isize wrong
        unresolved ref dir 4698 index 0 namelen 69 name f filetype 1 errors 6, no dir index, no inode ref
        unresolved ref dir 5506 index 0 namelen 53 name f filetype 1 errors 6, no dir index, no inode ref
        unresolved ref dir 5546 index 0 namelen 57 name f filetype 1 errors 6, no dir index, no inode ref 
root 164629 inode 39921 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 759824384, len: 2143326208
        unresolved ref dir 10205 index 356 namelen 34 name f filetype 0 errors 3, no dir item, no dir index
root 164629 inode 40537 errors 1400, nbytes wrong, some csum missing
root 164629 inode 72418 errors 1400, nbytes wrong, some csum missing
root 164629 inode 72429 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 1333788672, len: 3217154048
        unresolved ref dir 72438 index 4 namelen 49 name f filetype 0 errors 3, no dir item, no dir index
root 164629 inode 72433 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 892338176, len: 14957748224
        unresolved ref dir 34951 index 13 namelen 46 name f filetype 0 errors 3, no dir item, no dir index
root 164629 inode 72587 errors 200, dir isize wrong
root 164629 inode 72588 errors 1000, some csum missing
root 164629 inode 72592 errors 200, dir isize wrong
root 164629 inode 72593 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 42074112, len: 413642752
        unresolved ref dir 75036 index 19 namelen 45 name f filetype 0 errors 3, no dir item, no dir index
root 164629 inode 72639 errors 200, dir isize wrong
root 164629 inode 72640 errors 200, dir isize wrong
root 164629 inode 72672 errors 200, dir isize wrong
        unresolved ref dir 73103 index 1386 namelen 5 name f filetype mismatch
root 164629 inode 73001 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73006 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73009 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 13869056, len: 53100544
        unresolved ref dir 72672 index 134 namelen 26 name f filetype 1 errors 1, no dir item
root 164629 inode 73045 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73066 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73067 errors 1000, some csum missing
root 164629 inode 73082 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73083 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 197263360, len: 399777792 
        unresolved ref dir 3747 index 30 namelen 70 name f filetype 0 errors 3, no dir item, no dir index
root 164629 inode 73086 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73094 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes:
        start: 0, len: 524288 
        unresolved ref dir 74963 index 32 namelen 56 name f filetype 0 errors 3, no dir item, no dir index 
root 164629 inode 73097 errors 3500, file extent discount, nbytes wrong, some csum missing, link count wrong
Found file extent holes: 
        start: 0, len: 524288
        unresolved ref dir 74963 index 36 namelen 56 name f filetype 0 errors 3, no dir item, no dir index
root 164629 inode 73099 errors 1400, nbytes wrong, some csum missing
root 164629 inode 73100 errors 1400, nbytes wrong, some csum missing
        unresolved ref dir 791 index 0 namelen 25 name f filetype 1 errors 6, no dir index, no inode ref
        unresolved ref dir 3676 index 0 namelen 62 name f filetype 1 errors 6, no dir index, no inode ref
ERROR: errors found in fs roots
Opening filesystem to check...
Checking filesystem on /dev/mapper/dshelf1
UUID: 96539b8c-ccc9-47bf-9e6c-29305890941e
cache and super generation don't match, space cache will be invalidated
found 21916315648 bytes used, error(s) found
total csum bytes: 0
total tree bytes: 8880128
total fs tree bytes: 6799360
total extent tree bytes: 606208
btree space waste bytes: 2518033
file data blocks allocated: 36729012224
 referenced 36727418880

-- 
"A mouse is a device used to point at the xterm you want to type in" - A.S.R.
 
Home page: http://marc.merlins.org/