linux-btrfs.vger.kernel.org archive mirror
 help / color / mirror / Atom feed
* parent transid verify failed / FS wont mount / help please!
@ 2019-03-25 22:26 berodual_xyz
  2019-03-25 22:38 ` Hugo Mills
  2019-03-26  3:47 ` Andrei Borzenkov
  0 siblings, 2 replies; 7+ messages in thread
From: berodual_xyz @ 2019-03-25 22:26 UTC (permalink / raw)
  To: linux-btrfs

Dear all,

on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...) I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).

Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.

"btrfs check" gives the following error:

##
$ btrfs check -b /dev/sdd
Opening filesystem to check...
parent transid verify failed on 1048576 wanted 60234 found 60230
parent transid verify failed on 1048576 wanted 60234 found 60230
Ignoring transid failure
parent transid verify failed on 55432763981824 wanted 60233 found 60235
parent transid verify failed on 55432763981824 wanted 60233 found 60235
Ignoring transid failure
parent transid verify failed on 55432753725440 wanted 60232 found 60235
parent transid verify failed on 55432753725440 wanted 60232 found 60235
Ignoring transid failure
parent transid verify failed on 55432764063744 wanted 60233 found 60235
parent transid verify failed on 55432764063744 wanted 60233 found 60235
Ignoring transid failure
Checking filesystem on /dev/sdd
UUID: 8b19ff46-3f42-4f51-be6b-5fc8a7d8f2cd
[1/7] checking root items
Error: could not find extent items for root 268
ERROR: failed to repair root items: No such file or directory
##

I have a complete "dump tree" zip but its a couple of GB.

Some sources on the net say to run "btrfs check --init-extent-tree" but I would like to reach out first.

btrfs progs version is 4.20.2 and kernel is 4.20.17

Thank you for any help! Much appreciated!

Marcel


Sent with ProtonMail Secure Email.


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: parent transid verify failed / FS wont mount / help please!
  2019-03-25 22:26 parent transid verify failed / FS wont mount / help please! berodual_xyz
@ 2019-03-25 22:38 ` Hugo Mills
       [not found]   ` <kxQglfkZm7A_i8CXh0JVrJ9H0MF4vFVJWiT8BWRBc8NPM24nZC-vc6sIrts3ZR4T8HEvRTfPv1yz4lPotIuO1JiPYOk60cHMQqHcTFkDfZ4=@protonmail.com>
  2019-03-26  3:47 ` Andrei Borzenkov
  1 sibling, 1 reply; 7+ messages in thread
From: Hugo Mills @ 2019-03-25 22:38 UTC (permalink / raw)
  To: berodual_xyz; +Cc: linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 2830 bytes --]

On Mon, Mar 25, 2019 at 10:26:29PM +0000, berodual_xyz wrote:
> Dear all,
> 
> on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...) 

   Ouch. I think the only thing you could have done to make the FS
more fragile is mounting with nobarrier(*). Frankly, anything you're
getting off it is a bonus. RAID-0 gives you no duplicate copy,
nodatacow implies nodatasum, and nodatasum doesn't even give you the
ability to *detect* data corruption, let alone fix it.

   With that configuration, I'd say pretty much by definition the
contents of the FS are considered to be discardable.

   Restoring from backups is the recommended approach with transid
failures.

(*) Don't do that.

> I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
> The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).

   Given how close the transids are, have you tried
"ro,usebackuproot"? That's about your only other option at this
point. But, if btrfs restore isn't working, then usebacuproot probably
won't either.

> Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.
> 
> "btrfs check" gives the following error:
> 
> ##
> $ btrfs check -b /dev/sdd
> Opening filesystem to check...
> parent transid verify failed on 1048576 wanted 60234 found 60230
> parent transid verify failed on 1048576 wanted 60234 found 60230
> Ignoring transid failure
> parent transid verify failed on 55432763981824 wanted 60233 found 60235
> parent transid verify failed on 55432763981824 wanted 60233 found 60235
> Ignoring transid failure
> parent transid verify failed on 55432753725440 wanted 60232 found 60235
> parent transid verify failed on 55432753725440 wanted 60232 found 60235
> Ignoring transid failure
> parent transid verify failed on 55432764063744 wanted 60233 found 60235
> parent transid verify failed on 55432764063744 wanted 60233 found 60235
> Ignoring transid failure
> Checking filesystem on /dev/sdd
> UUID: 8b19ff46-3f42-4f51-be6b-5fc8a7d8f2cd
> [1/7] checking root items
> Error: could not find extent items for root 268
> ERROR: failed to repair root items: No such file or directory
> ##
> 
> I have a complete "dump tree" zip but its a couple of GB.
> 
> Some sources on the net say to run "btrfs check --init-extent-tree" but I would like to reach out first.

   Probably not wise. "Sources on the net" are frequently wrong when
it comes to btrfs recovery.

> btrfs progs version is 4.20.2 and kernel is 4.20.17

   At least those aren't out of date. The only positive thing here...

   Hugo.

-- 
Hugo Mills             | I gave up smoking, drinking and sex once. It was the
hugo@... carfax.org.uk | scariest 20 minutes of my life.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: parent transid verify failed / FS wont mount / help please!
       [not found]   ` <kxQglfkZm7A_i8CXh0JVrJ9H0MF4vFVJWiT8BWRBc8NPM24nZC-vc6sIrts3ZR4T8HEvRTfPv1yz4lPotIuO1JiPYOk60cHMQqHcTFkDfZ4=@protonmail.com>
@ 2019-03-25 22:51     ` berodual_xyz
  2019-03-25 22:56       ` Hugo Mills
  2019-03-25 22:55     ` Hugo Mills
  1 sibling, 1 reply; 7+ messages in thread
From: berodual_xyz @ 2019-03-25 22:51 UTC (permalink / raw)
  To: linux-btrfs

Running "btrfs check" on the 3rd of the 4 devices the volume consists of crashes with a trace:

##
$ btrfs check --readonly /dev/sdd
Opening filesystem to check...
parent transid verify failed on 1048576 wanted 60234 found 60230
parent transid verify failed on 1048576 wanted 60234 found 60230
Ignoring transid failure
volumes.c:1762: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1
btrfs[0x426fdc]
btrfs(btrfs_chunk_readonly+0x98)[0x429acd]
btrfs(btrfs_read_block_groups+0x1c1)[0x41cd44]
btrfs(btrfs_setup_all_roots+0x368)[0x416540]
btrfs[0x416a8a]
btrfs(open_ctree_fs_info+0xd0)[0x416bcc]
btrfs(cmd_check+0x591)[0x45f431]
btrfs(main+0x24a)[0x40ca02]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa320a44b35]
btrfs[0x40c509]
[1]    22848 abort      btrfs check --readonly /dev/sdd
##

Trying to mount "ro,usebackuproot" shows bad superblock and following errors in /var/log/messages:

##
33814.360633] BTRFS info (device sdd): trying to use backup root at mount time
[33814.360637] BTRFS info (device sdd): using free space tree
[33814.360638] BTRFS info (device sdd): has skinny extents
[33814.361708] BTRFS error (device sdd): parent transid verify failed on 1048576 wanted 60234 found 60230
[33814.361764] BTRFS error (device sdd): failed to read chunk root
[33814.373140] BTRFS error (device sdd): open_ctree failed
##


Again, thank you very much for all help!



Sent with ProtonMail Secure Email.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, March 25, 2019 11:44 PM, berodual_xyz <berodual_xyz@protonmail.com> wrote:

> Thank you very much Hugo,
>
> the underlying devices are based on HW raid6 and effectively "stitched" together. Loosing any of those would mean loosing all data, so much is clear.
>
> My concern was not so much bitrod / silent data corruption but I would not have expected disabled data checksumming to be a disadvantage at recovering from the supposed corruption now.
>
> Does anyone have any input on how to restore files based on inode no. from the tree dump that I have?
>
> "usebackuproot,ro" did not succeed either.
>
> Much appreciate the input!
>
> Sent with ProtonMail Secure Email.
>
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Monday, March 25, 2019 11:38 PM, Hugo Mills hugo@carfax.org.uk wrote:
>
> > On Mon, Mar 25, 2019 at 10:26:29PM +0000, berodual_xyz wrote:
> >
> > > Dear all,
> > > on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...)
> >
> > Ouch. I think the only thing you could have done to make the FS
> > more fragile is mounting with nobarrier(). Frankly, anything you're
> > getting off it is a bonus. RAID-0 gives you no duplicate copy,
> > nodatacow implies nodatasum, and nodatasum doesn't even give you the
> > ability to detect data corruption, let alone fix it.
> > With that configuration, I'd say pretty much by definition the
> > contents of the FS are considered to be discardable.
> > Restoring from backups is the recommended approach with transid
> > failures.
> > () Don't do that.
> >
> > > I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
> > > The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).
> >
> > Given how close the transids are, have you tried
> > "ro,usebackuproot"? That's about your only other option at this
> > point. But, if btrfs restore isn't working, then usebacuproot probably
> > won't either.
> >
> > > Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.
> > > "btrfs check" gives the following error:
> > > --
> > > $ btrfs check -b /dev/sdd
> > > Opening filesystem to check...
> > > parent transid verify failed on 1048576 wanted 60234 found 60230
> > > parent transid verify failed on 1048576 wanted 60234 found 60230
> > > Ignoring transid failure
> > > parent transid verify failed on 55432763981824 wanted 60233 found 60235
> > > parent transid verify failed on 55432763981824 wanted 60233 found 60235
> > > Ignoring transid failure
> > > parent transid verify failed on 55432753725440 wanted 60232 found 60235
> > > parent transid verify failed on 55432753725440 wanted 60232 found 60235
> > > Ignoring transid failure
> > > parent transid verify failed on 55432764063744 wanted 60233 found 60235
> > > parent transid verify failed on 55432764063744 wanted 60233 found 60235
> > > Ignoring transid failure
> > > Checking filesystem on /dev/sdd
> > > UUID: 8b19ff46-3f42-4f51-be6b-5fc8a7d8f2cd
> > > [1/7] checking root items
> > > Error: could not find extent items for root 268
> > > ERROR: failed to repair root items: No such file or directory
> > > --
> > > I have a complete "dump tree" zip but its a couple of GB.
> > > Some sources on the net say to run "btrfs check --init-extent-tree" but I would like to reach out first.
> >
> > Probably not wise. "Sources on the net" are frequently wrong when
> > it comes to btrfs recovery.
> >
> > > btrfs progs version is 4.20.2 and kernel is 4.20.17
> >
> > At least those aren't out of date. The only positive thing here...
> > Hugo.
> >
> > Hugo Mills | I gave up smoking, drinking and sex once. It was the
> > hugo@... carfax.org.uk | scariest 20 minutes of my life.
> > http://carfax.org.uk/ |
> > PGP: E2AB1DE4 |


^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: parent transid verify failed / FS wont mount / help please!
       [not found]   ` <kxQglfkZm7A_i8CXh0JVrJ9H0MF4vFVJWiT8BWRBc8NPM24nZC-vc6sIrts3ZR4T8HEvRTfPv1yz4lPotIuO1JiPYOk60cHMQqHcTFkDfZ4=@protonmail.com>
  2019-03-25 22:51     ` berodual_xyz
@ 2019-03-25 22:55     ` Hugo Mills
  1 sibling, 0 replies; 7+ messages in thread
From: Hugo Mills @ 2019-03-25 22:55 UTC (permalink / raw)
  To: berodual_xyz; +Cc: Btrfs mailing list

[-- Attachment #1: Type: text/plain, Size: 3416 bytes --]

On Mon, Mar 25, 2019 at 10:44:15PM +0000, berodual_xyz wrote:
> Thank you very much Hugo,
> 
> the underlying devices are based on HW raid6 and effectively "stitched" together. Loosing any of those would mean loosing all data, so much is clear.
> 
> My concern was not so much bitrod / silent data corruption but I would not have expected disabled data checksumming to be a disadvantage at recovering from the supposed corruption now.

   OK, so it's not quite as bad a case as I painted. Turning off all
of the btrfs data-protection features still isn't something you'd do
to data you're friends with. However, it shouldn't directly affect the
recoverability of the data (assuming you had RAID-1 metadata).

   The main problem is that you've had a transid error, which is
pretty much universally fatal. There's a description of what that
means in the FAQ here:
https://btrfs.wiki.kernel.org/index.php/FAQ#What_does_.22parent_transid_verify_failed.22_mean.3F

> Does anyone have any input on how to restore files based on inode no. from the tree dump that I have?

   I'm not sure what you mean by "tree dump". Do you mean
btrfs-debug-tree? Or btrfs-image? Or something else? In any case, none
of those are likely to help all that much. The metadata is
corrupted in a way that shouldn't ever happen, and where it's really
hard to work out how to fix it, even with an actual human expert
involved.  (It's why there's no btrfs check fix for this situation --
you simply can't take the metadata broken in this way and make much
sense out of it).

   Hugo.

> "usebackuproot,ro" did not succeed either.
> 
> Much appreciate the input!
> 
> 
> Sent with ProtonMail Secure Email.
> 
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Monday, March 25, 2019 11:38 PM, Hugo Mills <hugo@carfax.org.uk> wrote:
> 
> > On Mon, Mar 25, 2019 at 10:26:29PM +0000, berodual_xyz wrote:
> >
> > > Dear all,
> > > on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...)
> >
> > Ouch. I think the only thing you could have done to make the FS
> > more fragile is mounting with nobarrier(). Frankly, anything you're
> > getting off it is a bonus. RAID-0 gives you no duplicate copy,
> > nodatacow implies nodatasum, and nodatasum doesn't even give you the
> > ability to detect data corruption, let alone fix it.
> > With that configuration, I'd say pretty much by definition the
> > contents of the FS are considered to be discardable.
> > Restoring from backups is the recommended approach with transid
> > failures.
> > () Don't do that.
> >
> > > I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
> > > The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).
> >
> > Given how close the transids are, have you tried
> > "ro,usebackuproot"? That's about your only other option at this
> > point. But, if btrfs restore isn't working, then usebacuproot probably
> > won't either.
> >
> > > Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.
> > > "btrfs check" gives the following error:
> > >

-- 
Hugo Mills             | I gave up smoking, drinking and sex once. It was the
hugo@... carfax.org.uk | scariest 20 minutes of my life.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: parent transid verify failed / FS wont mount / help please!
  2019-03-25 22:51     ` berodual_xyz
@ 2019-03-25 22:56       ` Hugo Mills
  2019-03-25 23:07         ` berodual_xyz
  0 siblings, 1 reply; 7+ messages in thread
From: Hugo Mills @ 2019-03-25 22:56 UTC (permalink / raw)
  To: berodual_xyz; +Cc: linux-btrfs

[-- Attachment #1: Type: text/plain, Size: 4462 bytes --]

On Mon, Mar 25, 2019 at 10:51:24PM +0000, berodual_xyz wrote:
> Running "btrfs check" on the 3rd of the 4 devices the volume consists of crashes with a trace:

   Just for the record, it doesn't matter which device you use for
btrfs check. You're running it on the whole filesystem, not just one
device. The device just serves to identify which FS you're running it
on. (The btrfs check code will scan all the available block devices
for the other pieces of the FS).

   Hugo.

> ##
> $ btrfs check --readonly /dev/sdd
> Opening filesystem to check...
> parent transid verify failed on 1048576 wanted 60234 found 60230
> parent transid verify failed on 1048576 wanted 60234 found 60230
> Ignoring transid failure
> volumes.c:1762: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1
> btrfs[0x426fdc]
> btrfs(btrfs_chunk_readonly+0x98)[0x429acd]
> btrfs(btrfs_read_block_groups+0x1c1)[0x41cd44]
> btrfs(btrfs_setup_all_roots+0x368)[0x416540]
> btrfs[0x416a8a]
> btrfs(open_ctree_fs_info+0xd0)[0x416bcc]
> btrfs(cmd_check+0x591)[0x45f431]
> btrfs(main+0x24a)[0x40ca02]
> /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa320a44b35]
> btrfs[0x40c509]
> [1]    22848 abort      btrfs check --readonly /dev/sdd
> ##
> 
> Trying to mount "ro,usebackuproot" shows bad superblock and following errors in /var/log/messages:
> 
> ##
> 33814.360633] BTRFS info (device sdd): trying to use backup root at mount time
> [33814.360637] BTRFS info (device sdd): using free space tree
> [33814.360638] BTRFS info (device sdd): has skinny extents
> [33814.361708] BTRFS error (device sdd): parent transid verify failed on 1048576 wanted 60234 found 60230
> [33814.361764] BTRFS error (device sdd): failed to read chunk root
> [33814.373140] BTRFS error (device sdd): open_ctree failed
> ##
> 
> 
> Again, thank you very much for all help!
> 
> 
> 
> Sent with ProtonMail Secure Email.
> 
> ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> On Monday, March 25, 2019 11:44 PM, berodual_xyz <berodual_xyz@protonmail.com> wrote:
> 
> > Thank you very much Hugo,
> >
> > the underlying devices are based on HW raid6 and effectively "stitched" together. Loosing any of those would mean loosing all data, so much is clear.
> >
> > My concern was not so much bitrod / silent data corruption but I would not have expected disabled data checksumming to be a disadvantage at recovering from the supposed corruption now.
> >
> > Does anyone have any input on how to restore files based on inode no. from the tree dump that I have?
> >
> > "usebackuproot,ro" did not succeed either.
> >
> > Much appreciate the input!
> >
> > Sent with ProtonMail Secure Email.
> >
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Monday, March 25, 2019 11:38 PM, Hugo Mills hugo@carfax.org.uk wrote:
> >
> > > On Mon, Mar 25, 2019 at 10:26:29PM +0000, berodual_xyz wrote:
> > >
> > > > Dear all,
> > > > on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...)
> > >
> > > Ouch. I think the only thing you could have done to make the FS
> > > more fragile is mounting with nobarrier(). Frankly, anything you're
> > > getting off it is a bonus. RAID-0 gives you no duplicate copy,
> > > nodatacow implies nodatasum, and nodatasum doesn't even give you the
> > > ability to detect data corruption, let alone fix it.
> > > With that configuration, I'd say pretty much by definition the
> > > contents of the FS are considered to be discardable.
> > > Restoring from backups is the recommended approach with transid
> > > failures.
> > > () Don't do that.
> > >
> > > > I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
> > > > The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).
> > >
> > > Given how close the transids are, have you tried
> > > "ro,usebackuproot"? That's about your only other option at this
> > > point. But, if btrfs restore isn't working, then usebacuproot probably
> > > won't either.
> > >
> > > > Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.
> > > > "btrfs check" gives the following error:

-- 
Hugo Mills             | I gave up smoking, drinking and sex once. It was the
hugo@... carfax.org.uk | scariest 20 minutes of my life.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: parent transid verify failed / FS wont mount / help please!
  2019-03-25 22:56       ` Hugo Mills
@ 2019-03-25 23:07         ` berodual_xyz
  0 siblings, 0 replies; 7+ messages in thread
From: berodual_xyz @ 2019-03-25 23:07 UTC (permalink / raw)
  To: linux-btrfs, Hugo Mills



Dear Hugo, please see below.

‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
On Monday, 25. March 2019 23:56, Hugo Mills <hugo@carfax.org.uk> wrote:

> On Mon, Mar 25, 2019 at 10:51:24PM +0000, berodual_xyz wrote:
>
> > Running "btrfs check" on the 3rd of the 4 devices the volume consists of crashes with a trace:
>
> Just for the record, it doesn't matter which device you use for
> btrfs check. You're running it on the whole filesystem, not just one
> device. The device just serves to identify which FS you're running it
> on. (The btrfs check code will scan all the available block devices
> for the other pieces of the FS).
>
> Hugo.

I know and would have expected every device to give me the same output, but it does not:

##
$ btrfs filesystem show
Label: 'FS1'  uuid: 8b00ff46-3f42-4f51-be6b-5ec8a7d8f2cd
	Total devices 4 FS bytes used 54.91TiB
	devid    1 size 21.83TiB used 13.75TiB path /dev/sdb
	devid    2 size 21.83TiB used 13.75TiB path /dev/sdc
	devid    3 size 21.83TiB used 13.75TiB path /dev/sdd
	devid    4 size 21.83TiB used 13.75TiB path /dev/sde

##
$ btrfs check --readonly /dev/sdd
Opening filesystem to check...
parent transid verify failed on 1048576 wanted 60234 found 60230
parent transid verify failed on 1048576 wanted 60234 found 60230
Ignoring transid failure
volumes.c:1762: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1
btrfs[0x426fdc]
btrfs(btrfs_chunk_readonly+0x98)[0x429acd]
btrfs(btrfs_read_block_groups+0x1c1)[0x41cd44]
btrfs(btrfs_setup_all_roots+0x368)[0x416540]
btrfs[0x416a8a]
btrfs(open_ctree_fs_info+0xd0)[0x416bcc]
btrfs(cmd_check+0x591)[0x45f431]
btrfs(main+0x24a)[0x40ca02]
/lib64/libc.so.6(__libc_start_main+0xf5)[0x7f8480705b35]
btrfs[0x40c509]
[1]    479 abort      btrfs check --readonly /dev/sdd
##

Running "btrfs check --readonly /dev/sdc" (sdc is the 2nd device) returns:

##
$ btrfs check --readonly /dev/sdc
Opening filesystem to check...
parent transid verify failed on 55432763981824 wanted 60233 found 60235
parent transid verify failed on 55432763981824 wanted 60233 found 60235
Ignoring transid failure
parent transid verify failed on 55432753725440 wanted 60232 found 60235
parent transid verify failed on 55432753725440 wanted 60232 found 60235
Ignoring transid failure
parent transid verify failed on 55432764063744 wanted 60233 found 60235
parent transid verify failed on 55432764063744 wanted 60233 found 60235
Ignoring transid failure
parent transid verify failed on 55432747958272 wanted 60234 found 60235
parent transid verify failed on 55432747958272 wanted 60234 found 60235
Ignoring transid failure
Checking filesystem on /dev/sdc
UUID: 8b19ff46-3f42-4f51-be6b-5fc8a7d8f2cd
[1/7] checking root items
Error: could not find extent items for root 268
ERROR: failed to repair root items: No such file or directory
##

By "tree dump" I refered to the output of "btrfs inspect-internal dump-tree"

My hope is that I can restore certain files by specifying their inode number based on the tree info that I have, but I struggle to find a clear explanation on how to do this.

Thank you, Hugo and all!





>
> > --
> >
> > $ btrfs check --readonly /dev/sdd
> > Opening filesystem to check...
> > parent transid verify failed on 1048576 wanted 60234 found 60230
> > parent transid verify failed on 1048576 wanted 60234 found 60230
> > Ignoring transid failure
> > volumes.c:1762: btrfs_chunk_readonly: BUG_ON `!ce` triggered, value 1
> > btrfs[0x426fdc]
> > btrfs(btrfs_chunk_readonly+0x98)[0x429acd]
> > btrfs(btrfs_read_block_groups+0x1c1)[0x41cd44]
> > btrfs(btrfs_setup_all_roots+0x368)[0x416540]
> > btrfs[0x416a8a]
> > btrfs(open_ctree_fs_info+0xd0)[0x416bcc]
> > btrfs(cmd_check+0x591)[0x45f431]
> > btrfs(main+0x24a)[0x40ca02]
> > /lib64/libc.so.6(__libc_start_main+0xf5)[0x7fa320a44b35]
> > btrfs[0x40c509]
> > [1] 22848 abort btrfs check --readonly /dev/sdd
> >
> > --
> >
> > Trying to mount "ro,usebackuproot" shows bad superblock and following errors in /var/log/messages:
> >
> > --
> >
> > 33814.360633] BTRFS info (device sdd): trying to use backup root at mount time
> > [33814.360637] BTRFS info (device sdd): using free space tree
> > [33814.360638] BTRFS info (device sdd): has skinny extents
> > [33814.361708] BTRFS error (device sdd): parent transid verify failed on 1048576 wanted 60234 found 60230
> > [33814.361764] BTRFS error (device sdd): failed to read chunk root
> > [33814.373140] BTRFS error (device sdd): open_ctree failed
> >
> > --
> >
> > Again, thank you very much for all help!
> > Sent with ProtonMail Secure Email.
> > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > On Monday, March 25, 2019 11:44 PM, berodual_xyz berodual_xyz@protonmail.com wrote:
> >
> > > Thank you very much Hugo,
> > > the underlying devices are based on HW raid6 and effectively "stitched" together. Loosing any of those would mean loosing all data, so much is clear.
> > > My concern was not so much bitrod / silent data corruption but I would not have expected disabled data checksumming to be a disadvantage at recovering from the supposed corruption now.
> > > Does anyone have any input on how to restore files based on inode no. from the tree dump that I have?
> > > "usebackuproot,ro" did not succeed either.
> > > Much appreciate the input!
> > > Sent with ProtonMail Secure Email.
> > > ‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐
> > > On Monday, March 25, 2019 11:38 PM, Hugo Mills hugo@carfax.org.uk wrote:
> > >
> > > > On Mon, Mar 25, 2019 at 10:26:29PM +0000, berodual_xyz wrote:
> > > >
> > > > > Dear all,
> > > > > on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...)
> > > >
> > > > Ouch. I think the only thing you could have done to make the FS
> > > > more fragile is mounting with nobarrier(). Frankly, anything you're
> > > > getting off it is a bonus. RAID-0 gives you no duplicate copy,
> > > > nodatacow implies nodatasum, and nodatasum doesn't even give you the
> > > > ability to detect data corruption, let alone fix it.
> > > > With that configuration, I'd say pretty much by definition the
> > > > contents of the FS are considered to be discardable.
> > > > Restoring from backups is the recommended approach with transid
> > > > failures.
> > > > () Don't do that.
> > > >
> > > > > I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
> > > > > The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).
> > > >
> > > > Given how close the transids are, have you tried
> > > > "ro,usebackuproot"? That's about your only other option at this
> > > > point. But, if btrfs restore isn't working, then usebacuproot probably
> > > > won't either.
> > > >
> > > > > Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.
> > > > > "btrfs check" gives the following error:
>
> --
>
> Hugo Mills | I gave up smoking, drinking and sex once. It was the
> hugo@... carfax.org.uk | scariest 20 minutes of my life.
> http://carfax.org.uk/ |
> PGP: E2AB1DE4 |



^ permalink raw reply	[flat|nested] 7+ messages in thread

* Re: parent transid verify failed / FS wont mount / help please!
  2019-03-25 22:26 parent transid verify failed / FS wont mount / help please! berodual_xyz
  2019-03-25 22:38 ` Hugo Mills
@ 2019-03-26  3:47 ` Andrei Borzenkov
  1 sibling, 0 replies; 7+ messages in thread
From: Andrei Borzenkov @ 2019-03-26  3:47 UTC (permalink / raw)
  To: berodual_xyz, linux-btrfs

26.03.2019 1:26, berodual_xyz пишет:
> Dear all,
> 
> on a large btrfs based filesystem (multi-device raid0 - all devices okay, nodatacow,nodatasum...) I experienced severe filesystem corruption, most likely due to a hard reset with inflight data.
> The system cannot mount (also not with "ro,nologreplay" / "nospace_cache" etc.).
> 
> Running "btrfs restore" I got a reasonable amount of data backed up, but a large chunk is missing.
> 

You may be interested in

https://lore.kernel.org/linux-btrfs/CAH+WbHw1cm96YNjVhmdFP4qZ_HR3mzfmojna=d-MeydCtunG1g@mail.gmail.com/


^ permalink raw reply	[flat|nested] 7+ messages in thread

end of thread, other threads:[~2019-03-26  3:47 UTC | newest]

Thread overview: 7+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2019-03-25 22:26 parent transid verify failed / FS wont mount / help please! berodual_xyz
2019-03-25 22:38 ` Hugo Mills
     [not found]   ` <kxQglfkZm7A_i8CXh0JVrJ9H0MF4vFVJWiT8BWRBc8NPM24nZC-vc6sIrts3ZR4T8HEvRTfPv1yz4lPotIuO1JiPYOk60cHMQqHcTFkDfZ4=@protonmail.com>
2019-03-25 22:51     ` berodual_xyz
2019-03-25 22:56       ` Hugo Mills
2019-03-25 23:07         ` berodual_xyz
2019-03-25 22:55     ` Hugo Mills
2019-03-26  3:47 ` Andrei Borzenkov

This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).