fs unreadable after powercycle: BTRFS (device sda): parent transid verify failed on 427084513280 wanted 390924 found 390922

fs unreadable after powercycle: BTRFS (device sda): parent transid verify failed on 427084513280 wanted 390924 found 390922

[Martin Tippmann]

Hi, after a hard reboot (powercycle) a btrfs volume did not come up again:

It's a single 4TB disk - only btrfs with lzo - data=single,metadata=dup

[  121.831814] BTRFS info (device sda): disk space caching is enabled
[  121.857820] BTRFS (device sda): parent transid verify failed on
427084513280 wanted 390924 found 390922
[  121.861607] BTRFS (device sda): parent transid verify failed on
427084513280 wanted 390924 found 390922
[  121.861715] BTRFS: failed to read tree root on sda
[  121.878111] BTRFS: open_ctree failed

btrfs-progs v4.0
Kernel: 4.1.4

I'm quite sure that the HDD is fine (no SMART Problems, Disk Errorlog
is empty, It's a new Enterprise-Drive that worked well in the past
days/weeks).

So I'm kind at loss what to do:

How can I recover from that problem? I've found just a note in the
FAQ[1] but no solution to the problem.

Maybe someone can give some clues why does this happen in the first
place? Is it unfortunate timing due to the abrupt power cycle?
Shouldn't CoW protect against this somewhat?

Thanks for any hints!

Additional info:

# btrfs check /dev/sda
parent transid verify failed on 427084513280 wanted 390924 found 390922
parent transid verify failed on 427084513280 wanted 390924 found 390922
parent transid verify failed on 427084513280 wanted 390924 found 390922
parent transid verify failed on 427084513280 wanted 390924 found 390922
Ignoring transid failure
Couldn't setup extent tree
Couldn't open file system

Not sure what it does but it looks not too good:

# btrfs-find-root /dev/sda
parent transid verify failed on 427084513280 wanted 390924 found 390922
parent transid verify failed on 427084513280 wanted 390924 found 390922
parent transid verify failed on 427084513280 wanted 390924 found 390922
parent transid verify failed on 427084513280 wanted 390924 found 390922
Ignoring transid failure
Couldn't setup extent tree
Couldn't setup device tree
Superblock thinks the generation is 390924
Superblock thinks the level is 1
  Well block 427084988416(gen: 390923 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427084021760(gen: 390923 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427084431360(gen: 390915 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427084398592(gen: 390915 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427083988992(gen: 390915 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427038621696(gen: 390914 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427031035904(gen: 390913 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427285069824(gen: 390912 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427060887552(gen: 390912 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427013128192(gen: 390912 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 427001872384(gen: 390909 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426965237760(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426965221376(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426965188608(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426965172224(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426965155840(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426964271104(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426964156416(gen: 390906 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426950377472(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426944512000(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426940841984(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426940612608(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426940465152(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426940153856(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426939809792(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426939711488(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426939465728(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426939351040(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426939006976(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426938564608(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426938351616(gen: 390905 level: 0) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 426347577344(gen: 390904 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 168491106304(gen: 390903 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1340608053248(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1340295020544(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1264696688640(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1136507092992(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1136226516992(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 998473711616(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 998114721792(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 667127316480(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 667048116224(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 667041382400(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 667034255360(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 667028701184(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 667018526720(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 666950320128(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 23074570240(gen: 390902 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 22614048768(gen: 390813 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 22605398016(gen: 390812 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 22599712768(gen: 390811 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 22586949632(gen: 390809 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1340463906816(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1340435660800(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1340394799104(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1264894115840(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 1136142614528(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 666915078144(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1
Well block 666902560768(gen: 390808 level: 1) seems good, but
generation/level doesn't match, want gen: 390924 level: 1


regards
Martin


1: https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#parent_transid_verify_failed

Re: fs unreadable after powercycle: BTRFS (device sda): parent transid verify failed on 427084513280 wanted 390924 found 390922

[Hugo Mills]

[-- Attachment #1: Type: text/plain, Size: 10456 bytes --]

On Sat, Aug 08, 2015 at 08:43:34PM +0200, Martin Tippmann wrote:
> Hi, after a hard reboot (powercycle) a btrfs volume did not come up again:
> 
> It's a single 4TB disk - only btrfs with lzo - data=single,metadata=dup
> 
> [  121.831814] BTRFS info (device sda): disk space caching is enabled
> [  121.857820] BTRFS (device sda): parent transid verify failed on
> 427084513280 wanted 390924 found 390922
> [  121.861607] BTRFS (device sda): parent transid verify failed on
> 427084513280 wanted 390924 found 390922
> [  121.861715] BTRFS: failed to read tree root on sda
> [  121.878111] BTRFS: open_ctree failed
> 
> btrfs-progs v4.0
> Kernel: 4.1.4
> 
> I'm quite sure that the HDD is fine (no SMART Problems, Disk Errorlog
> is empty, It's a new Enterprise-Drive that worked well in the past
> days/weeks).
> 
> So I'm kind at loss what to do:
> 
> How can I recover from that problem? I've found just a note in the
> FAQ[1] but no solution to the problem.
> 
> Maybe someone can give some clues why does this happen in the first
> place? Is it unfortunate timing due to the abrupt power cycle?
> Shouldn't CoW protect against this somewhat?

   Not "somewhat": it should protect it completely. There are two ways
that this can happen: it's a bug in btrfs, or there's something
stopping barriers from working. That latter case can be either a bug
in the kernel's block layer (pretty unlikely), or the hardware is
behaving badly and ignoring the barriers (more likely, particularly if
it's on a USB/SATA converter).

   I don't think there's a good solution to transid failures, I'm
afraid. The best that I'm aware of is to use btrfs restore to grab the
pieces of your FS that aren't up to date in your backups, and then
restore from them.

> Thanks for any hints!
> 
> Additional info:
> 
> # btrfs check /dev/sda
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> Ignoring transid failure
> Couldn't setup extent tree
> Couldn't open file system
> 
> Not sure what it does but it looks not too good:

   Actually, it's pretty good, other than the transid failure, which
is a real problem.

   Hugo.

> # btrfs-find-root /dev/sda
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> Ignoring transid failure
> Couldn't setup extent tree
> Couldn't setup device tree
> Superblock thinks the generation is 390924
> Superblock thinks the level is 1
>   Well block 427084988416(gen: 390923 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427084021760(gen: 390923 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427084431360(gen: 390915 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427084398592(gen: 390915 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427083988992(gen: 390915 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427038621696(gen: 390914 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427031035904(gen: 390913 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427285069824(gen: 390912 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427060887552(gen: 390912 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427013128192(gen: 390912 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427001872384(gen: 390909 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965237760(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965221376(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965188608(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965172224(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965155840(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426964271104(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426964156416(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426950377472(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426944512000(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940841984(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940612608(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940465152(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940153856(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939809792(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939711488(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939465728(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939351040(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939006976(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426938564608(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426938351616(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426347577344(gen: 390904 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 168491106304(gen: 390903 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340608053248(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340295020544(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1264696688640(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1136507092992(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1136226516992(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 998473711616(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 998114721792(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667127316480(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667048116224(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667041382400(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667034255360(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667028701184(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667018526720(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 666950320128(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 23074570240(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22614048768(gen: 390813 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22605398016(gen: 390812 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22599712768(gen: 390811 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22586949632(gen: 390809 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340463906816(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340435660800(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340394799104(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1264894115840(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1136142614528(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 666915078144(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 666902560768(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> 
> 
> regards
> Martin
> 
> 
> 1: https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#parent_transid_verify_failed

-- 
Hugo Mills             | This chap Anon is writing some perfectly lovely
hugo@... carfax.org.uk | stuff at the moment.
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]

Re: fs unreadable after powercycle: BTRFS (device sda): parent transid verify failed on 427084513280 wanted 390924 found 390922

[Martin Tippmann]

2015-08-08 21:05 GMT+02:00 Hugo Mills <hugo@carfax.org.uk>:
>> Maybe someone can give some clues why does this happen in the first
>> place? Is it unfortunate timing due to the abrupt power cycle?
>> Shouldn't CoW protect against this somewhat?
>
>    Not "somewhat": it should protect it completely. There are two ways
> that this can happen: it's a bug in btrfs, or there's something
> stopping barriers from working. That latter case can be either a bug
> in the kernel's block layer (pretty unlikely), or the hardware is
> behaving badly and ignoring the barriers (more likely, particularly if
> it's on a USB/SATA converter).

Thanks for the information. The setup is nothing out of the ordinary.
The disks are HGST HUS724040ALA640 running on a Dell H310 SATA
controller and configured as JBOD. It's all running on defaults on a
Dell PowerEdge R720. SMART says the disk write cache is enabled -
maybe that's part of the problem?

>    I don't think there's a good solution to transid failures, I'm
> afraid. The best that I'm aware of is to use btrfs restore to grab the
> pieces of your FS that aren't up to date in your backups, and then
> restore from them.

Okay, fortunately can I dismiss the data - or is the broken Image of
any use for anyone? It's a 4TB disk but I guess I could create a
compressed (partial) image if it's of interest to anyone.

regards
Martin

Re: fs unreadable after powercycle: BTRFS (device sda): parent transid verify failed on 427084513280 wanted 390924 found 390922

[Duncan]

Martin Tippmann posted on Sat, 08 Aug 2015 20:43:34 +0200 as excerpted:

> Hi, after a hard reboot (powercycle) a btrfs volume did not come up
> again:
> 
> It's a single 4TB disk - only btrfs with lzo - data=single,metadata=dup
> 
> [  121.831814] BTRFS info (device sda): disk space caching is enabled [
> 121.857820] BTRFS (device sda): parent transid verify failed on
> 427084513280 wanted 390924 found 390922 [  121.861607] BTRFS (device
> sda):
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> [ 121.861715] BTRFS: failed to read tree root on sda [  121.878111]
> BTRFS: open_ctree failed
> 
> btrfs-progs v4.0 Kernel: 4.1.4
> 
> I'm quite sure that the HDD is fine (no SMART Problems, Disk Errorlog is
> empty, It's a new Enterprise-Drive that worked well in the past
> days/weeks).
> 
> So I'm kind at loss what to do:
> 
> How can I recover from that problem? I've found just a note in the
> FAQ[1] but no solution to the problem.

[The FAQ reference was to the wiki problem faq, transid failure 
explanation, but it didn't say what to do about it.]

Did you try the recovery mount option suggested earlier in the problem-faq 
under mount problems?

https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_can.27t_mount_my_filesystem.2

For transid failures, that's what I'd try first, since that scans 
previous tree-roots and tries to use the first one it can read.  Since 
the transid it wants (390924) is only a couple ahead of what it finds 
(390922), and the recover mount option scans backward in the tree-root 
history to see if it can find any that work, that could well solve the 
problem.

If not, as Hugo mentions, given find-tree-root looks good, btrfs restore 
has a good chance of working.  I've used that myself to good effect a 
couple times when a btrfs refused to mount (I have backups if I have to 
use 'em, but recovery or restore, when they work, will normally leave me 
with more current copies, since I tend to let my backups get somewhat 
stale).  There's a page on the wiki for using it with find-root if 
necessary, but the wiki page is a bit dated.  The btrfs-restore manpage 
should be current, but doesn't have the detail about using it with find-
root that the wiki page has.

> Maybe someone can give some clues why does this happen in the first
> place?
> Is it unfortunate timing due to the abrupt power cycle?
> Shouldn't CoW protect against this somewhat?

As Hugo says, in theory cow should protect against this, but the 
combination of possible bugs in a still not yet fully stable and mature 
btrfs, and possibly buggy hardware, means theory and practice don't 
always line up as well as they should, in theory. (How's that for an 
ouroboros, aka snake eating it's tail circular-reference, explanation? 
=:^)

But the recovery mount option is a reasonable first recovery (now 
ouroboroi =:^) option, and btrfs restore not too bad to work with if that 
fails.

Referencing the hardware write-caching option you mentioned later, yes, 
turning that off can help... in theory... but it also tends to have a 
DRAMATICALLY bad effect on spinning rust write performance (I don't know 
enough about SSD write caching to venture a guess), and in some cases 
voids warranties due to the additional thrashing it's likely to cause as 
well, so do your research before turning it off.  In general, it's not a 
good idea as it's simply not worth it.  Both Linux at the generic IO 
level and the various filesystem stacks are designed to work around all 
but the worst hardware IO barrier failures, and the write slowdown and 
increased disk thrashing are simply not worth it, in most cases.  If the 
hardware is actually bad enough that it's worth it, I'd strongly consider 
different hardware.





-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: fs unreadable after powercycle: BTRFS (device sda): parent transid verify failed on 427084513280 wanted 390924 found 390922

[Qu Wenruo]

Martin Tippmann wrote on 2015/08/08 20:43 +0200:
> Hi, after a hard reboot (powercycle) a btrfs volume did not come up again:
>
> It's a single 4TB disk - only btrfs with lzo - data=single,metadata=dup
>
> [  121.831814] BTRFS info (device sda): disk space caching is enabled
> [  121.857820] BTRFS (device sda): parent transid verify failed on
> 427084513280 wanted 390924 found 390922
> [  121.861607] BTRFS (device sda): parent transid verify failed on
> 427084513280 wanted 390924 found 390922
> [  121.861715] BTRFS: failed to read tree root on sda
> [  121.878111] BTRFS: open_ctree failed
>
> btrfs-progs v4.0
> Kernel: 4.1.4
>
> I'm quite sure that the HDD is fine (no SMART Problems, Disk Errorlog
> is empty, It's a new Enterprise-Drive that worked well in the past
> days/weeks).
>
> So I'm kind at loss what to do:
>
> How can I recover from that problem? I've found just a note in the
> FAQ[1] but no solution to the problem.
>
> Maybe someone can give some clues why does this happen in the first
> place? Is it unfortunate timing due to the abrupt power cycle?
> Shouldn't CoW protect against this somewhat?
>
> Thanks for any hints!
>
> Additional info:
>
> # btrfs check /dev/sda
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> Ignoring transid failure
> Couldn't setup extent tree
Seems extent tree or tree root is corrupted.

> Couldn't open file system
>
> Not sure what it does but it looks not too good:
>
> # btrfs-find-root /dev/sda
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> parent transid verify failed on 427084513280 wanted 390924 found 390922
> Ignoring transid failure
> Couldn't setup extent tree
> Couldn't setup device tree
> Superblock thinks the generation is 390924
> Superblock thinks the level is 1
>    Well block 427084988416(gen: 390923 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427084021760(gen: 390923 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
These two block seems to be good, but not sure why there will be two of 
them.

Try "btrfsck --tree-root 427084988416" and "btrfsck --tree-root 
427084021760" to see which produce the least number of error.

Thanks,
Qu
> Well block 427084431360(gen: 390915 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427084398592(gen: 390915 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427083988992(gen: 390915 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427038621696(gen: 390914 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427031035904(gen: 390913 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427285069824(gen: 390912 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427060887552(gen: 390912 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427013128192(gen: 390912 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 427001872384(gen: 390909 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965237760(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965221376(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965188608(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965172224(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426965155840(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426964271104(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426964156416(gen: 390906 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426950377472(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426944512000(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940841984(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940612608(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940465152(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426940153856(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939809792(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939711488(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939465728(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939351040(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426939006976(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426938564608(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426938351616(gen: 390905 level: 0) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 426347577344(gen: 390904 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 168491106304(gen: 390903 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340608053248(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340295020544(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1264696688640(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1136507092992(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1136226516992(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 998473711616(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 998114721792(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667127316480(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667048116224(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667041382400(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667034255360(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667028701184(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 667018526720(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 666950320128(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 23074570240(gen: 390902 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22614048768(gen: 390813 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22605398016(gen: 390812 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22599712768(gen: 390811 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 22586949632(gen: 390809 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340463906816(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340435660800(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1340394799104(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1264894115840(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 1136142614528(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 666915078144(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
> Well block 666902560768(gen: 390808 level: 1) seems good, but
> generation/level doesn't match, want gen: 390924 level: 1
>
>
> regards
> Martin
>
>
> 1: https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#parent_transid_verify_failed
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>