superblock corruption, cannot obtain data

superblock corruption, cannot obtain data

[Shaun Reich]

at some point, my /home randomly(?) went into -ro as i noticed writes
were not working. Checked dmesg which had some backtraces which i
ignored. So I simply rebooted, only to find out it wouldn't come back.

so now my /home is stuck on here, I was literally going to do my round
of backups tonight. It's not extremely critical..but to reproduce my
data it'll still be several hours of lost time..and the annoyances of
redoing it.

mounting results in this, as below. mounting recovery doesn't help.
mounting ro recovery doesn't help. btrfs fsck basically reaffirms what
I know so far, that I'm screwed because my superblock is bad.

show super: http://pastie.org/private/b1k2famcfmtquaqgpyfozg
mount dmesg: http://pastie.org/private/8zsczfustrmeg6a2bbfxag

btrfs-find-root: http://bpaste.net/show/326346

it's a 250gb fs.

so, how can i (a) extract my data, (b) try to get it working again,
without needing to redo the filesystem?

I could not understand how to use btrfs-recover with my problem. It
mostly seems to hate me because of said superblock issue.

I wasn't aware that the superblock was extremely critical (surely one
can recreate such data from existing data, theoretically?)

need any other info, just ask.

(please keep me cc'd, not sub'ed)


-- 
Shaun Reich,
KDE Software Developer (kde.org)

Re: superblock corruption, cannot obtain data

[Duncan]

Shaun Reich posted on Sat, 31 May 2014 23:51:26 -0400 as excerpted:

> at some point, my /home randomly(?) went into -ro as i noticed writes
> were not working. Checked dmesg which had some backtraces which i
> ignored. So I simply rebooted, only to find out it wouldn't come back.
> 
> so now my /home is stuck on here, I was literally going to do my round
> of backups tonight. It's not extremely critical..but to reproduce my
> data it'll still be several hours of lost time..and the annoyances of
> redoing it.
> 
> mounting results in this, as below. mounting recovery doesn't help.
> mounting ro recovery doesn't help. btrfs fsck basically reaffirms what I
> know so far, that I'm screwed because my superblock is bad.
> 
> show super: http://pastie.org/private/b1k2famcfmtquaqgpyfozg mount
> dmesg: http://pastie.org/private/8zsczfustrmeg6a2bbfxag
> 
> btrfs-find-root: http://bpaste.net/show/326346

Hi, Shaun.  KDE user here, gentoo, USE=-semantic-desktop, active on the
kde general and linux lists (as well as here and the gentoo lists).  =:^)

How do you know it's a bad superblock?  While I'm not a dev just a list
regular, show-super looks reasonable from here, and find-root does find
what appears to be a good root.  From here the problem seems to be a bad
ctree (of several), not a bad superblock.

First thing to try is mounting with nospace_cache.  If that works, try
mounting with clear_cache and letting it work for a bit to rebuild.

If that doesn't work, it may simply be the log tree.  btrfs-zero-log is
used to fix that, but before you try that, read this entry in the FAQ and
follow the instructions for making a filesystem image to turn in to the
devs to help them fix such problems, and consider making a backup of the
damaged filesystem using dd/ddrescue, since the warning about making it
possibly more difficult to recover if this is NOT the problem applies:

https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_can.27t_mount_my_filesystem.2C_and_I_get_a_kernel_oops.21

With luck one of those options will help.  I know I had a space_cache
issue here myself, some time ago.  I've not had to run zero-log, but
I've seen a number of posters who found it helped.

Another thing to try is mounting recovery,ro.  A number of people have
reported that would work when simple recovery wouldn't, because when
it was writable btrfs would immediately try to fix the problem and
immediately fail.

Beyond that, you'll need more expert help.  But most of the time one
of those three works, so with luck...

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: superblock corruption, cannot obtain data

[Shaun Reich]

> How do you know it's a bad superblock?  While I'm not a dev just a list
> regular, show-super looks reasonable from here, and find-root does find
> what appears to be a good root.  From here the problem seems to be a bad
> ctree (of several), not a bad superblock.

yes i think you're right. i think that was just the initial thinking of #btrfs


>
> First thing to try is mounting with nospace_cache.  If that works, try
> mounting with clear_cache and letting it work for a bit to rebuild.

nope, same error.

>
> If that doesn't work, it may simply be the log tree.  btrfs-zero-log is
> used to fix that, but before you try that, read this entry in the FAQ
> and follow the instructions for making a filesystem image to turn in to
> the devs to help them fix such problems, and consider making a backup
> of the damaged filesystem using dd/ddrescue, since the warning about
> making it possibly more difficult to recover if this is NOT the problem
> applies:

tried this, btrfs-image complains about roughly the same error,
btrfs-zero-log said the following: http://bpaste.net/show/328495


> Another thing to try is mounting recovery,ro.  A number of people have
> reported that would work when simple recovery wouldn't, because when
> it was writable btrfs would immediately try to fix the problem and
> immediately fail.

nope, same error.

here's what happens when i run fsck http://cwillu.com:8080/74.84.235.22/3

"Critical roots corrupted, unable to fsck the FS" sounds like
especially good news ;p


--
Shaun Reich,
KDE Software Developer (kde.org)

Re: superblock corruption, cannot obtain data

[Shaun Reich]

Tl;dr tried nearly everything, couldn't get it to recover, gave up and
restored my old backup. I was running the newest Archlinux kernel
release.

.. Extracting data with Btrfs restore was quite useless, Btrfs find
root didn't list any object id's either, which I was unsure why. I
guess I was just unlucky or the utility got changed since all the
other people I've seen had used it.

Still have a full dd image of the disk that I'll keep for a week or
so, and a Btrfs-image, if anyone is at all interested.