Restoring BTRFS partition

Restoring BTRFS partition

[Alli Quaknaa]

Hello,
Yesterday I have messed up my partition table and lost the information
of my BTRFS partition. Luckily I was able to find it's beginning, or
at least I think so. This is what I did:
- created a blank 300M file, formated with btrfs, examined with
hexedit. After 65536 zeros (at the position 0x10000) a sequence "1F 55
A8 7C" comes and at a position 0x10040 an ASCII sequence _BHRfS_M_ is
found.
- I found the same sequence ("_BHRfS_M_") in the raw image of my drive
and I've made another image to make my original look more like the
300M file - so in the beginning 65k of zeros and the sequence
_BHRfS_M_ is at the same position (basically I just deleted whatever
was before this sequence and prepended those zeros).

I don't think I have done anything that could have damaged the
filesystem itself and when viewing with hexedit I still see ELF data,
include files and so on, but I can't get the new image to mount or
btrfsck, it is simply not recognized as a btrfs image. Could anyone
please help me fix this? I have very little actual C/Linux knowledge
(from the programmers point of view) so I'd rather not go reading the
code.

Of course I can send whatever more information you need, just as a
starter - I have no idea what the sequence at 0x10000 is, but it is
different than the one of my blank 300M file - in my partition it says
"CB EE AE 02".

Thanks in advance,
al-Quaknaa

Re: Restoring BTRFS partition

[Wengang Wang]

Hi al-Quaknaa,

I guess the reason is that the 300M file btrfs and the one on your
partition have different block size. Thus 65k zeros on your file image
doesn't mean 65k on the partition. So maybe you will try with blocks
instead of bytes.

The man page for mkfs.btrfs is bad for having no description about what
will be going on if an option(-s this case) is not provided. A fix block
size such as 1k, 4k or calculated per the partition(s) size.

thanks,
wengang.

On 10-04-20 16:56, Alli Quaknaa wrote:
> Hello,
> Yesterday I have messed up my partition table and lost the information
> of my BTRFS partition. Luckily I was able to find it's beginning, or
> at least I think so. This is what I did:
> - created a blank 300M file, formated with btrfs, examined with
> hexedit. After 65536 zeros (at the position 0x10000) a sequence "1F 55
> A8 7C" comes and at a position 0x10040 an ASCII sequence _BHRfS_M_ is
> found.
> - I found the same sequence ("_BHRfS_M_") in the raw image of my drive
> and I've made another image to make my original look more like the
> 300M file - so in the beginning 65k of zeros and the sequence
> _BHRfS_M_ is at the same position (basically I just deleted whatever
> was before this sequence and prepended those zeros).
> 
> I don't think I have done anything that could have damaged the
> filesystem itself and when viewing with hexedit I still see ELF data,
> include files and so on, but I can't get the new image to mount or
> btrfsck, it is simply not recognized as a btrfs image. Could anyone
> please help me fix this? I have very little actual C/Linux knowledge
> (from the programmers point of view) so I'd rather not go reading the
> code.
> 
> Of course I can send whatever more information you need, just as a
> starter - I have no idea what the sequence at 0x10000 is, but it is
> different than the one of my blank 300M file - in my partition it says
> "CB EE AE 02".
> 
> Thanks in advance,
> al-Quaknaa
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Restoring BTRFS partition

[Sean Bartell]

On Tue, Apr 20, 2010 at 11:55:38PM +0800, Wengang Wang wrote:
> I guess the reason is that the 300M file btrfs and the one on your
> partition have different block size. Thus 65k zeros on your file image
> doesn't mean 65k on the partition. So maybe you will try with blocks
> instead of bytes.

Actually, the block size doesn't matter for this--the superblock is
always at 0x10000. Alli, I think you'll have to upload the start of the
partition so someone can take a look at it.

Re: Restoring BTRFS partition

[Alli Quaknaa]

I will do so soon. How big of a chunk would be most useful? The
original partition was about 15GB.
Thanks,
al-Quaknaa

Re: Restoring BTRFS partition

[Alli Quaknaa]

So here are first ~12M of the partition. There was some junk preceding
what is in the file, but it mostly looked like my swap or something
(cached css, javascript and webpages I've recently visited - though I
hope the beginning of the partition isn't somewhere else. Hopefuly you
would be able to tell from the dump).

http://pub.yweb.cz/sda7_head.dump

Thank you all for your time and effort!
al-Quaknaa

Re: Restoring BTRFS partition

[Sean Bartell]

On Tue, Apr 20, 2010 at 06:13:41PM +0000, Alli Quaknaa wrote:
> So here are first ~12M of the partition. There was some junk preceding
> what is in the file, but it mostly looked like my swap or something
> (cached css, javascript and webpages I've recently visited - though I
> hope the beginning of the partition isn't somewhere else. Hopefuly you
> would be able to tell from the dump).
> 
> http://pub.yweb.cz/sda7_head.dump

The superblock in that file (starting at byte 0x10) is actually a mirror
of the real superblock. Aside from the real superblock at 0x10000,
Btrfs stores mirror copies of the superblock at 0x4000000 (64 MiB),
0x4000000000 (256 GiB), and 0x4000000000000 (1 PiB). Each superblock has
a field that indicates where it is; when you made your image, you put
the mirror superblock where the real superblock was supposed to be, and
btrfs refused to mount it because that field was wrong. 

The real start of the btrfs partition is 0x4000000 bytes (64 MiB) before
the place you found that mirror superblock; the real superblock should
be 0x3ff0000 bytes before the mirror. Even if the real superblock is
corrupt, if the mirror is at 0x4000000, where it's supposed to be, you
should be able to get btrfs to mount it (though I think you might need a
mount option or a patch).

Re: Restoring BTRFS partition

[Alli Quaknaa]

Oh, thank you very much!

I think I have found the real superblock you are talking about, but
I'm afraid I may have written something in the first 64MiB. Is there a
chance btrfsck will recover it?

Also, I think there's gotta be a better way to manipulate those huge
files then dd and hexedit for examination - I'd like to take the raw
file, open it in some hex editor and be able to cut of some of it's
beginning - I can't seem to be able to do it with hexedit. Is there a
tool you'd recomment?

al-Quaknaa

Re: Restoring BTRFS partition

[Sean Bartell]

On Tue, Apr 20, 2010 at 10:25:34PM +0000, Alli Quaknaa wrote:
> I think I have found the real superblock you are talking about, but
> I'm afraid I may have written something in the first 64MiB. Is there a
> chance btrfsck will recover it?
btrfsck is currently very limited; it only detects a limited number of
problems, and it can't fix anything. Btrfs focuses on handling problems
when they are discovered while using the FS; generally, it should handle
corruption relatively gracefully. However, if anything really crucial
was overwritten and the FS can't be mounted, there aren't any tools to
repair it.

> Also, I think there's gotta be a better way to manipulate those huge
> files then dd and hexedit for examination - I'd like to take the raw
> file, open it in some hex editor and be able to cut of some of it's
> beginning - I can't seem to be able to do it with hexedit. Is there a
> tool you'd recomment?
For viewing, you can use less, head, and tail with hexdump:
    tail -c +$((0x10000+1)) /dev/sda1|hexdump -C|less
will view the disk starting at the superblock. For editing, dd is
probably best, though you could use a hex editor like Okteta. I've also
heard of Radare, supposedly a very advanced command-line tool. Keep in
mind that any tool that deletes the first part of a huge file will be
forced to rewrite the entire file.

Re: Restoring BTRFS partition

[Wengang Wang]

On 10-04-20 12:50, Sean Bartell wrote:
> On Tue, Apr 20, 2010 at 11:55:38PM +0800, Wengang Wang wrote:
> > I guess the reason is that the 300M file btrfs and the one on your
> > partition have different block size. Thus 65k zeros on your file image
> > doesn't mean 65k on the partition. So maybe you will try with blocks
> > instead of bytes.
> 
> Actually, the block size doesn't matter for this--the superblock is
> always at 0x10000. Alli, I think you'll have to upload the start of the
> partition so someone can take a look at it.

I see.
Thanks all for my knowledge!

regards,
wengang.

Re: Restoring BTRFS partition

[Alli Quaknaa]

Okay,
I've got it working! Huge thanks to everybody and especially to Sean
for identifying that what I was seeing wasn't the real superblock.
So my steps to fix it were (for future generations searching through
the archives):
1. Find the real superblock
2. play with dd ibs=1 skip=nnnnnnn for a while to get an image where
the checksum is at 0x0 and the _BHRfS_ starts at 0x40
3. create the zeros - dd if=/dev/zero of=pad bs=1 count=65536
4. append the data with dd if=/dev/sda of=pad oflag=append
conv=notrunc skip=nnnnnnn ibs=1
~ that's basically it. Converting the skip offset to some bigger units
(512 in my case) helps speed, also some garbage at the end of the file
doesn't seem to cause any harm, so if one doesn't know the exact end
of the partition, it is better to get some more but not necessary to
compute the exact size.
al-Quaknaa