filesystem finder / fixer

filesystem finder / fixer

[serialhex]

Hi all, so I've filled out an awesome ID10T form this weekend (all
starting with a forded update) and completely annialated my partitions...
I was running btrfs and was wondering 2 things:

1) is there a tool to help me recover data from my fs? I don't have a
backup of my partition table and so I have about 500GB of space where a
few partitionns might reside... GPT partitions mind you

...and...

2) if no such tool yet exsists, what do I need to learn to make one? I
know C/C++ okayish... and filesystems are *REALLY FREAKING COOL* to me, so
if someone can point me in a good direction / mentor me or something that
would be awesome!

...or I could just suffer. Though I'm probably going to do that anyway...
thanks in advance!!!

Justin

github.com/serialhex snt frum my awesum ansroid... txt mite b rong

Re: filesystem finder / fixer

[Roman Mamedov]

[-- Attachment #1: Type: text/plain, Size: 644 bytes --]

On Mon, 30 Jul 2012 23:26:42 -0400 (EDT)
serialhex@lavabit.com wrote:

> 1) is there a tool to help me recover data from my fs? I don't have a
> backup of my partition table and so I have about 500GB of space where a
> few partitionns might reside... GPT partitions mind you

If you only lost the partition table, there's a tool (strangely)named
TestDisk, which can find the actual partitions on disk and restore it.
Don't know if it supports GPT and BTRFS, though.

-- 
With respect,
Roman

~~~~~~~~~~~~~~~~~~~~~~~~~~~
"Stallman had a printer,
with code he could not see.
So he began to tinker,
and set the software free."

[-- Attachment #2: signature.asc --]
[-- Type: application/pgp-signature, Size: 198 bytes --]

Re: filesystem finder / fixer

[Calvin Walton]

On Mon, 2012-07-30 at 23:26 -0400, serialhex@lavabit.com wrote:
> Hi all, so I've filled out an awesome ID10T form this weekend (all
> starting with a forded update) and completely annialated my partitions...
> I was running btrfs and was wondering 2 things:
> 
> 1) is there a tool to help me recover data from my fs? I don't have a
> backup of my partition table and so I have about 500GB of space where a
> few partitionns might reside... GPT partitions mind you

One of the nice things about GPT is that it stores a backup partition
table in a second spot near the end of the disk. It may be possible to
recover it.

Try running 'gdisk' on your drive, then hit 'r' to switch to recovery
mode. Try the 'c' recovery command to load the backup partition table
from disk.
More details and some other things you might want to try are at
http://www.rodsbooks.com/gdisk/repairing.html

Do keep in mind that if you have the space, you should make an image of
your disk before trying any recovery tools. If you don't have the space,
then you should buy, say, an external USB drive to *make* the space, and
use it for backups once you're done recovering your data (if that's
possible.)

This wouldn't have been a major problem if you had backups in the first
place :)

-- 
Calvin Walton <calvin.walton@kepstin.ca>

Re: filesystem finder / fixer

[serialhex]

> On Mon, 2012-07-30 at 23:26 -0400, serialhex@lavabit.com wrote:
>> Hi all, so I've filled out an awesome ID10T form this weekend (all
>> starting with a forded update) and completely annialated my
>> partitions...
>> I was running btrfs and was wondering 2 things:
>>
>> 1) is there a tool to help me recover data from my fs? I don't have a
>> backup of my partition table and so I have about 500GB of space where a
>> few partitionns might reside... GPT partitions mind you
>
> One of the nice things about GPT is that it stores a backup partition
> table in a second spot near the end of the disk. It may be possible to
> recover it.
>
> Try running 'gdisk' on your drive, then hit 'r' to switch to recovery
> mode. Try the 'c' recovery command to load the backup partition table
> from disk.
> More details and some other things you might want to try are at
> http://www.rodsbooks.com/gdisk/repairing.html
>

i've already tried some of the recovery options... i'm going to try the
MBR recovery later on, but i doubt that will work.  i typically do a lot
of the data recovery at the place where i work, but it's all windows-based
stuff and thus very easy (i use a bunch of linuxy tools... because they
rock!)  unfortunately, i havn't found anything that will work with btrfs,
which is why i was asking for direction for a fs-finder (scan the drive,
look for data structures, pull out data) or what i need to learn to build
one.  i've never worked on something like that, but i wouldnt mind doing
it (i'd need help of course, but that's what the community is for).  :)

> This wouldn't have been a major problem if you had backups in the first
> place :)
> --
> Calvin Walton <calvin.walton@kepstin.ca>

you know, i scold people for not having backups, and my latest backup is
3+months old... :'(  being a hypocrite sucks!!  :P

thanks!
Justin

Re: filesystem finder / fixer

[Alex Elsayed]

Just realized I messed up sending this to the list.

Roman Mamedov wrote:

> On Mon, 30 Jul 2012 23:26:42 -0400 (EDT)
> serialhex@lavabit.com wrote:
> 
>> 1) is there a tool to help me recover data from my fs? I don't have a
>> backup of my partition table and so I have about 500GB of space where a
>> few partitionns might reside... GPT partitions mind you
> 
> If you only lost the partition table, there's a tool (strangely)named
> TestDisk, which can find the actual partitions on disk and restore it.
> Don't know if it supports GPT and BTRFS, though.
> 

If TestDisk doesn't support it, then you may be able to do it manually with 
some trial and error.

I just dumped the first 4 megabytes of my disk, and it looks like at offset
0x10040 (64K + 64 bytes) there's the string BHRfS (hex 5F 42 48 52 66 53 
5f). That matches the documentation (the first superblock should be at 64K). 
Therefore, if you can find that string, subtract 64 from the byte offset (to 
get the offset of the block it's in in bytes), divide that by 512, and 
subtract 125 (64 kilobytes in 512-byte blocks), you should have the number 
of 512-byte blocks from the start of the device that the partition should 
be.

Note: I have not actually done this. You may want to set the device read-
only (blockdev --setro) and use dm-linear directly to test whether the 
offset you find this way is correct.

Once you have a mapping or partition in the right place, you have two 
options for recovering your files: the btrfsck repair tool, or the btrfs-
restore extraction tool. The repair tool still doesn't fix all problems, but 
using btrfs-restore with btrfs-find-root has worked well for me in the past 
in one case where btrfsck didn't. Another benefit is that you can use it 
even while the disk is still in read-only mode and mapped via dm-linear, 
with essentially zero risk of damaging your data further. The downside is 
that you need a second drive that's big enough for all of your data.

Re: filesystem finder / fixer

[Alex Elsayed]

Alex Elsayed wrote:
> offset 0x10040 (64K + 64 bytes) there's the string BHRfS (hex 5F 42 48 52
> 66 53 5f). That matches the documentation (the first superblock should be

Ugh, and resending it stripped the underscores. _BHRfS_ is at 0x10040, not 
BHRfS (which would be at 0x10041)

Re: filesystem finder / fixer

[serialhex]

> Alex Elsayed wrote:
>
> Just realized I messed up sending this to the list.
>
> Roman Mamedov wrote:
>
>> On Mon, 30 Jul 2012 23:26:42 -0400 (EDT)
>> serialhex@lavabit.com wrote:
>>
>>> 1) is there a tool to help me recover data from my fs? I don't have a
>>> backup of my partition table and so I have about 500GB of space where a
>>> few partitionns might reside... GPT partitions mind you
>>
>> If you only lost the partition table, there's a tool (strangely)named
>> TestDisk, which can find the actual partitions on disk and restore it.
>> Don't know if it supports GPT and BTRFS, though.
>>
>
> If TestDisk doesn't support it, then you may be able to do it manually
> with
> some trial and error.
>
> I just dumped the first 4 megabytes of my disk, and it looks like at
> offset
> 0x10040 (64K + 64 bytes) there's the string BHRfS (hex 5F 42 48 52 66 53
> 5f). That matches the documentation (the first superblock should be at
> 64K).

ok, so it's been a few days, because it took me a few days to get
everything back up and running, but here's what i did:

(tl;dr - i got my data back!)

i did as Alex suggested.  but i was getting a *lot* of false positives. 
the string _BHRfS_H is in the source code, some binaries, or something....
so the fact that bits are bits when reading the device as a file, i have
to ignore all of those.  then there were a handful of other ones that
looked to be the first superblock, but every time i tried it didn't
work...  so i read up on btrfs some more.  the _BHRfS_H string starts at
0x40 (64 bytes) and the first bytes are a checksum of everything after. 
so i started with that, and found a number of the _BHRfS_H strings to have
00's before it, so a checksum of zero... which is *HIGHLY* unlikely.  so i
looked for that, and the first one i found with a checksum >0 i tried, and
succeeded!!

Thanks for the help & advice!!  i've now got my stuff back (and need to do
an actual backup :P )

hex

Re: filesystem finder / fixer

[Martin Steigerwald]

Am Freitag, 3. August 2012 schrieb serialhex@lavabit.com:
> > Alex Elsayed wrote:
> > 
> > Just realized I messed up sending this to the list.
> > 
> > Roman Mamedov wrote:
> >> On Mon, 30 Jul 2012 23:26:42 -0400 (EDT)
> >> 
> >> serialhex@lavabit.com wrote:
> >>> 1) is there a tool to help me recover data from my fs? I don't have
> >>> a backup of my partition table and so I have about 500GB of space
> >>> where a few partitionns might reside... GPT partitions mind you
> >> 
> >> If you only lost the partition table, there's a tool
> >> (strangely)named TestDisk, which can find the actual partitions on
> >> disk and restore it. Don't know if it supports GPT and BTRFS,
> >> though.
> > 
> > If TestDisk doesn't support it, then you may be able to do it
> > manually with
> > some trial and error.
> > 
> > I just dumped the first 4 megabytes of my disk, and it looks like at
> > offset
> > 0x10040 (64K + 64 bytes) there's the string BHRfS (hex 5F 42 48 52 66
> > 53 5f). That matches the documentation (the first superblock should
> > be at 64K).
> 
> ok, so it's been a few days, because it took me a few days to get
> everything back up and running, but here's what i did:
> 
> (tl;dr - i got my data back!)
> 
> i did as Alex suggested.  but i was getting a *lot* of false positives.
> the string _BHRfS_H is in the source code, some binaries, or
> something.... so the fact that bits are bits when reading the device
> as a file, i have to ignore all of those.  then there were a handful
> of other ones that looked to be the first superblock, but every time i
> tried it didn't work...  so i read up on btrfs some more.  the
> _BHRfS_H string starts at 0x40 (64 bytes) and the first bytes are a
> checksum of everything after. so i started with that, and found a
> number of the _BHRfS_H strings to have 00's before it, so a checksum
> of zero... which is *HIGHLY* unlikely.  so i looked for that, and the
> first one i found with a checksum >0 i tried, and succeeded!!
> 
> Thanks for the help & advice!!  i've now got my stuff back (and need to
> do an actual backup :P )

Now that would be either a good addition to testdisk or gpart if not 
already there or a nice thing for a btrfs recovery tool. Find my BTRFS 
partition. ;)

I think I will update my backup script to do sfdisk -d or gdisk -something 
> /root/partitioning before backuping ;). Always wanted to do that ;).

-- 
Martin 'Helios' Steigerwald - http://www.Lichtvoll.de
GPG: 03B0 0D6C 0040 0710 4AFA  B82F 991B EAAC A599 84C7