btrfs restore fails because of NO SPACE

btrfs restore fails because of NO SPACE

[Wolf Bublitz]

Hallo,

I have a confusing problem with my btrfs raid. Currently I am using the following setup:

btrfs fi show
Label: none  uuid: 93000933-e46d-403b-80d7-60475855e3f3
	Total devices 2 FS bytes used 2.56TiB
	devid    1 size 2.73TiB used 2.71TiB path /dev/sda
	devid    4 size 2.73TiB used 2.71TiB path /dev/sdb

As you can see both disks are full.

Actually I cannot mount my raid, even with recovery options enabled:

mount /dev/sda /mnt/Data -t btrfs -onospace_cache,clear_cache,enospc_debug,nodatacow
mount: wrong fs type, bad option, bad superblock on /dev/sda,
       missing codepage or helper program, or other error

       In some cases useful info is found in syslog - try
       dmesg | tail or so.

dmesg shows:

[ 1066.221696] BTRFS info (device sda): disabling disk space caching
[ 1066.227990] BTRFS info (device sda): force clearing of disk cache
[ 1066.234331] BTRFS info (device sda): setting nodatacow, compression disabled
[ 1066.243813] BTRFS error (device sda): parent transid verify failed on 9676657786880 wanted 242139 found 0
[ 1066.253672] BTRFS error (device sda): parent transid verify failed on 9676657786880 wanted 242139 found 0
[ 1066.263450] BTRFS error (device sda): parent transid verify failed on 9676657786880 wanted 242139 found 0
[ 1066.273234] BTRFS: failed to read chunk root on sda
[ 1066.279675] BTRFS warning (device sda): page private not zero on page 9676657786880
[ 1066.287482] BTRFS warning (device sda): page private not zero on page 9676657790976
[ 1066.295361] BTRFS warning (device sda): page private not zero on page 9676657795072
[ 1066.303204] BTRFS warning (device sda): page private not zero on page 9676657799168
[ 1066.369266] BTRFS: open_ctree failed

After spending some time with Google I found a possible solution for my problem by running:

btrfs restore -v /dev/sda /mnt/Data

Actually this operation fails silently (computer freezes). After examine the kernel logs I have found out that the operations fails because of „NO SPACE LEFT ON DEVICE“. Can anybody please give me a solution for this problem?

Greetings

Wolf Bublitz

Re: btrfs restore fails because of NO SPACE

[Chris Murphy]

What versions for kernel and btrfs-progs?

Have you tried only '-o ro,recovery' ? What kernel messages do you get for this?

Failure to read chunk tree message is usually bad. If you have a
recent enough btrfs-progs, try 'btrfs check' on the volume without
--repair and post the results; recent would be 4.4.1 or better.

Also a good idea is to post btrfs-show-super -f, there's a scant
possibility there's more than one backup chunk root and possible to
explicitly use an older one (but btrfs check--chunk-root is only in
v4.5 of btrfs-progs).



Chris Murphy

Re: btrfs restore fails because of NO SPACE

[Chris Murphy]

>btrfs fi show
>Label: none  uuid: 93000933-e46d-403b-80d7-60475855e3f3
 >       Total devices 2 FS bytes used 2.56TiB
  >      devid    1 size 2.73TiB used 2.71TiB path /dev/sda
   >     devid    4 size 2.73TiB used 2.71TiB path /dev/sdb


OK so why does it only list two devices? This is a three drive or four
drive raid10? This is Btrfs raid10 specifically? I'm confused now
about the setup and why btrfs fi show isn't saying there are missing
devices, there is no such thing as two drive btrfs raid10.


Chris Murphy

Re: btrfs restore fails because of NO SPACE

[Duncan]

Chris Murphy posted on Fri, 20 May 2016 15:53:07 -0600 as excerpted:

>>btrfs fi show Label: none  uuid: 93000933-e46d-403b-80d7-60475855e3f3
>>         Total devices 2 FS bytes used 2.56TiB
>>         devid    1 size 2.73TiB used 2.71TiB path /dev/sda
>>         devid    4 size 2.73TiB used 2.71TiB path /dev/sdb
> 
> 
> OK so why does it only list two devices? This is a three drive or four
> drive raid10? This is Btrfs raid10 specifically? I'm confused now about
> the setup and why btrfs fi show isn't saying there are missing devices,
> there is no such thing as two drive btrfs raid10.

I'm confused about where you got that it was a raid10?  I don't see it in 
anything he posted, that got here to gmane, at least.  In fact, I see 
only his initial thread-root post, and it doesn't mention raid10 at all 
that I can see.

So given that fi show says two devices, none missing, I'd say it can't be 
a raid10, and further, given that he didn't specify the raid type, the 
btrfs default for a two-device btrfs must be assumed, which is raid1 
system and metadata, single mode data.

But I think that's beside the point in terms of the original question.  I 
think the problem is with his understanding of restore.  See the reply 
directly to his post, that I'll be making after this one.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

Re: btrfs restore fails because of NO SPACE

[Duncan]

Wolf Bublitz posted on Fri, 20 May 2016 19:38:55 +0200 as excerpted:

> Hallo,
> 
> I have a confusing problem with my btrfs raid. Currently I am using the
> following setup:
> 
> btrfs fi show
> Label: none  uuid: 93000933-e46d-403b-80d7-60475855e3f3
> 	Total devices 2 FS bytes used 2.56TiB
>       devid    1 size 2.73TiB used 2.71TiB path /dev/sda
>       devid    4 size 2.73TiB used 2.71TiB path /dev/sdb
> 
> As you can see both disks are full.
> 
> Actually I cannot mount my raid, even with recovery options enabled:
> 
> mount /dev/sda /mnt/Data -t btrfs
> -onospace_cache,clear_cache,enospc_debug,nodatacow

[generic mount error]

> dmesg shows:
> 
> [ 1066.243813] BTRFS error (device sda): parent transid verify
> failed on 9676657786880 wanted 242139 found 0
[...]
> [ 1066.273234] BTRFS: failed to read chunk root on sda

None of those options are likely to help, there.

What /might/ help is the "usebackuproot" mount option, if your kernel is 
reasonably current, or the "recovery" mount option, if it's a bit older.  
Btrfs mount options are documented in the btrfs (5) manpage (not the 
btrfs (8) manpage, specify the 5), tho again, usebackuproot will only 
appear in the manpage if you're running a reasonably current btrfs-progs 
version (recovery should be listed in both new and old, but it's listed 
as deprecated in new, referring readers to the usebackuproot entry).  Or 
alternatively to the manpage, you can check the mount options listing on 
the wiki.

> After spending some time with Google I found a possible solution for my
> problem by running:
> 
> btrfs restore -v /dev/sda /mnt/Data
> 
> Actually this operation fails silently (computer freezes). After examine
> the kernel logs I have found out that the operations fails because of
> „NO SPACE LEFT ON DEVICE“. Can anybody please give me a solution for
> this problem?

You don't explicitly say what you expect btrfs restore to do, but given 
the specific command you use, I suspect that you misunderstand what it 
does, and it's actually working, but you are running out of space as a 
result of using restore incorrectly, because of that misunderstanding.

What btrfs restore does is provide you a read-only method to try to 
restore your files from a filesystem that won't mount, by rewriting what 
it can recover to an entirely different location on an entirely 
different, mounted, filesystem, which of course must contain enough space 
to hold a new copy of all those restored files.

And if the filesystem in question isn't mounted to its usual mountpoint,
/mnt/Data, that means you're trying to write all the recovered files to 
whatever filesystem actually contains the mountpoint itself, almost 
certainly the root filesystem (/), in this case.

And I'll place money on a bet that whatever your root filesystem is, it 
doesn't have the terabytes of free space that are likely to be necessary 
to restore all of that multi-device multi-TB per device btrfs!  
Otherwise, you /likely/ wouldn't be running the separate btrfs in the 
first place, but storing it on your main filesystem, instead.  So when 
btrfs restore runs out of room on / ... everything freezes.


IOW, in ordered to successfully use btrfs restore, you have to have a 
filesystem with enough free space available mounted somewhere in ordered 
to write the files to that btrfs restore is restoring!  If you don't, 
yes, you /will/ run into problems! =:^)

That said, it is possible to use pattern matching to tell btrfs restore 
to only restore say one directory at a time, and by using that if you 
don't have enough space for everything but are willing to give up some of 
what was stored, you can tell btrfs restore to only restore the vitally 
important stuff that will fit, and not bother trying to restore the less 
important stuff that won't fit.  Again, see the btrfs-restore manpage, 
for that and other details.

And, unless you want those restored files all written as root, you'll 
probably want to use the restore metadata option as well, to restore 
timestamps and owner/perms information.  Similarly, there's an option to 
restore symlinks as well, without which they'll be missing.  So you 
probably do want to check that manpage.  Just sayin'. =:^)

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman