* Unmountable btrfs filesystem - 'unable to find logical' / 'no mapping'
@ 2014-07-15 13:35 Gareth Clay
2014-07-16 8:00 ` Duncan
0 siblings, 1 reply; 4+ messages in thread
From: Gareth Clay @ 2014-07-15 13:35 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1672 bytes --]
Hi there,
I noticed yesterday that the mount points on my btrfs RAID1 filesystem
had become read-only. On a reboot, the filesystem fails to mount. I
wondered if someone here might be able offer any advice on how to
recover (if possible) from this position?
dmesg is showing the following errors
[61355.392226] btrfs: bdev /dev/sdd errs: wr 3175131, rd 2633223,
flush 0, corrupt 0, gen 0
[61356.205444] BTRFS critical (device sdd): unable to find logical
457140613120 len 4096
[61356.207007] BTRFS critical (device sdd): No mapping for
457140613120-457140617216
[61356.207007]
[61356.210005] BTRFS critical (device sdd): unable to find logical
457140613120 len 4096
[61356.211532] BTRFS critical (device sdd): No mapping for
457140613120-457140617216
[61356.211532]
[61356.214587] Failed to read block groups: -5
[61356.234306] btrfs: open_ctree failed
[61380.234793] btrfs: device label server-mirror devid 3 transid 40496 /dev/sdc
[61380.829916] btrfs: allowing degraded mounts
[61380.829918] btrfs: disk space caching is enabled
btrfsck /dev/sdc:
btrfsck: volumes.c:1104: btrfs_num_copies: Assertion '!(ce->start >
logical || ce->start + ce->size < logical)' failed.
Aborted
btrfs fi show:
Label: 'server-mirror' uuid: 2fec5bfc-aa24-4e1d-a18d-18da768c5ea5
Total devices 2 FS bytes used 962.99GiB
devid 2 size 1.82TiB used 977.03GiB path /dev/sdd
devid 3 size 1.82TiB used 977.03GiB path /dev/sdc
Btrfs v3.12
uname -a:
Linux ubuntu-pc 3.13.0-24-generic #47-Ubuntu SMP Fri May 2 23:30:00
UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
I've attached my dmesg.log gzipped to keep within the mailing list size limits.
Thanks for any help you might be able to offer!
[-- Attachment #2: dmesg.log.gz --]
[-- Type: application/x-gzip, Size: 26498 bytes --]
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Unmountable btrfs filesystem - 'unable to find logical' / 'no mapping'
2014-07-15 13:35 Unmountable btrfs filesystem - 'unable to find logical' / 'no mapping' Gareth Clay
@ 2014-07-16 8:00 ` Duncan
2014-07-17 23:09 ` Gareth Clay
0 siblings, 1 reply; 4+ messages in thread
From: Duncan @ 2014-07-16 8:00 UTC (permalink / raw)
To: linux-btrfs
Gareth Clay posted on Tue, 15 Jul 2014 14:35:22 +0100 as excerpted:
> I noticed yesterday that the mount points on my btrfs RAID1 filesystem
> had become read-only. On a reboot, the filesystem fails to mount. I
> wondered if someone here might be able offer any advice on how to
> recover (if possible) from this position?
I had a similar (but I think different) issue some weeks ago. It was my
first real experience with btrfs troubleshooting and recovery.
First, the recommendation is do NOT do btrfs check --repair except either
at the recommendation of a dev after they've seen the details and
determined it can fix them, or if your next step would be a new mkfs of
the filesystem, thus blowing away what's there anyway, so you've nothing
to lose. You can try btrfs check (aka btrfsck) without --repair to see
what it reports as that's read-only and thus won't break anything
further, but similarly, won't repair anything either.
Also, as a general recommendation, try a current kernel as btrfs is still
developing fast enough that if you're a kernel series behind, there's
fixes in the new version that you won't have in older kernels. I see
you're on an ubuntu 3.13 series kernel, and the recommendation would be
the latest 3.15 series stable kernel, if not the 3.16-rc series
development kernel, since that's past rc5 now and thus getting close to
release.
The userspace, btrfs-progs, isn't quite as critical, but running at least
v3.12 (which you are), is recommended. FWIW, v3.14.2 is current (as of
when I last checked a couple days ago anyway) and is what I am running
here.
In general, you can try mounting with recovery and then with recovery,ro
options, but that didn't work here. You can also try with the degraded
option (tho I didn't), to see if it'll mount with just one of the pair.
Of course, btrfs is still not fully stable and keeping current backups is
recommended. I did have backups, but they weren't as current as I wanted.
Beyond that, there's btrfs restore (a separate btrfs-restore executable
in older btrfs-progs, part of the main btrfs executable in newer
versions), which is what I ended up using and is what the rest of this
reply is about. That does NOT mount or write to the filesystem, but DOES
let you pull files off the unmounted filesystem and write them to a
working filesystem (btrfs or other, it was reiserfs here) in ordered to
recover what you can. You can use --dry-run to list files that would be
recovered in ordered to get an idea of how much it can recover.
There's a page on the wiki about using btrfs recover in combination with
btrfs-find-root, if the current root is damaged and won't let you recover
much. Note that "generation" and "transid" refer to the same thing, and
you want to specify the root (using the -t location option, with the
location found using find-root) that lets you recover the most. The -l
(list tree roots) option is also useful in this context.
https://btrfs.wiki.kernel.org/index.php/Restore
Of course restoring in this manner means you have to have somewhere else
to put what you restore, which was fine for me as I'm using relatively
small independent btrfs filesystems and could restore to a larger
reiserfs on a different device, but could be rather tougher for large
multi-terabyte filesystems, unless you have (or purchase) a spare disk to
put it on.
One thing I did NOT realize until later, however, is that btrfs restore
loses the user and permissions information (at least without -x, which
says it restores extended attributes, I didn't try it with that). I
hacked up a find script to compare the restore to the backup and set
ownership/permissions appropriately based on the files in the backup, but
of course that didn't help for files that were new since the backup, and
I had to set their ownership/permissions manually.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Unmountable btrfs filesystem - 'unable to find logical' / 'no mapping'
2014-07-16 8:00 ` Duncan
@ 2014-07-17 23:09 ` Gareth Clay
2014-07-18 10:01 ` Duncan
0 siblings, 1 reply; 4+ messages in thread
From: Gareth Clay @ 2014-07-17 23:09 UTC (permalink / raw)
To: linux-btrfs
Duncan <1i5t5.duncan <at> cox.net> writes:
>
> Gareth Clay posted on Tue, 15 Jul 2014 14:35:22 +0100 as excerpted:
>
> > I noticed yesterday that the mount points on my btrfs RAID1 filesystem
> > had become read-only. On a reboot, the filesystem fails to mount. I
> > wondered if someone here might be able offer any advice on how to
> > recover (if possible) from this position?
>
> I had a similar (but I think different) issue some weeks ago. It was my
> first real experience with btrfs troubleshooting and recovery.
>
> First, the recommendation is do NOT do btrfs check --repair except either
> at the recommendation of a dev after they've seen the details and
> determined it can fix them, or if your next step would be a new mkfs of
> the filesystem, thus blowing away what's there anyway, so you've nothing
> to lose. You can try btrfs check (aka btrfsck) without --repair to see
> what it reports as that's read-only and thus won't break anything
> further, but similarly, won't repair anything either.
>
> Also, as a general recommendation, try a current kernel as btrfs is still
> developing fast enough that if you're a kernel series behind, there's
> fixes in the new version that you won't have in older kernels. I see
> you're on an ubuntu 3.13 series kernel, and the recommendation would be
> the latest 3.15 series stable kernel, if not the 3.16-rc series
> development kernel, since that's past rc5 now and thus getting close to
> release.
>
> The userspace, btrfs-progs, isn't quite as critical, but running at least
> v3.12 (which you are), is recommended. FWIW, v3.14.2 is current (as of
> when I last checked a couple days ago anyway) and is what I am running
> here.
>
> In general, you can try mounting with recovery and then with recovery,ro
> options, but that didn't work here. You can also try with the degraded
> option (tho I didn't), to see if it'll mount with just one of the pair.
>
> Of course, btrfs is still not fully stable and keeping current backups is
> recommended. I did have backups, but they weren't as current as I wanted.
>
> Beyond that, there's btrfs restore (a separate btrfs-restore executable
> in older btrfs-progs, part of the main btrfs executable in newer
> versions), which is what I ended up using and is what the rest of this
> reply is about. That does NOT mount or write to the filesystem, but DOES
> let you pull files off the unmounted filesystem and write them to a
> working filesystem (btrfs or other, it was reiserfs here) in ordered to
> recover what you can. You can use --dry-run to list files that would be
> recovered in ordered to get an idea of how much it can recover.
>
> There's a page on the wiki about using btrfs recover in combination with
> btrfs-find-root, if the current root is damaged and won't let you recover
> much. Note that "generation" and "transid" refer to the same thing, and
> you want to specify the root (using the -t location option, with the
> location found using find-root) that lets you recover the most. The -l
> (list tree roots) option is also useful in this context.
>
> https://btrfs.wiki.kernel.org/index.php/Restore
>
> Of course restoring in this manner means you have to have somewhere else
> to put what you restore, which was fine for me as I'm using relatively
> small independent btrfs filesystems and could restore to a larger
> reiserfs on a different device, but could be rather tougher for large
> multi-terabyte filesystems, unless you have (or purchase) a spare disk to
> put it on.
>
> One thing I did NOT realize until later, however, is that btrfs restore
> loses the user and permissions information (at least without -x, which
> says it restores extended attributes, I didn't try it with that). I
> hacked up a find script to compare the restore to the backup and set
> ownership/permissions appropriately based on the files in the backup, but
> of course that didn't help for files that were new since the backup, and
> I had to set their ownership/permissions manually.
>
Hi Duncan,
Thanks for your thorough response and the tips - sorry to hear you've had
issues too. Point taken on the kernel updates! I'm in a similar situation to
you - this is my first btrfs recovery experience. I've been playing with the fs
for some time and have had no apparent issues, but this has been a useful
reality check. Read / write error counts were high so there's a suggestion that
it might be down to drive failure.
In the end I had a lot of help from xaba on the #btrfs IRC channel, whose
suggestions got me to the point where, with a bang up to date version of the
userspace utils, I could get a successful btrfsck run using the -b option (3.12
only got part way). At that point btrfs restore still couldn't be run, degraded
mounting also wouldn't work, and I'd spent about as much time as I was prepared
to spend on recovering this fs, so I took a deep breath and ran btrfsck
--repair. That's got me to the point where btrfs restore can now be run, so I'm
going to dump as much as I can to a spare drive and then blow the file system
(and maybe a dodgy disk) away.
I'm not fully sure about the btrfs restore -x behaviour either. Ownership of the
restored files is still incorrect, but maybe it affects r/w/x permissions,
which look fairly sensible for the small set of files I've looked at so far...
^ permalink raw reply [flat|nested] 4+ messages in thread
* Re: Unmountable btrfs filesystem - 'unable to find logical' / 'no mapping'
2014-07-17 23:09 ` Gareth Clay
@ 2014-07-18 10:01 ` Duncan
0 siblings, 0 replies; 4+ messages in thread
From: Duncan @ 2014-07-18 10:01 UTC (permalink / raw)
To: linux-btrfs
Gareth Clay posted on Thu, 17 Jul 2014 23:09:08 +0000 as excerpted:
> I'm not fully sure about the btrfs restore -x behaviour either.
> Ownership of the restored files is still incorrect, but maybe it
> affects r/w/x permissions, which look fairly sensible for the small set
> of files I've looked at so far...
Thanks and good to read that you eventually able to successfully restore
most of the files too. A wakeup call indeed! I've always stressed
backups with btrfs and did have them, so wouldn't have been too bad off
if I had to revert to them. I'd simply let them get inconveniently
outdated, and between that and the chance it gave me to actually get real
experience with btrfs restore, I decided to try it first.
But your reply reminded me...
Something think I forgot to mention is that btrfs restore didn't restore
symlinks, at all, not as symlinks and not as copies of the files they
pointed at. It was as if the symlinks simply didn't exist on the source
filesystem I was restoring from, so I'm guessing the implementation
simply overlooked symlinks as something it needed to deal with.
Meanwhile, on ownership/permissions I think btrfs restore must simply
find the data and write it out as the user (presumably root) it is run
as, using the existing umask, just as a normal user file write would do
by default. So if your root and user umasks are identical (presumably
0022), you probably won't notice the permissions difference. My root
umask is 0022 while my user umask is 0027, so I noticed.
--
Duncan - List replies preferred. No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master." Richard Stallman
^ permalink raw reply [flat|nested] 4+ messages in thread
end of thread, other threads:[~2014-07-18 10:01 UTC | newest]
Thread overview: 4+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2014-07-15 13:35 Unmountable btrfs filesystem - 'unable to find logical' / 'no mapping' Gareth Clay
2014-07-16 8:00 ` Duncan
2014-07-17 23:09 ` Gareth Clay
2014-07-18 10:01 ` Duncan
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.