Need advice for fixing a btrfs volume

Need advice for fixing a btrfs volume

[UGlee]

Dear:

We are using btrfs in an embedded arm/linux device.

The bootloader (u-boot) has only limited support for btrfs.
Occasionally the device lost power supply unexpectedly, leaving an
inconsistent file system on eMMC. If I dump the partition image from
eMMC and mount it on linux desktop, the file system is perfectly
usable.

My guess is that the linux kernel can fully handle the journalled
update and garbage data. But the u-boot cannot. So I consider to add a
minimal ext4 rootfs partition as a fallback. When u-boot cannot read
file from btrfs partition, it can switch to a minimal Linux system
booting from an ext4 fs.

Then I have a chance to use some tool to fix btrfs volume and reboot
the system. My question is which tools is recommended for this
purpose? According to the following page:

https://btrfs.wiki.kernel.org/index.php/Btrfsck

btrfsck is said to be deprecated. `btrfs check --repair` seems to be a
full volume check and time-consuming. All I need is just a good
superblock and a few files could be loaded. Most frequently complaints
from u-boot is the superblock issue such as `root_backup not found`.
It there a way to just fix the superblock, settle all journalled
update, and make sure the required several files is OK?

Tianfu Ma

Re: Need advice for fixing a btrfs volume

[Hugo Mills]

On Thu, Aug 29, 2019 at 10:45:37PM +0800, UGlee wrote:
> Dear:
> 
> We are using btrfs in an embedded arm/linux device.
> 
> The bootloader (u-boot) has only limited support for btrfs.
> Occasionally the device lost power supply unexpectedly, leaving an
> inconsistent file system on eMMC. If I dump the partition image from
> eMMC and mount it on linux desktop, the file system is perfectly
> usable.
> 
> My guess is that the linux kernel can fully handle the journalled
> update and garbage data.

   btrfs doesn't have a journal -- if the hardware is telling the
truth about barriers, and about written data reaching permanent
storage, then the FS structures on disk are always consistent. It's
got a log tree which is used for recovery of partial writes in the
case of a crash midway through a transaction, but that doesn't affect
the primary FS structures.

> But the u-boot cannot. So I consider to add a minimal ext4 rootfs
> partition as a fallback. When u-boot cannot read file from btrfs
> partition, it can switch to a minimal Linux system booting from an
> ext4 fs.

> Then I have a chance to use some tool to fix btrfs volume and reboot
> the system. My question is which tools is recommended for this
> purpose?

   It depends on the nature of the failure (if there is one, and why
uboot can't read the FS. Maybe it's saying that if there's a non-empty
log tree, it's not going to handle it (but there would be additional
code needed to check that). If that were the case, then simply
mounting the FS and unmounting it cleanly would work.

> According to the following page:
> 
> https://btrfs.wiki.kernel.org/index.php/Btrfsck
> 
> btrfsck is said to be deprecated. `btrfs check --repair` seems to be a
> full volume check and time-consuming.

   btrfs check --repair *is* btrfsck, under a different name. They're
the same code.

> All I need is just a good superblock and a few files could be
> loaded. Most frequently complaints from u-boot is the superblock
> issue such as `root_backup not found`.  It there a way to just fix
> the superblock, settle all journalled update, and make sure the
> required several files is OK?

   Mount the FS and unmount it cleanly?

   Hugo.

-- 
Hugo Mills             | Prisoner unknown: Return to Zenda.
hugo@... carfax.org.uk |
http://carfax.org.uk/  |
PGP: E2AB1DE4          |

Re: Need advice for fixing a btrfs volume

[Chris Murphy]

On Thu, Aug 29, 2019 at 8:57 AM Hugo Mills <hugo@carfax.org.uk> wrote:
>
> On Thu, Aug 29, 2019 at 10:45:37PM +0800, UGlee wrote:
> > Dear:
> >
> > We are using btrfs in an embedded arm/linux device.
> >
> > The bootloader (u-boot) has only limited support for btrfs.
> > Occasionally the device lost power supply unexpectedly, leaving an
> > inconsistent file system on eMMC. If I dump the partition image from
> > eMMC and mount it on linux desktop, the file system is perfectly
> > usable.

What kernel messages do you get when trying to mount the mmc device?
And also, does it mount ok with '-o ro,norecovery' ? And also I wonder
if there is even a log tree, which you can discover with 'btrfs insp
dump-s -f <dev>' and see if the log tree address is something other
than 0. Anyway, point is, Btrfs gets to a certain point in the mount
process and will try to do a write, and if that write fails, it
complains. Writes could fail on the mmc device but succeed on the
image file.

Both eMMC and SD cards are dreadfully susceptible to failure with
abrupt power supply loss, perhaps especially if writes are happening
at the time of the loss (there probably is some research and better
anecdotal information to support this contention). This topic comes up
all the time on Hacker News and people who do this a lot swear by
getting industrial grade flash for such embedded devices. This stuff
is not cheap compared to consumer grade flash. It's decently likely
the consumer grade stuff is optimized for exFAT, and the kind of
sequential writes you get from photos and video being written out by a
camera, rather than a file system like Btrfs (or even ext4 or XFS)
containing an operating system, and writing system logs/journals, and
doing updates, and things like that.



> >
> > My guess is that the linux kernel can fully handle the journalled
> > update and garbage data.
>
>    btrfs doesn't have a journal -- if the hardware is telling the
> truth about barriers, and about written data reaching permanent
> storage, then the FS structures on disk are always consistent. It's
> got a log tree which is used for recovery of partial writes in the
> case of a crash midway through a transaction, but that doesn't affect
> the primary FS structures.

Yeah and the log tree is per subvolume. A possible work around might
be to separate things in their own subvolumes, thereby obviating the
problem with a bootloader's lack of support for log tree replay. In
that case rudimentary breakdown would be boot, root, home subvolumes
at the top level of the file system, that way subvolid 5 isn't getting
the log tree added, rather it'll be root or home - hopefully boot
would be spared a log tree if writes weren't happening at the time of
the power failure.

Also, it's worth 'chattr +C' on the boot subvolume to ensure it's not
subject to compression mount options, if the bootloader doesn't
explicitly support compression.

-- 
Chris Murphy