Re: btrfs check discovered possibly inconsistent journal and now the errors are gone

[Andreas Falk <mail@andreasfalk.se>]

Oh, and apologies if this is not a good forum for asking this type of
question. Additionally, the various versions and other information
that might be useful:

$ uname -a
Linux pandora 5.12.3-arch1-1 #1 SMP PREEMPT Wed, 12 May 2021 17:54:18
+0000 x86_64 GNU/Linux

$ btrfs --version
btrfs-progs v5.12.1

$ btrfs fi show
Label: '$LABEL'  uuid: f0aae703-a128-42e7-8ad7-16ebd57833cb
        Total devices 4 FS bytes used 6.85TiB
        devid    1 size 7.28TiB used 5.91TiB path /dev/sdf
        devid    3 size 2.73TiB used 1.39TiB path /dev/sdb
        devid    4 size 1.82TiB used 489.00GiB path /dev/sdc
        devid    5 size 7.28TiB used 5.93TiB path /dev/sde

$ btrfs fi df /home
Data, RAID1: total=6.84TiB, used=6.84TiB
System, RAID1: total=32.00MiB, used=1.12MiB
Metadata, RAID1: total=11.00GiB, used=9.45GiB
GlobalReserve, single: total=512.00MiB, used=0.00B

Cheers!
Andreas

On Sun, May 23, 2021 at 4:55 PM Andreas Falk <mail@andreasfalk.se> wrote:
>
> Hey,
>
> I want to start with clarifying that I've got backups of my important
> data so what I'm asking here is primarily for my own education to
> understand how btrfs works and to make restoring things more
> convenient.
>
> I'm running a small home server with family photos etc with btrfs in
> raid1 and we recently experienced a power cut. I wasn't around when it
> got turned back on and when I finally got to it everything had run for
> ~2h with the filesystem mounted in readwrite mode so I ran (after the
> fact I realized that I should have probably unmounted immediately and
> made sure /etc/fstab had everything in readonly mode):
>
> $ sudo btrfs check --readonly --force /dev/sdb
>
> and it seemed to mostly run fine but after a while it started printing
> messages like this along with what looked like some paths that were
> problematic (from what I remember, but these are not my exact
> messages):
>
> parent transid verify failed on 31302336512 wanted 62455 found 62456
> parent transid verify failed on 31302336512 wanted 62455 found 62456
> parent transid verify failed on 31302336512 wanted 62455 found 62456
>
> along with (these are my exact messages from a log that I saved):
>
> The following tree block(s) is corrupted in tree 259:
> tree block bytenr: 17047541170176, level: 0, node key:
> (18446744073709551606, 128, 25115695316992)
> The following tree block(s) is corrupted in tree 260:
> tree block bytenr: 17047541170176, level: 0, node key:
> (18446744073709551606, 128, 25115695316992)
> tree block bytenr: 17047547396096, level: 0, node key:
> (18446744073709551606, 128, 25187668619264)
> tree block bytenr: 17047547871232, level: 0, node key:
> (18446744073709551606, 128, 25124709920768)
> tree block bytenr: 17047549526016, level: 0, node key:
> (18446744073709551606, 128, 25195576877056)
> tree block bytenr: 17047551426560, level: 0, node key:
> (18446744073709551606, 128, 25162283048960)
> tree block bytenr: 17047551803392, level: 0, node key:
> (18446744073709551606, 128, 25177327333376)
>
> I didn't have time to look into it deeper at the time and decided to
> just shut it down cleanly until today when I'd have some time to look
> further at it. I booted it today (still in readwrite unfortunately)
> and immediately modified /etc/fstab to not mount any of the volumes,
> disabled services that might touch them and then rebooted it again to
> make sure it's not touching the disks anymore. Then I ran a check
> again:
>
> $ sudo btrfs check --readonly --progress /dev/sdb
>
> and now it's no longer finding any problems or printing any paths that
> are problematic.
>
> From what I've understood from this[a] article, the errors I saw were
> likely due to an inconsistent journal.
>
> Now for my questions:
>
> 1. I'm guessing that my reboots, in particular the ones where I still
> had it mounted in readwrite mode somehow cleared the journal and
> that's why I'm no longer seeing any errors. Does this sound
> plausible/correct? The errors being gone without me manually clearing
> them feels scary to me.
> 2. Is there any way to identify the paths again that were problematic
> based on the values in the log that I posted above so I can figure out
> what to restore from backups rather than doing a full restore?
>
> a. https://ownyourbits.com/2019/03/03/how-to-recover-a-btrfs-partition/
>
> Thank you,
> Andreas