* parent transid verify failed + level verify failed
@ 2023-11-19 11:34 Martin Steigerwald
2023-11-19 16:41 ` Martin Steigerwald
0 siblings, 1 reply; 3+ messages in thread
From: Martin Steigerwald @ 2023-11-19 11:34 UTC (permalink / raw)
To: linux-btrfs
Hi!
Have a great Sunday and absolutely feel free to reply at a later time :)
After having used linux-6.7-rc1 almost rc2, git commit
791c8ab095f71327899023223940dd52257a4173 in order to test out BCacheFS,
back on 6.6.1 after two attempts of non working hibernation and a strange
experience with GRUB presenting its command prompt instead of a boot menu¹,
I now got this funny stuff:
[ 1849.408572] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1178386432 mirror 1 wanted 538478 found 538901
[ 1849.408943] BTRFS error (device dm-1: state EA): level verify failed on logical 1132724224 mirror 1 wanted 0 found 1
[ 1849.414073] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1178386432 mirror 1 wanted 538478 found 538901
[ 1849.446144] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1204469760 mirror 1 wanted 538478 found 538904
[ 1849.448792] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1204469760 mirror 1 wanted 538478 found 538904
[ 1849.449515] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1204469760 mirror 1 wanted 538478 found 538904
[ 1849.451647] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1204469760 mirror 1 wanted 538478 found 538904
[ 1849.454860] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1204469760 mirror 1 wanted 538478 found 538904
[ 1849.455527] BTRFS error (device dm-1: state EA): parent transid verify failed on 2a006853-bec0-4026-94aa-8ade2e0c3674logical 1204469760 mirror 1 wanted 538478 found 538904
[ 1849.456286] BTRFS error (device dm-1: state EA): parent transid verify failed on logical 1204469760 mirror 1 wanted 538478 found 538904
Noticed as BTRFS for / went read only.
Kernel 6.6.1 on Devuan Ceres on ThinkPad T14 AMD Gen 1 32 GiB RAM with
2 TB Samsung 980 Pro NVME SSD.
I cannot do a scrub cause despite
WARNING: failed to write the progress status file: Read-only file system. Status recording disabled
scrub started on /, fsid […] (pid=31693)
scrub does not seem to do anything.
Will now boot into GRML and recover from backup, also checking the other
BTRFS filesystems.
While focus will be to get to a full working state as quickly as possible,
I intend to do a full dd copy for forensic analysis.
[1] Described in more detail at the end of:
Re: Questions related to BCacheFS
https://lore.kernel.org/linux-bcachefs/2273246.iZASKD2KPV@lichtvoll.de/T/#m3fc22fb92f1f2d160f2b3a2387dc1f10d067fb97
Best,
--
Martin
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: parent transid verify failed + level verify failed
2023-11-19 11:34 parent transid verify failed + level verify failed Martin Steigerwald
@ 2023-11-19 16:41 ` Martin Steigerwald
2023-11-19 18:41 ` Martin Steigerwald
0 siblings, 1 reply; 3+ messages in thread
From: Martin Steigerwald @ 2023-11-19 16:41 UTC (permalink / raw)
To: linux-btrfs
Martin Steigerwald - 19.11.23, 12:34:44 CET:
> After having used linux-6.7-rc1 almost rc2, git commit
> 791c8ab095f71327899023223940dd52257a4173 in order to test out BCacheFS,
> back on 6.6.1 after two attempts of non working hibernation and a
> strange experience with GRUB presenting its command prompt instead of a
> boot menu¹, I now got this funny stuff:
>
> [ 1849.408572] BTRFS error (device dm-1: state EA): parent transid
> verify failed on logical 1178386432 mirror 1 wanted 538478 found 538901
Finally recovered.
/ was not mountable anymore after booting into GRML. Made image with dd
and restored from backup onto a newly created BTRFS.
/home also had errors. Did not take a chance. Scrubbed it, updated a
backup with good old rsync, recreated filesystem and restored from updated
backup.
Another filesystem with larger files that only sees user space writes when I
write something to it, has been okay.
I have the following diagnostic data:
1) For /: dd image on external SSD with BTRFS, thus I can snapshot and/or
ref-link copy it around and run various diagnostic stuff on it, btrfs check
output (no repair), scrub was okay.
2) For /home: btrfs check output (no repair), scrub was okay.
3) I think I have some dmesg output from what caused the root filesystem
going read only, but I think that is the output I already posted.
SMART status of 2TB Samsung 980 Pro disk okay. I do not assume that there
is anything wrong with the device.
On what caused the corruption? No idea. Could be something wrong with that
6.7-almost-rc kernel, could be some corruption that happened on failed
hibernation attempts, could have to do something with BCacheFS albeit I
doubt it. Anyway, I am back on 6.6.1 for now and will not touch 6.7 kernel
again on this laptop before I am confident that it is is safe.
About diagnostic data: Willing to post more after the weekend. Just let me
know what you need. Of course I am not willing to upload complete dd image
somewhere, even not with just the operating system filesystem, but I can
run stuff on it and provide output. Otherwise you can have any of the btrfs
check output.
Best,
--
Martin
^ permalink raw reply [flat|nested] 3+ messages in thread
* Re: parent transid verify failed + level verify failed
2023-11-19 16:41 ` Martin Steigerwald
@ 2023-11-19 18:41 ` Martin Steigerwald
0 siblings, 0 replies; 3+ messages in thread
From: Martin Steigerwald @ 2023-11-19 18:41 UTC (permalink / raw)
To: linux-btrfs
Martin Steigerwald - 19.11.23, 17:41:17 CET:
> / was not mountable anymore after booting into GRML. Made image with dd
> and restored from backup onto a newly created BTRFS.
I at least have a possible explanation.
Yesterday I experienced 6.7-almost-rc2 not hibernating correctly, but
instead hanging with black screen. I rebooted into 6.6.1. Last thing I did
was remove 6.7-almost-rc2 and hibernate.
In the morning I was greeted by GRUB command line prompt for a reason I do
not understand yet either. I recovered from this with two GRML sessions.
For that at least I unlocked LUKS and mounted / and /boot in GRML. If my
memory is correct after I fixed booting up in these two GRML sessions,
kernel 6.6.1 resumed from hibernation. That does make some session as in
the GRML session I did not touch the swap volume. So I bet the hibernation
image was not invalidated. However that would explain the corruption on /
filesystem. Cause it was mounted in between and then the kernel resumed
from a hibernation image with outdated in-memory data structures for
BTRFS.
If my memory and analysis is correct this would easily explain the
corruption on /. Still not sure what happened to /home. I think I did not
touch it on attempting to repair GRUB. However I am not 100% sure about
that.
Would be interesting to know whether diagnostic data would fit that
explanation.
In case that is correct, then I think it would be good that in case I ever
need to fix up GRUB again I also activate swap volume within GRML in the
hope to invalidate the hibernation image. But I am not completely sure
whether activating swap would be enough for that.
> /home also had errors. Did not take a chance. Scrubbed it, updated a
> backup with good old rsync, recreated filesystem and restored from
> updated backup.
Best,
--
Martin
^ permalink raw reply [flat|nested] 3+ messages in thread
end of thread, other threads:[~2023-11-19 18:42 UTC | newest]
Thread overview: 3+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2023-11-19 11:34 parent transid verify failed + level verify failed Martin Steigerwald
2023-11-19 16:41 ` Martin Steigerwald
2023-11-19 18:41 ` Martin Steigerwald
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).