From: Richard Weinberger <richard@nod.at>
To: linux-btrfs@vger.kernel.org
Subject: Decoding "unable to fixup (regular)" errors
Date: Tue, 05 Nov 2019 23:03:01 +0100 [thread overview]
Message-ID: <1591390.YpsIS3gr9g@blindfold> (raw)
Hi!
One of my build servers logged the following:
[10511433.614135] BTRFS info (device md1): relocating block group 2931997933568 flags data
[10511441.887812] BTRFS info (device md1): found 135 extents
[10511466.539198] BTRFS info (device md1): found 135 extents
[10511472.805969] BTRFS info (device md1): found 1 extents
[10511480.786194] BTRFS info (device md1): relocating block group 2933071675392 flags data
[10511487.314283] BTRFS info (device md1): found 117 extents
[10511498.483226] BTRFS info (device md1): found 117 extents
[10511506.708389] BTRFS info (device md1): relocating block group 2930890637312 flags system|dup
[10511508.386025] BTRFS info (device md1): found 5 extents
[10511511.382986] BTRFS info (device md1): relocating block group 2935219159040 flags system|dup
[10511512.565190] BTRFS info (device md1): found 5 extents
[10511519.032713] BTRFS info (device md1): relocating block group 2935252713472 flags system|dup
[10511520.586222] BTRFS info (device md1): found 5 extents
[10511523.107052] BTRFS info (device md1): relocating block group 2935286267904 flags system|dup
[10511524.392271] BTRFS info (device md1): found 5 extents
[10511527.381846] BTRFS info (device md1): relocating block group 2935319822336 flags system|dup
[10511528.766564] BTRFS info (device md1): found 5 extents
[10857025.725121] BTRFS info (device md1): relocating block group 2934145417216 flags data
[10857057.071228] BTRFS info (device md1): found 1275 extents
[10857073.721609] BTRFS info (device md1): found 1231 extents
[10857086.237500] BTRFS info (device md1): relocating block group 2935386931200 flags data
[10857095.182532] BTRFS info (device md1): found 151 extents
[10857125.204024] BTRFS info (device md1): found 151 extents
[10857133.473086] BTRFS info (device md1): relocating block group 2935353376768 flags system|dup
[10857135.063924] BTRFS info (device md1): found 5 extents
[10857138.066852] BTRFS info (device md1): relocating block group 2937534414848 flags system|dup
[10857139.542984] BTRFS info (device md1): found 5 extents
[10857142.083035] BTRFS info (device md1): relocating block group 2937567969280 flags system|dup
[10857143.664667] BTRFS info (device md1): found 5 extents
[10857145.971518] BTRFS info (device md1): relocating block group 2937601523712 flags system|dup
[10857146.924543] BTRFS info (device md1): found 5 extents
[10857150.289957] BTRFS info (device md1): relocating block group 2937635078144 flags system|dup
[10857152.173086] BTRFS info (device md1): found 5 extents
[10860370.725465] scrub_handle_errored_block: 71 callbacks suppressed
[10860370.764356] btrfs_dev_stat_print_on_error: 71 callbacks suppressed
[10860370.764359] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2291, gen 0
[10860370.764593] scrub_handle_errored_block: 71 callbacks suppressed
[10860370.764595] BTRFS error (device md1): unable to fixup (regular) error at logical 593483341824 on dev /dev/md1
[10860395.236787] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2292, gen 0
[10860395.237267] BTRFS error (device md1): unable to fixup (regular) error at logical 595304841216 on dev /dev/md1
[10860395.506085] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2293, gen 0
[10860395.506560] BTRFS error (device md1): unable to fixup (regular) error at logical 595326820352 on dev /dev/md1
[10860395.511546] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2294, gen 0
[10860395.512061] BTRFS error (device md1): unable to fixup (regular) error at logical 595327647744 on dev /dev/md1
[10860395.664956] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2295, gen 0
[10860395.664959] BTRFS error (device md1): unable to fixup (regular) error at logical 595344850944 on dev /dev/md1
[10860395.677733] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2296, gen 0
[10860395.677736] BTRFS error (device md1): unable to fixup (regular) error at logical 595346452480 on dev /dev/md1
[10860395.770918] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2297, gen 0
[10860395.771523] BTRFS error (device md1): unable to fixup (regular) error at logical 595357601792 on dev /dev/md1
[10860395.789808] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2298, gen 0
[10860395.790455] BTRFS error (device md1): unable to fixup (regular) error at logical 595359870976 on dev /dev/md1
[10860395.806699] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2299, gen 0
[10860395.807381] BTRFS error (device md1): unable to fixup (regular) error at logical 595361865728 on dev /dev/md1
[10860395.918793] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2300, gen 0
[10860395.919513] BTRFS error (device md1): unable to fixup (regular) error at logical 595372343296 on dev /dev/md1
[10860395.993817] BTRFS error (device md1): bdev /dev/md1 errs: wr 0, rd 0, flush 0, corrupt 2301, gen 0
[10860395.994574] BTRFS error (device md1): unable to fixup (regular) error at logical 595384438784 on dev /dev/md1
[11033396.165434] md: data-check of RAID array md0
[11033396.273818] md: data-check of RAID array md2
[11033396.282822] md: delaying data-check of md1 until md0 has finished (they share one or more physical units)
[11033406.609033] md: md0: data-check done.
[11033406.623027] md: data-check of RAID array md1
[11035858.847538] md: md2: data-check done.
[11043788.746468] md: md1: data-check done.
For obvious reasons the "BTRFS error (device md1): unable to fixup (regular) error" lines made me nervous
and I would like to understand better what is going on.
The system has ECC memory with md1 being a RAID1 which passes all health checks.
I tried to find the inodes behind the erroneous addresses without success.
e.g.
$ btrfs inspect-internal logical-resolve -v -P 593483341824 /
ioctl ret=0, total_size=4096, bytes_left=4080, bytes_missing=0, cnt=0, missed=0
$ echo $?
1
My kernel is 4.12.14-lp150.12.64-default (OpenSUSE 15.0), so not super recent but AFAICT btrfs should be sane
there. :-)
What could cause the errors and how to dig further?
Thanks,
//richard
next reply other threads:[~2019-11-05 22:03 UTC|newest]
Thread overview: 15+ messages / expand[flat|nested] mbox.gz Atom feed top
2019-11-05 22:03 Richard Weinberger [this message]
2019-11-08 22:06 ` Decoding "unable to fixup (regular)" errors Richard Weinberger
2019-11-08 22:16 ` Zygo Blaxell
2019-11-08 22:09 ` Zygo Blaxell
2019-11-08 22:21 ` Richard Weinberger
2019-11-08 22:25 ` Zygo Blaxell
2019-11-08 22:31 ` Richard Weinberger
2019-11-08 23:39 ` Zygo Blaxell
2019-11-09 9:58 ` checksum errors in orphaned blocks on multiple systems (Was: Re: Decoding "unable to fixup (regular)" errors) Richard Weinberger
2019-11-13 3:34 ` Zygo Blaxell
2019-11-09 10:00 ` Decoding "unable to fixup (regular)" errors Richard Weinberger
2019-11-13 3:31 ` Zygo Blaxell
2019-11-13 18:17 ` Chris Murphy
2019-11-13 18:24 ` Chris Murphy
2019-11-16 6:16 ` Zygo Blaxell
Reply instructions:
You may reply publicly to this message via plain-text email
using any one of the following methods:
* Save the following mbox file, import it into your mail client,
and reply-to-all from there: mbox
Avoid top-posting and favor interleaved quoting:
https://en.wikipedia.org/wiki/Posting_style#Interleaved_style
* Reply using the --to, --cc, and --in-reply-to
switches of git-send-email(1):
git send-email \
--in-reply-to=1591390.YpsIS3gr9g@blindfold \
--to=richard@nod.at \
--cc=linux-btrfs@vger.kernel.org \
/path/to/YOUR_REPLY
https://kernel.org/pub/software/scm/git/docs/git-send-email.html
* If your mail client supports setting the In-Reply-To header
via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line
before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox;
as well as URLs for NNTP newsgroup(s).