* How to (attempt to) repair these btrfs errors
@ 2022-02-28 18:41 Carsten Grommel
2022-03-01 10:55 ` AW: " Carsten Grommel
0 siblings, 1 reply; 11+ messages in thread
From: Carsten Grommel @ 2022-02-28 18:41 UTC (permalink / raw)
To: linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 2325 bytes --]
Hi,
short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
But it was Kernel 5.4.121 before
btrfs --version
btrfs-progs v4.20.1
btrfs fi show
Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
Total devices 3 FS bytes used 56.74TiB
devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
btrfs fi df /vmbackup/
Data, RAID0: total=66.62TiB, used=56.45TiB
System, RAID1: total=8.00MiB, used=4.36MiB
Metadata, RAID1: total=750.00GiB, used=294.90GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
[Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
[Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
[Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
[Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
[Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
I am currently running a scrub over the filesystem.
The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
Kind regards
Carsten Grommel
[-- Attachment #2: dmesg.log --]
[-- Type: application/octet-stream, Size: 268471 bytes --]
[33963.001428] BTRFS warning (device sda1): checksum error at logical 763615379456 on dev /dev/sdd1, physical 254895652864, root 111184, inode 257, offset 98910003200, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33963.001429] BTRFS warning (device sda1): checksum error at logical 763612233728 on dev /dev/sdd1, physical 254894604288, root 111184, inode 257, offset 98906705920, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33963.001435] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 111778, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33963.001438] BTRFS warning (device sda1): checksum error at logical 763615379456 on dev /dev/sdd1, physical 254895652864, root 109849, inode 257, offset 98910003200, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33963.001440] BTRFS warning (device sda1): checksum error at logical 763612233728 on dev /dev/sdd1, physical 254894604288, root 109849, inode 257, offset 98906705920, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33963.001448] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 111184, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33963.001460] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 109849, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33963.001462] BTRFS warning (device sda1): checksum error at logical 763612233728 on dev /dev/sdd1, physical 254894604288, root 109373, inode 257, offset 98906705920, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33963.001463] BTRFS warning (device sda1): checksum error at logical 763615379456 on dev /dev/sdd1, physical 254895652864, root 109373, inode 257, offset 98910003200, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33963.001485] BTRFS warning (device sda1): checksum error at logical 763612233728 on dev /dev/sdd1, physical 254894604288, root 108899, inode 257, offset 98906705920, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33963.001487] BTRFS warning (device sda1): checksum error at logical 763615379456 on dev /dev/sdd1, physical 254895652864, root 108899, inode 257, offset 98910003200, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33963.001489] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 109373, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33963.001514] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 108899, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33963.001519] BTRFS warning (device sda1): checksum error at logical 763612233728 on dev /dev/sdd1, physical 254894604288, root 108424, inode 257, offset 98906705920, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33963.001520] BTRFS warning (device sda1): checksum error at logical 763615379456 on dev /dev/sdd1, physical 254895652864, root 108424, inode 257, offset 98910003200, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33963.001530] BTRFS warning (device sda1): checksum error at logical 763615379456 on dev /dev/sdd1, physical 254895652864, root 107950, inode 257, offset 98910003200, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33963.001531] BTRFS warning (device sda1): checksum error at logical 763612233728 on dev /dev/sdd1, physical 254894604288, root 107950, inode 257, offset 98906705920, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33963.001550] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 108424, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33963.001562] BTRFS warning (device sda1): checksum error at logical 763618525184 on dev /dev/sdd1, physical 254896701440, root 107950, inode 257, offset 98913222656, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33963.044218] BTRFS warning (device sda1): checksum error at logical 763610664960 on dev /dev/sdd1, physical 254894084096, root 111663, inode 257, offset 854224896, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[33963.044222] BTRFS warning (device sda1): checksum error at logical 763609108480 on dev /dev/sdd1, physical 254893576192, root 111663, inode 257, offset 849735680, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[33963.045181] BTRFS warning (device sda1): checksum error at logical 763620098048 on dev /dev/sdd1, physical 254897225728, root 108904, inode 257, offset 59052138496, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[33963.079451] BTRFS warning (device sda1): checksum error at logical 763620098048 on dev /dev/sdd1, physical 254897225728, root 108429, inode 257, offset 59052138496, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[33963.081716] BTRFS warning (device sda1): checksum error at logical 763620098048 on dev /dev/sdd1, physical 254897225728, root 107956, inode 257, offset 59052138496, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[33965.431503] btrfs_dev_stat_print_on_error: 14222 callbacks suppressed
[33965.431509] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788197, gen 184286
[33965.433591] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788198, gen 184286
[33965.434547] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788199, gen 184286
[33965.435712] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788200, gen 184286
[33965.436645] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788201, gen 184286
[33965.439781] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788202, gen 184286
[33965.443139] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788203, gen 184286
[33965.444135] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788204, gen 184286
[33965.450272] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788205, gen 184286
[33965.458840] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68788206, gen 184286
[33965.958222] scrub_handle_errored_block: 13356 callbacks suppressed
[33965.958228] BTRFS error (device sda1): unable to fixup (regular) error at logical 763868610560 on dev /dev/sdd1
[33965.960284] BTRFS error (device sda1): unable to fixup (regular) error at logical 763849760768 on dev /dev/sdd1
[33965.960491] BTRFS error (device sda1): unable to fixup (regular) error at logical 763849764864 on dev /dev/sdd1
[33965.961241] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851317248 on dev /dev/sdd1
[33965.961467] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851321344 on dev /dev/sdd1
[33965.961641] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851325440 on dev /dev/sdd1
[33965.961773] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851329536 on dev /dev/sdd1
[33965.961904] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851333632 on dev /dev/sdd1
[33965.962076] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851337728 on dev /dev/sdd1
[33965.962212] BTRFS error (device sda1): unable to fixup (regular) error at logical 763851341824 on dev /dev/sdd1
[33967.935935] scrub_handle_errored_block: 16756 callbacks suppressed
[33967.981719] BTRFS warning (device sda1): checksum error at logical 764221947904 on dev /dev/sdd1, physical 255097864192, root 5, inode 88709, offset 0, length 4096, links 1 (path: cloud8-vm385.de-nserver.de/2022-02-24_11:09.log)
[33968.000684] BTRFS warning (device sda1): checksum error at logical 764221952000 on dev /dev/sdd1, physical 255097868288, root 5, inode 88710, offset 0, length 4096, links 1 (path: cloud8-vm386.de-nserver.de/2022-02-24_11:09.log)
[33968.007814] BTRFS warning (device sda1): checksum error at logical 764221956096 on dev /dev/sdd1, physical 255097872384, root 5, inode 88711, offset 0, length 4096, links 1 (path: cloud8-vm382.de-nserver.de/2022-02-24_10:42.log)
[33968.015101] BTRFS warning (device sda1): checksum error at logical 764225650688 on dev /dev/sdd1, physical 255099076608, root 107729, inode 257, offset 947997773824, length 4096, links 1 (path: cephstor8_vm-167-disk-1-diff-base.img_1645151418)
[33968.015997] BTRFS warning (device sda1): checksum error at logical 764225060864 on dev /dev/sdd1, physical 255098880000, root 107729, inode 257, offset 947995537408, length 4096, links 1 (path: cephstor8_vm-167-disk-1-diff-base.img_1645151418)
[33968.037711] BTRFS warning (device sda1): checksum error at logical 764221915136 on dev /dev/sdd1, physical 255097831424, root 112139, inode 257, offset 9504526336, length 4096, links 1 (path: cephstor8_vm-386-disk-0-diff-base.img_1645767256)
[33968.047761] BTRFS warning (device sda1): checksum error at logical 764229976064 on dev /dev/sdd1, physical 255100518400, root 111660, inode 257, offset 2388013056, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[33968.047803] BTRFS warning (device sda1): checksum error at logical 764209926144 on dev /dev/sdd1, physical 255093837824, root 111665, inode 257, offset 9650753536, length 4096, links 1 (path: cephstor8_vm-386-disk-0-diff-base.img_1645697368)
[33968.047808] BTRFS warning (device sda1): checksum error at logical 764221915136 on dev /dev/sdd1, physical 255097831424, root 111665, inode 257, offset 9504526336, length 4096, links 1 (path: cephstor8_vm-386-disk-0-diff-base.img_1645697368)
[33968.047932] BTRFS warning (device sda1): checksum error at logical 764222504960 on dev /dev/sdd1, physical 255098028032, root 111660, inode 257, offset 2343383040, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[33968.047951] BTRFS warning (device sda1): checksum error at logical 764229779456 on dev /dev/sdd1, physical 255100452864, root 111660, inode 257, offset 2386137088, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[33970.437680] btrfs_dev_stat_print_on_error: 23427 callbacks suppressed
[33970.437686] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811649, gen 184286
[33970.444940] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811650, gen 184286
[33970.456111] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811651, gen 184286
[33970.482173] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811652, gen 184286
[33970.515529] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811653, gen 184286
[33970.516458] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811654, gen 184286
[33970.532764] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811655, gen 184286
[33970.537216] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811656, gen 184286
[33970.538714] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811657, gen 184286
[33970.540211] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68811658, gen 184286
[33970.959848] scrub_handle_errored_block: 23750 callbacks suppressed
[33970.959852] BTRFS error (device sda1): unable to fixup (regular) error at logical 764687482880 on dev /dev/sdd1
[33970.965698] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670779392 on dev /dev/sdd1
[33970.965777] BTRFS error (device sda1): unable to fixup (regular) error at logical 764672352256 on dev /dev/sdd1
[33970.967595] BTRFS error (device sda1): unable to fixup (regular) error at logical 764687486976 on dev /dev/sdd1
[33970.967599] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670783488 on dev /dev/sdd1
[33970.967779] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670787584 on dev /dev/sdd1
[33970.967939] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670791680 on dev /dev/sdd1
[33970.968097] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670795776 on dev /dev/sdd1
[33970.968253] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670799872 on dev /dev/sdd1
[33970.968406] BTRFS error (device sda1): unable to fixup (regular) error at logical 764670803968 on dev /dev/sdd1
[33972.942815] scrub_handle_errored_block: 20319 callbacks suppressed
[33972.999310] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 111778, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999316] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 111778, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999333] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 111184, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999351] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 111778, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999358] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 111778, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999371] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 111184, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999375] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 111184, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999386] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 109849, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33972.999389] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 109849, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33972.999413] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 111778, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999423] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 111778, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999427] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 109373, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33972.999430] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 109373, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33972.999434] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 111184, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999438] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 111184, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999452] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 109849, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33972.999454] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 109849, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33972.999460] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 111778, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999475] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 111184, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999486] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 109849, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33972.999492] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 109373, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33972.999495] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 109373, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33972.999535] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 109373, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33972.999674] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 111778, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33972.999688] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 111184, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33972.999700] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 109849, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33972.999728] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 109373, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33973.002439] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 111184, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33973.005510] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 109849, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33973.008556] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 109849, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33973.011626] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 109373, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33973.014696] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 109373, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33973.020557] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 108899, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.020566] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 108899, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.020569] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 108899, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.020572] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 108899, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.020576] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 108899, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.020579] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 108424, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.020585] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 108424, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.020588] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 108424, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.020591] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 108424, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.020594] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 108424, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.020604] BTRFS warning (device sda1): checksum error at logical 764898840576 on dev /dev/sdd1, physical 255323475968, root 107950, inode 257, offset 99094777856, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.020609] BTRFS warning (device sda1): checksum error at logical 764903555072 on dev /dev/sdd1, physical 255325044736, root 107950, inode 257, offset 99099541504, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.020612] BTRFS warning (device sda1): checksum error at logical 764887838720 on dev /dev/sdd1, physical 255319814144, root 107950, inode 257, offset 99083567104, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.020615] BTRFS warning (device sda1): checksum error at logical 764897267712 on dev /dev/sdd1, physical 255322951680, root 107950, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.020617] BTRFS warning (device sda1): checksum error at logical 764889411584 on dev /dev/sdd1, physical 255320338432, root 107950, inode 257, offset 99085189120, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.020909] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 111778, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33973.020923] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 111184, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33973.020937] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 109849, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33973.020966] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 109373, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33973.020986] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 111778, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33973.020999] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 111184, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33973.021011] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 109849, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33973.021016] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 108899, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.021031] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 108424, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.021038] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 109373, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33973.021045] BTRFS warning (device sda1): checksum error at logical 764898844672 on dev /dev/sdd1, physical 255323480064, root 107950, inode 257, offset 99094781952, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.021083] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 108899, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.021095] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 108424, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.021107] BTRFS warning (device sda1): checksum error at logical 764897271808 on dev /dev/sdd1, physical 255322955776, root 107950, inode 257, offset 99093053440, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.021752] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 108899, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.021932] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 108424, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.022079] BTRFS warning (device sda1): checksum error at logical 764894126080 on dev /dev/sdd1, physical 255321907200, root 107950, inode 257, offset 99090010112, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.034700] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 108899, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.079724] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 108899, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33973.099541] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 108424, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.103546] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 108424, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33973.119253] BTRFS warning (device sda1): checksum error at logical 764884692992 on dev /dev/sdd1, physical 255318765568, root 107950, inode 257, offset 99080372224, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33973.119925] BTRFS warning (device sda1): checksum error at logical 764900409344 on dev /dev/sdd1, physical 255323996160, root 107950, inode 257, offset 99096346624, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33975.443328] btrfs_dev_stat_print_on_error: 18194 callbacks suppressed
[33975.443334] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829868, gen 184286
[33975.452820] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829869, gen 184286
[33975.459480] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829870, gen 184286
[33975.467464] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829871, gen 184286
[33975.470689] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829872, gen 184286
[33975.470750] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829874, gen 184286
[33975.470757] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829873, gen 184286
[33975.470806] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829875, gen 184286
[33975.470821] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829876, gen 184286
[33975.470852] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68829877, gen 184286
[33975.963495] scrub_handle_errored_block: 19239 callbacks suppressed
[33975.963500] BTRFS error (device sda1): unable to fixup (regular) error at logical 765337710592 on dev /dev/sdd1
[33975.966387] BTRFS error (device sda1): unable to fixup (regular) error at logical 765342384128 on dev /dev/sdd1
[33975.966772] BTRFS error (device sda1): unable to fixup (regular) error at logical 765342388224 on dev /dev/sdd1
[33975.976916] BTRFS error (device sda1): unable to fixup (regular) error at logical 765345529856 on dev /dev/sdd1
[33975.992152] BTRFS error (device sda1): unable to fixup (regular) error at logical 765342392320 on dev /dev/sdd1
[33975.996417] BTRFS error (device sda1): unable to fixup (regular) error at logical 765349658624 on dev /dev/sdd1
[33976.004715] BTRFS error (device sda1): unable to fixup (regular) error at logical 765349662720 on dev /dev/sdd1
[33976.005051] BTRFS error (device sda1): unable to fixup (regular) error at logical 765337714688 on dev /dev/sdd1
[33976.005547] BTRFS error (device sda1): unable to fixup (regular) error at logical 765349666816 on dev /dev/sdd1
[33976.008699] BTRFS error (device sda1): unable to fixup (regular) error at logical 765342396416 on dev /dev/sdd1
[33977.945345] scrub_handle_errored_block: 20181 callbacks suppressed
[33977.960135] BTRFS warning (device sda1): checksum error at logical 765589327872 on dev /dev/sdd1, physical 255553638400, root 112074, inode 257, offset 419798712320, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.960141] BTRFS warning (device sda1): checksum error at logical 765584613376 on dev /dev/sdd1, physical 255552069632, root 112074, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.960150] BTRFS warning (device sda1): checksum error at logical 765586186240 on dev /dev/sdd1, physical 255552593920, root 111600, inode 257, offset 419793862656, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.960154] BTRFS warning (device sda1): checksum error at logical 765584613376 on dev /dev/sdd1, physical 255552069632, root 111600, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.960171] BTRFS warning (device sda1): checksum error at logical 765584613376 on dev /dev/sdd1, physical 255552069632, root 110180, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.960187] BTRFS warning (device sda1): checksum error at logical 765584613376 on dev /dev/sdd1, physical 255552069632, root 109671, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.960202] BTRFS warning (device sda1): checksum error at logical 765584613376 on dev /dev/sdd1, physical 255552069632, root 109195, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.960217] BTRFS warning (device sda1): checksum error at logical 765584613376 on dev /dev/sdd1, physical 255552069632, root 108721, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.960465] BTRFS warning (device sda1): checksum error at logical 765584617472 on dev /dev/sdd1, physical 255552073728, root 112074, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.960475] BTRFS warning (device sda1): checksum error at logical 765584617472 on dev /dev/sdd1, physical 255552073728, root 111600, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.960487] BTRFS warning (device sda1): checksum error at logical 765584617472 on dev /dev/sdd1, physical 255552073728, root 110180, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.960500] BTRFS warning (device sda1): checksum error at logical 765584617472 on dev /dev/sdd1, physical 255552073728, root 109671, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.960512] BTRFS warning (device sda1): checksum error at logical 765584617472 on dev /dev/sdd1, physical 255552073728, root 109195, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.960523] BTRFS warning (device sda1): checksum error at logical 765584617472 on dev /dev/sdd1, physical 255552073728, root 108721, inode 257, offset 419791765504, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.963576] BTRFS warning (device sda1): checksum error at logical 765589327872 on dev /dev/sdd1, physical 255553638400, root 111600, inode 257, offset 419798712320, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.966972] BTRFS warning (device sda1): checksum error at logical 765586186240 on dev /dev/sdd1, physical 255552593920, root 110180, inode 257, offset 419793862656, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.970236] BTRFS warning (device sda1): checksum error at logical 765589327872 on dev /dev/sdd1, physical 255553638400, root 110180, inode 257, offset 419798712320, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.970385] BTRFS warning (device sda1): checksum error at logical 765597188096 on dev /dev/sdd1, physical 255556255744, root 112074, inode 257, offset 419810508800, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.970393] BTRFS warning (device sda1): checksum error at logical 765597188096 on dev /dev/sdd1, physical 255556255744, root 111600, inode 257, offset 419810508800, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.970399] BTRFS warning (device sda1): checksum error at logical 765597188096 on dev /dev/sdd1, physical 255556255744, root 110180, inode 257, offset 419810508800, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.970408] BTRFS warning (device sda1): checksum error at logical 765597188096 on dev /dev/sdd1, physical 255556255744, root 109671, inode 257, offset 419810508800, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.970416] BTRFS warning (device sda1): checksum error at logical 765597188096 on dev /dev/sdd1, physical 255556255744, root 109195, inode 257, offset 419810508800, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.970424] BTRFS warning (device sda1): checksum error at logical 765597188096 on dev /dev/sdd1, physical 255556255744, root 108721, inode 257, offset 419810508800, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.970458] BTRFS warning (device sda1): checksum error at logical 765599744000 on dev /dev/sdd1, physical 255557107712, root 112074, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.970473] BTRFS warning (device sda1): checksum error at logical 765599744000 on dev /dev/sdd1, physical 255557107712, root 111600, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.970485] BTRFS warning (device sda1): checksum error at logical 765599744000 on dev /dev/sdd1, physical 255557107712, root 110180, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.970497] BTRFS warning (device sda1): checksum error at logical 765599744000 on dev /dev/sdd1, physical 255557107712, root 109671, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.970510] BTRFS warning (device sda1): checksum error at logical 765599744000 on dev /dev/sdd1, physical 255557107712, root 109195, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.970523] BTRFS warning (device sda1): checksum error at logical 765599744000 on dev /dev/sdd1, physical 255557107712, root 108721, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.970674] BTRFS warning (device sda1): checksum error at logical 765599748096 on dev /dev/sdd1, physical 255557111808, root 112074, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.970687] BTRFS warning (device sda1): checksum error at logical 765599748096 on dev /dev/sdd1, physical 255557111808, root 111600, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.970698] BTRFS warning (device sda1): checksum error at logical 765599748096 on dev /dev/sdd1, physical 255557111808, root 110180, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.970711] BTRFS warning (device sda1): checksum error at logical 765599748096 on dev /dev/sdd1, physical 255557111808, root 109671, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.970723] BTRFS warning (device sda1): checksum error at logical 765599748096 on dev /dev/sdd1, physical 255557111808, root 109195, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.970736] BTRFS warning (device sda1): checksum error at logical 765599748096 on dev /dev/sdd1, physical 255557111808, root 108721, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.970862] BTRFS warning (device sda1): checksum error at logical 765599752192 on dev /dev/sdd1, physical 255557115904, root 112074, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.970875] BTRFS warning (device sda1): checksum error at logical 765599752192 on dev /dev/sdd1, physical 255557115904, root 111600, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.970887] BTRFS warning (device sda1): checksum error at logical 765599752192 on dev /dev/sdd1, physical 255557115904, root 110180, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.970900] BTRFS warning (device sda1): checksum error at logical 765599752192 on dev /dev/sdd1, physical 255557115904, root 109671, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.970914] BTRFS warning (device sda1): checksum error at logical 765599752192 on dev /dev/sdd1, physical 255557115904, root 109195, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.970927] BTRFS warning (device sda1): checksum error at logical 765599752192 on dev /dev/sdd1, physical 255557115904, root 108721, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.971041] BTRFS warning (device sda1): checksum error at logical 765599756288 on dev /dev/sdd1, physical 255557120000, root 112074, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.971053] BTRFS warning (device sda1): checksum error at logical 765599756288 on dev /dev/sdd1, physical 255557120000, root 111600, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.971064] BTRFS warning (device sda1): checksum error at logical 765599756288 on dev /dev/sdd1, physical 255557120000, root 110180, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.971076] BTRFS warning (device sda1): checksum error at logical 765599756288 on dev /dev/sdd1, physical 255557120000, root 109671, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.971088] BTRFS warning (device sda1): checksum error at logical 765599756288 on dev /dev/sdd1, physical 255557120000, root 109195, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.971100] BTRFS warning (device sda1): checksum error at logical 765599756288 on dev /dev/sdd1, physical 255557120000, root 108721, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.971210] BTRFS warning (device sda1): checksum error at logical 765599760384 on dev /dev/sdd1, physical 255557124096, root 112074, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[33977.971222] BTRFS warning (device sda1): checksum error at logical 765599760384 on dev /dev/sdd1, physical 255557124096, root 111600, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[33977.971233] BTRFS warning (device sda1): checksum error at logical 765599760384 on dev /dev/sdd1, physical 255557124096, root 110180, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[33977.971245] BTRFS warning (device sda1): checksum error at logical 765599760384 on dev /dev/sdd1, physical 255557124096, root 109671, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.971258] BTRFS warning (device sda1): checksum error at logical 765599760384 on dev /dev/sdd1, physical 255557124096, root 109195, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.971270] BTRFS warning (device sda1): checksum error at logical 765599760384 on dev /dev/sdd1, physical 255557124096, root 108721, inode 257, offset 419814047744, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33977.973518] BTRFS warning (device sda1): checksum error at logical 765586186240 on dev /dev/sdd1, physical 255552593920, root 109671, inode 257, offset 419793862656, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.976821] BTRFS warning (device sda1): checksum error at logical 765589327872 on dev /dev/sdd1, physical 255553638400, root 109671, inode 257, offset 419798712320, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[33977.980099] BTRFS warning (device sda1): checksum error at logical 765586186240 on dev /dev/sdd1, physical 255552593920, root 109195, inode 257, offset 419793862656, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.983465] BTRFS warning (device sda1): checksum error at logical 765589327872 on dev /dev/sdd1, physical 255553638400, root 109195, inode 257, offset 419798712320, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[33977.986759] BTRFS warning (device sda1): checksum error at logical 765586186240 on dev /dev/sdd1, physical 255552593920, root 108721, inode 257, offset 419793862656, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[33980.443277] btrfs_dev_stat_print_on_error: 16500 callbacks suppressed
[33980.443281] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846389, gen 184286
[33980.443398] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846390, gen 184286
[33980.448781] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846391, gen 184286
[33980.449545] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846392, gen 184286
[33980.450678] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846393, gen 184286
[33980.450897] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846394, gen 184286
[33980.451050] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846395, gen 184286
[33980.451213] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846396, gen 184286
[33980.451358] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846397, gen 184286
[33980.451565] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68846398, gen 184286
[33980.971273] scrub_handle_errored_block: 15294 callbacks suppressed
[33980.971278] BTRFS error (device sda1): unable to fixup (regular) error at logical 765851992064 on dev /dev/sdd1
[33980.976068] BTRFS error (device sda1): unable to fixup (regular) error at logical 765842558976 on dev /dev/sdd1
[33980.977483] BTRFS error (device sda1): unable to fixup (regular) error at logical 765844135936 on dev /dev/sdd1
[33980.991712] BTRFS error (device sda1): unable to fixup (regular) error at logical 765853564928 on dev /dev/sdd1
[33980.996063] BTRFS error (device sda1): unable to fixup (regular) error at logical 765842563072 on dev /dev/sdd1
[33981.015294] BTRFS error (device sda1): unable to fixup (regular) error at logical 765848854528 on dev /dev/sdd1
[33981.017754] BTRFS error (device sda1): unable to fixup (regular) error at logical 765842567168 on dev /dev/sdd1
[33981.018719] BTRFS error (device sda1): unable to fixup (regular) error at logical 765842571264 on dev /dev/sdd1
[33981.019648] BTRFS error (device sda1): unable to fixup (regular) error at logical 765842575360 on dev /dev/sdd1
[33981.020564] BTRFS error (device sda1): unable to fixup (regular) error at logical 765842579456 on dev /dev/sdd1
[33982.955313] scrub_handle_errored_block: 12398 callbacks suppressed
[33982.978434] btrfs_printk: 1 callbacks suppressed
[33982.978440] BTRFS warning (device sda1): checksum error at logical 765999845376 on dev /dev/sdd1, physical 255690477568, root 109650, inode 257, offset 936282972160, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645503240)
[33982.978493] BTRFS warning (device sda1): checksum error at logical 765998276608 on dev /dev/sdd1, physical 255689957376, root 109650, inode 257, offset 936281268224, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645503240)
[33982.980001] BTRFS warning (device sda1): checksum error at logical 765999845376 on dev /dev/sdd1, physical 255690477568, root 109174, inode 257, offset 936282972160, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645408160)
[33982.983144] BTRFS warning (device sda1): checksum error at logical 765998276608 on dev /dev/sdd1, physical 255689957376, root 109174, inode 257, offset 936281268224, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645408160)
[33982.992670] BTRFS warning (device sda1): checksum error at logical 765998280704 on dev /dev/sdd1, physical 255689961472, root 109650, inode 257, offset 936281272320, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645503240)
[33982.995724] BTRFS warning (device sda1): checksum error at logical 765998280704 on dev /dev/sdd1, physical 255689961472, root 109174, inode 257, offset 936281272320, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645408160)
[33982.998961] BTRFS warning (device sda1): checksum error at logical 765998284800 on dev /dev/sdd1, physical 255689965568, root 109650, inode 257, offset 936281276416, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645503240)
[33983.001987] BTRFS warning (device sda1): checksum error at logical 765998284800 on dev /dev/sdd1, physical 255689965568, root 109174, inode 257, offset 936281276416, length 4096, links 1 (path: cephstor8_vm-209-disk-0-diff-base.img_1645408160)
[33983.094057] BTRFS warning (device sda1): checksum error at logical 766004563968 on dev /dev/sdd1, physical 255692050432, root 112138, inode 257, offset 17266880512, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33983.094085] BTRFS warning (device sda1): checksum error at logical 766007705600 on dev /dev/sdd1, physical 255693094912, root 112138, inode 257, offset 17274089472, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33983.094092] BTRFS warning (device sda1): checksum error at logical 766002991104 on dev /dev/sdd1, physical 255691526144, root 112138, inode 257, offset 17263341568, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33983.094098] BTRFS warning (device sda1): checksum error at logical 765996707840 on dev /dev/sdd1, physical 255689437184, root 112138, inode 257, offset 17258491904, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33983.094107] BTRFS warning (device sda1): checksum error at logical 765995155456 on dev /dev/sdd1, physical 255688933376, root 112138, inode 257, offset 17254952960, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33983.094112] BTRFS warning (device sda1): checksum error at logical 766006132736 on dev /dev/sdd1, physical 255692570624, root 112138, inode 257, offset 17270550528, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33983.143565] BTRFS warning (device sda1): checksum error at logical 766006132736 on dev /dev/sdd1, physical 255692570624, root 111664, inode 257, offset 17270550528, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33983.144457] BTRFS warning (device sda1): checksum error at logical 765995155456 on dev /dev/sdd1, physical 255688933376, root 111664, inode 257, offset 17254952960, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33983.145218] BTRFS warning (device sda1): checksum error at logical 766002991104 on dev /dev/sdd1, physical 255691526144, root 111664, inode 257, offset 17263341568, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33983.145529] BTRFS warning (device sda1): checksum error at logical 766004563968 on dev /dev/sdd1, physical 255692050432, root 111664, inode 257, offset 17266880512, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33983.145679] BTRFS warning (device sda1): checksum error at logical 765996707840 on dev /dev/sdd1, physical 255689437184, root 111664, inode 257, offset 17258491904, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33983.155319] BTRFS warning (device sda1): checksum error at logical 766007705600 on dev /dev/sdd1, physical 255693094912, root 111664, inode 257, offset 17274089472, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33985.447140] btrfs_dev_stat_print_on_error: 16130 callbacks suppressed
[33985.447144] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862546, gen 184286
[33985.447154] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862547, gen 184286
[33985.447159] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862548, gen 184286
[33985.447204] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862549, gen 184286
[33985.447251] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862550, gen 184286
[33985.447320] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862551, gen 184286
[33985.447432] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862552, gen 184286
[33985.447532] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862553, gen 184286
[33985.447637] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862554, gen 184286
[33985.447737] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68862555, gen 184286
[33985.978294] scrub_handle_errored_block: 17292 callbacks suppressed
[33985.978299] BTRFS error (device sda1): unable to fixup (regular) error at logical 766422962176 on dev /dev/sdd1
[33985.979273] BTRFS error (device sda1): unable to fixup (regular) error at logical 766424526848 on dev /dev/sdd1
[33985.980133] BTRFS error (device sda1): unable to fixup (regular) error at logical 766426099712 on dev /dev/sdd1
[33985.980400] BTRFS error (device sda1): unable to fixup (regular) error at logical 766426103808 on dev /dev/sdd1
[33985.984217] BTRFS error (device sda1): unable to fixup (regular) error at logical 766430806016 on dev /dev/sdd1
[33985.986957] BTRFS error (device sda1): unable to fixup (regular) error at logical 766422966272 on dev /dev/sdd1
[33985.987816] BTRFS error (device sda1): unable to fixup (regular) error at logical 766422970368 on dev /dev/sdd1
[33985.988468] BTRFS error (device sda1): unable to fixup (regular) error at logical 766422974464 on dev /dev/sdd1
[33985.988979] BTRFS error (device sda1): unable to fixup (regular) error at logical 766422978560 on dev /dev/sdd1
[33985.989468] BTRFS error (device sda1): unable to fixup (regular) error at logical 766422982656 on dev /dev/sdd1
[33987.962000] scrub_handle_errored_block: 20878 callbacks suppressed
[33987.994566] BTRFS warning (device sda1): checksum error at logical 766685618176 on dev /dev/sdd1, physical 255919071232, root 112095, inode 257, offset 157815808000, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[33987.994579] BTRFS warning (device sda1): checksum error at logical 766688763904 on dev /dev/sdd1, physical 255920119808, root 112095, inode 257, offset 157818953728, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[33987.996240] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 111763, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33987.996246] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 111763, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33987.996253] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 111763, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33987.996258] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 111763, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33987.997112] BTRFS warning (device sda1): checksum error at logical 766685618176 on dev /dev/sdd1, physical 255919071232, root 111622, inode 257, offset 157815808000, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[33988.000242] BTRFS warning (device sda1): checksum error at logical 766688763904 on dev /dev/sdd1, physical 255920119808, root 111622, inode 257, offset 157818953728, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[33988.003312] BTRFS warning (device sda1): checksum error at logical 766685618176 on dev /dev/sdd1, physical 255919071232, root 110333, inode 257, offset 157815808000, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[33988.006357] BTRFS warning (device sda1): checksum error at logical 766688763904 on dev /dev/sdd1, physical 255920119808, root 110333, inode 257, offset 157818953728, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[33988.009522] BTRFS warning (device sda1): checksum error at logical 766685618176 on dev /dev/sdd1, physical 255919071232, root 109693, inode 257, offset 157815808000, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[33988.009823] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 111066, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.009828] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 111066, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.009833] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 111066, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.009836] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 111066, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.009841] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 109833, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.009847] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 109833, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.009854] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 109833, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.009858] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 109833, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.009860] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 109359, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.009865] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 109359, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.009873] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 109359, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.009877] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 109359, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.009880] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 108884, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.009884] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 108884, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.009893] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 108884, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.009897] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 108884, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.009899] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 108409, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.009903] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 108409, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.009913] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 108409, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.009917] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 108409, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.009919] BTRFS warning (device sda1): checksum error at logical 766682480640 on dev /dev/sdd1, physical 255918030848, root 107934, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.009923] BTRFS warning (device sda1): checksum error at logical 766690332672 on dev /dev/sdd1, physical 255920640000, root 107934, inode 257, offset 142022148096, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.009932] BTRFS warning (device sda1): checksum error at logical 766684045312 on dev /dev/sdd1, physical 255918546944, root 107934, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.009937] BTRFS warning (device sda1): checksum error at logical 766680915968 on dev /dev/sdd1, physical 255917514752, root 107934, inode 257, offset 142015594496, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.012301] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 111763, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33988.013251] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 111763, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33988.014051] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 111066, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.014218] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 109833, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.014376] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 109359, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.014510] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 108884, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.014681] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 108409, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.014875] BTRFS warning (device sda1): checksum error at logical 766682484736 on dev /dev/sdd1, physical 255918034944, root 107934, inode 257, offset 142018084864, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.016408] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 111066, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.016997] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 109833, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.017477] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 109359, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.017911] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 108884, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.018326] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 108409, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.018732] BTRFS warning (device sda1): checksum error at logical 766684049408 on dev /dev/sdd1, physical 255918551040, root 107934, inode 257, offset 142020182016, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.019507] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 111763, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[33988.019663] BTRFS warning (device sda1): checksum error at logical 766695047168 on dev /dev/sdd1, physical 255922208768, root 111066, inode 257, offset 142030012416, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.019666] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 111066, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[33988.019681] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 109833, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.019685] BTRFS warning (device sda1): checksum error at logical 766695047168 on dev /dev/sdd1, physical 255922208768, root 109833, inode 257, offset 142030012416, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[33988.019698] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 109359, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.019701] BTRFS warning (device sda1): checksum error at logical 766695047168 on dev /dev/sdd1, physical 255922208768, root 109359, inode 257, offset 142030012416, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[33988.019715] BTRFS warning (device sda1): checksum error at logical 766695047168 on dev /dev/sdd1, physical 255922208768, root 108884, inode 257, offset 142030012416, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.019718] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 108884, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[33988.019731] BTRFS warning (device sda1): checksum error at logical 766695047168 on dev /dev/sdd1, physical 255922208768, root 108409, inode 257, offset 142030012416, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.019734] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 108409, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[33988.019749] BTRFS warning (device sda1): checksum error at logical 766691905536 on dev /dev/sdd1, physical 255921164288, root 107934, inode 257, offset 142025162752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.019751] BTRFS warning (device sda1): checksum error at logical 766695047168 on dev /dev/sdd1, physical 255922208768, root 107934, inode 257, offset 142030012416, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[33988.031985] BTRFS warning (device sda1): checksum error at logical 766688763904 on dev /dev/sdd1, physical 255920119808, root 109693, inode 257, offset 157818953728, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[33988.045548] BTRFS warning (device sda1): checksum error at logical 766685618176 on dev /dev/sdd1, physical 255919071232, root 109217, inode 257, offset 157815808000, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[33988.067036] BTRFS warning (device sda1): checksum error at logical 766688763904 on dev /dev/sdd1, physical 255920119808, root 109217, inode 257, offset 157818953728, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[33988.105750] BTRFS warning (device sda1): checksum error at logical 766685618176 on dev /dev/sdd1, physical 255919071232, root 108743, inode 257, offset 157815808000, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[33988.127140] BTRFS warning (device sda1): checksum error at logical 766688763904 on dev /dev/sdd1, physical 255920119808, root 108743, inode 257, offset 157818953728, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[33990.453460] btrfs_dev_stat_print_on_error: 18754 callbacks suppressed
[33990.453465] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881326, gen 184286
[33990.455640] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881327, gen 184286
[33990.457529] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881328, gen 184286
[33990.458353] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881329, gen 184286
[33990.462558] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881330, gen 184286
[33990.484416] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881331, gen 184286
[33990.488818] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881332, gen 184286
[33990.498302] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881333, gen 184286
[33990.498576] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881334, gen 184286
[33990.498752] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68881335, gen 184286
[33991.014582] scrub_handle_errored_block: 20563 callbacks suppressed
[33991.014587] BTRFS error (device sda1): unable to fixup (regular) error at logical 767105564672 on dev /dev/sdd1
[33991.015491] BTRFS error (device sda1): unable to fixup (regular) error at logical 767106547712 on dev /dev/sdd1
[33991.015950] BTRFS error (device sda1): unable to fixup (regular) error at logical 767107137536 on dev /dev/sdd1
[33991.018706] BTRFS error (device sda1): unable to fixup (regular) error at logical 767110283264 on dev /dev/sdd1
[33991.028129] BTRFS error (device sda1): unable to fixup (regular) error at logical 767100850176 on dev /dev/sdd1
[33991.031705] BTRFS error (device sda1): unable to fixup (regular) error at logical 767106551808 on dev /dev/sdd1
[33991.033104] BTRFS error (device sda1): unable to fixup (regular) error at logical 767111266304 on dev /dev/sdd1
[33991.033160] BTRFS error (device sda1): unable to fixup (regular) error at logical 767111270400 on dev /dev/sdd1
[33991.033198] BTRFS error (device sda1): unable to fixup (regular) error at logical 767111274496 on dev /dev/sdd1
[33991.033239] BTRFS error (device sda1): unable to fixup (regular) error at logical 767111278592 on dev /dev/sdd1
[33992.969129] scrub_handle_errored_block: 23860 callbacks suppressed
[33992.988071] BTRFS warning (device sda1): checksum error at logical 767481479168 on dev /dev/sdd1, physical 256184352768, root 111663, inode 257, offset 1935486976, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[33993.009904] BTRFS warning (device sda1): checksum error at logical 767479316480 on dev /dev/sdd1, physical 256183631872, root 111663, inode 257, offset 1927753728, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[33993.019258] BTRFS warning (device sda1): checksum error at logical 767487180800 on dev /dev/sdd1, physical 256186253312, root 112138, inode 257, offset 27531239424, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33993.019894] BTRFS warning (device sda1): checksum error at logical 767486197760 on dev /dev/sdd1, physical 256185925632, root 112138, inode 257, offset 27101315072, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33993.026891] BTRFS warning (device sda1): checksum error at logical 767476760576 on dev /dev/sdd1, physical 256182779904, root 111778, inode 257, offset 99402727424, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[33993.027021] BTRFS warning (device sda1): checksum error at logical 767475191808 on dev /dev/sdd1, physical 256182259712, root 111184, inode 257, offset 99401134080, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33993.027042] BTRFS warning (device sda1): checksum error at logical 767476760576 on dev /dev/sdd1, physical 256182779904, root 111184, inode 257, offset 99402727424, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[33993.027215] BTRFS warning (device sda1): checksum error at logical 767476760576 on dev /dev/sdd1, physical 256182779904, root 109849, inode 257, offset 99402727424, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33993.027222] BTRFS warning (device sda1): checksum error at logical 767475191808 on dev /dev/sdd1, physical 256182259712, root 109849, inode 257, offset 99401134080, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[33993.027944] BTRFS warning (device sda1): checksum error at logical 767475191808 on dev /dev/sdd1, physical 256182259712, root 109373, inode 257, offset 99401134080, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33993.027953] BTRFS warning (device sda1): checksum error at logical 767476760576 on dev /dev/sdd1, physical 256182779904, root 109373, inode 257, offset 99402727424, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[33993.029351] BTRFS warning (device sda1): checksum error at logical 767475191808 on dev /dev/sdd1, physical 256182259712, root 108899, inode 257, offset 99401134080, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[33993.029488] BTRFS warning (device sda1): checksum error at logical 767475191808 on dev /dev/sdd1, physical 256182259712, root 108424, inode 257, offset 99401134080, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33993.029499] BTRFS warning (device sda1): checksum error at logical 767476760576 on dev /dev/sdd1, physical 256182779904, root 108424, inode 257, offset 99402727424, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[33993.029540] BTRFS warning (device sda1): checksum error at logical 767475191808 on dev /dev/sdd1, physical 256182259712, root 107950, inode 257, offset 99401134080, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33993.029543] BTRFS warning (device sda1): checksum error at logical 767476760576 on dev /dev/sdd1, physical 256182779904, root 107950, inode 257, offset 99402727424, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[33993.029671] BTRFS warning (device sda1): checksum error at logical 767479320576 on dev /dev/sdd1, physical 256183635968, root 111663, inode 257, offset 1927753728, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[33993.029767] BTRFS warning (device sda1): checksum error at logical 767479324672 on dev /dev/sdd1, physical 256183640064, root 111663, inode 257, offset 1927753728, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[33993.036823] BTRFS warning (device sda1): checksum error at logical 767490326528 on dev /dev/sdd1, physical 256187301888, root 112138, inode 257, offset 28844593152, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33993.036862] BTRFS warning (device sda1): checksum error at logical 767490326528 on dev /dev/sdd1, physical 256187301888, root 111664, inode 257, offset 28844593152, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33993.039290] BTRFS warning (device sda1): checksum error at logical 767489343488 on dev /dev/sdd1, physical 256186974208, root 112138, inode 257, offset 27876421632, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645767241)
[33993.041042] BTRFS warning (device sda1): checksum error at logical 767487180800 on dev /dev/sdd1, physical 256186253312, root 111664, inode 257, offset 27531239424, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33993.076231] BTRFS warning (device sda1): checksum error at logical 767486197760 on dev /dev/sdd1, physical 256185925632, root 111664, inode 257, offset 27101315072, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33993.077373] BTRFS warning (device sda1): checksum error at logical 767489343488 on dev /dev/sdd1, physical 256186974208, root 111664, inode 257, offset 27876421632, length 4096, links 1 (path: cephstor8_vm-385-disk-0-diff-base.img_1645697365)
[33995.460096] btrfs_dev_stat_print_on_error: 22601 callbacks suppressed
[33995.460102] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903956, gen 184286
[33995.467537] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903957, gen 184286
[33995.481655] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903958, gen 184286
[33995.482055] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903959, gen 184286
[33995.482131] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903960, gen 184286
[33995.482173] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903961, gen 184286
[33995.482213] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903962, gen 184286
[33995.482254] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903963, gen 184286
[33995.482292] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903964, gen 184286
[33995.482331] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68903965, gen 184286
[33996.015403] scrub_handle_errored_block: 20516 callbacks suppressed
[33996.015408] BTRFS error (device sda1): unable to fixup (regular) error at logical 767733182464 on dev /dev/sdd1
[33996.017800] BTRFS error (device sda1): unable to fixup (regular) error at logical 767764623360 on dev /dev/sdd1
[33996.029890] BTRFS error (device sda1): unable to fixup (regular) error at logical 767764627456 on dev /dev/sdd1
[33996.036345] BTRFS error (device sda1): unable to fixup (regular) error at logical 767733186560 on dev /dev/sdd1
[33996.044124] BTRFS error (device sda1): unable to fixup (regular) error at logical 767766188032 on dev /dev/sdd1
[33996.044375] BTRFS error (device sda1): unable to fixup (regular) error at logical 767769333760 on dev /dev/sdd1
[33996.044517] BTRFS error (device sda1): unable to fixup (regular) error at logical 767778750464 on dev /dev/sdd1
[33996.044581] BTRFS error (device sda1): unable to fixup (regular) error at logical 767767760896 on dev /dev/sdd1
[33996.044685] BTRFS error (device sda1): unable to fixup (regular) error at logical 767772475392 on dev /dev/sdd1
[33996.046042] BTRFS error (device sda1): unable to fixup (regular) error at logical 767770906624 on dev /dev/sdd1
[33997.970839] scrub_handle_errored_block: 16610 callbacks suppressed
[33997.984600] BTRFS warning (device sda1): checksum error at logical 768024133632 on dev /dev/sdd1, physical 256365248512, root 108267, inode 257, offset 26245963776, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[33997.993248] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 111784, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[33997.993254] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 111784, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[33997.993270] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 111196, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[33997.996485] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 111196, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[33997.999578] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 109854, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[33998.001337] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 111784, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[33998.001355] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 111196, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[33998.001371] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 109854, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[33998.001389] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 109380, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[33998.001402] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 108904, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[33998.001415] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 108429, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[33998.001428] BTRFS warning (device sda1): checksum error at logical 768028844032 on dev /dev/sdd1, physical 256366813184, root 107956, inode 257, offset 59810668544, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[33998.002649] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 109854, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[33998.006749] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 109380, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[33998.006799] BTRFS warning (device sda1): checksum error at logical 768024133632 on dev /dev/sdd1, physical 256365248512, root 107792, inode 257, offset 26245963776, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[33998.008873] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 109380, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[33998.011423] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 111784, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[33998.011440] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 111196, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[33998.011456] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 109854, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[33998.011470] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 109380, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[33998.011485] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 108904, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[33998.011499] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 108429, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[33998.011513] BTRFS warning (device sda1): checksum error at logical 768031989760 on dev /dev/sdd1, physical 256367861760, root 107956, inode 257, offset 59813814272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[33998.012115] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 108904, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[33998.012179] BTRFS warning (device sda1): checksum error at logical 768019419136 on dev /dev/sdd1, physical 256363679744, root 108267, inode 257, offset 26239410176, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[33998.012230] BTRFS warning (device sda1): checksum error at logical 768019419136 on dev /dev/sdd1, physical 256363679744, root 107792, inode 257, offset 26239410176, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[33998.012534] BTRFS warning (device sda1): checksum error at logical 768019423232 on dev /dev/sdd1, physical 256363683840, root 108267, inode 257, offset 26239414272, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[33998.012559] BTRFS warning (device sda1): checksum error at logical 768019423232 on dev /dev/sdd1, physical 256363683840, root 107792, inode 257, offset 26239414272, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[33998.012768] BTRFS warning (device sda1): checksum error at logical 768019427328 on dev /dev/sdd1, physical 256363687936, root 108267, inode 257, offset 26239418368, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[33998.012791] BTRFS warning (device sda1): checksum error at logical 768019427328 on dev /dev/sdd1, physical 256363687936, root 107792, inode 257, offset 26239418368, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[33998.013001] BTRFS warning (device sda1): checksum error at logical 768019431424 on dev /dev/sdd1, physical 256363692032, root 108267, inode 257, offset 26239422464, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[33998.013024] BTRFS warning (device sda1): checksum error at logical 768019431424 on dev /dev/sdd1, physical 256363692032, root 107792, inode 257, offset 26239422464, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[33998.013263] BTRFS warning (device sda1): checksum error at logical 768020992000 on dev /dev/sdd1, physical 256364204032, root 108267, inode 257, offset 26240983040, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[33998.013299] BTRFS warning (device sda1): checksum error at logical 768020992000 on dev /dev/sdd1, physical 256364204032, root 107792, inode 257, offset 26240983040, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[33998.015209] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 108904, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[33998.018293] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 108429, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[33998.021381] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 108429, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[33998.024461] BTRFS warning (device sda1): checksum error at logical 768025706496 on dev /dev/sdd1, physical 256365772800, root 107956, inode 257, offset 59807006720, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[33998.027802] BTRFS warning (device sda1): checksum error at logical 768014684160 on dev /dev/sdd1, physical 256362090496, root 107956, inode 257, offset 59802632192, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34000.462677] btrfs_dev_stat_print_on_error: 19518 callbacks suppressed
[34000.462682] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923491, gen 184286
[34000.462896] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923492, gen 184286
[34000.462996] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923493, gen 184286
[34000.463184] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923494, gen 184286
[34000.463322] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923495, gen 184286
[34000.463439] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923496, gen 184286
[34000.463627] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923497, gen 184286
[34000.463706] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923498, gen 184286
[34000.463857] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923499, gen 184286
[34000.464051] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68923500, gen 184286
[34001.030595] scrub_handle_errored_block: 20615 callbacks suppressed
[34001.030600] BTRFS error (device sda1): unable to fixup (regular) error at logical 768453513216 on dev /dev/sdd1
[34001.031467] BTRFS error (device sda1): unable to fixup (regular) error at logical 768455086080 on dev /dev/sdd1
[34001.037090] BTRFS error (device sda1): unable to fixup (regular) error at logical 768467664896 on dev /dev/sdd1
[34001.037382] BTRFS error (device sda1): unable to fixup (regular) error at logical 768455090176 on dev /dev/sdd1
[34001.037574] BTRFS error (device sda1): unable to fixup (regular) error at logical 768469237760 on dev /dev/sdd1
[34001.046643] BTRFS error (device sda1): unable to fixup (regular) error at logical 768455094272 on dev /dev/sdd1
[34001.048698] BTRFS error (device sda1): unable to fixup (regular) error at logical 768455098368 on dev /dev/sdd1
[34001.050464] BTRFS error (device sda1): unable to fixup (regular) error at logical 768456654848 on dev /dev/sdd1
[34001.050753] BTRFS error (device sda1): unable to fixup (regular) error at logical 768455102464 on dev /dev/sdd1
[34001.050766] BTRFS error (device sda1): unable to fixup (regular) error at logical 768470220800 on dev /dev/sdd1
[34002.978466] scrub_handle_errored_block: 21350 callbacks suppressed
[34003.005079] BTRFS warning (device sda1): checksum error at logical 768730357760 on dev /dev/sdd1, physical 256600662016, root 109175, inode 257, offset 38610911232, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005085] BTRFS warning (device sda1): checksum error at logical 768720920576 on dev /dev/sdd1, physical 256597516288, root 109175, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005091] BTRFS warning (device sda1): checksum error at logical 768722493440 on dev /dev/sdd1, physical 256598040576, root 109175, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005098] BTRFS warning (device sda1): checksum error at logical 768725639168 on dev /dev/sdd1, physical 256599089152, root 109175, inode 257, offset 38604750848, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005105] BTRFS warning (device sda1): checksum error at logical 768724066304 on dev /dev/sdd1, physical 256598564864, root 108700, inode 257, offset 38602784768, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005109] BTRFS warning (device sda1): checksum error at logical 768720920576 on dev /dev/sdd1, physical 256597516288, root 108700, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005112] BTRFS warning (device sda1): checksum error at logical 768722493440 on dev /dev/sdd1, physical 256598040576, root 108700, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005115] BTRFS warning (device sda1): checksum error at logical 768725639168 on dev /dev/sdd1, physical 256599089152, root 108700, inode 257, offset 38604750848, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005124] BTRFS warning (device sda1): checksum error at logical 768720920576 on dev /dev/sdd1, physical 256597516288, root 108227, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.005127] BTRFS warning (device sda1): checksum error at logical 768722493440 on dev /dev/sdd1, physical 256598040576, root 108227, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.005130] BTRFS warning (device sda1): checksum error at logical 768725639168 on dev /dev/sdd1, physical 256599089152, root 108227, inode 257, offset 38604750848, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.005142] BTRFS warning (device sda1): checksum error at logical 768731906048 on dev /dev/sdd1, physical 256601161728, root 109175, inode 257, offset 38612877312, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005146] BTRFS warning (device sda1): checksum error at logical 768720920576 on dev /dev/sdd1, physical 256597516288, root 107751, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.005149] BTRFS warning (device sda1): checksum error at logical 768722493440 on dev /dev/sdd1, physical 256598040576, root 107751, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.005153] BTRFS warning (device sda1): checksum error at logical 768725639168 on dev /dev/sdd1, physical 256599089152, root 107751, inode 257, offset 38604750848, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.005162] BTRFS warning (device sda1): checksum error at logical 768731906048 on dev /dev/sdd1, physical 256601161728, root 108700, inode 257, offset 38612877312, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005177] BTRFS warning (device sda1): checksum error at logical 768731906048 on dev /dev/sdd1, physical 256601161728, root 108227, inode 257, offset 38612877312, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.005192] BTRFS warning (device sda1): checksum error at logical 768731906048 on dev /dev/sdd1, physical 256601161728, root 107751, inode 257, offset 38612877312, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.005436] BTRFS warning (device sda1): checksum error at logical 768720924672 on dev /dev/sdd1, physical 256597520384, root 109175, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005448] BTRFS warning (device sda1): checksum error at logical 768720924672 on dev /dev/sdd1, physical 256597520384, root 108700, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005459] BTRFS warning (device sda1): checksum error at logical 768720924672 on dev /dev/sdd1, physical 256597520384, root 108227, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.005472] BTRFS warning (device sda1): checksum error at logical 768720924672 on dev /dev/sdd1, physical 256597520384, root 107751, inode 257, offset 38598590464, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.005498] BTRFS warning (device sda1): checksum error at logical 768722497536 on dev /dev/sdd1, physical 256598044672, root 109175, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34003.005509] BTRFS warning (device sda1): checksum error at logical 768722497536 on dev /dev/sdd1, physical 256598044672, root 108700, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.005520] BTRFS warning (device sda1): checksum error at logical 768722497536 on dev /dev/sdd1, physical 256598044672, root 108227, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.005531] BTRFS warning (device sda1): checksum error at logical 768722497536 on dev /dev/sdd1, physical 256598044672, root 107751, inode 257, offset 38600687616, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.008564] BTRFS warning (device sda1): checksum error at logical 768730357760 on dev /dev/sdd1, physical 256600662016, root 108700, inode 257, offset 38610911232, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34003.010242] BTRFS warning (device sda1): checksum error at logical 768724066304 on dev /dev/sdd1, physical 256598564864, root 108227, inode 257, offset 38602784768, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.013497] BTRFS warning (device sda1): checksum error at logical 768730357760 on dev /dev/sdd1, physical 256600662016, root 108227, inode 257, offset 38610911232, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34003.014970] BTRFS warning (device sda1): checksum error at logical 768724066304 on dev /dev/sdd1, physical 256598564864, root 107751, inode 257, offset 38602784768, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.016930] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 111778, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[34003.016938] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 111184, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[34003.016946] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 109849, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[34003.016970] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 109373, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[34003.017015] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 108899, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[34003.017022] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 108424, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[34003.017031] BTRFS warning (device sda1): checksum error at logical 768714661888 on dev /dev/sdd1, physical 256595451904, root 107950, inode 257, offset 99533979648, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[34003.018296] BTRFS warning (device sda1): checksum error at logical 768730357760 on dev /dev/sdd1, physical 256600662016, root 107751, inode 257, offset 38610911232, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34003.164015] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 111778, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[34003.168018] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 111184, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[34003.173148] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 109849, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[34003.178343] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 109373, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[34003.207442] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 108899, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[34003.229199] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 108424, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[34003.254563] BTRFS warning (device sda1): checksum error at logical 768717766656 on dev /dev/sdd1, physical 256596459520, root 107950, inode 257, offset 99537154048, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[34005.479139] btrfs_dev_stat_print_on_error: 20282 callbacks suppressed
[34005.479144] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943793, gen 184286
[34005.479231] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943794, gen 184286
[34005.480003] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943795, gen 184286
[34005.490720] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943796, gen 184286
[34005.497472] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943797, gen 184286
[34005.497545] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943798, gen 184286
[34005.498527] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943799, gen 184286
[34005.499358] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943800, gen 184286
[34005.500096] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943801, gen 184286
[34005.502339] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68943802, gen 184286
[34006.030541] scrub_handle_errored_block: 18979 callbacks suppressed
[34006.030546] BTRFS error (device sda1): unable to fixup (regular) error at logical 769079541760 on dev /dev/sdd1
[34006.045442] BTRFS error (device sda1): unable to fixup (regular) error at logical 769090519040 on dev /dev/sdd1
[34006.050615] BTRFS error (device sda1): unable to fixup (regular) error at logical 769081085952 on dev /dev/sdd1
[34006.062953] BTRFS error (device sda1): unable to fixup (regular) error at logical 769079545856 on dev /dev/sdd1
[34006.077391] BTRFS error (device sda1): unable to fixup (regular) error at logical 769082658816 on dev /dev/sdd1
[34006.094460] BTRFS error (device sda1): unable to fixup (regular) error at logical 769084231680 on dev /dev/sdd1
[34006.120256] BTRFS error (device sda1): unable to fixup (regular) error at logical 769079549952 on dev /dev/sdd1
[34006.135356] BTRFS error (device sda1): unable to fixup (regular) error at logical 769087377408 on dev /dev/sdd1
[34006.139258] BTRFS error (device sda1): unable to fixup (regular) error at logical 769079554048 on dev /dev/sdd1
[34006.146862] BTRFS error (device sda1): unable to fixup (regular) error at logical 769088950272 on dev /dev/sdd1
[34007.982304] scrub_handle_errored_block: 15424 callbacks suppressed
[34008.009481] BTRFS warning (device sda1): checksum error at logical 769238409216 on dev /dev/sdd1, physical 256770023424, root 112095, inode 257, offset 158218321920, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34008.012689] BTRFS warning (device sda1): checksum error at logical 769238409216 on dev /dev/sdd1, physical 256770023424, root 111622, inode 257, offset 158218321920, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34008.015794] BTRFS warning (device sda1): checksum error at logical 769238409216 on dev /dev/sdd1, physical 256770023424, root 110333, inode 257, offset 158218321920, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34008.016137] BTRFS warning (device sda1): checksum error at logical 769249382400 on dev /dev/sdd1, physical 256773656576, root 112101, inode 257, offset 58748436480, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34008.018892] BTRFS warning (device sda1): checksum error at logical 769238409216 on dev /dev/sdd1, physical 256770023424, root 109693, inode 257, offset 158218321920, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34008.022086] BTRFS warning (device sda1): checksum error at logical 769249382400 on dev /dev/sdd1, physical 256773656576, root 111626, inode 257, offset 58748436480, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34008.025243] BTRFS warning (device sda1): checksum error at logical 769238409216 on dev /dev/sdd1, physical 256770023424, root 109217, inode 257, offset 158218321920, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34008.028412] BTRFS warning (device sda1): checksum error at logical 769249382400 on dev /dev/sdd1, physical 256773656576, root 110392, inode 257, offset 58748436480, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34008.031588] BTRFS warning (device sda1): checksum error at logical 769238409216 on dev /dev/sdd1, physical 256770023424, root 108743, inode 257, offset 158218321920, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34008.034759] BTRFS warning (device sda1): checksum error at logical 769249382400 on dev /dev/sdd1, physical 256773656576, root 109697, inode 257, offset 58748436480, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34008.041254] BTRFS warning (device sda1): checksum error at logical 769249382400 on dev /dev/sdd1, physical 256773656576, root 109219, inode 257, offset 58748436480, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34008.043635] BTRFS warning (device sda1): checksum error at logical 769239949312 on dev /dev/sdd1, physical 256770514944, root 112095, inode 257, offset 158226841600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34008.044381] BTRFS warning (device sda1): checksum error at logical 769249382400 on dev /dev/sdd1, physical 256773656576, root 108747, inode 257, offset 58748436480, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34008.046606] BTRFS warning (device sda1): checksum error at logical 769244663808 on dev /dev/sdd1, physical 256772083712, root 112095, inode 257, offset 158257774592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34008.046956] BTRFS warning (device sda1): checksum error at logical 769244663808 on dev /dev/sdd1, physical 256772083712, root 111622, inode 257, offset 158257774592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34008.047180] BTRFS warning (device sda1): checksum error at logical 769244663808 on dev /dev/sdd1, physical 256772083712, root 110333, inode 257, offset 158257774592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34008.047532] BTRFS warning (device sda1): checksum error at logical 769246236672 on dev /dev/sdd1, physical 256772608000, root 112095, inode 257, offset 158270357504, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34008.047816] BTRFS warning (device sda1): checksum error at logical 769239949312 on dev /dev/sdd1, physical 256770514944, root 111622, inode 257, offset 158226841600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34008.048220] BTRFS warning (device sda1): checksum error at logical 769243086848 on dev /dev/sdd1, physical 256771555328, root 112095, inode 257, offset 158251876352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34008.048310] BTRFS warning (device sda1): checksum error at logical 769246236672 on dev /dev/sdd1, physical 256772608000, root 111622, inode 257, offset 158270357504, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34008.048487] BTRFS warning (device sda1): checksum error at logical 769243086848 on dev /dev/sdd1, physical 256771555328, root 111622, inode 257, offset 158251876352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34008.048516] BTRFS warning (device sda1): checksum error at logical 769246236672 on dev /dev/sdd1, physical 256772608000, root 110333, inode 257, offset 158270357504, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34008.048680] BTRFS warning (device sda1): checksum error at logical 769243086848 on dev /dev/sdd1, physical 256771555328, root 110333, inode 257, offset 158251876352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34008.049226] BTRFS warning (device sda1): checksum error at logical 769243086848 on dev /dev/sdd1, physical 256771555328, root 109693, inode 257, offset 158251876352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34008.049248] BTRFS warning (device sda1): checksum error at logical 769244663808 on dev /dev/sdd1, physical 256772083712, root 109693, inode 257, offset 158257774592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34008.049541] BTRFS warning (device sda1): checksum error at logical 769244663808 on dev /dev/sdd1, physical 256772083712, root 109217, inode 257, offset 158257774592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34008.049554] BTRFS warning (device sda1): checksum error at logical 769243086848 on dev /dev/sdd1, physical 256771555328, root 109217, inode 257, offset 158251876352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34008.049878] BTRFS warning (device sda1): checksum error at logical 769244663808 on dev /dev/sdd1, physical 256772083712, root 108743, inode 257, offset 158257774592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34008.049887] BTRFS warning (device sda1): checksum error at logical 769243086848 on dev /dev/sdd1, physical 256771555328, root 108743, inode 257, offset 158251876352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34008.049973] BTRFS warning (device sda1): checksum error at logical 769246236672 on dev /dev/sdd1, physical 256772608000, root 109693, inode 257, offset 158270357504, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34008.050180] BTRFS warning (device sda1): checksum error at logical 769246236672 on dev /dev/sdd1, physical 256772608000, root 109217, inode 257, offset 158270357504, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34008.050400] BTRFS warning (device sda1): checksum error at logical 769246236672 on dev /dev/sdd1, physical 256772608000, root 108743, inode 257, offset 158270357504, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34008.051190] BTRFS warning (device sda1): checksum error at logical 769252528128 on dev /dev/sdd1, physical 256774705152, root 112101, inode 257, offset 58751975424, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34008.052536] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 111778, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[34008.052556] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 111184, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[34008.052574] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 109849, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[34008.052614] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 109373, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[34008.052687] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 108899, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[34008.052704] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 108424, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[34008.052719] BTRFS warning (device sda1): checksum error at logical 769255079936 on dev /dev/sdd1, physical 256775553024, root 107950, inode 257, offset 99673604096, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[34008.052856] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 111778, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[34008.052870] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 111184, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[34008.052884] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 109849, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[34008.052908] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 109373, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[34008.052948] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 108899, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[34008.052961] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 108424, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[34008.052975] BTRFS warning (device sda1): checksum error at logical 769255084032 on dev /dev/sdd1, physical 256775557120, root 107950, inode 257, offset 99673608192, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[34008.053079] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 111778, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645710409)
[34008.053092] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 111184, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645604506)
[34008.053105] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 109849, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645537704)
[34008.053129] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 109373, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645428182)
[34008.053167] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 108899, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645373281)
[34008.053180] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 108424, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645264980)
[34008.053193] BTRFS warning (device sda1): checksum error at logical 769255088128 on dev /dev/sdd1, physical 256775561216, root 107950, inode 257, offset 99673612288, length 4096, links 1 (path: cephstor8_vm-594-disk-0-diff-base.img_1645188088)
[34008.054545] BTRFS warning (device sda1): checksum error at logical 769239949312 on dev /dev/sdd1, physical 256770514944, root 110333, inode 257, offset 158226841600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34008.058241] BTRFS warning (device sda1): checksum error at logical 769252528128 on dev /dev/sdd1, physical 256774705152, root 111626, inode 257, offset 58751975424, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34008.062261] BTRFS warning (device sda1): checksum error at logical 769239949312 on dev /dev/sdd1, physical 256770514944, root 109693, inode 257, offset 158226841600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34008.066153] BTRFS warning (device sda1): checksum error at logical 769252528128 on dev /dev/sdd1, physical 256774705152, root 110392, inode 257, offset 58751975424, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34008.073544] BTRFS warning (device sda1): checksum error at logical 769239949312 on dev /dev/sdd1, physical 256770514944, root 109217, inode 257, offset 158226841600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34008.074692] BTRFS warning (device sda1): checksum error at logical 769252528128 on dev /dev/sdd1, physical 256774705152, root 109697, inode 257, offset 58751975424, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34008.079249] BTRFS warning (device sda1): checksum error at logical 769239949312 on dev /dev/sdd1, physical 256770514944, root 108743, inode 257, offset 158226841600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34008.083888] BTRFS warning (device sda1): checksum error at logical 769252528128 on dev /dev/sdd1, physical 256774705152, root 109219, inode 257, offset 58751975424, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34008.209916] BTRFS warning (device sda1): checksum error at logical 769252528128 on dev /dev/sdd1, physical 256774705152, root 108747, inode 257, offset 58751975424, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34010.482369] btrfs_dev_stat_print_on_error: 17276 callbacks suppressed
[34010.482374] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961085, gen 184286
[34010.482473] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961086, gen 184286
[34010.482567] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961087, gen 184286
[34010.482774] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961088, gen 184286
[34010.482908] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961089, gen 184286
[34010.483048] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961090, gen 184286
[34010.483181] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961091, gen 184286
[34010.483313] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961092, gen 184286
[34010.483439] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961093, gen 184286
[34010.483504] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68961094, gen 184286
[34011.038346] scrub_handle_errored_block: 17452 callbacks suppressed
[34011.038352] BTRFS error (device sda1): unable to fixup (regular) error at logical 769642643456 on dev /dev/sdd1
[34011.040531] BTRFS error (device sda1): unable to fixup (regular) error at logical 769666199552 on dev /dev/sdd1
[34011.041555] BTRFS error (device sda1): unable to fixup (regular) error at logical 769667768320 on dev /dev/sdd1
[34011.041837] BTRFS error (device sda1): unable to fixup (regular) error at logical 769667772416 on dev /dev/sdd1
[34011.042009] BTRFS error (device sda1): unable to fixup (regular) error at logical 769645776896 on dev /dev/sdd1
[34011.042191] BTRFS error (device sda1): unable to fixup (regular) error at logical 769648922624 on dev /dev/sdd1
[34011.042269] BTRFS error (device sda1): unable to fixup (regular) error at logical 769652064256 on dev /dev/sdd1
[34011.042327] BTRFS error (device sda1): unable to fixup (regular) error at logical 769644208128 on dev /dev/sdd1
[34011.042460] BTRFS error (device sda1): unable to fixup (regular) error at logical 769667776512 on dev /dev/sdd1
[34011.042570] BTRFS error (device sda1): unable to fixup (regular) error at logical 769645780992 on dev /dev/sdd1
[34012.985700] scrub_handle_errored_block: 25553 callbacks suppressed
[34013.016641] BTRFS warning (device sda1): checksum error at logical 770084569088 on dev /dev/sdd1, physical 257052049408, root 108267, inode 257, offset 26602418176, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34013.019942] BTRFS warning (device sda1): checksum error at logical 770084569088 on dev /dev/sdd1, physical 257052049408, root 107792, inode 257, offset 26602418176, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34013.040223] BTRFS warning (device sda1): checksum error at logical 770084573184 on dev /dev/sdd1, physical 257052053504, root 108267, inode 257, offset 26602422272, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34013.043392] BTRFS warning (device sda1): checksum error at logical 770084573184 on dev /dev/sdd1, physical 257052053504, root 107792, inode 257, offset 26602422272, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34013.050315] BTRFS warning (device sda1): checksum error at logical 770084577280 on dev /dev/sdd1, physical 257052057600, root 108267, inode 257, offset 26602426368, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34013.080817] BTRFS warning (device sda1): checksum error at logical 770084577280 on dev /dev/sdd1, physical 257052057600, root 107792, inode 257, offset 26602426368, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34013.091359] BTRFS warning (device sda1): checksum error at logical 770094006272 on dev /dev/sdd1, physical 257055195136, root 111661, inode 257, offset 48640462848, length 4096, links 1 (path: cephstor8_vm-379-disk-0-diff-base.img_1645695777)
[34013.093091] BTRFS warning (device sda1): checksum error at logical 770087124992 on dev /dev/sdd1, physical 257052901376, root 111661, inode 257, offset 48592588800, length 4096, links 1 (path: cephstor8_vm-379-disk-0-diff-base.img_1645695777)
[34013.093496] BTRFS warning (device sda1): checksum error at logical 770092433408 on dev /dev/sdd1, physical 257054670848, root 111661, inode 257, offset 48623214592, length 4096, links 1 (path: cephstor8_vm-379-disk-0-diff-base.img_1645695777)
[34013.094203] BTRFS warning (device sda1): checksum error at logical 770091843584 on dev /dev/sdd1, physical 257054474240, root 111661, inode 257, offset 48622297088, length 4096, links 1 (path: cephstor8_vm-379-disk-0-diff-base.img_1645695777)
[34013.094272] BTRFS warning (device sda1): checksum error at logical 770086141952 on dev /dev/sdd1, physical 257052573696, root 111661, inode 257, offset 48585854976, length 4096, links 1 (path: cephstor8_vm-379-disk-0-diff-base.img_1645695777)
[34015.487835] btrfs_dev_stat_print_on_error: 23514 callbacks suppressed
[34015.487841] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984621, gen 184286
[34015.492952] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984622, gen 184286
[34015.493094] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984623, gen 184286
[34015.493215] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984624, gen 184286
[34015.493541] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984625, gen 184286
[34015.494155] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984626, gen 184286
[34015.494166] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984627, gen 184286
[34015.494171] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984628, gen 184286
[34015.505598] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984629, gen 184286
[34015.507373] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 68984630, gen 184286
[34016.047191] scrub_handle_errored_block: 24057 callbacks suppressed
[34016.047196] BTRFS error (device sda1): unable to fixup (regular) error at logical 770436927488 on dev /dev/sdd1
[34016.064656] BTRFS error (device sda1): unable to fixup (regular) error at logical 770462060544 on dev /dev/sdd1
[34016.066127] BTRFS error (device sda1): unable to fixup (regular) error at logical 770463633408 on dev /dev/sdd1
[34016.067980] BTRFS error (device sda1): unable to fixup (regular) error at logical 770466775040 on dev /dev/sdd1
[34016.081633] BTRFS error (device sda1): unable to fixup (regular) error at logical 770436931584 on dev /dev/sdd1
[34016.082136] BTRFS error (device sda1): unable to fixup (regular) error at logical 770465206272 on dev /dev/sdd1
[34016.092192] BTRFS error (device sda1): unable to fixup (regular) error at logical 770458918912 on dev /dev/sdd1
[34016.099753] BTRFS error (device sda1): unable to fixup (regular) error at logical 770460491776 on dev /dev/sdd1
[34016.105453] BTRFS error (device sda1): unable to fixup (regular) error at logical 770436935680 on dev /dev/sdd1
[34016.122142] BTRFS error (device sda1): unable to fixup (regular) error at logical 770436939776 on dev /dev/sdd1
[34017.992612] scrub_handle_errored_block: 16210 callbacks suppressed
[34018.028205] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 111763, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028219] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 111763, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028226] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 111763, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028234] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 111763, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028238] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 111763, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028243] BTRFS warning (device sda1): checksum error at logical 770633498624 on dev /dev/sdd1, physical 257235025920, root 111066, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028246] BTRFS warning (device sda1): checksum error at logical 770630356992 on dev /dev/sdd1, physical 257233981440, root 111066, inode 257, offset 142414053376, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028250] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 111066, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028253] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 111066, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028257] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 111066, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028263] BTRFS warning (device sda1): checksum error at logical 770630356992 on dev /dev/sdd1, physical 257233981440, root 109833, inode 257, offset 142414053376, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028267] BTRFS warning (device sda1): checksum error at logical 770633498624 on dev /dev/sdd1, physical 257235025920, root 109833, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028270] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 109833, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028274] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 109833, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028277] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 109833, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028281] BTRFS warning (device sda1): checksum error at logical 770630356992 on dev /dev/sdd1, physical 257233981440, root 109359, inode 257, offset 142414053376, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028285] BTRFS warning (device sda1): checksum error at logical 770633498624 on dev /dev/sdd1, physical 257235025920, root 109359, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028289] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 109359, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028292] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 109359, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028296] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 109359, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028298] BTRFS warning (device sda1): checksum error at logical 770630356992 on dev /dev/sdd1, physical 257233981440, root 108884, inode 257, offset 142414053376, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028303] BTRFS warning (device sda1): checksum error at logical 770633498624 on dev /dev/sdd1, physical 257235025920, root 108884, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028307] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 108884, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028311] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 108884, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028314] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 108884, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028317] BTRFS warning (device sda1): checksum error at logical 770630356992 on dev /dev/sdd1, physical 257233981440, root 108409, inode 257, offset 142414053376, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028322] BTRFS warning (device sda1): checksum error at logical 770633498624 on dev /dev/sdd1, physical 257235025920, root 108409, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028326] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 108409, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028329] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 108409, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028333] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 108409, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028336] BTRFS warning (device sda1): checksum error at logical 770630356992 on dev /dev/sdd1, physical 257233981440, root 107934, inode 257, offset 142414053376, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.028340] BTRFS warning (device sda1): checksum error at logical 770633498624 on dev /dev/sdd1, physical 257235025920, root 107934, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.028344] BTRFS warning (device sda1): checksum error at logical 770624069632 on dev /dev/sdd1, physical 257231888384, root 107934, inode 257, offset 142405140480, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.028347] BTRFS warning (device sda1): checksum error at logical 770619375616 on dev /dev/sdd1, physical 257230340096, root 107934, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.028351] BTRFS warning (device sda1): checksum error at logical 770620928000 on dev /dev/sdd1, physical 257230843904, root 107934, inode 257, offset 142400815104, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.028677] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 111763, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028694] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 111066, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028710] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 109833, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028727] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 109359, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028743] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 108884, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028759] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 108409, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028763] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 111763, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34018.028775] BTRFS warning (device sda1): checksum error at logical 770633502720 on dev /dev/sdd1, physical 257235030016, root 107934, inode 257, offset 142418378752, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.028780] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 111066, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.028796] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 109833, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.028812] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 109359, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.028824] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 108884, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.028835] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 108409, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.028846] BTRFS warning (device sda1): checksum error at logical 770619379712 on dev /dev/sdd1, physical 257230344192, root 107934, inode 257, offset 142398849024, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.031599] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 111066, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.035112] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 111066, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34018.038362] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 109833, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.039887] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 109833, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34018.043138] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 109359, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.044575] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 109359, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34018.047819] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 108884, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.049250] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 108884, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34018.052495] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 108409, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.053924] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 108409, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34018.057181] BTRFS warning (device sda1): checksum error at logical 770627211264 on dev /dev/sdd1, physical 257232932864, root 107934, inode 257, offset 142409728000, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.058619] BTRFS warning (device sda1): checksum error at logical 770625638400 on dev /dev/sdd1, physical 257232408576, root 107934, inode 257, offset 142407368704, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34018.071221] BTRFS warning (device sda1): checksum error at logical 770636644352 on dev /dev/sdd1, physical 257236074496, root 112074, inode 257, offset 420316786688, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645758034)
[34019.163763] BTRFS warning (device sda1): checksum error at logical 770636644352 on dev /dev/sdd1, physical 257236074496, root 111600, inode 257, offset 420316786688, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645674568)
[34019.188293] BTRFS warning (device sda1): checksum error at logical 770636644352 on dev /dev/sdd1, physical 257236074496, root 110180, inode 257, offset 420316786688, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645585019)
[34019.214045] BTRFS warning (device sda1): checksum error at logical 770636644352 on dev /dev/sdd1, physical 257236074496, root 109671, inode 257, offset 420316786688, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645509396)
[34019.236416] BTRFS warning (device sda1): checksum error at logical 770636644352 on dev /dev/sdd1, physical 257236074496, root 109195, inode 257, offset 420316786688, length 4096, links 1 (path: cephstor8_vm-255-disk-0-diff-base.img_1645411324)
[34019.263076] BTRFS warning (device sda1): checksum error at logical 770636644352 on dev /dev/sdd1, physical 257236074496, root 108721, inode 257, offset 420316786688, length 4096, links 1 (path: cephstor8_vm-255-disk-0-base.img_1645332198)
[34020.490586] btrfs_dev_stat_print_on_error: 16206 callbacks suppressed
[34020.490592] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000853, gen 184286
[34020.502719] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000854, gen 184286
[34020.506505] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000855, gen 184286
[34020.507593] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000856, gen 184286
[34020.511524] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000857, gen 184286
[34020.511750] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000858, gen 184286
[34020.531533] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000859, gen 184286
[34020.540361] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000860, gen 184286
[34020.545939] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000861, gen 184286
[34020.549449] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69000862, gen 184286
[34021.050485] scrub_handle_errored_block: 15007 callbacks suppressed
[34021.050490] BTRFS error (device sda1): unable to fixup (regular) error at logical 770952794112 on dev /dev/sdd1
[34021.052936] BTRFS error (device sda1): unable to fixup (regular) error at logical 770949652480 on dev /dev/sdd1
[34021.053895] BTRFS error (device sda1): unable to fixup (regular) error at logical 770954366976 on dev /dev/sdd1
[34021.054541] BTRFS error (device sda1): unable to fixup (regular) error at logical 770955939840 on dev /dev/sdd1
[34021.054646] BTRFS error (device sda1): unable to fixup (regular) error at logical 770959085568 on dev /dev/sdd1
[34021.054860] BTRFS error (device sda1): unable to fixup (regular) error at logical 770963800064 on dev /dev/sdd1
[34021.055008] BTRFS error (device sda1): unable to fixup (regular) error at logical 770960658432 on dev /dev/sdd1
[34021.056088] BTRFS error (device sda1): unable to fixup (regular) error at logical 770962227200 on dev /dev/sdd1
[34021.060916] BTRFS error (device sda1): unable to fixup (regular) error at logical 770949656576 on dev /dev/sdd1
[34021.063731] BTRFS error (device sda1): unable to fixup (regular) error at logical 770952798208 on dev /dev/sdd1
[34022.993929] scrub_handle_errored_block: 16495 callbacks suppressed
[34023.011519] BTRFS warning (device sda1): checksum error at logical 771169849344 on dev /dev/sdd1, physical 257413812224, root 112095, inode 257, offset 158344937472, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.014750] BTRFS warning (device sda1): checksum error at logical 771169849344 on dev /dev/sdd1, physical 257413812224, root 111622, inode 257, offset 158344937472, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.016433] BTRFS warning (device sda1): checksum error at logical 771165130752 on dev /dev/sdd1, physical 257412239360, root 112095, inode 257, offset 158327783424, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.017616] BTRFS warning (device sda1): checksum error at logical 771166707712 on dev /dev/sdd1, physical 257412767744, root 112095, inode 257, offset 158329360384, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.017636] BTRFS warning (device sda1): checksum error at logical 771166707712 on dev /dev/sdd1, physical 257412767744, root 111622, inode 257, offset 158329360384, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.017642] BTRFS warning (device sda1): checksum error at logical 771174563840 on dev /dev/sdd1, physical 257415380992, root 112095, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.017652] BTRFS warning (device sda1): checksum error at logical 771166707712 on dev /dev/sdd1, physical 257412767744, root 110333, inode 257, offset 158329360384, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.017658] BTRFS warning (device sda1): checksum error at logical 771171463168 on dev /dev/sdd1, physical 257414377472, root 112095, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.017661] BTRFS warning (device sda1): checksum error at logical 771174563840 on dev /dev/sdd1, physical 257415380992, root 111622, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.017668] BTRFS warning (device sda1): checksum error at logical 771166707712 on dev /dev/sdd1, physical 257412767744, root 109693, inode 257, offset 158329360384, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.017673] BTRFS warning (device sda1): checksum error at logical 771171463168 on dev /dev/sdd1, physical 257414377472, root 111622, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.017677] BTRFS warning (device sda1): checksum error at logical 771174563840 on dev /dev/sdd1, physical 257415380992, root 110333, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.017686] BTRFS warning (device sda1): checksum error at logical 771166707712 on dev /dev/sdd1, physical 257412767744, root 109217, inode 257, offset 158329360384, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.017689] BTRFS warning (device sda1): checksum error at logical 771171463168 on dev /dev/sdd1, physical 257414377472, root 110333, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.017692] BTRFS warning (device sda1): checksum error at logical 771174563840 on dev /dev/sdd1, physical 257415380992, root 109693, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.017701] BTRFS warning (device sda1): checksum error at logical 771171463168 on dev /dev/sdd1, physical 257414377472, root 109693, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.017705] BTRFS warning (device sda1): checksum error at logical 771166707712 on dev /dev/sdd1, physical 257412767744, root 108743, inode 257, offset 158329360384, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.017708] BTRFS warning (device sda1): checksum error at logical 771174563840 on dev /dev/sdd1, physical 257415380992, root 109217, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.017713] BTRFS warning (device sda1): checksum error at logical 771171463168 on dev /dev/sdd1, physical 257414377472, root 109217, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.017722] BTRFS warning (device sda1): checksum error at logical 771174563840 on dev /dev/sdd1, physical 257415380992, root 108743, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.017725] BTRFS warning (device sda1): checksum error at logical 771171463168 on dev /dev/sdd1, physical 257414377472, root 108743, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.017851] BTRFS warning (device sda1): checksum error at logical 771169849344 on dev /dev/sdd1, physical 257413812224, root 110333, inode 257, offset 158344937472, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.017913] BTRFS warning (device sda1): checksum error at logical 771171467264 on dev /dev/sdd1, physical 257414381568, root 112095, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.017922] BTRFS warning (device sda1): checksum error at logical 771172995072 on dev /dev/sdd1, physical 257414860800, root 112095, inode 257, offset 158360928256, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.017926] BTRFS warning (device sda1): checksum error at logical 771171467264 on dev /dev/sdd1, physical 257414381568, root 111622, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.017933] BTRFS warning (device sda1): checksum error at logical 771172995072 on dev /dev/sdd1, physical 257414860800, root 111622, inode 257, offset 158360928256, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.017951] BTRFS warning (device sda1): checksum error at logical 771171467264 on dev /dev/sdd1, physical 257414381568, root 110333, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.017953] BTRFS warning (device sda1): checksum error at logical 771172995072 on dev /dev/sdd1, physical 257414860800, root 110333, inode 257, offset 158360928256, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.017965] BTRFS warning (device sda1): checksum error at logical 771172995072 on dev /dev/sdd1, physical 257414860800, root 109693, inode 257, offset 158360928256, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.017967] BTRFS warning (device sda1): checksum error at logical 771171467264 on dev /dev/sdd1, physical 257414381568, root 109693, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.017978] BTRFS warning (device sda1): checksum error at logical 771172995072 on dev /dev/sdd1, physical 257414860800, root 109217, inode 257, offset 158360928256, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.017981] BTRFS warning (device sda1): checksum error at logical 771171467264 on dev /dev/sdd1, physical 257414381568, root 109217, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.017993] BTRFS warning (device sda1): checksum error at logical 771172995072 on dev /dev/sdd1, physical 257414860800, root 108743, inode 257, offset 158360928256, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.017996] BTRFS warning (device sda1): checksum error at logical 771171467264 on dev /dev/sdd1, physical 257414381568, root 108743, inode 257, offset 158355554304, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.018107] BTRFS warning (device sda1): checksum error at logical 771174567936 on dev /dev/sdd1, physical 257415385088, root 112095, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.018123] BTRFS warning (device sda1): checksum error at logical 771174567936 on dev /dev/sdd1, physical 257415385088, root 111622, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.018132] BTRFS warning (device sda1): checksum error at logical 771174567936 on dev /dev/sdd1, physical 257415385088, root 110333, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.018146] BTRFS warning (device sda1): checksum error at logical 771174567936 on dev /dev/sdd1, physical 257415385088, root 109693, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.018158] BTRFS warning (device sda1): checksum error at logical 771174567936 on dev /dev/sdd1, physical 257415385088, root 109217, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.018176] BTRFS warning (device sda1): checksum error at logical 771174567936 on dev /dev/sdd1, physical 257415385088, root 108743, inode 257, offset 158366564352, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.019535] BTRFS warning (device sda1): checksum error at logical 771165130752 on dev /dev/sdd1, physical 257412239360, root 111622, inode 257, offset 158327783424, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.023112] BTRFS warning (device sda1): checksum error at logical 771169849344 on dev /dev/sdd1, physical 257413812224, root 109693, inode 257, offset 158344937472, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.024689] BTRFS warning (device sda1): checksum error at logical 771165130752 on dev /dev/sdd1, physical 257412239360, root 110333, inode 257, offset 158327783424, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.028062] BTRFS warning (device sda1): checksum error at logical 771169849344 on dev /dev/sdd1, physical 257413812224, root 109217, inode 257, offset 158344937472, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.029641] BTRFS warning (device sda1): checksum error at logical 771165130752 on dev /dev/sdd1, physical 257412239360, root 109693, inode 257, offset 158327783424, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.032972] BTRFS warning (device sda1): checksum error at logical 771169849344 on dev /dev/sdd1, physical 257413812224, root 108743, inode 257, offset 158344937472, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.034532] BTRFS warning (device sda1): checksum error at logical 771163570176 on dev /dev/sdd1, physical 257411727360, root 112095, inode 257, offset 158321999872, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.034553] BTRFS warning (device sda1): checksum error at logical 771161980928 on dev /dev/sdd1, physical 257411186688, root 112095, inode 257, offset 158309679104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645762263)
[34023.034557] BTRFS warning (device sda1): checksum error at logical 771163570176 on dev /dev/sdd1, physical 257411727360, root 111622, inode 257, offset 158321999872, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.034572] BTRFS warning (device sda1): checksum error at logical 771161980928 on dev /dev/sdd1, physical 257411186688, root 111622, inode 257, offset 158309679104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645682786)
[34023.034575] BTRFS warning (device sda1): checksum error at logical 771163570176 on dev /dev/sdd1, physical 257411727360, root 110333, inode 257, offset 158321999872, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.034584] BTRFS warning (device sda1): checksum error at logical 771161980928 on dev /dev/sdd1, physical 257411186688, root 110333, inode 257, offset 158309679104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645588073)
[34023.034594] BTRFS warning (device sda1): checksum error at logical 771165130752 on dev /dev/sdd1, physical 257412239360, root 109217, inode 257, offset 158327783424, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.034652] BTRFS warning (device sda1): checksum error at logical 771163570176 on dev /dev/sdd1, physical 257411727360, root 109693, inode 257, offset 158321999872, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.034658] BTRFS warning (device sda1): checksum error at logical 771161980928 on dev /dev/sdd1, physical 257411186688, root 109693, inode 257, offset 158309679104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645515646)
[34023.034679] BTRFS warning (device sda1): checksum error at logical 771163570176 on dev /dev/sdd1, physical 257411727360, root 109217, inode 257, offset 158321999872, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.034682] BTRFS warning (device sda1): checksum error at logical 771161980928 on dev /dev/sdd1, physical 257411186688, root 109217, inode 257, offset 158309679104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645416022)
[34023.034696] BTRFS warning (device sda1): checksum error at logical 771161980928 on dev /dev/sdd1, physical 257411186688, root 108743, inode 257, offset 158309679104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.034700] BTRFS warning (device sda1): checksum error at logical 771163570176 on dev /dev/sdd1, physical 257411727360, root 108743, inode 257, offset 158321999872, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34023.144759] BTRFS warning (device sda1): checksum error at logical 771165130752 on dev /dev/sdd1, physical 257412239360, root 108743, inode 257, offset 158327783424, length 4096, links 1 (path: cephstor8_vm-288-disk-0-base.img_1645336604)
[34025.508478] btrfs_dev_stat_print_on_error: 16663 callbacks suppressed
[34025.508483] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017541, gen 184286
[34025.510629] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017542, gen 184286
[34025.513823] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017543, gen 184286
[34025.517198] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017544, gen 184286
[34025.534036] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017545, gen 184286
[34025.541834] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017546, gen 184286
[34025.542400] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017547, gen 184286
[34025.542914] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017548, gen 184286
[34025.543366] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017549, gen 184286
[34025.543768] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69017550, gen 184286
[34026.060232] scrub_handle_errored_block: 17025 callbacks suppressed
[34026.060236] BTRFS error (device sda1): unable to fixup (regular) error at logical 771512733696 on dev /dev/sdd1
[34026.061151] BTRFS error (device sda1): unable to fixup (regular) error at logical 771514306560 on dev /dev/sdd1
[34026.061252] BTRFS error (device sda1): unable to fixup (regular) error at logical 771509592064 on dev /dev/sdd1
[34026.061329] BTRFS error (device sda1): unable to fixup (regular) error at logical 771511164928 on dev /dev/sdd1
[34026.065838] BTRFS error (device sda1): unable to fixup (regular) error at logical 771504877568 on dev /dev/sdd1
[34026.077381] BTRFS error (device sda1): unable to fixup (regular) error at logical 771520598016 on dev /dev/sdd1
[34026.077785] BTRFS error (device sda1): unable to fixup (regular) error at logical 771509596160 on dev /dev/sdd1
[34026.077850] BTRFS error (device sda1): unable to fixup (regular) error at logical 771508019200 on dev /dev/sdd1
[34026.078045] BTRFS error (device sda1): unable to fixup (regular) error at logical 771509600256 on dev /dev/sdd1
[34026.078188] BTRFS error (device sda1): unable to fixup (regular) error at logical 771509604352 on dev /dev/sdd1
[34028.005069] scrub_handle_errored_block: 15532 callbacks suppressed
[34028.005246] BTRFS warning (device sda1): checksum error at logical 771673169920 on dev /dev/sdd1, physical 257581588480, root 112101, inode 257, offset 59153227776, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.009989] BTRFS warning (device sda1): checksum error at logical 771673169920 on dev /dev/sdd1, physical 257581588480, root 111626, inode 257, offset 59153227776, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.013057] BTRFS warning (device sda1): checksum error at logical 771673169920 on dev /dev/sdd1, physical 257581588480, root 110392, inode 257, offset 59153227776, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.016137] BTRFS warning (device sda1): checksum error at logical 771673169920 on dev /dev/sdd1, physical 257581588480, root 109697, inode 257, offset 59153227776, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.017139] BTRFS warning (device sda1): checksum error at logical 771677884416 on dev /dev/sdd1, physical 257583157248, root 112101, inode 257, offset 59157942272, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.019234] BTRFS warning (device sda1): checksum error at logical 771673169920 on dev /dev/sdd1, physical 257581588480, root 109219, inode 257, offset 59153227776, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.019511] BTRFS warning (device sda1): checksum error at logical 771670044672 on dev /dev/sdd1, physical 257580560384, root 112101, inode 257, offset 59150102528, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.019530] BTRFS warning (device sda1): checksum error at logical 771670044672 on dev /dev/sdd1, physical 257580560384, root 111626, inode 257, offset 59150102528, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.019546] BTRFS warning (device sda1): checksum error at logical 771670044672 on dev /dev/sdd1, physical 257580560384, root 110392, inode 257, offset 59150102528, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.019562] BTRFS warning (device sda1): checksum error at logical 771670044672 on dev /dev/sdd1, physical 257580560384, root 109697, inode 257, offset 59150102528, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.019577] BTRFS warning (device sda1): checksum error at logical 771670044672 on dev /dev/sdd1, physical 257580560384, root 109219, inode 257, offset 59150102528, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.019590] BTRFS warning (device sda1): checksum error at logical 771670044672 on dev /dev/sdd1, physical 257580560384, root 108747, inode 257, offset 59150102528, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.019957] BTRFS warning (device sda1): checksum error at logical 771670048768 on dev /dev/sdd1, physical 257580564480, root 112101, inode 257, offset 59150106624, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.019970] BTRFS warning (device sda1): checksum error at logical 771670048768 on dev /dev/sdd1, physical 257580564480, root 111626, inode 257, offset 59150106624, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.019983] BTRFS warning (device sda1): checksum error at logical 771670048768 on dev /dev/sdd1, physical 257580564480, root 110392, inode 257, offset 59150106624, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.019995] BTRFS warning (device sda1): checksum error at logical 771670048768 on dev /dev/sdd1, physical 257580564480, root 109697, inode 257, offset 59150106624, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.020007] BTRFS warning (device sda1): checksum error at logical 771670048768 on dev /dev/sdd1, physical 257580564480, root 109219, inode 257, offset 59150106624, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.020019] BTRFS warning (device sda1): checksum error at logical 771670048768 on dev /dev/sdd1, physical 257580564480, root 108747, inode 257, offset 59150106624, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.020334] BTRFS warning (device sda1): checksum error at logical 771670052864 on dev /dev/sdd1, physical 257580568576, root 112101, inode 257, offset 59150110720, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.020347] BTRFS warning (device sda1): checksum error at logical 771670052864 on dev /dev/sdd1, physical 257580568576, root 111626, inode 257, offset 59150110720, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.020360] BTRFS warning (device sda1): checksum error at logical 771670052864 on dev /dev/sdd1, physical 257580568576, root 110392, inode 257, offset 59150110720, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.020372] BTRFS warning (device sda1): checksum error at logical 771670052864 on dev /dev/sdd1, physical 257580568576, root 109697, inode 257, offset 59150110720, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.020385] BTRFS warning (device sda1): checksum error at logical 771670052864 on dev /dev/sdd1, physical 257580568576, root 109219, inode 257, offset 59150110720, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.020397] BTRFS warning (device sda1): checksum error at logical 771670052864 on dev /dev/sdd1, physical 257580568576, root 108747, inode 257, offset 59150110720, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.020661] BTRFS warning (device sda1): checksum error at logical 771670056960 on dev /dev/sdd1, physical 257580572672, root 112101, inode 257, offset 59150114816, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.020674] BTRFS warning (device sda1): checksum error at logical 771670056960 on dev /dev/sdd1, physical 257580572672, root 111626, inode 257, offset 59150114816, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.020686] BTRFS warning (device sda1): checksum error at logical 771670056960 on dev /dev/sdd1, physical 257580572672, root 110392, inode 257, offset 59150114816, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.020699] BTRFS warning (device sda1): checksum error at logical 771670056960 on dev /dev/sdd1, physical 257580572672, root 109697, inode 257, offset 59150114816, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.020711] BTRFS warning (device sda1): checksum error at logical 771670056960 on dev /dev/sdd1, physical 257580572672, root 109219, inode 257, offset 59150114816, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.020723] BTRFS warning (device sda1): checksum error at logical 771670056960 on dev /dev/sdd1, physical 257580572672, root 108747, inode 257, offset 59150114816, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.020960] BTRFS warning (device sda1): checksum error at logical 771670061056 on dev /dev/sdd1, physical 257580576768, root 112101, inode 257, offset 59150118912, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.020973] BTRFS warning (device sda1): checksum error at logical 771670061056 on dev /dev/sdd1, physical 257580576768, root 111626, inode 257, offset 59150118912, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.020986] BTRFS warning (device sda1): checksum error at logical 771670061056 on dev /dev/sdd1, physical 257580576768, root 110392, inode 257, offset 59150118912, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.020998] BTRFS warning (device sda1): checksum error at logical 771670061056 on dev /dev/sdd1, physical 257580576768, root 109697, inode 257, offset 59150118912, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.021009] BTRFS warning (device sda1): checksum error at logical 771670061056 on dev /dev/sdd1, physical 257580576768, root 109219, inode 257, offset 59150118912, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.021020] BTRFS warning (device sda1): checksum error at logical 771670061056 on dev /dev/sdd1, physical 257580576768, root 108747, inode 257, offset 59150118912, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.021262] BTRFS warning (device sda1): checksum error at logical 771670065152 on dev /dev/sdd1, physical 257580580864, root 112101, inode 257, offset 59150123008, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.021274] BTRFS warning (device sda1): checksum error at logical 771670065152 on dev /dev/sdd1, physical 257580580864, root 111626, inode 257, offset 59150123008, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.021286] BTRFS warning (device sda1): checksum error at logical 771670065152 on dev /dev/sdd1, physical 257580580864, root 110392, inode 257, offset 59150123008, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.021299] BTRFS warning (device sda1): checksum error at logical 771670065152 on dev /dev/sdd1, physical 257580580864, root 109697, inode 257, offset 59150123008, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.021311] BTRFS warning (device sda1): checksum error at logical 771670065152 on dev /dev/sdd1, physical 257580580864, root 109219, inode 257, offset 59150123008, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.021323] BTRFS warning (device sda1): checksum error at logical 771670065152 on dev /dev/sdd1, physical 257580580864, root 108747, inode 257, offset 59150123008, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.021553] BTRFS warning (device sda1): checksum error at logical 771670069248 on dev /dev/sdd1, physical 257580584960, root 112101, inode 257, offset 59150127104, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.021566] BTRFS warning (device sda1): checksum error at logical 771670069248 on dev /dev/sdd1, physical 257580584960, root 111626, inode 257, offset 59150127104, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.021578] BTRFS warning (device sda1): checksum error at logical 771670069248 on dev /dev/sdd1, physical 257580584960, root 110392, inode 257, offset 59150127104, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.021590] BTRFS warning (device sda1): checksum error at logical 771670069248 on dev /dev/sdd1, physical 257580584960, root 109697, inode 257, offset 59150127104, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.021603] BTRFS warning (device sda1): checksum error at logical 771670069248 on dev /dev/sdd1, physical 257580584960, root 109219, inode 257, offset 59150127104, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.021619] BTRFS warning (device sda1): checksum error at logical 771670069248 on dev /dev/sdd1, physical 257580584960, root 108747, inode 257, offset 59150127104, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.021850] BTRFS warning (device sda1): checksum error at logical 771670073344 on dev /dev/sdd1, physical 257580589056, root 112101, inode 257, offset 59150131200, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645762291)
[34028.021863] BTRFS warning (device sda1): checksum error at logical 771670073344 on dev /dev/sdd1, physical 257580589056, root 111626, inode 257, offset 59150131200, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.021875] BTRFS warning (device sda1): checksum error at logical 771670073344 on dev /dev/sdd1, physical 257580589056, root 110392, inode 257, offset 59150131200, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34028.021888] BTRFS warning (device sda1): checksum error at logical 771670073344 on dev /dev/sdd1, physical 257580589056, root 109697, inode 257, offset 59150131200, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34028.021903] BTRFS warning (device sda1): checksum error at logical 771670073344 on dev /dev/sdd1, physical 257580589056, root 109219, inode 257, offset 59150131200, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34028.021916] BTRFS warning (device sda1): checksum error at logical 771670073344 on dev /dev/sdd1, physical 257580589056, root 108747, inode 257, offset 59150131200, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.022650] BTRFS warning (device sda1): checksum error at logical 771677884416 on dev /dev/sdd1, physical 257583157248, root 111626, inode 257, offset 59157942272, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645684920)
[34028.025763] BTRFS warning (device sda1): checksum error at logical 771673169920 on dev /dev/sdd1, physical 257581588480, root 108747, inode 257, offset 59153227776, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34028.028880] BTRFS warning (device sda1): checksum error at logical 771677884416 on dev /dev/sdd1, physical 257583157248, root 110392, inode 257, offset 59157942272, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645589093)
[34029.410547] BTRFS warning (device sda1): checksum error at logical 771677884416 on dev /dev/sdd1, physical 257583157248, root 109697, inode 257, offset 59157942272, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645516572)
[34029.441709] BTRFS warning (device sda1): checksum error at logical 771677884416 on dev /dev/sdd1, physical 257583157248, root 109219, inode 257, offset 59157942272, length 4096, links 1 (path: cephstor8_vm-297-disk-0-diff-base.img_1645416069)
[34029.468718] BTRFS warning (device sda1): checksum error at logical 771677884416 on dev /dev/sdd1, physical 257583157248, root 108747, inode 257, offset 59157942272, length 4096, links 1 (path: cephstor8_vm-297-disk-0-base.img_1645337735)
[34030.510291] btrfs_dev_stat_print_on_error: 16089 callbacks suppressed
[34030.510296] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033647, gen 184286
[34030.510350] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033648, gen 184286
[34030.510548] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033649, gen 184286
[34030.510628] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033650, gen 184286
[34030.519986] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033651, gen 184286
[34030.520568] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033652, gen 184286
[34030.521224] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033653, gen 184286
[34030.521269] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033654, gen 184286
[34030.521481] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033655, gen 184286
[34030.522168] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69033656, gen 184286
[34031.062013] scrub_handle_errored_block: 17563 callbacks suppressed
[34031.062019] BTRFS error (device sda1): unable to fixup (regular) error at logical 772104126464 on dev /dev/sdd1
[34031.064129] BTRFS error (device sda1): unable to fixup (regular) error at logical 772106682368 on dev /dev/sdd1
[34031.088142] BTRFS error (device sda1): unable to fixup (regular) error at logical 772106686464 on dev /dev/sdd1
[34031.102454] BTRFS error (device sda1): unable to fixup (regular) error at logical 772106690560 on dev /dev/sdd1
[34031.105241] BTRFS error (device sda1): unable to fixup (regular) error at logical 772105699328 on dev /dev/sdd1
[34031.107917] BTRFS error (device sda1): unable to fixup (regular) error at logical 772097839104 on dev /dev/sdd1
[34031.116976] BTRFS error (device sda1): unable to fixup (regular) error at logical 772106694656 on dev /dev/sdd1
[34031.124499] BTRFS error (device sda1): unable to fixup (regular) error at logical 772097843200 on dev /dev/sdd1
[34031.125656] BTRFS error (device sda1): unable to fixup (regular) error at logical 772099411968 on dev /dev/sdd1
[34031.127591] BTRFS error (device sda1): unable to fixup (regular) error at logical 772100984832 on dev /dev/sdd1
[34033.005583] scrub_handle_errored_block: 16198 callbacks suppressed
[34033.019704] BTRFS warning (device sda1): checksum error at logical 772225294336 on dev /dev/sdd1, physical 257765662720, root 109175, inode 257, offset 38758162432, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34033.019959] BTRFS warning (device sda1): checksum error at logical 772228382720 on dev /dev/sdd1, physical 257766653952, root 109175, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34033.020027] BTRFS warning (device sda1): checksum error at logical 772226867200 on dev /dev/sdd1, physical 257766187008, root 108700, inode 257, offset 38760390656, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34033.020086] BTRFS warning (device sda1): checksum error at logical 772228382720 on dev /dev/sdd1, physical 257766653952, root 108700, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34033.020274] BTRFS warning (device sda1): checksum error at logical 772228382720 on dev /dev/sdd1, physical 257766653952, root 108227, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34033.020448] BTRFS warning (device sda1): checksum error at logical 772228382720 on dev /dev/sdd1, physical 257766653952, root 107751, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34033.025593] BTRFS warning (device sda1): checksum error at logical 772228386816 on dev /dev/sdd1, physical 257766658048, root 109175, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34033.025614] BTRFS warning (device sda1): checksum error at logical 772228386816 on dev /dev/sdd1, physical 257766658048, root 108700, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34033.025794] BTRFS warning (device sda1): checksum error at logical 772228386816 on dev /dev/sdd1, physical 257766658048, root 108227, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34033.026011] BTRFS warning (device sda1): checksum error at logical 772228386816 on dev /dev/sdd1, physical 257766658048, root 107751, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34033.029594] BTRFS warning (device sda1): checksum error at logical 772228390912 on dev /dev/sdd1, physical 257766662144, root 109175, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645408234)
[34033.029695] BTRFS warning (device sda1): checksum error at logical 772228390912 on dev /dev/sdd1, physical 257766662144, root 108700, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34033.029829] BTRFS warning (device sda1): checksum error at logical 772228390912 on dev /dev/sdd1, physical 257766662144, root 108227, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34033.029975] BTRFS warning (device sda1): checksum error at logical 772228390912 on dev /dev/sdd1, physical 257766662144, root 107751, inode 257, offset 38762356736, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34033.030737] BTRFS warning (device sda1): checksum error at logical 772215808000 on dev /dev/sdd1, physical 257762467840, root 111661, inode 257, offset 63910649856, length 4096, links 1 (path: cephstor8_vm-379-disk-0-diff-base.img_1645695777)
[34033.041578] BTRFS warning (device sda1): checksum error at logical 772218945536 on dev /dev/sdd1, physical 257763508224, root 111663, inode 257, offset 13016621056, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[34033.044883] BTRFS warning (device sda1): checksum error at logical 772221710336 on dev /dev/sdd1, physical 257764438016, root 111663, inode 257, offset 13029896192, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[34033.046046] BTRFS warning (device sda1): checksum error at logical 772225294336 on dev /dev/sdd1, physical 257765662720, root 108700, inode 257, offset 38758162432, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645325781)
[34033.046070] BTRFS warning (device sda1): checksum error at logical 772225294336 on dev /dev/sdd1, physical 257765662720, root 108227, inode 257, offset 38758162432, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34033.046113] BTRFS warning (device sda1): checksum error at logical 772225294336 on dev /dev/sdd1, physical 257765662720, root 107751, inode 257, offset 38758162432, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34033.047265] BTRFS warning (device sda1): checksum error at logical 772222091264 on dev /dev/sdd1, physical 257764556800, root 111663, inode 257, offset 13060333568, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[34033.050327] BTRFS warning (device sda1): checksum error at logical 772220579840 on dev /dev/sdd1, physical 257764093952, root 111663, inode 257, offset 13024751616, length 4096, links 1 (path: cephstor8_vm-383-disk-0-diff-base.img_1645697355)
[34033.062490] BTRFS warning (device sda1): checksum error at logical 772226867200 on dev /dev/sdd1, physical 257766187008, root 108227, inode 257, offset 38760390656, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645240102)
[34033.525801] BTRFS warning (device sda1): checksum error at logical 772226867200 on dev /dev/sdd1, physical 257766187008, root 107751, inode 257, offset 38760390656, length 4096, links 1 (path: cephstor8_vm-211-disk-0-diff-base.img_1645157205)
[34035.517833] btrfs_dev_stat_print_on_error: 15848 callbacks suppressed
[34035.517839] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049515, gen 184286
[34035.519851] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049516, gen 184286
[34035.524209] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049517, gen 184286
[34035.533562] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049518, gen 184286
[34035.535649] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049519, gen 184286
[34035.542736] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049520, gen 184286
[34035.542830] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049521, gen 184286
[34035.549413] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049522, gen 184286
[34035.552632] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049523, gen 184286
[34035.553367] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69049524, gen 184286
[34036.066146] scrub_handle_errored_block: 14004 callbacks suppressed
[34036.066151] BTRFS error (device sda1): unable to fixup (regular) error at logical 772565958656 on dev /dev/sdd1
[34036.066224] BTRFS error (device sda1): unable to fixup (regular) error at logical 772555542528 on dev /dev/sdd1
[34036.066329] BTRFS error (device sda1): unable to fixup (regular) error at logical 772558688256 on dev /dev/sdd1
[34036.066657] BTRFS error (device sda1): unable to fixup (regular) error at logical 772566548480 on dev /dev/sdd1
[34036.069972] BTRFS error (device sda1): unable to fixup (regular) error at logical 772557119488 on dev /dev/sdd1
[34036.070469] BTRFS error (device sda1): unable to fixup (regular) error at logical 772563406848 on dev /dev/sdd1
[34036.070590] BTRFS error (device sda1): unable to fixup (regular) error at logical 772555546624 on dev /dev/sdd1
[34036.070743] BTRFS error (device sda1): unable to fixup (regular) error at logical 772555550720 on dev /dev/sdd1
[34036.070884] BTRFS error (device sda1): unable to fixup (regular) error at logical 772555554816 on dev /dev/sdd1
[34036.071027] BTRFS error (device sda1): unable to fixup (regular) error at logical 772555558912 on dev /dev/sdd1
[34038.021049] scrub_handle_errored_block: 19390 callbacks suppressed
[34038.039585] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 111784, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.042764] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 111196, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.045884] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 109854, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.048946] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 109380, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.050331] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 111784, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.050336] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 111784, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.050352] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 111196, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.050366] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 109854, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.050381] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 109380, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.050394] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 108904, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.050407] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 108429, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.050420] BTRFS warning (device sda1): checksum error at logical 772859113472 on dev /dev/sdd1, physical 257976905728, root 107956, inode 257, offset 60333232128, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.050736] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 111784, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.050749] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 111196, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.050760] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 109854, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.050772] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 109380, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.050783] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 108904, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.050794] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 108429, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.050805] BTRFS warning (device sda1): checksum error at logical 772859117568 on dev /dev/sdd1, physical 257976909824, root 107956, inode 257, offset 60333236224, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.051123] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 111784, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.051134] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 111196, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.051147] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 109854, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.051158] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 109380, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.051170] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 108904, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.051181] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 108429, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.051192] BTRFS warning (device sda1): checksum error at logical 772859121664 on dev /dev/sdd1, physical 257976913920, root 107956, inode 257, offset 60333240320, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.051471] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 111784, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.051482] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 111196, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.051494] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 109854, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.051505] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 109380, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.051516] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 108904, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.051527] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 108429, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.051538] BTRFS warning (device sda1): checksum error at logical 772859125760 on dev /dev/sdd1, physical 257976918016, root 107956, inode 257, offset 60333244416, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.052017] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 108904, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.055478] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 111196, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.056067] BTRFS warning (device sda1): checksum error at logical 772868538368 on dev /dev/sdd1, physical 257980039168, root 111784, inode 257, offset 60342657024, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.056079] BTRFS warning (device sda1): checksum error at logical 772868538368 on dev /dev/sdd1, physical 257980039168, root 111196, inode 257, offset 60342657024, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.056089] BTRFS warning (device sda1): checksum error at logical 772868538368 on dev /dev/sdd1, physical 257980039168, root 109854, inode 257, offset 60342657024, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.056104] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 111784, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34038.056107] BTRFS warning (device sda1): checksum error at logical 772868538368 on dev /dev/sdd1, physical 257980039168, root 108904, inode 257, offset 60342657024, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.056120] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 111196, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34038.056122] BTRFS warning (device sda1): checksum error at logical 772868538368 on dev /dev/sdd1, physical 257980039168, root 108429, inode 257, offset 60342657024, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.056132] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 109854, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.056135] BTRFS warning (device sda1): checksum error at logical 772868538368 on dev /dev/sdd1, physical 257980039168, root 107956, inode 257, offset 60342657024, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.056145] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 109380, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.056157] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 108904, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.056170] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 108429, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.056182] BTRFS warning (device sda1): checksum error at logical 772866965504 on dev /dev/sdd1, physical 257979514880, root 107956, inode 257, offset 60341084160, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.058678] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 108429, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.061970] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 109854, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34038.065155] BTRFS warning (device sda1): checksum error at logical 772849680384 on dev /dev/sdd1, physical 257973764096, root 107956, inode 257, offset 60330315776, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34038.068416] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 109380, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34038.075769] BTRFS warning (device sda1): checksum error at logical 772854386688 on dev /dev/sdd1, physical 257975324672, root 108267, inode 257, offset 27183685632, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34038.081453] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 108904, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34038.086436] BTRFS warning (device sda1): checksum error at logical 772851249152 on dev /dev/sdd1, physical 257974284288, root 108267, inode 257, offset 27159961600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34038.086676] BTRFS warning (device sda1): checksum error at logical 772851249152 on dev /dev/sdd1, physical 257974284288, root 107792, inode 257, offset 27159961600, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34038.099944] BTRFS warning (device sda1): checksum error at logical 772854386688 on dev /dev/sdd1, physical 257975324672, root 107792, inode 257, offset 27183685632, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34038.102384] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 108429, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34038.102401] BTRFS warning (device sda1): checksum error at logical 772857540608 on dev /dev/sdd1, physical 257976381440, root 107956, inode 257, offset 60331659264, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34040.521377] btrfs_dev_stat_print_on_error: 24968 callbacks suppressed
[34040.521384] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074508, gen 184286
[34040.521480] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074509, gen 184286
[34040.521588] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074510, gen 184286
[34040.521726] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074511, gen 184286
[34040.521827] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074512, gen 184286
[34040.521952] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074513, gen 184286
[34040.522069] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074514, gen 184286
[34040.522180] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074515, gen 184286
[34040.522294] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
[34040.522405] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
[34041.069317] scrub_handle_errored_block: 26558 callbacks suppressed
[34041.069321] BTRFS error (device sda1): unable to fixup (regular) error at logical 773443637248 on dev /dev/sdd1
[34041.069907] BTRFS error (device sda1): unable to fixup (regular) error at logical 773445189632 on dev /dev/sdd1
[34041.070207] BTRFS error (device sda1): unable to fixup (regular) error at logical 773436346368 on dev /dev/sdd1
[34041.070230] BTRFS error (device sda1): unable to fixup (regular) error at logical 773446762496 on dev /dev/sdd1
[34041.070244] BTRFS error (device sda1): unable to fixup (regular) error at logical 773437919232 on dev /dev/sdd1
[34041.070305] BTRFS error (device sda1): unable to fixup (regular) error at logical 773446766592 on dev /dev/sdd1
[34041.070350] BTRFS error (device sda1): unable to fixup (regular) error at logical 773446770688 on dev /dev/sdd1
[34041.070465] BTRFS error (device sda1): unable to fixup (regular) error at logical 773439492096 on dev /dev/sdd1
[34041.070542] BTRFS error (device sda1): unable to fixup (regular) error at logical 773443641344 on dev /dev/sdd1
[34041.070579] BTRFS error (device sda1): unable to fixup (regular) error at logical 773443645440 on dev /dev/sdd1
[34043.021368] scrub_handle_errored_block: 24817 callbacks suppressed
[34043.062028] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 111763, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.062034] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 111763, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.062040] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 111763, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.062049] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 111763, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.062173] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 111066, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.062176] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 111066, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.062188] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 109833, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.062190] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 109833, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.062203] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 109359, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.062205] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 109359, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.062218] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 108884, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.062220] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 108884, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.062232] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 108409, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.062235] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 108409, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.062248] BTRFS warning (device sda1): checksum error at logical 773672271872 on dev /dev/sdd1, physical 258247950336, root 107934, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.062250] BTRFS warning (device sda1): checksum error at logical 773675417600 on dev /dev/sdd1, physical 258248998912, root 107934, inode 257, offset 142867038208, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.062479] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 111763, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.062547] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 111066, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.062559] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 109833, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.062570] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 109359, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.062580] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 108884, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.062591] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 108409, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.062602] BTRFS warning (device sda1): checksum error at logical 773672275968 on dev /dev/sdd1, physical 258247954432, root 107934, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.062839] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 111763, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.062903] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 111066, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.062913] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 109833, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.062924] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 109359, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.062935] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 108884, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.062946] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 108409, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.062964] BTRFS warning (device sda1): checksum error at logical 773672280064 on dev /dev/sdd1, physical 258247958528, root 107934, inode 257, offset 142862188544, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.063114] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 111763, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.063164] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 111763, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.063176] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 111066, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.063187] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 109833, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.063193] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 111066, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.063198] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 109359, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.063202] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 111763, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.063204] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 109833, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.063209] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 108884, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.063217] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 111066, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.063219] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 109359, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.063222] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 108409, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.063229] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 109833, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.063232] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 108884, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.063234] BTRFS warning (device sda1): checksum error at logical 773678563328 on dev /dev/sdd1, physical 258250047488, root 107934, inode 257, offset 142871101440, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.063242] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 109359, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.063245] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 108409, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.063255] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 108884, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.063260] BTRFS warning (device sda1): checksum error at logical 773676990464 on dev /dev/sdd1, physical 258249523200, root 107934, inode 257, offset 142869135360, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.063268] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 108409, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.063281] BTRFS warning (device sda1): checksum error at logical 773680136192 on dev /dev/sdd1, physical 258250571776, root 107934, inode 257, offset 142873067520, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.065220] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 111763, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645709034)
[34043.065266] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 111066, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.065275] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 109833, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.065326] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 109359, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.065404] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 111066, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.065418] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 109833, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.065431] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 109359, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.065443] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 108884, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.065455] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 108409, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.065467] BTRFS warning (device sda1): checksum error at logical 773673844736 on dev /dev/sdd1, physical 258248474624, root 107934, inode 257, offset 142864285696, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.086560] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 111066, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645602506)
[34043.108399] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 108884, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34043.133361] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 109833, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645535785)
[34043.157268] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 108409, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34043.180613] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 109359, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645426816)
[34043.197278] BTRFS warning (device sda1): checksum error at logical 773653397504 on dev /dev/sdd1, physical 258241658880, root 107934, inode 257, offset 142832697344, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34043.213259] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 108884, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645371920)
[34044.820958] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 108409, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645264083)
[34044.850240] BTRFS warning (device sda1): checksum error at logical 773670723584 on dev /dev/sdd1, physical 258247450624, root 107934, inode 257, offset 142860222464, length 4096, links 1 (path: cephstor8_vm-570-disk-0-diff-base.img_1645186729)
[34045.533269] btrfs_dev_stat_print_on_error: 22978 callbacks suppressed
[34045.533274] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097506, gen 184286
[34045.533977] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097507, gen 184286
[34045.545219] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097508, gen 184286
[34045.546114] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097509, gen 184286
[34045.547645] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097510, gen 184286
[34045.549990] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097511, gen 184286
[34045.552363] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097512, gen 184286
[34045.554587] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097513, gen 184286
[34045.556880] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097514, gen 184286
[34045.559071] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69097515, gen 184286
[34046.076741] scrub_handle_errored_block: 22936 callbacks suppressed
[34046.076746] BTRFS error (device sda1): unable to fixup (regular) error at logical 774153617408 on dev /dev/sdd1
[34046.078824] BTRFS error (device sda1): unable to fixup (regular) error at logical 774188183552 on dev /dev/sdd1
[34046.087175] BTRFS error (device sda1): unable to fixup (regular) error at logical 774188187648 on dev /dev/sdd1
[34046.088766] BTRFS error (device sda1): unable to fixup (regular) error at logical 774194462720 on dev /dev/sdd1
[34046.089163] BTRFS error (device sda1): unable to fixup (regular) error at logical 774195445760 on dev /dev/sdd1
[34046.089706] BTRFS error (device sda1): unable to fixup (regular) error at logical 774195449856 on dev /dev/sdd1
[34046.093007] BTRFS error (device sda1): unable to fixup (regular) error at logical 774188191744 on dev /dev/sdd1
[34046.093253] BTRFS error (device sda1): unable to fixup (regular) error at logical 774188195840 on dev /dev/sdd1
[34046.093444] BTRFS error (device sda1): unable to fixup (regular) error at logical 774188199936 on dev /dev/sdd1
[34046.093588] BTRFS error (device sda1): unable to fixup (regular) error at logical 774188204032 on dev /dev/sdd1
[34048.053131] scrub_handle_errored_block: 21305 callbacks suppressed
[34048.083834] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 111784, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34048.087027] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 111196, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.090108] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 109854, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.090217] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 111784, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34048.093196] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 109380, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.096369] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 111196, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.099533] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 108904, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.099909] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 111784, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34048.099923] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 111196, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.099936] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 109854, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.099948] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 111784, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34048.099952] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 109380, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.099960] BTRFS warning (device sda1): checksum error at logical 774367485952 on dev /dev/sdd1, physical 258479693824, root 111196, inode 257, offset 60545564672, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.099964] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 111196, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.099967] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 108904, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.099979] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 109854, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.099983] BTRFS warning (device sda1): checksum error at logical 774367485952 on dev /dev/sdd1, physical 258479693824, root 109854, inode 257, offset 60545564672, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.099986] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 108429, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.099996] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 109380, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.099999] BTRFS warning (device sda1): checksum error at logical 774367485952 on dev /dev/sdd1, physical 258479693824, root 109380, inode 257, offset 60545564672, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.100003] BTRFS warning (device sda1): checksum error at logical 774369054720 on dev /dev/sdd1, physical 258480214016, root 107956, inode 257, offset 60601270272, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.100014] BTRFS warning (device sda1): checksum error at logical 774367485952 on dev /dev/sdd1, physical 258479693824, root 108904, inode 257, offset 60545564672, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.100016] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 108904, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.100030] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 108429, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.100033] BTRFS warning (device sda1): checksum error at logical 774367485952 on dev /dev/sdd1, physical 258479693824, root 108429, inode 257, offset 60545564672, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.100047] BTRFS warning (device sda1): checksum error at logical 774364340224 on dev /dev/sdd1, physical 258478645248, root 107956, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.100049] BTRFS warning (device sda1): checksum error at logical 774367485952 on dev /dev/sdd1, physical 258479693824, root 107956, inode 257, offset 60545564672, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.102673] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 109854, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.105833] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 108429, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.105848] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 111784, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34048.105861] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 111196, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.105874] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 109854, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.105886] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 109380, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.105898] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 108904, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.105909] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 108429, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.105921] BTRFS warning (device sda1): checksum error at logical 774369058816 on dev /dev/sdd1, physical 258480218112, root 107956, inode 257, offset 60601401344, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.108990] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 109380, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.111546] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 111784, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645711022)
[34048.111559] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 111196, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645604752)
[34048.111571] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 109854, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645538257)
[34048.111583] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 109380, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645428821)
[34048.111595] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 108904, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.111608] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 108429, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.111619] BTRFS warning (device sda1): checksum error at logical 774364344320 on dev /dev/sdd1, physical 258478649344, root 107956, inode 257, offset 60541370368, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.112148] BTRFS warning (device sda1): checksum error at logical 774372200448 on dev /dev/sdd1, physical 258481262592, root 107956, inode 257, offset 60638232576, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.115372] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 108904, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645373735)
[34048.179610] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 111786, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645711048)
[34048.180956] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 111786, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645711048)
[34048.181687] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 108429, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645265252)
[34048.186928] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 111786, inode 257, offset 46408974336, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645711048)
[34048.188409] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 111786, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645711048)
[34048.188412] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 111198, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645604821)
[34048.188415] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 111786, inode 257, offset 46385266688, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645711048)
[34048.188421] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 111198, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645604821)
[34048.188426] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 111198, inode 257, offset 46385266688, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645604821)
[34048.189249] BTRFS warning (device sda1): checksum error at logical 774370627584 on dev /dev/sdd1, physical 258480738304, root 107956, inode 257, offset 60618047488, length 4096, links 1 (path: cephstor8_vm-607-disk-0-diff-base.img_1645188425)
[34048.194351] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 111786, inode 257, offset 46409007104, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645711048)
[34048.203934] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 109856, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645538268)
[34048.203936] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 109856, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645538268)
[34048.203966] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 109381, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645429218)
[34048.206930] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 111198, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645604821)
[34048.209187] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 109856, inode 257, offset 46385266688, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645538268)
[34048.214266] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 111198, inode 257, offset 46408974336, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645604821)
[34048.215832] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 108907, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645374632)
[34048.215861] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 108431, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645265349)
[34048.216546] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 109381, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645429218)
[34048.221721] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 111198, inode 257, offset 46409007104, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645604821)
[34048.223993] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 108907, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645374632)
[34048.229091] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 109856, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645538268)
[34048.229648] BTRFS warning (device sda1): checksum error at logical 774376914944 on dev /dev/sdd1, physical 258482831360, root 107957, inode 257, offset 46379110400, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645188615)
[34048.231293] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 108431, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645265349)
[34048.236296] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 109856, inode 257, offset 46409007104, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645538268)
[34048.238525] BTRFS warning (device sda1): checksum error at logical 774378487808 on dev /dev/sdd1, physical 258483355648, root 107957, inode 257, offset 46385201152, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645188615)
[34048.243641] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 109381, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645429218)
[34048.323178] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 109381, inode 257, offset 46409007104, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645429218)
[34048.325529] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 108907, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645374632)
[34048.327823] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 108431, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645265349)
[34048.330085] BTRFS warning (device sda1): checksum error at logical 774380060672 on dev /dev/sdd1, physical 258483879936, root 107957, inode 257, offset 46408925184, length 4096, links 1 (path: cephstor8_vm-609-disk-0-diff-base.img_1645188615)
[34050.539015] btrfs_dev_stat_print_on_error: 19689 callbacks suppressed
[34050.539021] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117217, gen 184286
[34050.546162] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117218, gen 184286
[34050.548622] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117219, gen 184286
[34050.548732] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117220, gen 184286
[34050.548882] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117221, gen 184286
[34050.548994] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117222, gen 184286
[34050.549033] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117223, gen 184286
[34050.551188] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117224, gen 184286
[34050.552498] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117225, gen 184286
[34050.557028] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69117226, gen 184286
[34051.091718] scrub_handle_errored_block: 18980 callbacks suppressed
[34051.091722] BTRFS error (device sda1): unable to fixup (regular) error at logical 774803169280 on dev /dev/sdd1
[34051.091799] BTRFS error (device sda1): unable to fixup (regular) error at logical 774796881920 on dev /dev/sdd1
[34051.092029] BTRFS error (device sda1): unable to fixup (regular) error at logical 774806315008 on dev /dev/sdd1
[34051.092121] BTRFS error (device sda1): unable to fixup (regular) error at logical 774798454784 on dev /dev/sdd1
[34051.092213] BTRFS error (device sda1): unable to fixup (regular) error at logical 774800027648 on dev /dev/sdd1
[34051.092302] BTRFS error (device sda1): unable to fixup (regular) error at logical 774806319104 on dev /dev/sdd1
[34051.092427] BTRFS error (device sda1): unable to fixup (regular) error at logical 774798458880 on dev /dev/sdd1
[34051.092534] BTRFS error (device sda1): unable to fixup (regular) error at logical 774800031744 on dev /dev/sdd1
[34051.092640] BTRFS error (device sda1): unable to fixup (regular) error at logical 774806323200 on dev /dev/sdd1
[34051.092753] BTRFS error (device sda1): unable to fixup (regular) error at logical 774798462976 on dev /dev/sdd1
[34053.057331] scrub_handle_errored_block: 20145 callbacks suppressed
[34053.107550] BTRFS warning (device sda1): checksum error at logical 775032803328 on dev /dev/sdd1, physical 258701463552, root 111660, inode 257, offset 59300290560, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107559] BTRFS warning (device sda1): checksum error at logical 775043219456 on dev /dev/sdd1, physical 258704932864, root 111660, inode 257, offset 59324145664, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107589] BTRFS warning (device sda1): checksum error at logical 775035944960 on dev /dev/sdd1, physical 258702508032, root 111660, inode 257, offset 59307499520, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107608] BTRFS warning (device sda1): checksum error at logical 775038500864 on dev /dev/sdd1, physical 258703360000, root 111660, inode 257, offset 59313397760, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107632] BTRFS warning (device sda1): checksum error at logical 775034376192 on dev /dev/sdd1, physical 258701987840, root 111660, inode 257, offset 59303960576, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107659] BTRFS warning (device sda1): checksum error at logical 775042236416 on dev /dev/sdd1, physical 258704605184, root 111660, inode 257, offset 59321917440, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107664] BTRFS warning (device sda1): checksum error at logical 775043223552 on dev /dev/sdd1, physical 258704936960, root 111660, inode 257, offset 59324145664, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107720] BTRFS warning (device sda1): checksum error at logical 775038504960 on dev /dev/sdd1, physical 258703364096, root 111660, inode 257, offset 59313397760, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34053.107778] BTRFS warning (device sda1): checksum error at logical 775043227648 on dev /dev/sdd1, physical 258704941056, root 111660, inode 257, offset 59324145664, length 4096, links 1 (path: cephstor8_vm-382-disk-0-diff-base.img_1645695777)
[34055.562139] btrfs_dev_stat_print_on_error: 19560 callbacks suppressed
[34055.562145] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136794, gen 184286
[34055.572860] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136795, gen 184286
[34055.581771] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136796, gen 184286
[34055.582209] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136797, gen 184286
[34055.582277] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136798, gen 184286
[34055.582549] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136799, gen 184286
[34055.583324] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136800, gen 184286
[34055.596684] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136801, gen 184286
[34055.598882] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136802, gen 184286
[34055.602154] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69136803, gen 184286
[34056.094227] scrub_handle_errored_block: 19077 callbacks suppressed
[34056.094232] BTRFS error (device sda1): unable to fixup (regular) error at logical 775422873600 on dev /dev/sdd1
[34056.106277] BTRFS error (device sda1): unable to fixup (regular) error at logical 775426015232 on dev /dev/sdd1
[34056.114175] BTRFS error (device sda1): unable to fixup (regular) error at logical 775422877696 on dev /dev/sdd1
[34056.114790] BTRFS error (device sda1): unable to fixup (regular) error at logical 775426019328 on dev /dev/sdd1
[34056.125923] BTRFS error (device sda1): unable to fixup (regular) error at logical 775426023424 on dev /dev/sdd1
[34056.127561] BTRFS error (device sda1): unable to fixup (regular) error at logical 775426027520 on dev /dev/sdd1
[34056.128071] BTRFS error (device sda1): unable to fixup (regular) error at logical 775422881792 on dev /dev/sdd1
[34056.136414] BTRFS error (device sda1): unable to fixup (regular) error at logical 775426031616 on dev /dev/sdd1
[34056.139774] BTRFS error (device sda1): unable to fixup (regular) error at logical 775442726912 on dev /dev/sdd1
[34056.140264] BTRFS error (device sda1): unable to fixup (regular) error at logical 775433879552 on dev /dev/sdd1
[34058.068950] scrub_handle_errored_block: 21766 callbacks suppressed
[34058.166274] BTRFS warning (device sda1): checksum error at logical 775750029312 on dev /dev/sdd1, physical 258940538880, root 111667, inode 257, offset 11053379584, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166282] BTRFS warning (device sda1): checksum error at logical 775743143936 on dev /dev/sdd1, physical 258938241024, root 111667, inode 257, offset 11038175232, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166292] BTRFS warning (device sda1): checksum error at logical 775753175040 on dev /dev/sdd1, physical 258941587456, root 111667, inode 257, offset 11060326400, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166295] BTRFS warning (device sda1): checksum error at logical 775759462400 on dev /dev/sdd1, physical 258943680512, root 111667, inode 257, offset 11289980928, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166301] BTRFS warning (device sda1): checksum error at logical 775762018304 on dev /dev/sdd1, physical 258944532480, root 111667, inode 257, offset 11295879168, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166306] BTRFS warning (device sda1): checksum error at logical 775743741952 on dev /dev/sdd1, physical 258938445824, root 111667, inode 257, offset 11039354880, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166401] BTRFS warning (device sda1): checksum error at logical 775743148032 on dev /dev/sdd1, physical 258938245120, root 111667, inode 257, offset 11038175232, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.166436] BTRFS warning (device sda1): checksum error at logical 775762022400 on dev /dev/sdd1, physical 258944536576, root 111667, inode 257, offset 11295879168, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
[34058.169293] BTRFS warning (device sda1): checksum error at logical 775742160896 on dev /dev/sdd1, physical 258937913344, root 111668, inode 257, offset 40809275392, length 4096, links 1 (path: cephstor8_vm-396-disk-0-diff-base.img_1645697379)
[34060.571689] btrfs_dev_stat_print_on_error: 18933 callbacks suppressed
[34060.571695] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155747, gen 184286
[34060.575050] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155748, gen 184286
[34060.576791] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155749, gen 184286
[34060.577474] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155750, gen 184286
[34060.577635] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155751, gen 184286
[34060.577705] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155752, gen 184286
[34060.577825] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155753, gen 184286
[34060.577970] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155754, gen 184286
[34060.577981] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155755, gen 184286
[34060.577985] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69155756, gen 184286
[34061.102560] scrub_handle_errored_block: 19676 callbacks suppressed
[34061.102566] BTRFS error (device sda1): unable to fixup (regular) error at logical 776074096640 on dev /dev/sdd1
[34061.104652] BTRFS error (device sda1): unable to fixup (regular) error at logical 776085086208 on dev /dev/sdd1
[34061.139300] BTRFS error (device sda1): unable to fixup (regular) error at logical 776085090304 on dev /dev/sdd1
[34061.141290] BTRFS error (device sda1): unable to fixup (regular) error at logical 776089763840 on dev /dev/sdd1
[34061.143355] BTRFS error (device sda1): unable to fixup (regular) error at logical 776091336704 on dev /dev/sdd1
[34061.150932] BTRFS error (device sda1): unable to fixup (regular) error at logical 776085094400 on dev /dev/sdd1
[34061.161860] BTRFS error (device sda1): unable to fixup (regular) error at logical 776096055296 on dev /dev/sdd1
[34061.171491] BTRFS error (device sda1): unable to fixup (regular) error at logical 776097628160 on dev /dev/sdd1
[34061.180211] BTRFS error (device sda1): unable to fixup (regular) error at logical 776085098496 on dev /dev/sdd1
[34061.189325] BTRFS error (device sda1): unable to fixup (regular) error at logical 776098611200 on dev /dev/sdd1
[34063.072646] scrub_handle_errored_block: 15913 callbacks suppressed
[34063.081854] BTRFS warning (device sda1): checksum error at logical 776275390464 on dev /dev/sdd1, physical 259115675648, root 108267, inode 257, offset 27935342592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.081860] BTRFS warning (device sda1): checksum error at logical 776276578304 on dev /dev/sdd1, physical 259116077056, root 108267, inode 257, offset 27936550912, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.081867] BTRFS warning (device sda1): checksum error at logical 776273805312 on dev /dev/sdd1, physical 259115139072, root 107792, inode 257, offset 27933757440, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.081873] BTRFS warning (device sda1): checksum error at logical 776276578304 on dev /dev/sdd1, physical 259116077056, root 107792, inode 257, offset 27936550912, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.081992] BTRFS warning (device sda1): checksum error at logical 776276582400 on dev /dev/sdd1, physical 259116081152, root 108267, inode 257, offset 27936555008, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.082003] BTRFS warning (device sda1): checksum error at logical 776276582400 on dev /dev/sdd1, physical 259116081152, root 107792, inode 257, offset 27936555008, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.082081] BTRFS warning (device sda1): checksum error at logical 776276586496 on dev /dev/sdd1, physical 259116085248, root 108267, inode 257, offset 27936559104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.082091] BTRFS warning (device sda1): checksum error at logical 776276586496 on dev /dev/sdd1, physical 259116085248, root 107792, inode 257, offset 27936559104, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.082166] BTRFS warning (device sda1): checksum error at logical 776276590592 on dev /dev/sdd1, physical 259116089344, root 108267, inode 257, offset 27936563200, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.082176] BTRFS warning (device sda1): checksum error at logical 776276590592 on dev /dev/sdd1, physical 259116089344, root 107792, inode 257, offset 27936563200, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.082249] BTRFS warning (device sda1): checksum error at logical 776276594688 on dev /dev/sdd1, physical 259116093440, root 108267, inode 257, offset 27936567296, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.082261] BTRFS warning (device sda1): checksum error at logical 776276594688 on dev /dev/sdd1, physical 259116093440, root 107792, inode 257, offset 27936567296, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.082331] BTRFS warning (device sda1): checksum error at logical 776276598784 on dev /dev/sdd1, physical 259116097536, root 108267, inode 257, offset 27936571392, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.082341] BTRFS warning (device sda1): checksum error at logical 776276598784 on dev /dev/sdd1, physical 259116097536, root 107792, inode 257, offset 27936571392, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.082412] BTRFS warning (device sda1): checksum error at logical 776276602880 on dev /dev/sdd1, physical 259116101632, root 108267, inode 257, offset 27936575488, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645245078)
[34063.082423] BTRFS warning (device sda1): checksum error at logical 776276602880 on dev /dev/sdd1, physical 259116101632, root 107792, inode 257, offset 27936575488, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.084848] BTRFS warning (device sda1): checksum error at logical 776275390464 on dev /dev/sdd1, physical 259115675648, root 107792, inode 257, offset 27935342592, length 4096, links 1 (path: cephstor8_vm-288-disk-0-diff-base.img_1645166992)
[34063.113583] BTRFS warning (device sda1): checksum error at logical 776292077568 on dev /dev/sdd1, physical 259121221632, root 112142, inode 257, offset 35163701248, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645767339)
[34063.144845] BTRFS warning (device sda1): checksum error at logical 776292077568 on dev /dev/sdd1, physical 259121221632, root 111667, inode 257, offset 35163701248, length 4096, links 1 (path: cephstor8_vm-391-disk-0-diff-base.img_1645697379)
^ permalink raw reply [flat|nested] 11+ messages in thread
* AW: How to (attempt to) repair these btrfs errors
2022-02-28 18:41 How to (attempt to) repair these btrfs errors Carsten Grommel
@ 2022-03-01 10:55 ` Carsten Grommel
2022-03-06 1:36 ` Zygo Blaxell
0 siblings, 1 reply; 11+ messages in thread
From: Carsten Grommel @ 2022-03-01 10:55 UTC (permalink / raw)
To: linux-btrfs
Follow-up pastebin with the most recent errors in dmesg:
https://pastebin.com/4yJJdQPJ
________________________________________
Von: Carsten Grommel
Gesendet: Montag, 28. Februar 2022 19:41
An: linux-btrfs@vger.kernel.org
Betreff: How to (attempt to) repair these btrfs errors
Hi,
short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
But it was Kernel 5.4.121 before
btrfs --version
btrfs-progs v4.20.1
btrfs fi show
Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
Total devices 3 FS bytes used 56.74TiB
devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
btrfs fi df /vmbackup/
Data, RAID0: total=66.62TiB, used=56.45TiB
System, RAID1: total=8.00MiB, used=4.36MiB
Metadata, RAID1: total=750.00GiB, used=294.90GiB
GlobalReserve, single: total=512.00MiB, used=0.00B
Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
[Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
[Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
[Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
[Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
[Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
I am currently running a scrub over the filesystem.
The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
Kind regards
Carsten Grommel
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: How to (attempt to) repair these btrfs errors
2022-03-01 10:55 ` AW: " Carsten Grommel
@ 2022-03-06 1:36 ` Zygo Blaxell
2022-03-07 7:03 ` AW: " Carsten Grommel
0 siblings, 1 reply; 11+ messages in thread
From: Zygo Blaxell @ 2022-03-06 1:36 UTC (permalink / raw)
To: Carsten Grommel; +Cc: linux-btrfs
On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
> Follow-up pastebin with the most recent errors in dmesg:
>
> https://pastebin.com/4yJJdQPJ
This seems to have expired.
> ________________________________________
> Von: Carsten Grommel
> Gesendet: Montag, 28. Februar 2022 19:41
> An: linux-btrfs@vger.kernel.org
> Betreff: How to (attempt to) repair these btrfs errors
>
> Hi,
>
> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>
> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>
> But it was Kernel 5.4.121 before
>
> btrfs --version
> btrfs-progs v4.20.1
>
> btrfs fi show
> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
> Total devices 3 FS bytes used 56.74TiB
> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>
> btrfs fi df /vmbackup/
> Data, RAID0: total=66.62TiB, used=56.45TiB
> System, RAID1: total=8.00MiB, used=4.36MiB
> Metadata, RAID1: total=750.00GiB, used=294.90GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>
> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>
> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>
> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>
> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>
> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>
> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
> I am currently running a scrub over the filesystem.
>
> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>
> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
The corruption and generation event counts indicate sdd1 (or one of its
component devices) was offline for a long time or suffered corruption
on a large scale.
Data is raid0, so data repair is not possible. Delete all the files
that contain corrupt data.
If you are using space_cache=v1, now is a good time to upgrade to
space_cache=v2. v1 space cache is stored in the data profile, and it has
likely been corrupted. btrfs will usually detect and repair corruption
in space_cache=v1, but there is no need to take any such risk here
when you can easily use v2 instead (or at least clear the v1 cache).
I don't see any errors in these logs that would indicate a metadata issue,
but huge numbers of messages are suppressed. Perhaps a log closer
to the moment when the filesystem goes read-only will be more useful.
I would expect that if there are no problems on sda1 or sdb1 then it
should be possible to repair the metadata errors on sdd1 by scrubbing
that device.
> Kind regards
> Carsten Grommel
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* AW: How to (attempt to) repair these btrfs errors
2022-03-06 1:36 ` Zygo Blaxell
@ 2022-03-07 7:03 ` Carsten Grommel
2022-03-07 7:11 ` Qu Wenruo
0 siblings, 1 reply; 11+ messages in thread
From: Carsten Grommel @ 2022-03-07 7:03 UTC (permalink / raw)
To: Zygo Blaxell; +Cc: linux-btrfs
Thank you for the answer. We are using space_cache v2:
/dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>Data is raid0, so data repair is not possible. Delete all the files
>that contain corrupt data.
I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
>I don't see any errors in these logs that would indicate a metadata issue,
>but huge numbers of messages are suppressed. Perhaps a log closer
>to the moment when the filesystem goes read-only will be more useful.
>I would expect that if there are no problems on sda1 or sdb1 then it
>should be possible to repair the metadata errors on sdd1 by scrubbing
that device.
I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
I did not yet try to scrub specifically on sdd though, gonna try that.
Should it remount again i will provide the most recent dmesg's right before it crashes.
________________________________________
Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
Gesendet: Sonntag, 6. März 2022 02:36
An: Carsten Grommel
Cc: linux-btrfs@vger.kernel.org
Betreff: Re: How to (attempt to) repair these btrfs errors
On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
> Follow-up pastebin with the most recent errors in dmesg:
>
> https://pastebin.com/4yJJdQPJ
This seems to have expired.
> ________________________________________
> Von: Carsten Grommel
> Gesendet: Montag, 28. Februar 2022 19:41
> An: linux-btrfs@vger.kernel.org
> Betreff: How to (attempt to) repair these btrfs errors
>
> Hi,
>
> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>
> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>
> But it was Kernel 5.4.121 before
>
> btrfs --version
> btrfs-progs v4.20.1
>
> btrfs fi show
> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
> Total devices 3 FS bytes used 56.74TiB
> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>
> btrfs fi df /vmbackup/
> Data, RAID0: total=66.62TiB, used=56.45TiB
> System, RAID1: total=8.00MiB, used=4.36MiB
> Metadata, RAID1: total=750.00GiB, used=294.90GiB
> GlobalReserve, single: total=512.00MiB, used=0.00B
>
> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>
> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>
> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>
> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>
> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>
> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>
> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
> I am currently running a scrub over the filesystem.
>
> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>
> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
The corruption and generation event counts indicate sdd1 (or one of its
component devices) was offline for a long time or suffered corruption
on a large scale.
Data is raid0, so data repair is not possible. Delete all the files
that contain corrupt data.
If you are using space_cache=v1, now is a good time to upgrade to
space_cache=v2. v1 space cache is stored in the data profile, and it has
likely been corrupted. btrfs will usually detect and repair corruption
in space_cache=v1, but there is no need to take any such risk here
when you can easily use v2 instead (or at least clear the v1 cache).
I don't see any errors in these logs that would indicate a metadata issue,
but huge numbers of messages are suppressed. Perhaps a log closer
to the moment when the filesystem goes read-only will be more useful.
I would expect that if there are no problems on sda1 or sdb1 then it
should be possible to repair the metadata errors on sdd1 by scrubbing
that device.
> Kind regards
> Carsten Grommel
>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:03 ` AW: " Carsten Grommel
@ 2022-03-07 7:11 ` Qu Wenruo
2022-03-07 7:25 ` AW: " Carsten Grommel
0 siblings, 1 reply; 11+ messages in thread
From: Qu Wenruo @ 2022-03-07 7:11 UTC (permalink / raw)
To: Carsten Grommel, Zygo Blaxell; +Cc: linux-btrfs
On 2022/3/7 15:03, Carsten Grommel wrote:
> Thank you for the answer. We are using space_cache v2:
>
> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>
>> Data is raid0, so data repair is not possible. Delete all the files
>> that contain corrupt data.
>
> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
Btrfs only falls back to RO for very critical errors (which could affect
on-disk metadata consistency).
Thus plain data corruption should not cause the RO.
Mind to share a dmesg just after the RO fallback?
Thanks,
Qu
>
>> I don't see any errors in these logs that would indicate a metadata issue,
>> but huge numbers of messages are suppressed. Perhaps a log closer
>> to the moment when the filesystem goes read-only will be more useful.
>
>> I would expect that if there are no problems on sda1 or sdb1 then it
>> should be possible to repair the metadata errors on sdd1 by scrubbing
> that device.
>
> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
> I did not yet try to scrub specifically on sdd though, gonna try that.
>
> Should it remount again i will provide the most recent dmesg's right before it crashes.
>
> ________________________________________
> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
> Gesendet: Sonntag, 6. März 2022 02:36
> An: Carsten Grommel
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: How to (attempt to) repair these btrfs errors
>
> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>> Follow-up pastebin with the most recent errors in dmesg:
>>
>> https://pastebin.com/4yJJdQPJ
>
> This seems to have expired.
>
>> ________________________________________
>> Von: Carsten Grommel
>> Gesendet: Montag, 28. Februar 2022 19:41
>> An: linux-btrfs@vger.kernel.org
>> Betreff: How to (attempt to) repair these btrfs errors
>>
>> Hi,
>>
>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>
>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>
>> But it was Kernel 5.4.121 before
>>
>> btrfs --version
>> btrfs-progs v4.20.1
>>
>> btrfs fi show
>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>> Total devices 3 FS bytes used 56.74TiB
>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>
>> btrfs fi df /vmbackup/
>> Data, RAID0: total=66.62TiB, used=56.45TiB
>> System, RAID1: total=8.00MiB, used=4.36MiB
>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>
>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>
>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>
>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>
>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>
>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>
>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>
>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>> I am currently running a scrub over the filesystem.
>>
>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>
>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>
> The corruption and generation event counts indicate sdd1 (or one of its
> component devices) was offline for a long time or suffered corruption
> on a large scale.
>
> Data is raid0, so data repair is not possible. Delete all the files
> that contain corrupt data.
>
> If you are using space_cache=v1, now is a good time to upgrade to
> space_cache=v2. v1 space cache is stored in the data profile, and it has
> likely been corrupted. btrfs will usually detect and repair corruption
> in space_cache=v1, but there is no need to take any such risk here
> when you can easily use v2 instead (or at least clear the v1 cache).
>
> I don't see any errors in these logs that would indicate a metadata issue,
> but huge numbers of messages are suppressed. Perhaps a log closer
> to the moment when the filesystem goes read-only will be more useful.
>
> I would expect that if there are no problems on sda1 or sdb1 then it
> should be possible to repair the metadata errors on sdd1 by scrubbing
> that device.
>
>> Kind regards
>> Carsten Grommel
>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* AW: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:11 ` Qu Wenruo
@ 2022-03-07 7:25 ` Carsten Grommel
2022-03-07 7:27 ` Carsten Grommel
2022-03-07 7:34 ` Qu Wenruo
0 siblings, 2 replies; 11+ messages in thread
From: Carsten Grommel @ 2022-03-07 7:25 UTC (permalink / raw)
To: Qu Wenruo, Zygo Blaxell; +Cc: linux-btrfs
Hi Qu,
>Mind to share a dmesg just after the RO fallback?
the most recent crash:
Mar 4 01:43:15 cloud8-1550 kernel: [45667.191649] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
Mar 4 01:43:15 cloud8-1550 kernel: [45667.194011] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
Mar 4 01:43:15 cloud8-1550 kernel: [45667.195395] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
Mar 4 01:43:15 cloud8-1550 kernel: [45667.196620] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
Mar 4 01:43:15 cloud8-1550 kernel: [45667.197920] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
Mar 4 01:43:15 cloud8-1550 kernel: [45667.212980] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500953600 (dev /dev/sde1 sector 10546500256)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.214413] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500957696 (dev /dev/sde1 sector 10546500264)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.215204] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
Mar 4 01:43:15 cloud8-1550 kernel: [45667.215656] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500961792 (dev /dev/sde1 sector 10546500272)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.230156] BTRFS: error (device sdc1) in btrfs_finish_ordered_io:2736: errno=-5 IO failure
Mar 4 01:43:15 cloud8-1550 kernel: [45667.233127] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500965888 (dev /dev/sde1 sector 10546500280)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.247096] BTRFS info (device sdc1): forced readonly
________________________________________
Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
Gesendet: Montag, 7. März 2022 08:11
An: Carsten Grommel; Zygo Blaxell
Cc: linux-btrfs@vger.kernel.org
Betreff: Re: AW: How to (attempt to) repair these btrfs errors
On 2022/3/7 15:03, Carsten Grommel wrote:
> Thank you for the answer. We are using space_cache v2:
>
> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>
>> Data is raid0, so data repair is not possible. Delete all the files
>> that contain corrupt data.
>
> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
Btrfs only falls back to RO for very critical errors (which could affect
on-disk metadata consistency).
Thus plain data corruption should not cause the RO.
Mind to share a dmesg just after the RO fallback?
Thanks,
Qu
>
>> I don't see any errors in these logs that would indicate a metadata issue,
>> but huge numbers of messages are suppressed. Perhaps a log closer
>> to the moment when the filesystem goes read-only will be more useful.
>
>> I would expect that if there are no problems on sda1 or sdb1 then it
>> should be possible to repair the metadata errors on sdd1 by scrubbing
> that device.
>
> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
> I did not yet try to scrub specifically on sdd though, gonna try that.
>
> Should it remount again i will provide the most recent dmesg's right before it crashes.
>
> ________________________________________
> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
> Gesendet: Sonntag, 6. März 2022 02:36
> An: Carsten Grommel
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: How to (attempt to) repair these btrfs errors
>
> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>> Follow-up pastebin with the most recent errors in dmesg:
>>
>> https://pastebin.com/4yJJdQPJ
>
> This seems to have expired.
>
>> ________________________________________
>> Von: Carsten Grommel
>> Gesendet: Montag, 28. Februar 2022 19:41
>> An: linux-btrfs@vger.kernel.org
>> Betreff: How to (attempt to) repair these btrfs errors
>>
>> Hi,
>>
>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>
>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>
>> But it was Kernel 5.4.121 before
>>
>> btrfs --version
>> btrfs-progs v4.20.1
>>
>> btrfs fi show
>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>> Total devices 3 FS bytes used 56.74TiB
>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>
>> btrfs fi df /vmbackup/
>> Data, RAID0: total=66.62TiB, used=56.45TiB
>> System, RAID1: total=8.00MiB, used=4.36MiB
>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>
>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>
>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>
>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>
>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>
>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>
>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>
>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>> I am currently running a scrub over the filesystem.
>>
>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>
>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>
> The corruption and generation event counts indicate sdd1 (or one of its
> component devices) was offline for a long time or suffered corruption
> on a large scale.
>
> Data is raid0, so data repair is not possible. Delete all the files
> that contain corrupt data.
>
> If you are using space_cache=v1, now is a good time to upgrade to
> space_cache=v2. v1 space cache is stored in the data profile, and it has
> likely been corrupted. btrfs will usually detect and repair corruption
> in space_cache=v1, but there is no need to take any such risk here
> when you can easily use v2 instead (or at least clear the v1 cache).
>
> I don't see any errors in these logs that would indicate a metadata issue,
> but huge numbers of messages are suppressed. Perhaps a log closer
> to the moment when the filesystem goes read-only will be more useful.
>
> I would expect that if there are no problems on sda1 or sdb1 then it
> should be possible to repair the metadata errors on sdd1 by scrubbing
> that device.
>
>> Kind regards
>> Carsten Grommel
>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* AW: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:25 ` AW: " Carsten Grommel
@ 2022-03-07 7:27 ` Carsten Grommel
2022-03-07 7:39 ` Qu Wenruo
2022-03-07 7:34 ` Qu Wenruo
1 sibling, 1 reply; 11+ messages in thread
From: Carsten Grommel @ 2022-03-07 7:27 UTC (permalink / raw)
To: Qu Wenruo, Zygo Blaxell; +Cc: linux-btrfs
These seems to be the most critical afaic:
Mar 4 01:25:51 cloud8-1550 kernel: [44623.523395] BTRFS critical (device sdc1): corrupt leaf: root=111550 block=849874468864 slot=0 ino=32633089 file_offset=7805042688, invalid compression for file extent, have 15 expect range [0, 3]
Mar 4 01:25:51 cloud8-1550 kernel: [44623.527109] BTRFS error (device sdc1): block=849874468864 read time tree block corruption detected
Mar 4 01:25:52 cloud8-1550 kernel: [44623.643308] BTRFS critical (device sdc1): corrupt leaf: root=50979 block=849880268800 slot=2, bad key order, prev (18446744073709551606 128 1269917216768) current (18446744073709551606 128 1269916291
072)
Mar 4 01:25:52 cloud8-1550 kernel: [44623.648078] BTRFS error (device sdc1): block=849880268800 read time tree block corruption detected
Mar 4 01:26:30 cloud8-1550 kernel: [44662.087943] BTRFS critical (device sdc1): corrupt leaf: root=7 block=83188697202688 slot=0, unexpected item end, have 16139 expect 16283
Mar 4 01:26:30 cloud8-1550 kernel: [44662.092173] BTRFS error (device sdc1): block=83188697202688 read time tree block corruption detected
Mar 4 01:26:30 cloud8-1550 kernel: [44662.094775] repair_io_failure: 170 callbacks suppressed
Mar 4 01:26:30 cloud8-1550 kernel: [44662.094779] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697202688 (dev /dev/sde1 sector 380092288)
Mar 4 01:26:30 cloud8-1550 kernel: [44662.144525] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697206784 (dev /dev/sde1 sector 380092296)
Mar 4 01:26:30 cloud8-1550 kernel: [44662.147691] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697210880 (dev /dev/sde1 sector 380092304)
Mar 4 01:26:30 cloud8-1550 kernel: [44662.149705] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697214976 (dev /dev/sde1 sector 380092312)
Mar 4 01:26:30 cloud8-1550 kernel: [44662.156236] BTRFS critical (device sdc1): corrupt leaf: root=96207 block=83188697219072 slot=0, unexpected item end, have 16395 expect 16283
________________________________________
Von: Carsten Grommel <c.grommel@profihost.ag>
Gesendet: Montag, 7. März 2022 08:25
An: Qu Wenruo; Zygo Blaxell
Cc: linux-btrfs@vger.kernel.org
Betreff: AW: AW: How to (attempt to) repair these btrfs errors
Hi Qu,
>Mind to share a dmesg just after the RO fallback?
the most recent crash:
Mar 4 01:43:15 cloud8-1550 kernel: [45667.191649] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
Mar 4 01:43:15 cloud8-1550 kernel: [45667.194011] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
Mar 4 01:43:15 cloud8-1550 kernel: [45667.195395] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
Mar 4 01:43:15 cloud8-1550 kernel: [45667.196620] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
Mar 4 01:43:15 cloud8-1550 kernel: [45667.197920] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
Mar 4 01:43:15 cloud8-1550 kernel: [45667.212980] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500953600 (dev /dev/sde1 sector 10546500256)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.214413] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500957696 (dev /dev/sde1 sector 10546500264)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.215204] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
Mar 4 01:43:15 cloud8-1550 kernel: [45667.215656] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500961792 (dev /dev/sde1 sector 10546500272)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.230156] BTRFS: error (device sdc1) in btrfs_finish_ordered_io:2736: errno=-5 IO failure
Mar 4 01:43:15 cloud8-1550 kernel: [45667.233127] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500965888 (dev /dev/sde1 sector 10546500280)
Mar 4 01:43:15 cloud8-1550 kernel: [45667.247096] BTRFS info (device sdc1): forced readonly
________________________________________
Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
Gesendet: Montag, 7. März 2022 08:11
An: Carsten Grommel; Zygo Blaxell
Cc: linux-btrfs@vger.kernel.org
Betreff: Re: AW: How to (attempt to) repair these btrfs errors
On 2022/3/7 15:03, Carsten Grommel wrote:
> Thank you for the answer. We are using space_cache v2:
>
> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>
>> Data is raid0, so data repair is not possible. Delete all the files
>> that contain corrupt data.
>
> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
Btrfs only falls back to RO for very critical errors (which could affect
on-disk metadata consistency).
Thus plain data corruption should not cause the RO.
Mind to share a dmesg just after the RO fallback?
Thanks,
Qu
>
>> I don't see any errors in these logs that would indicate a metadata issue,
>> but huge numbers of messages are suppressed. Perhaps a log closer
>> to the moment when the filesystem goes read-only will be more useful.
>
>> I would expect that if there are no problems on sda1 or sdb1 then it
>> should be possible to repair the metadata errors on sdd1 by scrubbing
> that device.
>
> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
> I did not yet try to scrub specifically on sdd though, gonna try that.
>
> Should it remount again i will provide the most recent dmesg's right before it crashes.
>
> ________________________________________
> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
> Gesendet: Sonntag, 6. März 2022 02:36
> An: Carsten Grommel
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: How to (attempt to) repair these btrfs errors
>
> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>> Follow-up pastebin with the most recent errors in dmesg:
>>
>> https://pastebin.com/4yJJdQPJ
>
> This seems to have expired.
>
>> ________________________________________
>> Von: Carsten Grommel
>> Gesendet: Montag, 28. Februar 2022 19:41
>> An: linux-btrfs@vger.kernel.org
>> Betreff: How to (attempt to) repair these btrfs errors
>>
>> Hi,
>>
>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>
>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>
>> But it was Kernel 5.4.121 before
>>
>> btrfs --version
>> btrfs-progs v4.20.1
>>
>> btrfs fi show
>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>> Total devices 3 FS bytes used 56.74TiB
>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>
>> btrfs fi df /vmbackup/
>> Data, RAID0: total=66.62TiB, used=56.45TiB
>> System, RAID1: total=8.00MiB, used=4.36MiB
>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>
>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>
>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>
>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>
>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>
>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>
>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>
>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>> I am currently running a scrub over the filesystem.
>>
>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>
>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>
> The corruption and generation event counts indicate sdd1 (or one of its
> component devices) was offline for a long time or suffered corruption
> on a large scale.
>
> Data is raid0, so data repair is not possible. Delete all the files
> that contain corrupt data.
>
> If you are using space_cache=v1, now is a good time to upgrade to
> space_cache=v2. v1 space cache is stored in the data profile, and it has
> likely been corrupted. btrfs will usually detect and repair corruption
> in space_cache=v1, but there is no need to take any such risk here
> when you can easily use v2 instead (or at least clear the v1 cache).
>
> I don't see any errors in these logs that would indicate a metadata issue,
> but huge numbers of messages are suppressed. Perhaps a log closer
> to the moment when the filesystem goes read-only will be more useful.
>
> I would expect that if there are no problems on sda1 or sdb1 then it
> should be possible to repair the metadata errors on sdd1 by scrubbing
> that device.
>
>> Kind regards
>> Carsten Grommel
>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: AW: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:25 ` AW: " Carsten Grommel
2022-03-07 7:27 ` Carsten Grommel
@ 2022-03-07 7:34 ` Qu Wenruo
2022-03-07 7:48 ` AW: " Carsten Grommel
1 sibling, 1 reply; 11+ messages in thread
From: Qu Wenruo @ 2022-03-07 7:34 UTC (permalink / raw)
To: Carsten Grommel, Zygo Blaxell; +Cc: linux-btrfs
On 2022/3/7 15:25, Carsten Grommel wrote:
> Hi Qu,
>
>> Mind to share a dmesg just after the RO fallback?
>
> the most recent crash:
>
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.191649] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.194011] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.195395] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.196620] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.197920] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.212980] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500953600 (dev /dev/sde1 sector 10546500256)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.214413] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500957696 (dev /dev/sde1 sector 10546500264)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215204] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215656] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500961792 (dev /dev/sde1 sector 10546500272)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.230156] BTRFS: error (device sdc1) in btrfs_finish_ordered_io:2736: errno=-5 IO failure
OK, this explains the reason.
Some tree blocks in subvolume trees are corrupted, and by the worst
possible way, metadata transid mismatch.
Although it looks like some metadata read can be repaired, but I guess
since btrfs_finish_ordered_io() still failed, it means some can not be
repaired.
Did the fs go through some split-brain cases? E.g. some devices got
degraded mount, then the missing device come back?
Thanks,
Qu
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.233127] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500965888 (dev /dev/sde1 sector 10546500280)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.247096] BTRFS info (device sdc1): forced readonly
>
> ________________________________________
> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
> Gesendet: Montag, 7. März 2022 08:11
> An: Carsten Grommel; Zygo Blaxell
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: AW: How to (attempt to) repair these btrfs errors
>
>
>
> On 2022/3/7 15:03, Carsten Grommel wrote:
>> Thank you for the answer. We are using space_cache v2:
>>
>> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>>
>>> Data is raid0, so data repair is not possible. Delete all the files
>>> that contain corrupt data.
>>
>> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
>
> Btrfs only falls back to RO for very critical errors (which could affect
> on-disk metadata consistency).
>
> Thus plain data corruption should not cause the RO.
>
> Mind to share a dmesg just after the RO fallback?
>
> Thanks,
> Qu
>
>>
>>> I don't see any errors in these logs that would indicate a metadata issue,
>>> but huge numbers of messages are suppressed. Perhaps a log closer
>>> to the moment when the filesystem goes read-only will be more useful.
>>
>>> I would expect that if there are no problems on sda1 or sdb1 then it
>>> should be possible to repair the metadata errors on sdd1 by scrubbing
>> that device.
>>
>> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
>> I did not yet try to scrub specifically on sdd though, gonna try that.
>>
>> Should it remount again i will provide the most recent dmesg's right before it crashes.
>>
>> ________________________________________
>> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
>> Gesendet: Sonntag, 6. März 2022 02:36
>> An: Carsten Grommel
>> Cc: linux-btrfs@vger.kernel.org
>> Betreff: Re: How to (attempt to) repair these btrfs errors
>>
>> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>>> Follow-up pastebin with the most recent errors in dmesg:
>>>
>>> https://pastebin.com/4yJJdQPJ
>>
>> This seems to have expired.
>>
>>> ________________________________________
>>> Von: Carsten Grommel
>>> Gesendet: Montag, 28. Februar 2022 19:41
>>> An: linux-btrfs@vger.kernel.org
>>> Betreff: How to (attempt to) repair these btrfs errors
>>>
>>> Hi,
>>>
>>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>>
>>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>>
>>> But it was Kernel 5.4.121 before
>>>
>>> btrfs --version
>>> btrfs-progs v4.20.1
>>>
>>> btrfs fi show
>>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>>> Total devices 3 FS bytes used 56.74TiB
>>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>>
>>> btrfs fi df /vmbackup/
>>> Data, RAID0: total=66.62TiB, used=56.45TiB
>>> System, RAID1: total=8.00MiB, used=4.36MiB
>>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>>
>>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>>
>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>>
>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>>
>>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>>
>>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>>
>>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>>
>>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>>> I am currently running a scrub over the filesystem.
>>>
>>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>>
>>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>>
>> The corruption and generation event counts indicate sdd1 (or one of its
>> component devices) was offline for a long time or suffered corruption
>> on a large scale.
>>
>> Data is raid0, so data repair is not possible. Delete all the files
>> that contain corrupt data.
>>
>> If you are using space_cache=v1, now is a good time to upgrade to
>> space_cache=v2. v1 space cache is stored in the data profile, and it has
>> likely been corrupted. btrfs will usually detect and repair corruption
>> in space_cache=v1, but there is no need to take any such risk here
>> when you can easily use v2 instead (or at least clear the v1 cache).
>>
>> I don't see any errors in these logs that would indicate a metadata issue,
>> but huge numbers of messages are suppressed. Perhaps a log closer
>> to the moment when the filesystem goes read-only will be more useful.
>>
>> I would expect that if there are no problems on sda1 or sdb1 then it
>> should be possible to repair the metadata errors on sdd1 by scrubbing
>> that device.
>>
>>> Kind regards
>>> Carsten Grommel
>>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: AW: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:27 ` Carsten Grommel
@ 2022-03-07 7:39 ` Qu Wenruo
0 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2022-03-07 7:39 UTC (permalink / raw)
To: Carsten Grommel, Zygo Blaxell; +Cc: linux-btrfs
On 2022/3/7 15:27, Carsten Grommel wrote:
> These seems to be the most critical afaic:
>
> Mar 4 01:25:51 cloud8-1550 kernel: [44623.523395] BTRFS critical (device sdc1): corrupt leaf: root=111550 block=849874468864 slot=0 ino=32633089 file_offset=7805042688, invalid compression for file extent, have 15 expect range [0, 3]
OK, this would explain the problem much better than the repairable
metadata read.
There should be no way we have compression type 0xf.
> Mar 4 01:25:51 cloud8-1550 kernel: [44623.527109] BTRFS error (device sdc1): block=849874468864 read time tree block corruption detected
> Mar 4 01:25:52 cloud8-1550 kernel: [44623.643308] BTRFS critical (device sdc1): corrupt leaf: root=50979 block=849880268800 slot=2, bad key order, prev (18446744073709551606 128 1269917216768) current (18446744073709551606 128 1269916291
> 072)
And bad tree key order, even more serious.
hex(1269917216768) = 0x127acf6f000
hex(1269916291072) = 0x127ace8d000
Doesn't look like a simple bitflip, nor does the preivous 0xf
compression type.
I have no idea how things can be so terribly wrong...
Thanks,
Qu
> Mar 4 01:25:52 cloud8-1550 kernel: [44623.648078] BTRFS error (device sdc1): block=849880268800 read time tree block corruption detected
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.087943] BTRFS critical (device sdc1): corrupt leaf: root=7 block=83188697202688 slot=0, unexpected item end, have 16139 expect 16283
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.092173] BTRFS error (device sdc1): block=83188697202688 read time tree block corruption detected
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.094775] repair_io_failure: 170 callbacks suppressed
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.094779] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697202688 (dev /dev/sde1 sector 380092288)
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.144525] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697206784 (dev /dev/sde1 sector 380092296)
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.147691] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697210880 (dev /dev/sde1 sector 380092304)
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.149705] BTRFS info (device sdc1): read error corrected: ino 0 off 83188697214976 (dev /dev/sde1 sector 380092312)
> Mar 4 01:26:30 cloud8-1550 kernel: [44662.156236] BTRFS critical (device sdc1): corrupt leaf: root=96207 block=83188697219072 slot=0, unexpected item end, have 16395 expect 16283
>
> ________________________________________
> Von: Carsten Grommel <c.grommel@profihost.ag>
> Gesendet: Montag, 7. März 2022 08:25
> An: Qu Wenruo; Zygo Blaxell
> Cc: linux-btrfs@vger.kernel.org
> Betreff: AW: AW: How to (attempt to) repair these btrfs errors
>
> Hi Qu,
>
>> Mind to share a dmesg just after the RO fallback?
>
> the most recent crash:
>
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.191649] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.194011] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.195395] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.196620] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.197920] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.212980] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500953600 (dev /dev/sde1 sector 10546500256)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.214413] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500957696 (dev /dev/sde1 sector 10546500264)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215204] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215656] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500961792 (dev /dev/sde1 sector 10546500272)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.230156] BTRFS: error (device sdc1) in btrfs_finish_ordered_io:2736: errno=-5 IO failure
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.233127] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500965888 (dev /dev/sde1 sector 10546500280)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.247096] BTRFS info (device sdc1): forced readonly
>
> ________________________________________
> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
> Gesendet: Montag, 7. März 2022 08:11
> An: Carsten Grommel; Zygo Blaxell
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: AW: How to (attempt to) repair these btrfs errors
>
>
>
> On 2022/3/7 15:03, Carsten Grommel wrote:
>> Thank you for the answer. We are using space_cache v2:
>>
>> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>>
>>> Data is raid0, so data repair is not possible. Delete all the files
>>> that contain corrupt data.
>>
>> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
>
> Btrfs only falls back to RO for very critical errors (which could affect
> on-disk metadata consistency).
>
> Thus plain data corruption should not cause the RO.
>
> Mind to share a dmesg just after the RO fallback?
>
> Thanks,
> Qu
>
>>
>>> I don't see any errors in these logs that would indicate a metadata issue,
>>> but huge numbers of messages are suppressed. Perhaps a log closer
>>> to the moment when the filesystem goes read-only will be more useful.
>>
>>> I would expect that if there are no problems on sda1 or sdb1 then it
>>> should be possible to repair the metadata errors on sdd1 by scrubbing
>> that device.
>>
>> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
>> I did not yet try to scrub specifically on sdd though, gonna try that.
>>
>> Should it remount again i will provide the most recent dmesg's right before it crashes.
>>
>> ________________________________________
>> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
>> Gesendet: Sonntag, 6. März 2022 02:36
>> An: Carsten Grommel
>> Cc: linux-btrfs@vger.kernel.org
>> Betreff: Re: How to (attempt to) repair these btrfs errors
>>
>> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>>> Follow-up pastebin with the most recent errors in dmesg:
>>>
>>> https://pastebin.com/4yJJdQPJ
>>
>> This seems to have expired.
>>
>>> ________________________________________
>>> Von: Carsten Grommel
>>> Gesendet: Montag, 28. Februar 2022 19:41
>>> An: linux-btrfs@vger.kernel.org
>>> Betreff: How to (attempt to) repair these btrfs errors
>>>
>>> Hi,
>>>
>>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>>
>>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>>
>>> But it was Kernel 5.4.121 before
>>>
>>> btrfs --version
>>> btrfs-progs v4.20.1
>>>
>>> btrfs fi show
>>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>>> Total devices 3 FS bytes used 56.74TiB
>>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>>
>>> btrfs fi df /vmbackup/
>>> Data, RAID0: total=66.62TiB, used=56.45TiB
>>> System, RAID1: total=8.00MiB, used=4.36MiB
>>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>>
>>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>>
>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>>
>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>>
>>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>>
>>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>>
>>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>>
>>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>>> I am currently running a scrub over the filesystem.
>>>
>>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>>
>>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>>
>> The corruption and generation event counts indicate sdd1 (or one of its
>> component devices) was offline for a long time or suffered corruption
>> on a large scale.
>>
>> Data is raid0, so data repair is not possible. Delete all the files
>> that contain corrupt data.
>>
>> If you are using space_cache=v1, now is a good time to upgrade to
>> space_cache=v2. v1 space cache is stored in the data profile, and it has
>> likely been corrupted. btrfs will usually detect and repair corruption
>> in space_cache=v1, but there is no need to take any such risk here
>> when you can easily use v2 instead (or at least clear the v1 cache).
>>
>> I don't see any errors in these logs that would indicate a metadata issue,
>> but huge numbers of messages are suppressed. Perhaps a log closer
>> to the moment when the filesystem goes read-only will be more useful.
>>
>> I would expect that if there are no problems on sda1 or sdb1 then it
>> should be possible to repair the metadata errors on sdd1 by scrubbing
>> that device.
>>
>>> Kind regards
>>> Carsten Grommel
>>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* AW: AW: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:34 ` Qu Wenruo
@ 2022-03-07 7:48 ` Carsten Grommel
2022-03-07 8:00 ` Qu Wenruo
0 siblings, 1 reply; 11+ messages in thread
From: Carsten Grommel @ 2022-03-07 7:48 UTC (permalink / raw)
To: Qu Wenruo, Zygo Blaxell; +Cc: linux-btrfs
>OK, this explains the reason.
>Some tree blocks in subvolume trees are corrupted, and by the worst
>possible way, metadata transid mismatch.
>
>Although it looks like some metadata read can be repaired, but I guess
>since btrfs_finish_ordered_io() still failed, it means some can not be
>repaired.
>
>Did the fs go through some split-brain cases? E.g. some devices got
>degraded mount, then the missing device come back?
Indeed something like this seems to have happened.
One of the raid6 had two disks failures with one disk being able to rejoin the raid.
This concerned me because in a raid6 there should be no problem with two devices leaving the raid.
At this point there seemed to be some corruption happening. I suspect that this resonated in some kind of
corruption loop causing garbage writes during the heavy write io the backups are causing on the filesystem.
> These seems to be the most critical afaic:
>
> Mar 4 01:25:51 cloud8-1550 kernel: [44623.523395] BTRFS critical (device sdc1): corrupt leaf: root=111550 >block=849874468864 slot=0 ino=32633089 file_offset=7805042688, invalid compression for file extent, have 15 expect range [0, 3]
>
>OK, this would explain the problem much better than the repairable
>metadata read.
>
>There should be no way we have compression type 0xf.
> Mar 4 01:25:51 cloud8-1550 kernel: [44623.527109] BTRFS error (device sdc1): block=849874468864 read time tree block corruption detected
> Mar 4 01:25:52 cloud8-1550 kernel: [44623.643308] BTRFS critical (device sdc1): corrupt leaf: root=50979 block=849880268800 slot=2, bad key order, prev (18446744073709551606 128 1269917216768) current (18446744073709551606 128 1269916291
> 072)
>
>And bad tree key order, even more serious.
>
>hex(1269917216768) = 0x127acf6f000
>hex(1269916291072) = 0x127ace8d000
>
>Doesn't look like a simple bitflip, nor does the preivous 0xf
>compression type.
>
>I have no idea how things can be so terribly wrong...
This is where i try to wrap my head around, i just can not explain how this cascade of errors happened
Do you see a problem in trying to restore as much data as possible with btrfs restore / btrfs send | recieve?
I fear that the corruption could wander, any experiences on this?
Thanks!
________________________________________
Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
Gesendet: Montag, 7. März 2022 08:34
An: Carsten Grommel; Zygo Blaxell
Cc: linux-btrfs@vger.kernel.org
Betreff: Re: AW: AW: How to (attempt to) repair these btrfs errors
On 2022/3/7 15:25, Carsten Grommel wrote:
> Hi Qu,
>
>> Mind to share a dmesg just after the RO fallback?
>
> the most recent crash:
>
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.191649] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.194011] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.195395] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.196620] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.197920] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.212980] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500953600 (dev /dev/sde1 sector 10546500256)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.214413] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500957696 (dev /dev/sde1 sector 10546500264)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215204] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215656] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500961792 (dev /dev/sde1 sector 10546500272)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.230156] BTRFS: error (device sdc1) in btrfs_finish_ordered_io:2736: errno=-5 IO failure
OK, this explains the reason.
Some tree blocks in subvolume trees are corrupted, and by the worst
possible way, metadata transid mismatch.
Although it looks like some metadata read can be repaired, but I guess
since btrfs_finish_ordered_io() still failed, it means some can not be
repaired.
Did the fs go through some split-brain cases? E.g. some devices got
degraded mount, then the missing device come back?
Thanks,
Qu
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.233127] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500965888 (dev /dev/sde1 sector 10546500280)
> Mar 4 01:43:15 cloud8-1550 kernel: [45667.247096] BTRFS info (device sdc1): forced readonly
>
> ________________________________________
> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
> Gesendet: Montag, 7. März 2022 08:11
> An: Carsten Grommel; Zygo Blaxell
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: AW: How to (attempt to) repair these btrfs errors
>
>
>
> On 2022/3/7 15:03, Carsten Grommel wrote:
>> Thank you for the answer. We are using space_cache v2:
>>
>> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>>
>>> Data is raid0, so data repair is not possible. Delete all the files
>>> that contain corrupt data.
>>
>> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
>
> Btrfs only falls back to RO for very critical errors (which could affect
> on-disk metadata consistency).
>
> Thus plain data corruption should not cause the RO.
>
> Mind to share a dmesg just after the RO fallback?
>
> Thanks,
> Qu
>
>>
>>> I don't see any errors in these logs that would indicate a metadata issue,
>>> but huge numbers of messages are suppressed. Perhaps a log closer
>>> to the moment when the filesystem goes read-only will be more useful.
>>
>>> I would expect that if there are no problems on sda1 or sdb1 then it
>>> should be possible to repair the metadata errors on sdd1 by scrubbing
>> that device.
>>
>> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
>> I did not yet try to scrub specifically on sdd though, gonna try that.
>>
>> Should it remount again i will provide the most recent dmesg's right before it crashes.
>>
>> ________________________________________
>> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
>> Gesendet: Sonntag, 6. März 2022 02:36
>> An: Carsten Grommel
>> Cc: linux-btrfs@vger.kernel.org
>> Betreff: Re: How to (attempt to) repair these btrfs errors
>>
>> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>>> Follow-up pastebin with the most recent errors in dmesg:
>>>
>>> https://pastebin.com/4yJJdQPJ
>>
>> This seems to have expired.
>>
>>> ________________________________________
>>> Von: Carsten Grommel
>>> Gesendet: Montag, 28. Februar 2022 19:41
>>> An: linux-btrfs@vger.kernel.org
>>> Betreff: How to (attempt to) repair these btrfs errors
>>>
>>> Hi,
>>>
>>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>>
>>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>>
>>> But it was Kernel 5.4.121 before
>>>
>>> btrfs --version
>>> btrfs-progs v4.20.1
>>>
>>> btrfs fi show
>>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>>> Total devices 3 FS bytes used 56.74TiB
>>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>>
>>> btrfs fi df /vmbackup/
>>> Data, RAID0: total=66.62TiB, used=56.45TiB
>>> System, RAID1: total=8.00MiB, used=4.36MiB
>>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>>
>>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>>
>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>>
>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>>
>>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>>
>>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>>
>>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>>
>>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>>> I am currently running a scrub over the filesystem.
>>>
>>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>>
>>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>>
>> The corruption and generation event counts indicate sdd1 (or one of its
>> component devices) was offline for a long time or suffered corruption
>> on a large scale.
>>
>> Data is raid0, so data repair is not possible. Delete all the files
>> that contain corrupt data.
>>
>> If you are using space_cache=v1, now is a good time to upgrade to
>> space_cache=v2. v1 space cache is stored in the data profile, and it has
>> likely been corrupted. btrfs will usually detect and repair corruption
>> in space_cache=v1, but there is no need to take any such risk here
>> when you can easily use v2 instead (or at least clear the v1 cache).
>>
>> I don't see any errors in these logs that would indicate a metadata issue,
>> but huge numbers of messages are suppressed. Perhaps a log closer
>> to the moment when the filesystem goes read-only will be more useful.
>>
>> I would expect that if there are no problems on sda1 or sdb1 then it
>> should be possible to repair the metadata errors on sdd1 by scrubbing
>> that device.
>>
>>> Kind regards
>>> Carsten Grommel
>>>
^ permalink raw reply [flat|nested] 11+ messages in thread
* Re: AW: AW: AW: How to (attempt to) repair these btrfs errors
2022-03-07 7:48 ` AW: " Carsten Grommel
@ 2022-03-07 8:00 ` Qu Wenruo
0 siblings, 0 replies; 11+ messages in thread
From: Qu Wenruo @ 2022-03-07 8:00 UTC (permalink / raw)
To: Carsten Grommel, Zygo Blaxell; +Cc: linux-btrfs
On 2022/3/7 15:48, Carsten Grommel wrote:
>
>
>> OK, this explains the reason.
>
>> Some tree blocks in subvolume trees are corrupted, and by the worst
>> possible way, metadata transid mismatch.
>>
>> Although it looks like some metadata read can be repaired, but I guess
>> since btrfs_finish_ordered_io() still failed, it means some can not be
>> repaired.
>>
>> Did the fs go through some split-brain cases? E.g. some devices got
>> degraded mount, then the missing device come back?
>
> Indeed something like this seems to have happened.
> One of the raid6 had two disks failures with one disk being able to rejoin the raid.
> This concerned me because in a raid6 there should be no problem with two devices leaving the raid.
Oh, RAID56, it won't end up well due to write hole.
Thus we may have some weird corruption due to write hole then.
> At this point there seemed to be some corruption happening. I suspect that this resonated in some kind of
> corruption loop causing garbage writes during the heavy write io the backups are causing on the filesystem.
>
>> These seems to be the most critical afaic:
>>
>> Mar 4 01:25:51 cloud8-1550 kernel: [44623.523395] BTRFS critical (device sdc1): corrupt leaf: root=111550 >block=849874468864 slot=0 ino=32633089 file_offset=7805042688, invalid compression for file extent, have 15 expect range [0, 3]
>>
>> OK, this would explain the problem much better than the repairable
>> metadata read.
>>
>> There should be no way we have compression type 0xf.
>
>> Mar 4 01:25:51 cloud8-1550 kernel: [44623.527109] BTRFS error (device sdc1): block=849874468864 read time tree block corruption detected
>> Mar 4 01:25:52 cloud8-1550 kernel: [44623.643308] BTRFS critical (device sdc1): corrupt leaf: root=50979 block=849880268800 slot=2, bad key order, prev (18446744073709551606 128 1269917216768) current (18446744073709551606 128 1269916291
>> 072)
>>
>> And bad tree key order, even more serious.
>>
>> hex(1269917216768) = 0x127acf6f000
>> hex(1269916291072) = 0x127ace8d000
>>
>> Doesn't look like a simple bitflip, nor does the preivous 0xf
>> compression type.
>>
>> I have no idea how things can be so terribly wrong...
>
> This is where i try to wrap my head around, i just can not explain how this cascade of errors happened
> Do you see a problem in trying to restore as much data as possible with btrfs restore / btrfs send | recieve?
> I fear that the corruption could wander, any experiences on this?
For data salvage, the new rescue=all mount option would become pretty
handy I guess.
Thanks,
Qu
>
> Thanks!
>
> ________________________________________
> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
> Gesendet: Montag, 7. März 2022 08:34
> An: Carsten Grommel; Zygo Blaxell
> Cc: linux-btrfs@vger.kernel.org
> Betreff: Re: AW: AW: How to (attempt to) repair these btrfs errors
>
>
>
> On 2022/3/7 15:25, Carsten Grommel wrote:
>> Hi Qu,
>>
>>> Mind to share a dmesg just after the RO fallback?
>>
>> the most recent crash:
>>
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.191649] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.194011] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.195395] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.196620] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.197920] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 126097
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.212980] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500953600 (dev /dev/sde1 sector 10546500256)
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.214413] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500957696 (dev /dev/sde1 sector 10546500264)
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215204] BTRFS error (device sdc1): parent transid verify failed on 16155500953600 wanted 126097 found 94652
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.215656] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500961792 (dev /dev/sde1 sector 10546500272)
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.230156] BTRFS: error (device sdc1) in btrfs_finish_ordered_io:2736: errno=-5 IO failure
>
> OK, this explains the reason.
>
> Some tree blocks in subvolume trees are corrupted, and by the worst
> possible way, metadata transid mismatch.
>
> Although it looks like some metadata read can be repaired, but I guess
> since btrfs_finish_ordered_io() still failed, it means some can not be
> repaired.
>
> Did the fs go through some split-brain cases? E.g. some devices got
> degraded mount, then the missing device come back?
>
> Thanks,
> Qu
>
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.233127] BTRFS info (device sdc1): read error corrected: ino 0 off 16155500965888 (dev /dev/sde1 sector 10546500280)
>> Mar 4 01:43:15 cloud8-1550 kernel: [45667.247096] BTRFS info (device sdc1): forced readonly
>>
>> ________________________________________
>> Von: Qu Wenruo <quwenruo.btrfs@gmx.com>
>> Gesendet: Montag, 7. März 2022 08:11
>> An: Carsten Grommel; Zygo Blaxell
>> Cc: linux-btrfs@vger.kernel.org
>> Betreff: Re: AW: How to (attempt to) repair these btrfs errors
>>
>>
>>
>> On 2022/3/7 15:03, Carsten Grommel wrote:
>>> Thank you for the answer. We are using space_cache v2:
>>>
>>> /dev/sdc1 on /vmbackup type btrfs (rw,noatime,nobarrier,compress-force=zlib:3,ssd_spread,noacl,space_cache=v2,skip_balance,subvolid=5,subvol=/,x-systemd.mount-timeout=4h)
>>>
>>>> Data is raid0, so data repair is not possible. Delete all the files
>>>> that contain corrupt data.
>>>
>>> I tried but as soon as I access the broken blocks btrfs fails into readonly so I am kind of in a deadlock there.
>>
>> Btrfs only falls back to RO for very critical errors (which could affect
>> on-disk metadata consistency).
>>
>> Thus plain data corruption should not cause the RO.
>>
>> Mind to share a dmesg just after the RO fallback?
>>
>> Thanks,
>> Qu
>>
>>>
>>>> I don't see any errors in these logs that would indicate a metadata issue,
>>>> but huge numbers of messages are suppressed. Perhaps a log closer
>>>> to the moment when the filesystem goes read-only will be more useful.
>>>
>>>> I would expect that if there are no problems on sda1 or sdb1 then it
>>>> should be possible to repair the metadata errors on sdd1 by scrubbing
>>> that device.
>>>
>>> I ran a number of scrubs now, at some point it always fails and btrfs remounts into readonly.
>>> I did not yet try to scrub specifically on sdd though, gonna try that.
>>>
>>> Should it remount again i will provide the most recent dmesg's right before it crashes.
>>>
>>> ________________________________________
>>> Von: Zygo Blaxell <ce3g8jdj@umail.furryterror.org>
>>> Gesendet: Sonntag, 6. März 2022 02:36
>>> An: Carsten Grommel
>>> Cc: linux-btrfs@vger.kernel.org
>>> Betreff: Re: How to (attempt to) repair these btrfs errors
>>>
>>> On Tue, Mar 01, 2022 at 10:55:50AM +0000, Carsten Grommel wrote:
>>>> Follow-up pastebin with the most recent errors in dmesg:
>>>>
>>>> https://pastebin.com/4yJJdQPJ
>>>
>>> This seems to have expired.
>>>
>>>> ________________________________________
>>>> Von: Carsten Grommel
>>>> Gesendet: Montag, 28. Februar 2022 19:41
>>>> An: linux-btrfs@vger.kernel.org
>>>> Betreff: How to (attempt to) repair these btrfs errors
>>>>
>>>> Hi,
>>>>
>>>> short buildup: btrfs filesystem used for storing ceph rbd backups within subvolumes got corrupted.
>>>> Underlying 3 RAID 6es, btrfs is mounted on Top as RAID 0 over these Raids for performance ( we have to store massive Data)
>>>>
>>>> Linux cloud8-1550 5.10.93+2-ph #1 SMP Fri Jan 21 07:52:51 UTC 2022 x86_64 GNU/Linux
>>>>
>>>> But it was Kernel 5.4.121 before
>>>>
>>>> btrfs --version
>>>> btrfs-progs v4.20.1
>>>>
>>>> btrfs fi show
>>>> Label: none uuid: b634a011-28fa-41d7-8d6e-3f68ccb131d0
>>>> Total devices 3 FS bytes used 56.74TiB
>>>> devid 1 size 25.46TiB used 22.70TiB path /dev/sda1
>>>> devid 2 size 25.46TiB used 22.69TiB path /dev/sdb1
>>>> devid 3 size 25.46TiB used 22.70TiB path /dev/sdd1
>>>>
>>>> btrfs fi df /vmbackup/
>>>> Data, RAID0: total=66.62TiB, used=56.45TiB
>>>> System, RAID1: total=8.00MiB, used=4.36MiB
>>>> Metadata, RAID1: total=750.00GiB, used=294.90GiB
>>>> GlobalReserve, single: total=512.00MiB, used=0.00B
>>>>
>>>> Attached the dmesg.log, a few dmesg messages following regarding the different errors (some informations redacted):
>>>>
>>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074516, gen 184286
>>>>
>>>> [Mon Feb 28 18:53:57 2022] BTRFS error (device sda1): bdev /dev/sdd1 errs: wr 0, rd 0, flush 0, corrupt 69074517, gen 184286
>>>>
>>>> [Mon Feb 28 18:54:23 2022] BTRFS error (device sda1): unable to fixup (regular) error at logical 776693776384 on dev /dev/sdd1
>>>>
>>>> [Mon Feb 28 18:54:25 2022] scrub_handle_errored_block: 21812 callbacks suppressed
>>>>
>>>> [Mon Feb 28 18:54:31 2022] BTRFS warning (device sda1): checksum error at logical 777752285184 on dev /dev/sdd1, physical 259607957504, root 108747, inode 257, offset 59804737536, length 4096, links 1 (path: cephstorX_vm-XXX-disk-X-base.img_1645337735)
>>>>
>>>> I am able to mount the filesystem in read-write mode but accessing specific blocks seems to crash btrfs to remount into read-only
>>>> I am currently running a scrub over the filesystem.
>>>>
>>>> The system got rebooted and the fs got remounted 2-3 times. I made the experience that usually btrfs would and could fix these kinds of errors after a remount, not this time though.
>>>>
>>>> Before I ran “btrfs check –repair” I would like some advice at how to tackle theses errors.
>>>
>>> The corruption and generation event counts indicate sdd1 (or one of its
>>> component devices) was offline for a long time or suffered corruption
>>> on a large scale.
>>>
>>> Data is raid0, so data repair is not possible. Delete all the files
>>> that contain corrupt data.
>>>
>>> If you are using space_cache=v1, now is a good time to upgrade to
>>> space_cache=v2. v1 space cache is stored in the data profile, and it has
>>> likely been corrupted. btrfs will usually detect and repair corruption
>>> in space_cache=v1, but there is no need to take any such risk here
>>> when you can easily use v2 instead (or at least clear the v1 cache).
>>>
>>> I don't see any errors in these logs that would indicate a metadata issue,
>>> but huge numbers of messages are suppressed. Perhaps a log closer
>>> to the moment when the filesystem goes read-only will be more useful.
>>>
>>> I would expect that if there are no problems on sda1 or sdb1 then it
>>> should be possible to repair the metadata errors on sdd1 by scrubbing
>>> that device.
>>>
>>>> Kind regards
>>>> Carsten Grommel
>>>>
^ permalink raw reply [flat|nested] 11+ messages in thread
end of thread, other threads:[~2022-03-07 8:00 UTC | newest]
Thread overview: 11+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2022-02-28 18:41 How to (attempt to) repair these btrfs errors Carsten Grommel
2022-03-01 10:55 ` AW: " Carsten Grommel
2022-03-06 1:36 ` Zygo Blaxell
2022-03-07 7:03 ` AW: " Carsten Grommel
2022-03-07 7:11 ` Qu Wenruo
2022-03-07 7:25 ` AW: " Carsten Grommel
2022-03-07 7:27 ` Carsten Grommel
2022-03-07 7:39 ` Qu Wenruo
2022-03-07 7:34 ` Qu Wenruo
2022-03-07 7:48 ` AW: " Carsten Grommel
2022-03-07 8:00 ` Qu Wenruo
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.