* BTRFS: Transaction aborted (error -5)
@ 2015-01-04 14:50 Dyweni - BTRFS
2015-01-04 23:24 ` Chris Murphy
0 siblings, 1 reply; 6+ messages in thread
From: Dyweni - BTRFS @ 2015-01-04 14:50 UTC (permalink / raw)
To: linux-btrfs
Hi All,
Just wondering of a few things:
1. Has anyone seen a problem like this before (see below)? This occurs
when mounting the file system. It mounts, but goes straight to
read-only mode.
2. I've scanned the device (/dev/sdd) using 'badblocks -s v', which
reports no bad blocks found.
3. What can be done to recover from this? (I'm planning to upgrade to
3.18 before trying recovery/repairs)
Thanks,
Dyweni
[32079.815291] BTRFS info (device sdd1): disk space caching is enabled
[32082.419524] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[32114.418433] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[32125.951446] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[32125.959497] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 24492BB3 level 0
[32125.959505] ------------[ cut here ]------------
[32125.959517] WARNING: CPU: 0 PID: 2601 at fs/btrfs/super.c:259
__btrfs_abort_transaction+0x43/0x110()
[32125.959518] BTRFS: Transaction aborted (error -5)
[32125.959519] Modules linked in: iscsi_trgt(O)
[32125.959527] CPU: 0 PID: 2601 Comm: btrfs-cleaner Tainted: G
O 3.16.5-gentoo-20150103-0758 #1
[32125.959528] Hardware name: /MS-6570, BIOS 6.00 PG 11/07/2003
[32125.959530] ec009e68 ec009e68 ec009e28 c14d581f ec009e58 c10333d0
c15bcea0 ec009e84
[32125.959533] 00000a29 c15c9417 00000103 c11c8723 c11c8723 fffffffb
f1c5d400 c14fd856
[32125.959537] ec009e70 c103341e 00000009 ec009e68 c15bcea0 ec009e84
ec009e9c c11c8723
[32125.959541] Call Trace:
[32125.959548] [<c14d581f>] dump_stack+0x16/0x18
[32125.959553] [<c10333d0>] warn_slowpath_common+0x70/0x90
[32125.959556] [<c11c8723>] ? __btrfs_abort_transaction+0x43/0x110
[32125.959559] [<c11c8723>] ? __btrfs_abort_transaction+0x43/0x110
[32125.959562] [<c103341e>] warn_slowpath_fmt+0x2e/0x30
[32125.959565] [<c11c8723>] __btrfs_abort_transaction+0x43/0x110
[32125.959571] [<c11e2cc4>] btrfs_run_delayed_refs.part.74+0xd4/0x1d0
[32125.959574] [<c11e2dcf>] btrfs_run_delayed_refs+0xf/0x20
[32125.959578] [<c11f1854>] btrfs_should_end_transaction+0x34/0x50
[32125.959581] [<c11e1461>] btrfs_drop_snapshot+0x1d1/0x6a0
[32125.959585] [<c104fbfb>] ? default_wake_function+0xb/0x10
[32125.959590] [<c105497f>] ? __wake_up_common+0x3f/0x70
[32125.959593] [<c11f3222>] btrfs_clean_one_deleted_snapshot+0x62/0x90
[32125.959595] [<c11ead3e>] cleaner_kthread+0xce/0x100
[32125.959597] [<c11eac70>] ? btrfs_destroy_pinned_extent+0x120/0x120
[32125.959603] [<c1048cf5>] kthread+0x95/0xb0
[32125.959606] [<c14da0c0>] ret_from_kernel_thread+0x20/0x30
[32125.959609] [<c1048c60>] ? kthread_worker_fn+0xb0/0xb0
[32125.959611] ---[ end trace 1e2cdbbae61fcdc1 ]---
[32125.959614] BTRFS: error (device sdd1) in
btrfs_run_delayed_refs:2806: errno=-5 IO failure
[32125.959616] BTRFS info (device sdd1): forced readonly
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BTRFS: Transaction aborted (error -5)
2015-01-04 14:50 BTRFS: Transaction aborted (error -5) Dyweni - BTRFS
@ 2015-01-04 23:24 ` Chris Murphy
2015-01-06 14:58 ` Dyweni - BTRFS
2015-01-06 15:04 ` Dyweni - BTRFS
0 siblings, 2 replies; 6+ messages in thread
From: Chris Murphy @ 2015-01-04 23:24 UTC (permalink / raw)
To: Y4BwxfPC4k5h; +Cc: Btrfs BTRFS
On Sun, Jan 4, 2015 at 7:50 AM, Dyweni - BTRFS <Y4BwxfPC4k5h@dyweni.com> wrote:
> Hi All,
>
> Just wondering of a few things:
>
> 1. Has anyone seen a problem like this before (see below)? This occurs when
> mounting the file system. It mounts, but goes straight to read-only mode.
> 2. I've scanned the device (/dev/sdd) using 'badblocks -s v', which reports
> no bad blocks found.
> 3. What can be done to recover from this? (I'm planning to upgrade to 3.18
> before trying recovery/repairs)
Try to mount with -o recovery with either kernel (newer is pretty much
always better). If that doesn't work then you should try upgrading
btrfs-progs to 3.18 (released dozens of hours ago) and run 'btrfs
check' on the volume and report the results. I don't recommend using
--repair option just yet.
> [32079.815291] BTRFS info (device sdd1): disk space caching is enabled
> [32082.419524] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 6E0D3115 level 0
> [32114.418433] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 6E0D3115 level 0
> [32125.951446] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 6E0D3115 level 0
> [32125.959497] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 24492BB3 level 0
Well I'm no expert, but it seems suspicious to me it doesn't find what
it wants on a particular block twice, but then on the 3rd attempt it
finds something different on the same block which also isn't what it
wants. So that sounds like a device problem to me. Is this an SSD?
What are your mount options (are you using discard)? And what's the
metadata profile, is it single or DUP? I'm gonna guess it's an SSD
with single copy of metadata which is why this isn't self-correcting.
--
Chris Murphy
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BTRFS: Transaction aborted (error -5)
2015-01-04 23:24 ` Chris Murphy
@ 2015-01-06 14:58 ` Dyweni - BTRFS
2015-01-06 15:04 ` Dyweni - BTRFS
1 sibling, 0 replies; 6+ messages in thread
From: Dyweni - BTRFS @ 2015-01-06 14:58 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS, chris
Hi,
>> [32079.815291] BTRFS info (device sdd1): disk space caching is enabled
>> [32082.419524] BTRFS: sdd1 checksum verify failed on 588447744 wanted
>> F90C810B found 6E0D3115 level 0
>> [32114.418433] BTRFS: sdd1 checksum verify failed on 588447744 wanted
>> F90C810B found 6E0D3115 level 0
>> [32125.951446] BTRFS: sdd1 checksum verify failed on 588447744 wanted
>> F90C810B found 6E0D3115 level 0
>> [32125.959497] BTRFS: sdd1 checksum verify failed on 588447744 wanted
>> F90C810B found 24492BB3 level 0
>
> Well I'm no expert, but it seems suspicious to me it doesn't find what
> it wants on a particular block twice, but then on the 3rd attempt it
> finds something different on the same block which also isn't what it
> wants. So that sounds like a device problem to me. Is this an SSD?
> What are your mount options (are you using discard)? And what's the
> metadata profile, is it single or DUP? I'm gonna guess it's an SSD
> with single copy of metadata which is why this isn't self-correcting.
>
So I finished testing the drive using 'badblocks -n -s -v' (the
non-destructive read-write mode). It came back clean, no bad blocks
found. This I did with the entire drive unmounted.
Yet, still, the file system reports the errors shortly after mounting.
(See below)
This drive is an older spinning type drive. This is the drive as
reported by 'lsscsi':
[3:0:0:0] disk ATA WDC WD1001FALS-0 1D05 /dev/sdd
Newegg lists it as a 'Western Digital WD Black WD1001FALS 1TB 7200 RPM
32MB Cache SATA 3.0Gb/s 3.5" Internal Hard Drive Bare Drive'
The disk is attached to the system via this, as reported by 'lspci':
01:09.0 RAID bus controller: Silicon Image, Inc. SiI 3124 PCI-X Serial
ATA Controller (rev 02)
(Not sure why it lists it as a raid controller or a pci-x controller, as
I used it a simple sata controller and it plugs into a regular 32bit
pci slot).
The motherboard is a Micro-Star MS-6570, with an AMD Athlon XP 3000+
(2171 MHz) processor and 2GB of RAM.
Mount options are only: noatime
BTRFS Profile is:
# btrfs fi df /var/lib/ceph/osd/ceph-1/
Data, single: total=185.01GiB, used=183.39GiB
System, DUP: total=8.00MiB, used=48.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=367.19MiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=128.00MiB, used=0.00B
[162288.768747] BTRFS info (device sdc1): disk space caching is enabled
[162290.463003] BTRFS info (device sdd1): disk space caching is enabled
[162335.594094] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[162335.595476] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[162335.602066] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 24492BB3 level 0
[162335.602075] ------------[ cut here ]------------
[162335.602085] WARNING: CPU: 0 PID: 31841 at fs/btrfs/super.c:260
__btrfs_abort_transaction+0x43/0x110()
[162335.602086] BTRFS: Transaction aborted (error -5)
[162335.602087] Modules linked in: iscsi_trgt(O)
[162335.602094] CPU: 0 PID: 31841 Comm: btrfs-cleaner Tainted: G
O 3.18.1-gentoo-20150104-0921 #1
[162335.602096] Hardware name: /MS-6570, BIOS 6.00 PG 11/07/2003
[162335.602097] e68a5e68 e68a5e68 e68a5e28 c14e48a4 e68a5e58 c10345a0
c15cbefc e68a5e84
[162335.602101] 00007c61 c15d895b 00000104 c11cff13 c11cff13 fffffffb
f4d23800 c150d330
[162335.602104] e68a5e70 c10345ee 00000009 e68a5e68 c15cbefc e68a5e84
e68a5e9c c11cff13
[162335.602108] Call Trace:
[162335.602117] [<c14e48a4>] dump_stack+0x16/0x18
[162335.602122] [<c10345a0>] warn_slowpath_common+0x70/0x90
[162335.602125] [<c11cff13>] ? __btrfs_abort_transaction+0x43/0x110
[162335.602127] [<c11cff13>] ? __btrfs_abort_transaction+0x43/0x110
[162335.602130] [<c10345ee>] warn_slowpath_fmt+0x2e/0x30
[162335.602133] [<c11cff13>] __btrfs_abort_transaction+0x43/0x110
[162335.602138] [<c11ea884>] btrfs_run_delayed_refs.part.73+0xd4/0x1d0
[162335.602140] [<c11ea98f>] btrfs_run_delayed_refs+0xf/0x20
[162335.602143] [<c11f96f4>] btrfs_should_end_transaction+0x34/0x50
[162335.602146] [<c11e8ef9>] btrfs_drop_snapshot+0x1c9/0x740
[162335.602149] [<c11fb152>] btrfs_clean_one_deleted_snapshot+0x62/0x90
[162335.602152] [<c11f2a49>] cleaner_kthread+0xd9/0x110
[162335.602155] [<c11f2970>] ? btrfs_destroy_pinned_extent+0x120/0x120
[162335.602160] [<c1047415>] kthread+0x95/0xb0
[162335.602164] [<c14e9100>] ret_from_kernel_thread+0x20/0x30
[162335.602166] [<c1047380>] ? kthread_worker_fn+0xb0/0xb0
[162335.602168] ---[ end trace ba640116f371d2ff ]---
[162335.602171] BTRFS: error (device sdd1) in
btrfs_run_delayed_refs:2792: errno=-5 IO failure
[162335.602173] BTRFS info (device sdd1): forced readonly
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BTRFS: Transaction aborted (error -5)
2015-01-04 23:24 ` Chris Murphy
2015-01-06 14:58 ` Dyweni - BTRFS
@ 2015-01-06 15:04 ` Dyweni - BTRFS
2015-01-06 15:15 ` Dyweni - BTRFS
1 sibling, 1 reply; 6+ messages in thread
From: Dyweni - BTRFS @ 2015-01-06 15:04 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS, chris
Hi,
> Try to mount with -o recovery with either kernel (newer is pretty much
> always better). If that doesn't work then you should try upgrading
> btrfs-progs to 3.18 (released dozens of hours ago) and run 'btrfs
> check' on the volume and report the results. I don't recommend using
> --repair option just yet.
mounting with -o recovery yields the same errors in dmesg output as
without
the -o recovery.
This is even after upgrading the kernel to 3.18.1 and btrfs-progs to
3.18.
BTRFS check yields this:
# btrfs check /dev/sdd1
Checking filesystem on /dev/sdd1
UUID: adad9bea-fc42-4411-bfda-345111934fda
checking extents
checksum verify failed on 588447744 found 6E0D3115 wanted F90C810B
checksum verify failed on 588447744 found 6E0D3115 wanted F90C810B
checksum verify failed on 588447744 found 24492BB3 wanted F90C810B
checksum verify failed on 588447744 found 6E0D3115 wanted F90C810B
Csum didn't match
owner ref check failed [588447744 16384]
ref mismatch on [151193784320 32768] extent item 0, found 1
Backref 151193784320 root 277 owner 36161 offset 3833856 num_refs 0 not
found in extent tree
Incorrect local backref count on 151193784320 root 277 owner 36161
offset 3833856 found 1 wanted 0 back 0xad8be18
backpointer mismatch on [151193784320 32768]
ref mismatch on [151193817088 32768] extent item 0, found 1
Backref 151193817088 root 277 owner 36161 offset 3915776 num_refs 0 not
found in extent tree
Incorrect local backref count on 151193817088 root 277 owner 36161
offset 3915776 found 1 wanted 0 back 0xad8bf00
backpointer mismatch on [151193817088 32768]
ref mismatch on [151193849856 180224] extent item 0, found 1
Backref 151193849856 root 277 owner 36355 offset 3112960 num_refs 0 not
found in extent tree
Incorrect local backref count on 151193849856 root 277 owner 36355
offset 3112960 found 1 wanted 0 back 0xab333f0
backpointer mismatch on [151193849856 180224]
ref mismatch on [151194030080 3145728] extent item 0, found 7
Backref 151194030080 root 277 owner 36187 offset 1048576 num_refs 0 not
found in extent tree
Incorrect local backref count on 151194030080 root 277 owner 36187
offset 1048576 found 7 wanted 0 back 0x9b82580
backpointer mismatch on [151194030080 3145728]
ref mismatch on [151197175808 32768] extent item 0, found 1
Backref 151197175808 root 277 owner 36361 offset 2523136 num_refs 0 not
found in extent tree
Incorrect local backref count on 151197175808 root 277 owner 36361
offset 2523136 found 1 wanted 0 back 0xa0f5568
backpointer mismatch on [151197175808 32768]
ref mismatch on [151197208576 32768] extent item 0, found 1
Backref 151197208576 root 277 owner 36361 offset 2572288 num_refs 0 not
found in extent tree
Incorrect local backref count on 151197208576 root 277 owner 36361
offset 2572288 found 1 wanted 0 back 0xa783490
backpointer mismatch on [151197208576 32768]
ref mismatch on [151197241344 32768] extent item 0, found 1
Backref 151197241344 root 277 owner 36361 offset 2621440 num_refs 0 not
found in extent tree
Incorrect local backref count on 151197241344 root 277 owner 36361
offset 2621440 found 1 wanted 0 back 0xa4d67e8
backpointer mismatch on [151197241344 32768]
ref mismatch on [151197274112 32768] extent item 0, found 1
Backref 151197274112 root 277 owner 36361 offset 2703360 num_refs 0 not
found in extent tree
Incorrect local backref count on 151197274112 root 277 owner 36361
offset 2703360 found 1 wanted 0 back 0x925de30
backpointer mismatch on [151197274112 32768]
ref mismatch on [151197306880 16384] extent item 0, found 1
Backref 151197306880 root 277 owner 36361 offset 3637248 num_refs 0 not
found in extent tree
Incorrect local backref count on 151197306880 root 277 owner 36361
offset 3637248 found 1 wanted 0 back 0x916a658
backpointer mismatch on [151197306880 16384]
ref mismatch on [151197323264 983040] extent item 0, found 3
Backref 151197323264 root 277 owner 36208 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151197323264 root 277 owner 36208
offset 0 found 3 wanted 0 back 0xb18a1e0
backpointer mismatch on [151197323264 983040]
ref mismatch on [151198306304 32768] extent item 0, found 1
Backref 151198306304 root 277 owner 36086 offset 3780608 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198306304 root 277 owner 36086
offset 3780608 found 1 wanted 0 back 0xb30f878
backpointer mismatch on [151198306304 32768]
ref mismatch on [151198339072 98304] extent item 0, found 1
Backref 151198339072 root 277 owner 36396 offset 901120 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198339072 root 277 owner 36396
offset 901120 found 1 wanted 0 back 0x99bfc58
backpointer mismatch on [151198339072 98304]
ref mismatch on [151198437376 16384] extent item 0, found 1
Backref 151198437376 root 277 owner 36396 offset 1015808 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198437376 root 277 owner 36396
offset 1015808 found 1 wanted 0 back 0x99bfd40
backpointer mismatch on [151198437376 16384]
ref mismatch on [151198453760 16384] extent item 0, found 1
Backref 151198453760 root 277 owner 36396 offset 1048576 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198453760 root 277 owner 36396
offset 1048576 found 1 wanted 0 back 0x99bfe28
backpointer mismatch on [151198453760 16384]
ref mismatch on [151198470144 114688] extent item 0, found 1
Backref 151198470144 root 277 owner 36396 offset 1081344 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198470144 root 277 owner 36396
offset 1081344 found 1 wanted 0 back 0x99bff10
backpointer mismatch on [151198470144 114688]
ref mismatch on [151198584832 16384] extent item 0, found 1
Backref 151198584832 root 277 owner 36396 offset 1294336 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198584832 root 277 owner 36396
offset 1294336 found 1 wanted 0 back 0x99bfff8
backpointer mismatch on [151198584832 16384]
ref mismatch on [151198601216 16384] extent item 0, found 1
Backref 151198601216 root 277 owner 36187 offset 1179648 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198601216 root 277 owner 36187
offset 1179648 found 1 wanted 0 back 0x9b82668
backpointer mismatch on [151198601216 16384]
ref mismatch on [151198617600 16384] extent item 0, found 1
Backref 151198617600 root 277 owner 36187 offset 1638400 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198617600 root 277 owner 36187
offset 1638400 found 1 wanted 0 back 0x9b82920
backpointer mismatch on [151198617600 16384]
ref mismatch on [151198633984 180224] extent item 0, found 1
Backref 151198633984 root 277 owner 39502 offset 81920 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198633984 root 277 owner 39502
offset 81920 found 1 wanted 0 back 0x9c208f0
backpointer mismatch on [151198633984 180224]
ref mismatch on [151198814208 53248] extent item 0, found 4
Backref 151198814208 parent 487440384 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198814208 parent 487440384 owner 0
offset 0 found 1 wanted 0 back 0xa67ed30
Backref 151198814208 parent 510558208 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198814208 parent 510558208 owner 0
offset 0 found 1 wanted 0 back 0xafe7ed0
Backref 151198814208 parent 558415872 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198814208 parent 558415872 owner 0
offset 0 found 1 wanted 0 back 0xa3c4408
Backref 151198814208 root 277 owner 41529 offset 3493888 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198814208 root 277 owner 41529
offset 3493888 found 1 wanted 0 back 0x94adf80
backpointer mismatch on [151198814208 53248]
ref mismatch on [151198867456 16384] extent item 0, found 4
Backref 151198867456 parent 487440384 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198867456 parent 487440384 owner 0
offset 0 found 1 wanted 0 back 0xa506050
Backref 151198867456 parent 510558208 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198867456 parent 510558208 owner 0
offset 0 found 1 wanted 0 back 0xafe7fa8
Backref 151198867456 parent 558415872 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198867456 parent 558415872 owner 0
offset 0 found 1 wanted 0 back 0xae64ba8
Backref 151198867456 root 277 owner 41529 offset 3584000 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198867456 root 277 owner 41529
offset 3584000 found 1 wanted 0 back 0x95aa098
backpointer mismatch on [151198867456 16384]
ref mismatch on [151198896128 81920] extent item 0, found 1
Backref 151198896128 root 277 owner 39498 offset 819200 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198896128 root 277 owner 39498
offset 819200 found 1 wanted 0 back 0x90ee3d8
backpointer mismatch on [151198896128 81920]
ref mismatch on [151198978048 40960] extent item 0, found 4
Backref 151198978048 parent 487440384 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198978048 parent 487440384 owner 0
offset 0 found 1 wanted 0 back 0xa505f68
Backref 151198978048 parent 510558208 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198978048 parent 510558208 owner 0
offset 0 found 1 wanted 0 back 0xafe7f60
Backref 151198978048 parent 558415872 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198978048 parent 558415872 owner 0
offset 0 found 1 wanted 0 back 0xae64b60
Backref 151198978048 root 277 owner 41529 offset 3547136 num_refs 0 not
found in extent tree
Incorrect local backref count on 151198978048 root 277 owner 41529
offset 3547136 found 1 wanted 0 back 0x949f190
backpointer mismatch on [151198978048 40960]
ref mismatch on [151199043584 32768] extent item 0, found 1
Backref 151199043584 root 277 owner 39429 offset 2981888 num_refs 0 not
found in extent tree
Incorrect local backref count on 151199043584 root 277 owner 39429
offset 2981888 found 1 wanted 0 back 0x93109f0
backpointer mismatch on [151199043584 32768]
ref mismatch on [151199076352 16384] extent item 0, found 1
Backref 151199076352 root 277 owner 39429 offset 3375104 num_refs 0 not
found in extent tree
Incorrect local backref count on 151199076352 root 277 owner 39429
offset 3375104 found 1 wanted 0 back 0x957bae0
backpointer mismatch on [151199076352 16384]
ref mismatch on [151199092736 16384] extent item 0, found 1
Backref 151199092736 root 277 owner 39444 offset 638976 num_refs 0 not
found in extent tree
Incorrect local backref count on 151199092736 root 277 owner 39444
offset 638976 found 1 wanted 0 back 0x9783238
backpointer mismatch on [151199092736 16384]
ref mismatch on [151199121408 4194304] extent item 0, found 1
Backref 151199121408 root 277 owner 32024 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151199121408 root 277 owner 32024
offset 0 found 1 wanted 0 back 0x90c4460
backpointer mismatch on [151199121408 4194304]
ref mismatch on [151203315712 4194304] extent item 0, found 1
Backref 151203315712 root 277 owner 32025 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151203315712 root 277 owner 32025
offset 0 found 1 wanted 0 back 0x90c44a8
backpointer mismatch on [151203315712 4194304]
ref mismatch on [151207510016 4194304] extent item 0, found 1
Backref 151207510016 root 277 owner 32026 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151207510016 root 277 owner 32026
offset 0 found 1 wanted 0 back 0x94abda8
backpointer mismatch on [151207510016 4194304]
ref mismatch on [151211704320 1540096] extent item 0, found 1
Backref 151211704320 root 277 owner 32027 offset 2654208 num_refs 0 not
found in extent tree
Incorrect local backref count on 151211704320 root 277 owner 32027
offset 2654208 found 1 wanted 0 back 0x94abdf0
backpointer mismatch on [151211704320 1540096]
ref mismatch on [151213244416 4194304] extent item 0, found 33
Backref 151213244416 root 277 owner 36195 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151213244416 root 277 owner 36195
offset 0 found 33 wanted 0 back 0x9dd6290
backpointer mismatch on [151213244416 4194304]
ref mismatch on [151217438720 704512] extent item 0, found 2
Backref 151217438720 root 277 owner 36376 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151217438720 root 277 owner 36376
offset 0 found 2 wanted 0 back 0xa400dd8
backpointer mismatch on [151217438720 704512]
ref mismatch on [151218143232 98304] extent item 0, found 1
Backref 151218143232 root 277 owner 36187 offset 1048576 num_refs 0 not
found in extent tree
Incorrect local backref count on 151218143232 root 277 owner 36187
offset 1048576 found 1 wanted 0 back 0x9b82498
backpointer mismatch on [151218143232 98304]
ref mismatch on [151218241536 16384] extent item 0, found 1
Backref 151218241536 root 277 owner 36195 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151218241536 root 277 owner 36195
offset 0 found 1 wanted 0 back 0x9dd61a8
backpointer mismatch on [151218241536 16384]
ref mismatch on [151218270208 3571712] extent item 0, found 1
Backref 151218270208 root 277 owner 34539 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151218270208 root 277 owner 34539
offset 0 found 1 wanted 0 back 0xa20f620
backpointer mismatch on [151218270208 3571712]
ref mismatch on [151221841920 81920] extent item 0, found 1
Backref 151221841920 root 277 owner 35699 offset 2949120 num_refs 0 not
found in extent tree
Incorrect local backref count on 151221841920 root 277 owner 35699
offset 2949120 found 1 wanted 0 back 0x95f99c8
backpointer mismatch on [151221841920 81920]
ref mismatch on [151221923840 32768] extent item 0, found 1
Backref 151221923840 root 277 owner 35645 offset 3457024 num_refs 0 not
found in extent tree
Incorrect local backref count on 151221923840 root 277 owner 35645
offset 3457024 found 1 wanted 0 back 0x9b0daa0
backpointer mismatch on [151221923840 32768]
ref mismatch on [151221960704 4194304] extent item 0, found 1
Backref 151221960704 root 277 owner 32035 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151221960704 root 277 owner 32035
offset 0 found 1 wanted 0 back 0xa95f580
backpointer mismatch on [151221960704 4194304]
ref mismatch on [151226155008 4194304] extent item 0, found 33
Backref 151226155008 root 277 owner 36196 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151226155008 root 277 owner 36196
offset 0 found 33 wanted 0 back 0x9dd8248
backpointer mismatch on [151226155008 4194304]
ref mismatch on [151230349312 4194304] extent item 0, found 9
Backref 151230349312 root 277 owner 36197 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151230349312 root 277 owner 36197
offset 0 found 9 wanted 0 back 0xaaf2780
backpointer mismatch on [151230349312 4194304]
ref mismatch on [151234543616 16384] extent item 0, found 1
Backref 151234543616 root 277 owner 36195 offset 32768 num_refs 0 not
found in extent tree
Incorrect local backref count on 151234543616 root 277 owner 36195
offset 32768 found 1 wanted 0 back 0x9dd6378
backpointer mismatch on [151234543616 16384]
ref mismatch on [151234560000 4194304] extent item 0, found 1
Backref 151234560000 root 277 owner 34542 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151234560000 root 277 owner 34542
offset 0 found 1 wanted 0 back 0xa4e4ca0
backpointer mismatch on [151234560000 4194304]
ref mismatch on [151238754304 4194304] extent item 0, found 1
Backref 151238754304 root 277 owner 34543 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151238754304 root 277 owner 34543
offset 0 found 1 wanted 0 back 0x967da38
backpointer mismatch on [151238754304 4194304]
ref mismatch on [151242948608 4194304] extent item 0, found 1
Backref 151242948608 root 277 owner 34544 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151242948608 root 277 owner 34544
offset 0 found 1 wanted 0 back 0x9f41888
backpointer mismatch on [151242948608 4194304]
ref mismatch on [151247142912 4194304] extent item 0, found 1
Backref 151247142912 root 277 owner 34545 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151247142912 root 277 owner 34545
offset 0 found 1 wanted 0 back 0xab1d5a0
backpointer mismatch on [151247142912 4194304]
ref mismatch on [151251337216 311296] extent item 0, found 1
Backref 151251337216 root 277 owner 39346 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151251337216 root 277 owner 39346
offset 0 found 1 wanted 0 back 0xa889be0
backpointer mismatch on [151251337216 311296]
ref mismatch on [151251648512 49152] extent item 0, found 1
Backref 151251648512 root 277 owner 39488 offset 524288 num_refs 0 not
found in extent tree
Incorrect local backref count on 151251648512 root 277 owner 39488
offset 524288 found 1 wanted 0 back 0xaa39008
backpointer mismatch on [151251648512 49152]
ref mismatch on [151251705856 65536] extent item 0, found 1
Backref 151251705856 root 277 owner 35701 offset 229376 num_refs 0 not
found in extent tree
Incorrect local backref count on 151251705856 root 277 owner 35701
offset 229376 found 1 wanted 0 back 0xa93e940
backpointer mismatch on [151251705856 65536]
ref mismatch on [151251779584 4194304] extent item 0, found 1
Backref 151251779584 root 277 owner 32036 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151251779584 root 277 owner 32036
offset 0 found 1 wanted 0 back 0xb2b3be8
backpointer mismatch on [151251779584 4194304]
ref mismatch on [151255973888 4194304] extent item 0, found 1
Backref 151255973888 root 277 owner 32037 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151255973888 root 277 owner 32037
offset 0 found 1 wanted 0 back 0xa413ea0
backpointer mismatch on [151255973888 4194304]
ref mismatch on [151260168192 4194304] extent item 0, found 1
Backref 151260168192 root 277 owner 32038 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151260168192 root 277 owner 32038
offset 0 found 1 wanted 0 back 0xa2a63c0
backpointer mismatch on [151260168192 4194304]
ref mismatch on [151264362496 4194304] extent item 0, found 1
Backref 151264362496 root 277 owner 32039 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151264362496 root 277 owner 32039
offset 0 found 1 wanted 0 back 0x96a09b8
backpointer mismatch on [151264362496 4194304]
ref mismatch on [151268556800 4194304] extent item 0, found 1
Backref 151268556800 root 277 owner 34546 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151268556800 root 277 owner 34546
offset 0 found 1 wanted 0 back 0x99e28c8
backpointer mismatch on [151268556800 4194304]
ref mismatch on [151272759296 4194304] extent item 0, found 1
Backref 151272759296 root 277 owner 32040 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151272759296 root 277 owner 32040
offset 0 found 1 wanted 0 back 0x9f57cb0
backpointer mismatch on [151272759296 4194304]
ref mismatch on [151276953600 4194304] extent item 0, found 1
Backref 151276953600 root 277 owner 32041 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151276953600 root 277 owner 32041
offset 0 found 1 wanted 0 back 0xadc69b0
backpointer mismatch on [151276953600 4194304]
ref mismatch on [151281147904 1966080] extent item 0, found 4
Backref 151281147904 root 277 owner 36213 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151281147904 root 277 owner 36213
offset 0 found 4 wanted 0 back 0x9739f10
backpointer mismatch on [151281147904 1966080]
ref mismatch on [151283113984 2228224] extent item 0, found 6
Backref 151283113984 root 277 owner 36213 offset 1966080 num_refs 0 not
found in extent tree
Incorrect local backref count on 151283113984 root 277 owner 36213
offset 1966080 found 6 wanted 0 back 0xa795710
backpointer mismatch on [151283113984 2228224]
ref mismatch on [151285350400 4194304] extent item 0, found 1
Backref 151285350400 root 277 owner 34547 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151285350400 root 277 owner 34547
offset 0 found 1 wanted 0 back 0xa02d120
backpointer mismatch on [151285350400 4194304]
ref mismatch on [151289552896 4194304] extent item 0, found 1
Backref 151289552896 root 277 owner 32050 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151289552896 root 277 owner 32050
offset 0 found 1 wanted 0 back 0x93309f8
backpointer mismatch on [151289552896 4194304]
ref mismatch on [151293747200 4194304] extent item 0, found 3
Backref 151293747200 root 277 owner 36221 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151293747200 root 277 owner 36221
offset 0 found 3 wanted 0 back 0xabf6e98
backpointer mismatch on [151293747200 4194304]
ref mismatch on [151297941504 4096] extent item 0, found 4
Backref 151297941504 parent 489537536 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151297941504 parent 489537536 owner 0
offset 0 found 1 wanted 0 back 0xb344348
Backref 151297941504 parent 531447808 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151297941504 parent 531447808 owner 0
offset 0 found 1 wanted 0 back 0xa1b0fa0
Backref 151297941504 parent 568475648 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151297941504 parent 568475648 owner 0
offset 0 found 1 wanted 0 back 0xa2ae548
Backref 151297941504 root 277 owner 41529 offset 1716224 num_refs 0 not
found in extent tree
Incorrect local backref count on 151297941504 root 277 owner 41529
offset 1716224 found 1 wanted 0 back 0x9484610
backpointer mismatch on [151297941504 4096]
ref mismatch on [151297949696 4194304] extent item 0, found 1
Backref 151297949696 root 277 owner 32051 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151297949696 root 277 owner 32051
offset 0 found 1 wanted 0 back 0xb069348
backpointer mismatch on [151297949696 4194304]
ref mismatch on [151302144000 4194304] extent item 0, found 5
Backref 151302144000 root 277 owner 36222 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151302144000 root 277 owner 36222
offset 0 found 5 wanted 0 back 0xaff87b0
backpointer mismatch on [151302144000 4194304]
ref mismatch on [151306338304 4194304] extent item 0, found 3
Backref 151306338304 root 277 owner 36223 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151306338304 root 277 owner 36223
offset 0 found 3 wanted 0 back 0x92ef1e8
backpointer mismatch on [151306338304 4194304]
ref mismatch on [151310532608 2818048] extent item 0, found 10
Backref 151310532608 root 277 owner 38278 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151310532608 root 277 owner 38278
offset 0 found 10 wanted 0 back 0xaae0200
backpointer mismatch on [151310532608 2818048]
ref mismatch on [151313350656 90112] extent item 0, found 4
Backref 151313350656 parent 489537536 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313350656 parent 489537536 owner 0
offset 0 found 1 wanted 0 back 0xb344178
Backref 151313350656 parent 531447808 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313350656 parent 531447808 owner 0
offset 0 found 1 wanted 0 back 0x9676670
Backref 151313350656 parent 568475648 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313350656 parent 568475648 owner 0
offset 0 found 1 wanted 0 back 0x9ed5a10
Backref 151313350656 root 277 owner 41529 offset 1622016 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313350656 root 277 owner 41529
offset 1622016 found 1 wanted 0 back 0x94a30b0
backpointer mismatch on [151313350656 90112]
ref mismatch on [151313444864 98304] extent item 0, found 3
Backref 151313444864 root 277 owner 38292 offset 671744 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313444864 root 277 owner 38292
offset 671744 found 3 wanted 0 back 0x9899ce8
backpointer mismatch on [151313444864 98304]
ref mismatch on [151313543168 49152] extent item 0, found 1
Backref 151313543168 root 277 owner 38292 offset 983040 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313543168 root 277 owner 38292
offset 983040 found 1 wanted 0 back 0xaa66ea8
backpointer mismatch on [151313543168 49152]
ref mismatch on [151313592320 16384] extent item 0, found 1
Backref 151313592320 root 277 owner 38141 offset 786432 num_refs 0 not
found in extent tree
Incorrect local backref count on 151313592320 root 277 owner 38141
offset 786432 found 1 wanted 0 back 0xab64720
backpointer mismatch on [151313592320 16384]
ref mismatch on [151313612800 933888] extent item 0, found 7
Backref 151313612800 root 277 owner 38281 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151313612800 root 277 owner 38281
offset 0 found 7 wanted 0 back 0x9fec2c8
backpointer mismatch on [151313612800 933888]
ref mismatch on [151314546688 49152] extent item 0, found 1
Backref 151314546688 root 277 owner 38292 offset 1064960 num_refs 0 not
found in extent tree
Incorrect local backref count on 151314546688 root 277 owner 38292
offset 1064960 found 1 wanted 0 back 0xad2fdb0
backpointer mismatch on [151314546688 49152]
ref mismatch on [151314595840 32768] extent item 0, found 1
Backref 151314595840 root 277 owner 38292 offset 1130496 num_refs 0 not
found in extent tree
Incorrect local backref count on 151314595840 root 277 owner 38292
offset 1130496 found 1 wanted 0 back 0x9a82110
backpointer mismatch on [151314595840 32768]
ref mismatch on [151314628608 49152] extent item 0, found 1
Backref 151314628608 root 277 owner 38292 offset 1179648 num_refs 0 not
found in extent tree
Incorrect local backref count on 151314628608 root 277 owner 38292
offset 1179648 found 1 wanted 0 back 0x91b56c0
backpointer mismatch on [151314628608 49152]
ref mismatch on [151314677760 32768] extent item 0, found 1
Backref 151314677760 root 277 owner 38292 offset 1245184 num_refs 0 not
found in extent tree
Incorrect local backref count on 151314677760 root 277 owner 38292
offset 1245184 found 1 wanted 0 back 0xa5153c0
backpointer mismatch on [151314677760 32768]
ref mismatch on [151314710528 32768] extent item 0, found 1
Backref 151314710528 root 277 owner 38292 offset 1376256 num_refs 0 not
found in extent tree
Incorrect local backref count on 151314710528 root 277 owner 38292
offset 1376256 found 1 wanted 0 back 0xa93e988
backpointer mismatch on [151314710528 32768]
ref mismatch on [151314743296 151552] extent item 0, found 1
Backref 151314743296 root 277 owner 36249 offset 4042752 num_refs 0 not
found in extent tree
Incorrect local backref count on 151314743296 root 277 owner 36249
offset 4042752 found 1 wanted 0 back 0x91e6f08
backpointer mismatch on [151314743296 151552]
ref mismatch on [151314894848 196608] extent item 0, found 2
Backref 151314894848 root 277 owner 36237 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151314894848 root 277 owner 36237
offset 0 found 2 wanted 0 back 0x9020908
backpointer mismatch on [151314894848 196608]
ref mismatch on [151315091456 65536] extent item 0, found 1
Backref 151315091456 root 277 owner 36237 offset 491520 num_refs 0 not
found in extent tree
Incorrect local backref count on 151315091456 root 277 owner 36237
offset 491520 found 1 wanted 0 back 0x9df6298
backpointer mismatch on [151315091456 65536]
ref mismatch on [151315156992 32768] extent item 0, found 1
Backref 151315156992 root 277 owner 36187 offset 1212416 num_refs 0 not
found in extent tree
Incorrect local backref count on 151315156992 root 277 owner 36187
offset 1212416 found 1 wanted 0 back 0x9b82750
backpointer mismatch on [151315156992 32768]
ref mismatch on [151315189760 49152] extent item 0, found 1
Backref 151315189760 root 277 owner 36187 offset 1294336 num_refs 0 not
found in extent tree
Incorrect local backref count on 151315189760 root 277 owner 36187
offset 1294336 found 1 wanted 0 back 0x9b82838
backpointer mismatch on [151315189760 49152]
ref mismatch on [151315238912 16384] extent item 0, found 1
Backref 151315238912 root 277 owner 36195 offset 409600 num_refs 0 not
found in extent tree
Incorrect local backref count on 151315238912 root 277 owner 36195
offset 409600 found 1 wanted 0 back 0x9dd6ab8
backpointer mismatch on [151315238912 16384]
ref mismatch on [151315255296 8192] extent item 0, found 1
Backref 151315255296 root 277 owner 36859 offset 4009984 num_refs 0 not
found in extent tree
Incorrect local backref count on 151315255296 root 277 owner 36859
offset 4009984 found 1 wanted 0 back 0x9e1ccf8
backpointer mismatch on [151315255296 8192]
ref mismatch on [151315267584 4022272] extent item 0, found 1
Backref 151315267584 root 277 owner 34584 offset 172032 num_refs 0 not
found in extent tree
Incorrect local backref count on 151315267584 root 277 owner 34584
offset 172032 found 1 wanted 0 back 0xa3eb7d8
backpointer mismatch on [151315267584 4022272]
ref mismatch on [151319298048 4194304] extent item 0, found 1
Backref 151319298048 root 277 owner 32053 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151319298048 root 277 owner 32053
offset 0 found 1 wanted 0 back 0x92ec170
backpointer mismatch on [151319298048 4194304]
ref mismatch on [151323492352 4194304] extent item 0, found 1
Backref 151323492352 root 277 owner 32052 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151323492352 root 277 owner 32052
offset 0 found 1 wanted 0 back 0x92e1328
backpointer mismatch on [151323492352 4194304]
ref mismatch on [151327686656 4194304] extent item 0, found 1
Backref 151327686656 root 277 owner 32054 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151327686656 root 277 owner 32054
offset 0 found 1 wanted 0 back 0x9072680
backpointer mismatch on [151327686656 4194304]
ref mismatch on [151331880960 4194304] extent item 0, found 1
Backref 151331880960 root 277 owner 32055 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151331880960 root 277 owner 32055
offset 0 found 1 wanted 0 back 0x9078560
backpointer mismatch on [151331880960 4194304]
ref mismatch on [151336075264 4194304] extent item 0, found 1
Backref 151336075264 root 277 owner 34554 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151336075264 root 277 owner 34554
offset 0 found 1 wanted 0 back 0x9d14b38
backpointer mismatch on [151336075264 4194304]
ref mismatch on [151340277760 4194304] extent item 0, found 1
Backref 151340277760 root 277 owner 32056 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151340277760 root 277 owner 32056
offset 0 found 1 wanted 0 back 0xad266a0
backpointer mismatch on [151340277760 4194304]
ref mismatch on [151344472064 4194304] extent item 0, found 1
Backref 151344472064 root 277 owner 34555 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151344472064 root 277 owner 34555
offset 0 found 1 wanted 0 back 0x9b01820
backpointer mismatch on [151344472064 4194304]
ref mismatch on [151348674560 4194304] extent item 0, found 1
Backref 151348674560 root 277 owner 32065 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151348674560 root 277 owner 32065
offset 0 found 1 wanted 0 back 0xa801938
backpointer mismatch on [151348674560 4194304]
ref mismatch on [151352868864 4194304] extent item 0, found 2
Backref 151352868864 root 277 owner 36237 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151352868864 root 277 owner 36237
offset 0 found 2 wanted 0 back 0xa4994b0
backpointer mismatch on [151352868864 4194304]
ref mismatch on [151357063168 2228224] extent item 0, found 1
Backref 151357063168 root 277 owner 36238 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151357063168 root 277 owner 36238
offset 0 found 1 wanted 0 back 0xabb7b48
backpointer mismatch on [151357063168 2228224]
ref mismatch on [151359291392 4194304] extent item 0, found 9
Backref 151359291392 root 277 owner 38292 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151359291392 root 277 owner 38292
offset 0 found 9 wanted 0 back 0x9bb46e8
backpointer mismatch on [151359291392 4194304]
ref mismatch on [151363485696 98304] extent item 0, found 1
Backref 151363485696 root 277 owner 38298 offset 589824 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363485696 root 277 owner 38298
offset 589824 found 1 wanted 0 back 0x9902320
backpointer mismatch on [151363485696 98304]
ref mismatch on [151363584000 32768] extent item 0, found 1
Backref 151363584000 root 277 owner 38298 offset 704512 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363584000 root 277 owner 38298
offset 704512 found 1 wanted 0 back 0x9902408
backpointer mismatch on [151363584000 32768]
ref mismatch on [151363616768 98304] extent item 0, found 1
Backref 151363616768 root 277 owner 38038 offset 589824 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363616768 root 277 owner 38038
offset 589824 found 1 wanted 0 back 0x8fac4b8
backpointer mismatch on [151363616768 98304]
ref mismatch on [151363715072 32768] extent item 0, found 1
Backref 151363715072 root 277 owner 38038 offset 704512 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363715072 root 277 owner 38038
offset 704512 found 1 wanted 0 back 0xb0a7cd8
backpointer mismatch on [151363715072 32768]
ref mismatch on [151363747840 45056] extent item 0, found 1
Backref 151363747840 root 277 owner 38165 offset 4030464 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363747840 root 277 owner 38165
offset 4030464 found 1 wanted 0 back 0xaed2600
backpointer mismatch on [151363747840 45056]
ref mismatch on [151363792896 32768] extent item 0, found 1
Backref 151363792896 root 277 owner 38141 offset 1212416 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363792896 root 277 owner 38141
offset 1212416 found 1 wanted 0 back 0xac24c08
backpointer mismatch on [151363792896 32768]
ref mismatch on [151363825664 32768] extent item 0, found 1
Backref 151363825664 root 277 owner 38141 offset 1261568 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363825664 root 277 owner 38141
offset 1261568 found 1 wanted 0 back 0x90f41f8
backpointer mismatch on [151363825664 32768]
ref mismatch on [151363858432 16384] extent item 0, found 1
Backref 151363858432 root 277 owner 38141 offset 1310720 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363858432 root 277 owner 38141
offset 1310720 found 1 wanted 0 back 0xa65be18
backpointer mismatch on [151363858432 16384]
ref mismatch on [151363874816 16384] extent item 0, found 1
Backref 151363874816 root 277 owner 38141 offset 1343488 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363874816 root 277 owner 38141
offset 1343488 found 1 wanted 0 back 0xa4845a8
backpointer mismatch on [151363874816 16384]
ref mismatch on [151363891200 49152] extent item 0, found 1
Backref 151363891200 root 277 owner 38141 offset 1376256 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363891200 root 277 owner 38141
offset 1376256 found 1 wanted 0 back 0xa5b3f10
backpointer mismatch on [151363891200 49152]
ref mismatch on [151363940352 49152] extent item 0, found 1
Backref 151363940352 root 277 owner 38141 offset 1458176 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363940352 root 277 owner 38141
offset 1458176 found 1 wanted 0 back 0x92f53e8
backpointer mismatch on [151363940352 49152]
ref mismatch on [151363989504 32768] extent item 0, found 1
Backref 151363989504 root 277 owner 38141 offset 1523712 num_refs 0 not
found in extent tree
Incorrect local backref count on 151363989504 root 277 owner 38141
offset 1523712 found 1 wanted 0 back 0x94dacb8
backpointer mismatch on [151363989504 32768]
ref mismatch on [151364022272 16384] extent item 0, found 1
Backref 151364022272 root 277 owner 38141 offset 1572864 num_refs 0 not
found in extent tree
Incorrect local backref count on 151364022272 root 277 owner 38141
offset 1572864 found 1 wanted 0 back 0xa9337d8
backpointer mismatch on [151364022272 16384]
ref mismatch on [151364038656 12288] extent item 0, found 1
Backref 151364038656 root 277 owner 38165 offset 3936256 num_refs 0 not
found in extent tree
Incorrect local backref count on 151364038656 root 277 owner 38165
offset 3936256 found 1 wanted 0 back 0x93548c8
backpointer mismatch on [151364038656 12288]
ref mismatch on [151364050944 405504] extent item 0, found 2
Backref 151364050944 root 277 owner 36859 offset 3723264 num_refs 0 not
found in extent tree
Incorrect local backref count on 151364050944 root 277 owner 36859
offset 3723264 found 2 wanted 0 back 0xb33f640
backpointer mismatch on [151364050944 405504]
ref mismatch on [151364456448 65536] extent item 0, found 1
Backref 151364456448 root 277 owner 36859 offset 4128768 num_refs 0 not
found in extent tree
Incorrect local backref count on 151364456448 root 277 owner 36859
offset 4128768 found 1 wanted 0 back 0x9e22df0
backpointer mismatch on [151364456448 65536]
ref mismatch on [151364521984 294912] extent item 0, found 3
Backref 151364521984 root 277 owner 36972 offset 327680 num_refs 0 not
found in extent tree
Incorrect local backref count on 151364521984 root 277 owner 36972
offset 327680 found 3 wanted 0 back 0x9bfd1a0
backpointer mismatch on [151364521984 294912]
ref mismatch on [151364816896 311296] extent item 0, found 2
Backref 151364816896 root 277 owner 36972 offset 671744 num_refs 0 not
found in extent tree
Incorrect local backref count on 151364816896 root 277 owner 36972
offset 671744 found 2 wanted 0 back 0xa0c6c88
backpointer mismatch on [151364816896 311296]
ref mismatch on [151365136384 32768] extent item 0, found 1
Backref 151365136384 root 277 owner 36250 offset 98304 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365136384 root 277 owner 36250
offset 98304 found 1 wanted 0 back 0xa07a250
backpointer mismatch on [151365136384 32768]
ref mismatch on [151365169152 32768] extent item 0, found 1
Backref 151365169152 root 277 owner 36446 offset 65536 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365169152 root 277 owner 36446
offset 65536 found 1 wanted 0 back 0xa8f0bd8
backpointer mismatch on [151365169152 32768]
ref mismatch on [151365201920 16384] extent item 0, found 1
Backref 151365201920 root 277 owner 36446 offset 131072 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365201920 root 277 owner 36446
offset 131072 found 1 wanted 0 back 0xaed0f10
backpointer mismatch on [151365201920 16384]
ref mismatch on [151365218304 16384] extent item 0, found 1
Backref 151365218304 root 277 owner 36250 offset 1490944 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365218304 root 277 owner 36250
offset 1490944 found 1 wanted 0 back 0x9474a30
backpointer mismatch on [151365218304 16384]
ref mismatch on [151365234688 16384] extent item 0, found 1
Backref 151365234688 root 277 owner 36250 offset 1785856 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365234688 root 277 owner 36250
offset 1785856 found 1 wanted 0 back 0xa8ca350
backpointer mismatch on [151365234688 16384]
ref mismatch on [151365251072 131072] extent item 0, found 1
Backref 151365251072 root 277 owner 36250 offset 2015232 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365251072 root 277 owner 36250
offset 2015232 found 1 wanted 0 back 0xad573b0
backpointer mismatch on [151365251072 131072]
ref mismatch on [151365382144 16384] extent item 0, found 1
Backref 151365382144 root 277 owner 36250 offset 2736128 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365382144 root 277 owner 36250
offset 2736128 found 1 wanted 0 back 0x9838898
backpointer mismatch on [151365382144 16384]
ref mismatch on [151365398528 32768] extent item 0, found 1
Backref 151365398528 root 277 owner 36187 offset 1769472 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365398528 root 277 owner 36187
offset 1769472 found 1 wanted 0 back 0x9b82a08
backpointer mismatch on [151365398528 32768]
ref mismatch on [151365431296 32768] extent item 0, found 1
Backref 151365431296 root 277 owner 36187 offset 2244608 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365431296 root 277 owner 36187
offset 2244608 found 1 wanted 0 back 0x9b82af0
backpointer mismatch on [151365431296 32768]
ref mismatch on [151365464064 12288] extent item 0, found 4
Backref 151365464064 parent 489537536 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365464064 parent 489537536 owner 0
offset 0 found 1 wanted 0 back 0xb344260
Backref 151365464064 parent 531447808 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365464064 parent 531447808 owner 0
offset 0 found 1 wanted 0 back 0xa1b0f58
Backref 151365464064 parent 568475648 owner 0 offset 0 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365464064 parent 568475648 owner 0
offset 0 found 1 wanted 0 back 0xa2ae500
Backref 151365464064 root 277 owner 41529 offset 1708032 num_refs 0 not
found in extent tree
Incorrect local backref count on 151365464064 root 277 owner 41529
offset 1708032 found 1 wanted 0 back 0x94845c8
backpointer mismatch on [151365464064 12288]
ref mismatch on [151365476352 4194304] extent item 0, found 1
Backref 151365476352 root 277 owner 32064 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151365476352 root 277 owner 32064
offset 0 found 1 wanted 0 back 0x91c1888
backpointer mismatch on [151365476352 4194304]
ref mismatch on [151369670656 167936] extent item 0, found 1
Backref 151369670656 root 277 owner 38299 offset 1437696 num_refs 0 not
found in extent tree
Incorrect local backref count on 151369670656 root 277 owner 38299
offset 1437696 found 1 wanted 0 back 0x99039c8
backpointer mismatch on [151369670656 167936]
ref mismatch on [151369838592 81920] extent item 0, found 1
Backref 151369838592 root 277 owner 38306 offset 229376 num_refs 0 not
found in extent tree
Incorrect local backref count on 151369838592 root 277 owner 38306
offset 229376 found 1 wanted 0 back 0x91a5008
backpointer mismatch on [151369838592 81920]
ref mismatch on [151369920512 32768] extent item 0, found 1
Backref 151369920512 root 277 owner 38362 offset 3325952 num_refs 0 not
found in extent tree
Incorrect local backref count on 151369920512 root 277 owner 38362
offset 3325952 found 1 wanted 0 back 0x973aad8
backpointer mismatch on [151369920512 32768]
ref mismatch on [151369965568 65536] extent item 0, found 1
Backref 151369965568 root 277 owner 38163 offset 1474560 num_refs 0 not
found in extent tree
Incorrect local backref count on 151369965568 root 277 owner 38163
offset 1474560 found 1 wanted 0 back 0xb321a78
backpointer mismatch on [151369965568 65536]
ref mismatch on [151370031104 49152] extent item 0, found 1
Backref 151370031104 root 277 owner 38163 offset 1753088 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370031104 root 277 owner 38163
offset 1753088 found 1 wanted 0 back 0xb321b60
backpointer mismatch on [151370031104 49152]
ref mismatch on [151370080256 16384] extent item 0, found 1
Backref 151370080256 root 277 owner 38163 offset 1933312 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370080256 root 277 owner 38163
offset 1933312 found 1 wanted 0 back 0xa6fdba0
backpointer mismatch on [151370080256 16384]
ref mismatch on [151370096640 16384] extent item 0, found 1
Backref 151370096640 root 277 owner 38163 offset 2031616 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370096640 root 277 owner 38163
offset 2031616 found 1 wanted 0 back 0xa6fdc88
backpointer mismatch on [151370096640 16384]
ref mismatch on [151370113024 16384] extent item 0, found 1
Backref 151370113024 root 277 owner 38163 offset 2195456 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370113024 root 277 owner 38163
offset 2195456 found 1 wanted 0 back 0xb345040
backpointer mismatch on [151370113024 16384]
ref mismatch on [151370129408 32768] extent item 0, found 1
Backref 151370129408 root 277 owner 38163 offset 3457024 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370129408 root 277 owner 38163
offset 3457024 found 1 wanted 0 back 0xb345128
backpointer mismatch on [151370129408 32768]
ref mismatch on [151370162176 16384] extent item 0, found 1
Backref 151370162176 root 277 owner 38163 offset 3506176 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370162176 root 277 owner 38163
offset 3506176 found 1 wanted 0 back 0xa5fe538
backpointer mismatch on [151370162176 16384]
ref mismatch on [151370178560 65536] extent item 0, found 1
Backref 151370178560 root 277 owner 38377 offset 1605632 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370178560 root 277 owner 38377
offset 1605632 found 1 wanted 0 back 0x92d9490
backpointer mismatch on [151370178560 65536]
ref mismatch on [151370244096 98304] extent item 0, found 1
Backref 151370244096 root 277 owner 38377 offset 1769472 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370244096 root 277 owner 38377
offset 1769472 found 1 wanted 0 back 0x94891e8
backpointer mismatch on [151370244096 98304]
ref mismatch on [151370342400 32768] extent item 0, found 1
Backref 151370342400 root 277 owner 38377 offset 1900544 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370342400 root 277 owner 38377
offset 1900544 found 1 wanted 0 back 0x949b038
backpointer mismatch on [151370342400 32768]
ref mismatch on [151370375168 32768] extent item 0, found 1
Backref 151370375168 root 277 owner 38377 offset 2277376 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370375168 root 277 owner 38377
offset 2277376 found 1 wanted 0 back 0xa677248
backpointer mismatch on [151370375168 32768]
ref mismatch on [151370407936 81920] extent item 0, found 1
Backref 151370407936 root 277 owner 38386 offset 950272 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370407936 root 277 owner 38386
offset 950272 found 1 wanted 0 back 0xad55d60
backpointer mismatch on [151370407936 81920]
ref mismatch on [151370489856 32768] extent item 0, found 1
Backref 151370489856 root 277 owner 38386 offset 1196032 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370489856 root 277 owner 38386
offset 1196032 found 1 wanted 0 back 0xa8d9a60
backpointer mismatch on [151370489856 32768]
ref mismatch on [151370522624 32768] extent item 0, found 1
Backref 151370522624 root 277 owner 38386 offset 1540096 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370522624 root 277 owner 38386
offset 1540096 found 1 wanted 0 back 0xa8d9ee8
backpointer mismatch on [151370522624 32768]
ref mismatch on [151370555392 32768] extent item 0, found 1
Backref 151370555392 root 277 owner 38386 offset 1654784 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370555392 root 277 owner 38386
offset 1654784 found 1 wanted 0 back 0xa9cc850
backpointer mismatch on [151370555392 32768]
ref mismatch on [151370588160 49152] extent item 0, found 1
Backref 151370588160 root 277 owner 38386 offset 1703936 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370588160 root 277 owner 38386
offset 1703936 found 1 wanted 0 back 0xa9cc938
backpointer mismatch on [151370588160 49152]
ref mismatch on [151370637312 49152] extent item 0, found 1
Backref 151370637312 root 277 owner 38386 offset 1785856 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370637312 root 277 owner 38386
offset 1785856 found 1 wanted 0 back 0xad56100
backpointer mismatch on [151370637312 49152]
ref mismatch on [151370686464 32768] extent item 0, found 1
Backref 151370686464 root 277 owner 38383 offset 2965504 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370686464 root 277 owner 38383
offset 2965504 found 1 wanted 0 back 0x9a5dcc8
backpointer mismatch on [151370686464 32768]
ref mismatch on [151370719232 32768] extent item 0, found 1
Backref 151370719232 root 277 owner 38383 offset 3014656 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370719232 root 277 owner 38383
offset 3014656 found 1 wanted 0 back 0xa9cbf40
backpointer mismatch on [151370719232 32768]
ref mismatch on [151370752000 81920] extent item 0, found 1
Backref 151370752000 root 277 owner 38387 offset 950272 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370752000 root 277 owner 38387
offset 950272 found 1 wanted 0 back 0xad56840
backpointer mismatch on [151370752000 81920]
ref mismatch on [151370833920 32768] extent item 0, found 1
Backref 151370833920 root 277 owner 38387 offset 1196032 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370833920 root 277 owner 38387
offset 1196032 found 1 wanted 0 back 0xa8d4d58
backpointer mismatch on [151370833920 32768]
ref mismatch on [151370866688 32768] extent item 0, found 1
Backref 151370866688 root 277 owner 38387 offset 1540096 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370866688 root 277 owner 38387
offset 1540096 found 1 wanted 0 back 0xaddcb10
backpointer mismatch on [151370866688 32768]
ref mismatch on [151370899456 32768] extent item 0, found 1
Backref 151370899456 root 277 owner 38387 offset 1654784 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370899456 root 277 owner 38387
offset 1654784 found 1 wanted 0 back 0xaaa6180
backpointer mismatch on [151370899456 32768]
ref mismatch on [151370932224 24576] extent item 0, found 1
Backref 151370932224 root 277 owner 38165 offset 3186688 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370932224 root 277 owner 38165
offset 3186688 found 1 wanted 0 back 0x9d48648
backpointer mismatch on [151370932224 24576]
ref mismatch on [151370956800 16384] extent item 0, found 1
Backref 151370956800 root 277 owner 38124 offset 4177920 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370956800 root 277 owner 38124
offset 4177920 found 1 wanted 0 back 0x92872e0
backpointer mismatch on [151370956800 16384]
ref mismatch on [151370981376 16384] extent item 0, found 1
Backref 151370981376 root 277 owner 38141 offset 1605632 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370981376 root 277 owner 38141
offset 1605632 found 1 wanted 0 back 0x968f768
backpointer mismatch on [151370981376 16384]
ref mismatch on [151370997760 16384] extent item 0, found 1
Backref 151370997760 root 277 owner 38141 offset 1638400 num_refs 0 not
found in extent tree
Incorrect local backref count on 151370997760 root 277 owner 38141
offset 1638400 found 1 wanted 0 back 0x9957e48
backpointer mismatch on [151370997760 16384]
ref mismatch on [151371014144 32768] extent item 0, found 1
Backref 151371014144 root 277 owner 38141 offset 1671168 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371014144 root 277 owner 38141
offset 1671168 found 1 wanted 0 back 0x9408878
backpointer mismatch on [151371014144 32768]
ref mismatch on [151371046912 49152] extent item 0, found 1
Backref 151371046912 root 277 owner 38141 offset 1720320 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371046912 root 277 owner 38141
offset 1720320 found 1 wanted 0 back 0x9ca09f8
backpointer mismatch on [151371046912 49152]
ref mismatch on [151371096064 16384] extent item 0, found 1
Backref 151371096064 root 277 owner 38141 offset 2211840 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371096064 root 277 owner 38141
offset 2211840 found 1 wanted 0 back 0xad785c8
backpointer mismatch on [151371096064 16384]
ref mismatch on [151371112448 32768] extent item 0, found 1
Backref 151371112448 root 277 owner 38141 offset 2408448 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371112448 root 277 owner 38141
offset 2408448 found 1 wanted 0 back 0x94caaa8
backpointer mismatch on [151371112448 32768]
ref mismatch on [151371145216 32768] extent item 0, found 1
Backref 151371145216 root 277 owner 38389 offset 2588672 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371145216 root 277 owner 38389
offset 2588672 found 1 wanted 0 back 0x9a46898
backpointer mismatch on [151371145216 32768]
ref mismatch on [151371177984 16384] extent item 0, found 1
Backref 151371177984 root 277 owner 38141 offset 2719744 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371177984 root 277 owner 38141
offset 2719744 found 1 wanted 0 back 0x9c98c48
backpointer mismatch on [151371177984 16384]
ref mismatch on [151371194368 16384] extent item 0, found 1
Backref 151371194368 root 277 owner 38141 offset 3276800 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371194368 root 277 owner 38141
offset 3276800 found 1 wanted 0 back 0x9090dc8
backpointer mismatch on [151371194368 16384]
ref mismatch on [151371210752 16384] extent item 0, found 1
Backref 151371210752 root 277 owner 38141 offset 3932160 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371210752 root 277 owner 38141
offset 3932160 found 1 wanted 0 back 0xaa97338
backpointer mismatch on [151371210752 16384]
ref mismatch on [151371227136 16384] extent item 0, found 1
Backref 151371227136 root 277 owner 38142 offset 786432 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371227136 root 277 owner 38142
offset 786432 found 1 wanted 0 back 0xafa77c8
backpointer mismatch on [151371227136 16384]
ref mismatch on [151371243520 49152] extent item 0, found 1
Backref 151371243520 root 277 owner 38142 offset 147456 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371243520 root 277 owner 38142
offset 147456 found 1 wanted 0 back 0x9dd2b48
backpointer mismatch on [151371243520 49152]
ref mismatch on [151371292672 16384] extent item 0, found 1
Backref 151371292672 root 277 owner 38181 offset 0 num_refs 0 not found
in extent tree
Incorrect local backref count on 151371292672 root 277 owner 38181
offset 0 found 1 wanted 0 back 0xb4346c8
backpointer mismatch on [151371292672 16384]
ref mismatch on [151371309056 65536] extent item 0, found 1
Backref 151371309056 root 277 owner 38181 offset 32768 num_refs 0 not
found in extent tree
Incorrect local backref count on 151371309056 root 277 owner 38181
offset 32768 found 1 wanted 0 back 0xa88ec40
backpointer mismatch on [151371309056 65536]
Errors found in extent allocation tree or chunk allocation
checksum verify failed on 588447744 found 6E0D3115 wanted F90C810B
checksum verify failed on 588447744 found 6E0D3115 wanted F90C810B
checksum verify failed on 588447744 found 24492BB3 wanted F90C810B
checksum verify failed on 588447744 found 6E0D3115 wanted F90C810B
Csum didn't match
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BTRFS: Transaction aborted (error -5)
2015-01-06 15:04 ` Dyweni - BTRFS
@ 2015-01-06 15:15 ` Dyweni - BTRFS
2015-01-08 15:02 ` Dyweni - BTRFS
0 siblings, 1 reply; 6+ messages in thread
From: Dyweni - BTRFS @ 2015-01-06 15:15 UTC (permalink / raw)
To: Chris Murphy; +Cc: Btrfs BTRFS, chris, linux-btrfs-owner
Hi,
BTRFS check on /dev/sdc1 reveals everything looks ok:
# btrfs check /dev/sdc1
Checking filesystem on /dev/sdc1
UUID: 26ed1033-429a-444f-97cc-ce8103db4c39
checking extents
checking free space cache
checking fs roots
checking csums
checking root refs
found 195515710524 bytes used err is 0
total csum bytes: 205915200
total tree bytes: 407355392
total fs tree bytes: 94830592
total extent tree bytes: 31588352
btree space waste bytes: 100867438
file data blocks allocated: 537492316160
referenced 195656101888
Btrfs v3.18
(/dev/sdd1 and /dev/sdc1 are the only two btrfs file systems in this
machine).
Oddly, when the problem with /dev/sdd1 started, problems with /dev/sdc1
were
also reported, but /dev/sdc1 managed to fix itself.
Below is the complete dmesg output from when problems first started
until /dev/sdd1 went readonly with errors.
The strangest part of all of this, is that the dmesg output shows no
errors about the drive being physically bad.
(I ran badblocks -nsv on both /dev/sdd and /dev/sdc, and it confirmed 0
bad blocks for both drives).
[25581.099684] BTRFS: sdd1 checksum verify failed on 521797632 wanted
8F2F5FEC found 3E879EFE level 0
[25581.105441] BTRFS: read error corrected: ino 1 off 521797632 (dev
/dev/sdd1 sector 1035520)
[25581.105612] BTRFS: read error corrected: ino 1 off 521801728 (dev
/dev/sdd1 sector 1035528)
[25581.105784] BTRFS: read error corrected: ino 1 off 521805824 (dev
/dev/sdd1 sector 1035536)
[25581.105956] BTRFS: read error corrected: ino 1 off 521809920 (dev
/dev/sdd1 sector 1035544)
[27777.799514] BTRFS: sdd1 checksum verify failed on 680296448 wanted
AB0E191F found 192D4134 level 0
[27777.856199] BTRFS: read error corrected: ino 1 off 680296448 (dev
/dev/sdd1 sector 1345088)
[27777.860571] BTRFS: read error corrected: ino 1 off 680300544 (dev
/dev/sdd1 sector 1345096)
[27777.909634] BTRFS: read error corrected: ino 1 off 680304640 (dev
/dev/sdd1 sector 1345104)
[27777.909876] BTRFS: read error corrected: ino 1 off 680308736 (dev
/dev/sdd1 sector 1345112)
[29292.777237] BTRFS: sdc1 checksum verify failed on 937738240 wanted
F4196CDA found AF30B394 level 0
[29292.778022] BTRFS: sdc1 checksum verify failed on 937738240 wanted
F4196CDA found AF30B394 level 0
[29292.781889] BTRFS: read error corrected: ino 1 off 937738240 (dev
/dev/sdc1 sector 1847904)
[29292.782054] BTRFS: read error corrected: ino 1 off 937742336 (dev
/dev/sdc1 sector 1847912)
[29292.782224] BTRFS: read error corrected: ino 1 off 937746432 (dev
/dev/sdc1 sector 1847920)
[29292.782399] BTRFS: read error corrected: ino 1 off 937750528 (dev
/dev/sdc1 sector 1847928)
[29691.731107] BTRFS: sdd1 checksum verify failed on 610877440 wanted
5A8006E7 found 1CFE4A20 level 0
[29691.791550] BTRFS: read error corrected: ino 1 off 610877440 (dev
/dev/sdd1 sector 1209504)
[29691.793252] BTRFS: read error corrected: ino 1 off 610881536 (dev
/dev/sdd1 sector 1209512)
[29691.793608] BTRFS: read error corrected: ino 1 off 610885632 (dev
/dev/sdd1 sector 1209520)
[29691.793797] BTRFS: read error corrected: ino 1 off 610889728 (dev
/dev/sdd1 sector 1209528)
[34626.017914] BTRFS: sdd1 checksum verify failed on 737181696 wanted
15D7099D found B6A2A7A9 level 0
[34626.022656] BTRFS: read error corrected: ino 1 off 737181696 (dev
/dev/sdd1 sector 1456192)
[34626.022867] BTRFS: read error corrected: ino 1 off 737185792 (dev
/dev/sdd1 sector 1456200)
[34626.023107] BTRFS: read error corrected: ino 1 off 737189888 (dev
/dev/sdd1 sector 1456208)
[34626.023314] BTRFS: read error corrected: ino 1 off 737193984 (dev
/dev/sdd1 sector 1456216)
[37057.349996] BTRFS: sdc1 checksum verify failed on 701792256 wanted
A7BD5067 found 87EF0602 level 0
[37057.424920] BTRFS: read error corrected: ino 1 off 701792256 (dev
/dev/sdc1 sector 1387072)
[37057.425178] BTRFS: read error corrected: ino 1 off 701796352 (dev
/dev/sdc1 sector 1387080)
[37057.450174] BTRFS: read error corrected: ino 1 off 701800448 (dev
/dev/sdc1 sector 1387088)
[37057.453476] BTRFS: read error corrected: ino 1 off 701804544 (dev
/dev/sdc1 sector 1387096)
[38283.714855] BTRFS: sdd1 checksum verify failed on 190169088 wanted
27D1E032 found 585B1651 level 0
[38283.715349] BTRFS: sdd1 checksum verify failed on 190169088 wanted
27D1E032 found 585B1651 level 0
[38283.724140] BTRFS: read error corrected: ino 1 off 190169088 (dev
/dev/sdd1 sector 387808)
[38283.724313] BTRFS: read error corrected: ino 1 off 190173184 (dev
/dev/sdd1 sector 387816)
[38283.724485] BTRFS: read error corrected: ino 1 off 190177280 (dev
/dev/sdd1 sector 387824)
[38283.724648] BTRFS: read error corrected: ino 1 off 190181376 (dev
/dev/sdd1 sector 387832)
[38385.874438] BTRFS: sdd1 checksum verify failed on 472825856 wanted
937078F5 found 7FCB4F87 level 0
[38385.897113] BTRFS: read error corrected: ino 1 off 472825856 (dev
/dev/sdd1 sector 939872)
[38385.897336] BTRFS: read error corrected: ino 1 off 472829952 (dev
/dev/sdd1 sector 939880)
[38385.897567] BTRFS: read error corrected: ino 1 off 472834048 (dev
/dev/sdd1 sector 939888)
[38385.897790] BTRFS: read error corrected: ino 1 off 472838144 (dev
/dev/sdd1 sector 939896)
[39015.821873] BTRFS: sdc1 checksum verify failed on 443613184 wanted
302CD9DE found ED697366 level 0
[39015.823523] BTRFS: read error corrected: ino 1 off 443613184 (dev
/dev/sdc1 sector 882816)
[39015.823755] BTRFS: read error corrected: ino 1 off 443617280 (dev
/dev/sdc1 sector 882824)
[39015.824015] BTRFS: read error corrected: ino 1 off 443621376 (dev
/dev/sdc1 sector 882832)
[39015.824273] BTRFS: read error corrected: ino 1 off 443625472 (dev
/dev/sdc1 sector 882840)
[41213.535718] BTRFS: sdd1 checksum verify failed on 397557760 wanted
EBCB7405 found C1365A5E level 0
[41213.579520] BTRFS: read error corrected: ino 1 off 397557760 (dev
/dev/sdd1 sector 792864)
[41213.579687] BTRFS: read error corrected: ino 1 off 397561856 (dev
/dev/sdd1 sector 792872)
[41213.579913] BTRFS: read error corrected: ino 1 off 397565952 (dev
/dev/sdd1 sector 792880)
[41213.580062] BTRFS: read error corrected: ino 1 off 397570048 (dev
/dev/sdd1 sector 792888)
[41535.669279] BTRFS: sdc1 checksum verify failed on 476348416 wanted
A645D195 found 21AD02F8 level 0
[41535.805928] BTRFS: read error corrected: ino 1 off 476348416 (dev
/dev/sdc1 sector 946752)
[41535.807125] BTRFS: read error corrected: ino 1 off 476352512 (dev
/dev/sdc1 sector 946760)
[41535.808775] BTRFS: read error corrected: ino 1 off 476356608 (dev
/dev/sdc1 sector 946768)
[41535.810561] BTRFS: read error corrected: ino 1 off 476360704 (dev
/dev/sdc1 sector 946776)
[42148.710341] BTRFS: sdc1 checksum verify failed on 807469056 wanted
8490C0D5 found FB751835 level 0
[42148.771176] BTRFS: sdc1 checksum verify failed on 807469056 wanted
8490C0D5 found FB751835 level 0
[42148.773574] BTRFS: read error corrected: ino 1 off 807469056 (dev
/dev/sdc1 sector 1593472)
[42148.780409] BTRFS: read error corrected: ino 1 off 807473152 (dev
/dev/sdc1 sector 1593480)
[42148.784868] BTRFS: read error corrected: ino 1 off 807477248 (dev
/dev/sdc1 sector 1593488)
[42148.785629] BTRFS: read error corrected: ino 1 off 807481344 (dev
/dev/sdc1 sector 1593496)
[42263.866339] BTRFS: sdc1 checksum verify failed on 1019822080 wanted
EE99B968 found 42D4359B level 0
[42263.867150] BTRFS: read error corrected: ino 1 off 1019822080 (dev
/dev/sdc1 sector 2008224)
[42263.867371] BTRFS: read error corrected: ino 1 off 1019826176 (dev
/dev/sdc1 sector 2008232)
[42263.867651] BTRFS: read error corrected: ino 1 off 1019830272 (dev
/dev/sdc1 sector 2008240)
[42263.867874] BTRFS: read error corrected: ino 1 off 1019834368 (dev
/dev/sdc1 sector 2008248)
[43510.239652] BTRFS: sdd1 checksum verify failed on 561528832 wanted
8E3D9918 found 462F00F8 level 0
[43510.277730] BTRFS: sdd1 checksum verify failed on 561528832 wanted
8E3D9918 found 462F00F8 level 0
[43510.284768] BTRFS: read error corrected: ino 1 off 561528832 (dev
/dev/sdd1 sector 1113120)
[43510.284951] BTRFS: read error corrected: ino 1 off 561532928 (dev
/dev/sdd1 sector 1113128)
[43510.285120] BTRFS: read error corrected: ino 1 off 561537024 (dev
/dev/sdd1 sector 1113136)
[43510.285292] BTRFS: read error corrected: ino 1 off 561541120 (dev
/dev/sdd1 sector 1113144)
[43665.199768] BTRFS: sdd1 checksum verify failed on 912867328 wanted
322FCD22 found E3B80F9F level 0
[43665.204858] BTRFS: read error corrected: ino 1 off 912867328 (dev
/dev/sdd1 sector 1799328)
[43665.205055] BTRFS: read error corrected: ino 1 off 912871424 (dev
/dev/sdd1 sector 1799336)
[43665.205249] BTRFS: read error corrected: ino 1 off 912875520 (dev
/dev/sdd1 sector 1799344)
[43665.205431] BTRFS: read error corrected: ino 1 off 912879616 (dev
/dev/sdd1 sector 1799352)
[44830.868804] BTRFS: sdc1 checksum verify failed on 177061888 wanted
91211FC3 found 9ED5C83C level 0
[44830.881691] BTRFS: read error corrected: ino 1 off 177061888 (dev
/dev/sdc1 sector 362208)
[44830.881890] BTRFS: read error corrected: ino 1 off 177065984 (dev
/dev/sdc1 sector 362216)
[44830.882090] BTRFS: read error corrected: ino 1 off 177070080 (dev
/dev/sdc1 sector 362224)
[44830.882271] BTRFS: read error corrected: ino 1 off 177074176 (dev
/dev/sdc1 sector 362232)
[45891.249508] BTRFS: sdd1 checksum verify failed on 738902016 wanted
39A87C68 found 26F7DDC0 level 0
[45891.250573] BTRFS: sdd1 checksum verify failed on 738902016 wanted
39A87C68 found 26F7DDC0 level 0
[45891.256723] BTRFS: read error corrected: ino 1 off 738902016 (dev
/dev/sdd1 sector 1459552)
[45891.256930] BTRFS: read error corrected: ino 1 off 738906112 (dev
/dev/sdd1 sector 1459560)
[45891.257149] BTRFS: read error corrected: ino 1 off 738910208 (dev
/dev/sdd1 sector 1459568)
[45891.257354] BTRFS: read error corrected: ino 1 off 738914304 (dev
/dev/sdd1 sector 1459576)
[46483.180506] BTRFS: sdd1 checksum verify failed on 629964800 wanted
4925982F found FDC8E201 level 0
[46484.513724] BTRFS: sdd1 checksum verify failed on 629964800 wanted
4925982F found FDC8E201 level 0
[46484.531887] BTRFS: read error corrected: ino 1 off 629964800 (dev
/dev/sdd1 sector 1246784)
[46484.532272] BTRFS: read error corrected: ino 1 off 629968896 (dev
/dev/sdd1 sector 1246792)
[46484.532435] BTRFS: read error corrected: ino 1 off 629972992 (dev
/dev/sdd1 sector 1246800)
[46484.532606] BTRFS: read error corrected: ino 1 off 629977088 (dev
/dev/sdd1 sector 1246808)
[47018.575521] BTRFS: sdd1 checksum verify failed on 1057505280 wanted
73DE9ACE found CCB9B94F level 0
[47018.575961] BTRFS: sdd1 checksum verify failed on 1057505280 wanted
73DE9ACE found CCB9B94F level 0
[47018.587562] BTRFS: read error corrected: ino 1 off 1057505280 (dev
/dev/sdd1 sector 2081824)
[47018.587727] BTRFS: read error corrected: ino 1 off 1057509376 (dev
/dev/sdd1 sector 2081832)
[47018.587985] BTRFS: read error corrected: ino 1 off 1057513472 (dev
/dev/sdd1 sector 2081840)
[47018.588171] BTRFS: read error corrected: ino 1 off 1057517568 (dev
/dev/sdd1 sector 2081848)
[47093.159882] BTRFS: sdd1 checksum verify failed on 355254272 wanted
6CCD79A6 found D82B8A63 level 0
[47120.658821] BTRFS: sdd1 checksum verify failed on 355254272 wanted
6CCD79A6 found D82B8A63 level 0
[47120.668197] BTRFS: read error corrected: ino 1 off 355254272 (dev
/dev/sdd1 sector 710240)
[47120.668385] BTRFS: read error corrected: ino 1 off 355258368 (dev
/dev/sdd1 sector 710248)
[47120.668537] BTRFS: read error corrected: ino 1 off 355262464 (dev
/dev/sdd1 sector 710256)
[47120.668707] BTRFS: read error corrected: ino 1 off 355266560 (dev
/dev/sdd1 sector 710264)
[47187.793809] BTRFS: sdd1 checksum verify failed on 610385920 wanted
8942E6D9 found 974441F7 level 0
[47187.803132] BTRFS: read error corrected: ino 1 off 610385920 (dev
/dev/sdd1 sector 1208544)
[47187.803321] BTRFS: read error corrected: ino 1 off 610390016 (dev
/dev/sdd1 sector 1208552)
[47187.803500] BTRFS: read error corrected: ino 1 off 610394112 (dev
/dev/sdd1 sector 1208560)
[47187.804405] BTRFS: read error corrected: ino 1 off 610398208 (dev
/dev/sdd1 sector 1208568)
[47293.372124] BTRFS: sdd1 checksum verify failed on 30064640 wanted
E532B4CB found 1F6BB1CF level 0
[47293.392624] BTRFS: read error corrected: ino 1 off 30064640 (dev
/dev/sdd1 sector 75104)
[47293.392931] BTRFS: read error corrected: ino 1 off 30068736 (dev
/dev/sdd1 sector 75112)
[47293.393143] BTRFS: read error corrected: ino 1 off 30072832 (dev
/dev/sdd1 sector 75120)
[47293.393367] BTRFS: read error corrected: ino 1 off 30076928 (dev
/dev/sdd1 sector 75128)
[47415.432548] BTRFS: sdc1 checksum verify failed on 483311616 wanted
2337659C found FE72CF24 level 0
[47415.806725] BTRFS: sdc1 checksum verify failed on 483311616 wanted
2337659C found FE72CF24 level 0
[47415.816027] BTRFS: read error corrected: ino 1 off 483311616 (dev
/dev/sdc1 sector 960352)
[47415.816199] BTRFS: read error corrected: ino 1 off 483315712 (dev
/dev/sdc1 sector 960360)
[47415.816372] BTRFS: read error corrected: ino 1 off 483319808 (dev
/dev/sdc1 sector 960368)
[47415.816838] BTRFS: read error corrected: ino 1 off 483323904 (dev
/dev/sdc1 sector 960376)
[47593.204118] BTRFS: sdd1 checksum verify failed on 131497984 wanted
C1B6312B found E5650C5B level 0
[47593.212026] BTRFS: read error corrected: ino 1 off 131497984 (dev
/dev/sdd1 sector 273216)
[47593.212182] BTRFS: read error corrected: ino 1 off 131502080 (dev
/dev/sdd1 sector 273224)
[47593.212355] BTRFS: read error corrected: ino 1 off 131506176 (dev
/dev/sdd1 sector 273232)
[47593.212526] BTRFS: read error corrected: ino 1 off 131510272 (dev
/dev/sdd1 sector 273240)
[48026.762834] BTRFS: sdc1 checksum verify failed on 856752128 wanted
640450F found B927668E level 0
[48026.802163] BTRFS: read error corrected: ino 1 off 856752128 (dev
/dev/sdc1 sector 1689728)
[48026.802810] BTRFS: read error corrected: ino 1 off 856756224 (dev
/dev/sdc1 sector 1689736)
[48026.803040] BTRFS: read error corrected: ino 1 off 856760320 (dev
/dev/sdc1 sector 1689744)
[48026.803240] BTRFS: read error corrected: ino 1 off 856764416 (dev
/dev/sdc1 sector 1689752)
[48157.352847] BTRFS: sdd1 checksum verify failed on 721502208 wanted
F58E06B7 found 28CBAC0F level 0
[48157.363809] BTRFS: read error corrected: ino 1 off 721502208 (dev
/dev/sdd1 sector 1425568)
[48157.364175] BTRFS: read error corrected: ino 1 off 721506304 (dev
/dev/sdd1 sector 1425576)
[48157.364341] BTRFS: read error corrected: ino 1 off 721510400 (dev
/dev/sdd1 sector 1425584)
[48157.364559] BTRFS: read error corrected: ino 1 off 721514496 (dev
/dev/sdd1 sector 1425592)
[48717.558596] BTRFS: sdd1 checksum verify failed on 425967616 wanted
8B488C01 found 52F6044C level 0
[48717.576610] BTRFS: read error corrected: ino 1 off 425967616 (dev
/dev/sdd1 sector 848352)
[48717.576856] BTRFS: read error corrected: ino 1 off 425971712 (dev
/dev/sdd1 sector 848360)
[48717.577651] BTRFS: read error corrected: ino 1 off 425975808 (dev
/dev/sdd1 sector 848368)
[48717.577877] BTRFS: read error corrected: ino 1 off 425979904 (dev
/dev/sdd1 sector 848376)
[48740.429310] BTRFS: sdd1 checksum verify failed on 570064896 wanted
4B343BE0 found BD92455F level 0
[48740.437713] BTRFS: read error corrected: ino 1 off 570064896 (dev
/dev/sdd1 sector 1129792)
[48740.437935] BTRFS: read error corrected: ino 1 off 570068992 (dev
/dev/sdd1 sector 1129800)
[48740.438098] BTRFS: read error corrected: ino 1 off 570073088 (dev
/dev/sdd1 sector 1129808)
[48740.438271] BTRFS: read error corrected: ino 1 off 570077184 (dev
/dev/sdd1 sector 1129816)
[48746.809311] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[48746.810381] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 6E0D3115 level 0
[48746.819699] BTRFS: sdd1 checksum verify failed on 588447744 wanted
F90C810B found 24492BB3 level 0
[48746.819733] ------------[ cut here ]------------
[48746.819746] WARNING: CPU: 0 PID: 31802 at fs/btrfs/super.c:259
__btrfs_abort_transaction+0x43/0x110()
[48746.819747] BTRFS: Transaction aborted (error -5)
[48746.819748] Modules linked in: iscsi_trgt(O)
[48746.819755] CPU: 0 PID: 31802 Comm: kworker/u2:11 Tainted: G
O 3.16.5-gentoo-20150103-0758 #1
[48746.819757] Hardware name: /MS-6570, BIOS 6.00 PG 11/07/2003
[48746.819762] Workqueue: btrfs-extent-refs btrfs_extent_refs_helper
[48746.819764] c9a9fe74 c9a9fe74 c9a9fe34 c14d581f c9a9fe64 c10333d0
c15bcea0 c9a9fe90
[48746.819767] 00007c3a c15c9417 00000103 c11c8723 c11c8723 fffffffb
f4c1a000 c14fd856
[48746.819771] c9a9fe7c c103341e 00000009 c9a9fe74 c15bcea0 c9a9fe90
c9a9fea8 c11c8723
[48746.819775] Call Trace:
[48746.819783] [<c14d581f>] dump_stack+0x16/0x18
[48746.819787] [<c10333d0>] warn_slowpath_common+0x70/0x90
[48746.819791] [<c11c8723>] ? __btrfs_abort_transaction+0x43/0x110
[48746.819793] [<c11c8723>] ? __btrfs_abort_transaction+0x43/0x110
[48746.819796] [<c103341e>] warn_slowpath_fmt+0x2e/0x30
[48746.819799] [<c11c8723>] __btrfs_abort_transaction+0x43/0x110
[48746.819805] [<c11e2cc4>] btrfs_run_delayed_refs.part.74+0xd4/0x1d0
[48746.819808] [<c11e2e4d>] delayed_ref_async_start+0x6d/0x80
[48746.819810] [<c1222558>] normal_work_helper+0x88/0x120
[48746.819813] [<c12227db>] btrfs_extent_refs_helper+0xb/0x10
[48746.819816] [<c1044081>] process_one_work+0xd1/0x300
[48746.819819] [<c1044779>] worker_thread+0xe9/0x430
[48746.819821] [<c1044690>] ? create_and_start_worker+0x40/0x40
[48746.819823] [<c1044690>] ? create_and_start_worker+0x40/0x40
[48746.819828] [<c1048cf5>] kthread+0x95/0xb0
[48746.819831] [<c14da0c0>] ret_from_kernel_thread+0x20/0x30
[48746.819834] [<c1048c60>] ? kthread_worker_fn+0xb0/0xb0
[48746.819835] ---[ end trace 554ae9ecb6239dc6 ]---
[48746.819838] BTRFS: error (device sdd1) in
btrfs_run_delayed_refs:2806: errno=-5 IO failure
[48746.819840] BTRFS info (device sdd1): forced readonly
^ permalink raw reply [flat|nested] 6+ messages in thread
* Re: BTRFS: Transaction aborted (error -5)
2015-01-06 15:15 ` Dyweni - BTRFS
@ 2015-01-08 15:02 ` Dyweni - BTRFS
0 siblings, 0 replies; 6+ messages in thread
From: Dyweni - BTRFS @ 2015-01-08 15:02 UTC (permalink / raw)
To: Btrfs BTRFS; +Cc: Chris Murphy, chris, linux-btrfs-owner
Hi All,
Just wondering if someone could help me figure out what happened to this
file system? As you see below, the kernel reports btrfs errors on both
drives, but was able to correct the errors on the first drive. The
second drive not so lucky. However, running 'badblocks -nsv' (the
non-destructive read-write test) on both drives shows both drive free of
bad blocks.
Both drives are 1TB 7200RPM spinners. And, as Chris indicated earlier,
its strange that the file system gets different check sums for the same
block...
Thanks,
Dyweni
On 2015-01-06 09:15, Dyweni - BTRFS wrote:
> Hi,
>
> BTRFS check on /dev/sdc1 reveals everything looks ok:
>
> # btrfs check /dev/sdc1
> Checking filesystem on /dev/sdc1
> UUID: 26ed1033-429a-444f-97cc-ce8103db4c39
> checking extents
> checking free space cache
> checking fs roots
> checking csums
> checking root refs
> found 195515710524 bytes used err is 0
> total csum bytes: 205915200
> total tree bytes: 407355392
> total fs tree bytes: 94830592
> total extent tree bytes: 31588352
> btree space waste bytes: 100867438
> file data blocks allocated: 537492316160
> referenced 195656101888
> Btrfs v3.18
>
>
>
>
> (/dev/sdd1 and /dev/sdc1 are the only two btrfs file systems in this
> machine).
>
>
>
> Oddly, when the problem with /dev/sdd1 started, problems with /dev/sdc1
> were
> also reported, but /dev/sdc1 managed to fix itself.
>
> Below is the complete dmesg output from when problems first started
> until /dev/sdd1 went readonly with errors.
>
> The strangest part of all of this, is that the dmesg output shows no
> errors about the drive being physically bad.
> (I ran badblocks -nsv on both /dev/sdd and /dev/sdc, and it confirmed
> 0 bad blocks for both drives).
>
>
>
>
> [25581.099684] BTRFS: sdd1 checksum verify failed on 521797632 wanted
> 8F2F5FEC found 3E879EFE level 0
> [25581.105441] BTRFS: read error corrected: ino 1 off 521797632 (dev
> /dev/sdd1 sector 1035520)
> [25581.105612] BTRFS: read error corrected: ino 1 off 521801728 (dev
> /dev/sdd1 sector 1035528)
> [25581.105784] BTRFS: read error corrected: ino 1 off 521805824 (dev
> /dev/sdd1 sector 1035536)
> [25581.105956] BTRFS: read error corrected: ino 1 off 521809920 (dev
> /dev/sdd1 sector 1035544)
> [27777.799514] BTRFS: sdd1 checksum verify failed on 680296448 wanted
> AB0E191F found 192D4134 level 0
> [27777.856199] BTRFS: read error corrected: ino 1 off 680296448 (dev
> /dev/sdd1 sector 1345088)
> [27777.860571] BTRFS: read error corrected: ino 1 off 680300544 (dev
> /dev/sdd1 sector 1345096)
> [27777.909634] BTRFS: read error corrected: ino 1 off 680304640 (dev
> /dev/sdd1 sector 1345104)
> [27777.909876] BTRFS: read error corrected: ino 1 off 680308736 (dev
> /dev/sdd1 sector 1345112)
> [29292.777237] BTRFS: sdc1 checksum verify failed on 937738240 wanted
> F4196CDA found AF30B394 level 0
> [29292.778022] BTRFS: sdc1 checksum verify failed on 937738240 wanted
> F4196CDA found AF30B394 level 0
> [29292.781889] BTRFS: read error corrected: ino 1 off 937738240 (dev
> /dev/sdc1 sector 1847904)
> [29292.782054] BTRFS: read error corrected: ino 1 off 937742336 (dev
> /dev/sdc1 sector 1847912)
> [29292.782224] BTRFS: read error corrected: ino 1 off 937746432 (dev
> /dev/sdc1 sector 1847920)
> [29292.782399] BTRFS: read error corrected: ino 1 off 937750528 (dev
> /dev/sdc1 sector 1847928)
> [29691.731107] BTRFS: sdd1 checksum verify failed on 610877440 wanted
> 5A8006E7 found 1CFE4A20 level 0
> [29691.791550] BTRFS: read error corrected: ino 1 off 610877440 (dev
> /dev/sdd1 sector 1209504)
> [29691.793252] BTRFS: read error corrected: ino 1 off 610881536 (dev
> /dev/sdd1 sector 1209512)
> [29691.793608] BTRFS: read error corrected: ino 1 off 610885632 (dev
> /dev/sdd1 sector 1209520)
> [29691.793797] BTRFS: read error corrected: ino 1 off 610889728 (dev
> /dev/sdd1 sector 1209528)
> [34626.017914] BTRFS: sdd1 checksum verify failed on 737181696 wanted
> 15D7099D found B6A2A7A9 level 0
> [34626.022656] BTRFS: read error corrected: ino 1 off 737181696 (dev
> /dev/sdd1 sector 1456192)
> [34626.022867] BTRFS: read error corrected: ino 1 off 737185792 (dev
> /dev/sdd1 sector 1456200)
> [34626.023107] BTRFS: read error corrected: ino 1 off 737189888 (dev
> /dev/sdd1 sector 1456208)
> [34626.023314] BTRFS: read error corrected: ino 1 off 737193984 (dev
> /dev/sdd1 sector 1456216)
> [37057.349996] BTRFS: sdc1 checksum verify failed on 701792256 wanted
> A7BD5067 found 87EF0602 level 0
> [37057.424920] BTRFS: read error corrected: ino 1 off 701792256 (dev
> /dev/sdc1 sector 1387072)
> [37057.425178] BTRFS: read error corrected: ino 1 off 701796352 (dev
> /dev/sdc1 sector 1387080)
> [37057.450174] BTRFS: read error corrected: ino 1 off 701800448 (dev
> /dev/sdc1 sector 1387088)
> [37057.453476] BTRFS: read error corrected: ino 1 off 701804544 (dev
> /dev/sdc1 sector 1387096)
> [38283.714855] BTRFS: sdd1 checksum verify failed on 190169088 wanted
> 27D1E032 found 585B1651 level 0
> [38283.715349] BTRFS: sdd1 checksum verify failed on 190169088 wanted
> 27D1E032 found 585B1651 level 0
> [38283.724140] BTRFS: read error corrected: ino 1 off 190169088 (dev
> /dev/sdd1 sector 387808)
> [38283.724313] BTRFS: read error corrected: ino 1 off 190173184 (dev
> /dev/sdd1 sector 387816)
> [38283.724485] BTRFS: read error corrected: ino 1 off 190177280 (dev
> /dev/sdd1 sector 387824)
> [38283.724648] BTRFS: read error corrected: ino 1 off 190181376 (dev
> /dev/sdd1 sector 387832)
> [38385.874438] BTRFS: sdd1 checksum verify failed on 472825856 wanted
> 937078F5 found 7FCB4F87 level 0
> [38385.897113] BTRFS: read error corrected: ino 1 off 472825856 (dev
> /dev/sdd1 sector 939872)
> [38385.897336] BTRFS: read error corrected: ino 1 off 472829952 (dev
> /dev/sdd1 sector 939880)
> [38385.897567] BTRFS: read error corrected: ino 1 off 472834048 (dev
> /dev/sdd1 sector 939888)
> [38385.897790] BTRFS: read error corrected: ino 1 off 472838144 (dev
> /dev/sdd1 sector 939896)
> [39015.821873] BTRFS: sdc1 checksum verify failed on 443613184 wanted
> 302CD9DE found ED697366 level 0
> [39015.823523] BTRFS: read error corrected: ino 1 off 443613184 (dev
> /dev/sdc1 sector 882816)
> [39015.823755] BTRFS: read error corrected: ino 1 off 443617280 (dev
> /dev/sdc1 sector 882824)
> [39015.824015] BTRFS: read error corrected: ino 1 off 443621376 (dev
> /dev/sdc1 sector 882832)
> [39015.824273] BTRFS: read error corrected: ino 1 off 443625472 (dev
> /dev/sdc1 sector 882840)
> [41213.535718] BTRFS: sdd1 checksum verify failed on 397557760 wanted
> EBCB7405 found C1365A5E level 0
> [41213.579520] BTRFS: read error corrected: ino 1 off 397557760 (dev
> /dev/sdd1 sector 792864)
> [41213.579687] BTRFS: read error corrected: ino 1 off 397561856 (dev
> /dev/sdd1 sector 792872)
> [41213.579913] BTRFS: read error corrected: ino 1 off 397565952 (dev
> /dev/sdd1 sector 792880)
> [41213.580062] BTRFS: read error corrected: ino 1 off 397570048 (dev
> /dev/sdd1 sector 792888)
> [41535.669279] BTRFS: sdc1 checksum verify failed on 476348416 wanted
> A645D195 found 21AD02F8 level 0
> [41535.805928] BTRFS: read error corrected: ino 1 off 476348416 (dev
> /dev/sdc1 sector 946752)
> [41535.807125] BTRFS: read error corrected: ino 1 off 476352512 (dev
> /dev/sdc1 sector 946760)
> [41535.808775] BTRFS: read error corrected: ino 1 off 476356608 (dev
> /dev/sdc1 sector 946768)
> [41535.810561] BTRFS: read error corrected: ino 1 off 476360704 (dev
> /dev/sdc1 sector 946776)
> [42148.710341] BTRFS: sdc1 checksum verify failed on 807469056 wanted
> 8490C0D5 found FB751835 level 0
> [42148.771176] BTRFS: sdc1 checksum verify failed on 807469056 wanted
> 8490C0D5 found FB751835 level 0
> [42148.773574] BTRFS: read error corrected: ino 1 off 807469056 (dev
> /dev/sdc1 sector 1593472)
> [42148.780409] BTRFS: read error corrected: ino 1 off 807473152 (dev
> /dev/sdc1 sector 1593480)
> [42148.784868] BTRFS: read error corrected: ino 1 off 807477248 (dev
> /dev/sdc1 sector 1593488)
> [42148.785629] BTRFS: read error corrected: ino 1 off 807481344 (dev
> /dev/sdc1 sector 1593496)
> [42263.866339] BTRFS: sdc1 checksum verify failed on 1019822080 wanted
> EE99B968 found 42D4359B level 0
> [42263.867150] BTRFS: read error corrected: ino 1 off 1019822080 (dev
> /dev/sdc1 sector 2008224)
> [42263.867371] BTRFS: read error corrected: ino 1 off 1019826176 (dev
> /dev/sdc1 sector 2008232)
> [42263.867651] BTRFS: read error corrected: ino 1 off 1019830272 (dev
> /dev/sdc1 sector 2008240)
> [42263.867874] BTRFS: read error corrected: ino 1 off 1019834368 (dev
> /dev/sdc1 sector 2008248)
> [43510.239652] BTRFS: sdd1 checksum verify failed on 561528832 wanted
> 8E3D9918 found 462F00F8 level 0
> [43510.277730] BTRFS: sdd1 checksum verify failed on 561528832 wanted
> 8E3D9918 found 462F00F8 level 0
> [43510.284768] BTRFS: read error corrected: ino 1 off 561528832 (dev
> /dev/sdd1 sector 1113120)
> [43510.284951] BTRFS: read error corrected: ino 1 off 561532928 (dev
> /dev/sdd1 sector 1113128)
> [43510.285120] BTRFS: read error corrected: ino 1 off 561537024 (dev
> /dev/sdd1 sector 1113136)
> [43510.285292] BTRFS: read error corrected: ino 1 off 561541120 (dev
> /dev/sdd1 sector 1113144)
> [43665.199768] BTRFS: sdd1 checksum verify failed on 912867328 wanted
> 322FCD22 found E3B80F9F level 0
> [43665.204858] BTRFS: read error corrected: ino 1 off 912867328 (dev
> /dev/sdd1 sector 1799328)
> [43665.205055] BTRFS: read error corrected: ino 1 off 912871424 (dev
> /dev/sdd1 sector 1799336)
> [43665.205249] BTRFS: read error corrected: ino 1 off 912875520 (dev
> /dev/sdd1 sector 1799344)
> [43665.205431] BTRFS: read error corrected: ino 1 off 912879616 (dev
> /dev/sdd1 sector 1799352)
> [44830.868804] BTRFS: sdc1 checksum verify failed on 177061888 wanted
> 91211FC3 found 9ED5C83C level 0
> [44830.881691] BTRFS: read error corrected: ino 1 off 177061888 (dev
> /dev/sdc1 sector 362208)
> [44830.881890] BTRFS: read error corrected: ino 1 off 177065984 (dev
> /dev/sdc1 sector 362216)
> [44830.882090] BTRFS: read error corrected: ino 1 off 177070080 (dev
> /dev/sdc1 sector 362224)
> [44830.882271] BTRFS: read error corrected: ino 1 off 177074176 (dev
> /dev/sdc1 sector 362232)
> [45891.249508] BTRFS: sdd1 checksum verify failed on 738902016 wanted
> 39A87C68 found 26F7DDC0 level 0
> [45891.250573] BTRFS: sdd1 checksum verify failed on 738902016 wanted
> 39A87C68 found 26F7DDC0 level 0
> [45891.256723] BTRFS: read error corrected: ino 1 off 738902016 (dev
> /dev/sdd1 sector 1459552)
> [45891.256930] BTRFS: read error corrected: ino 1 off 738906112 (dev
> /dev/sdd1 sector 1459560)
> [45891.257149] BTRFS: read error corrected: ino 1 off 738910208 (dev
> /dev/sdd1 sector 1459568)
> [45891.257354] BTRFS: read error corrected: ino 1 off 738914304 (dev
> /dev/sdd1 sector 1459576)
> [46483.180506] BTRFS: sdd1 checksum verify failed on 629964800 wanted
> 4925982F found FDC8E201 level 0
> [46484.513724] BTRFS: sdd1 checksum verify failed on 629964800 wanted
> 4925982F found FDC8E201 level 0
> [46484.531887] BTRFS: read error corrected: ino 1 off 629964800 (dev
> /dev/sdd1 sector 1246784)
> [46484.532272] BTRFS: read error corrected: ino 1 off 629968896 (dev
> /dev/sdd1 sector 1246792)
> [46484.532435] BTRFS: read error corrected: ino 1 off 629972992 (dev
> /dev/sdd1 sector 1246800)
> [46484.532606] BTRFS: read error corrected: ino 1 off 629977088 (dev
> /dev/sdd1 sector 1246808)
> [47018.575521] BTRFS: sdd1 checksum verify failed on 1057505280 wanted
> 73DE9ACE found CCB9B94F level 0
> [47018.575961] BTRFS: sdd1 checksum verify failed on 1057505280 wanted
> 73DE9ACE found CCB9B94F level 0
> [47018.587562] BTRFS: read error corrected: ino 1 off 1057505280 (dev
> /dev/sdd1 sector 2081824)
> [47018.587727] BTRFS: read error corrected: ino 1 off 1057509376 (dev
> /dev/sdd1 sector 2081832)
> [47018.587985] BTRFS: read error corrected: ino 1 off 1057513472 (dev
> /dev/sdd1 sector 2081840)
> [47018.588171] BTRFS: read error corrected: ino 1 off 1057517568 (dev
> /dev/sdd1 sector 2081848)
> [47093.159882] BTRFS: sdd1 checksum verify failed on 355254272 wanted
> 6CCD79A6 found D82B8A63 level 0
> [47120.658821] BTRFS: sdd1 checksum verify failed on 355254272 wanted
> 6CCD79A6 found D82B8A63 level 0
> [47120.668197] BTRFS: read error corrected: ino 1 off 355254272 (dev
> /dev/sdd1 sector 710240)
> [47120.668385] BTRFS: read error corrected: ino 1 off 355258368 (dev
> /dev/sdd1 sector 710248)
> [47120.668537] BTRFS: read error corrected: ino 1 off 355262464 (dev
> /dev/sdd1 sector 710256)
> [47120.668707] BTRFS: read error corrected: ino 1 off 355266560 (dev
> /dev/sdd1 sector 710264)
> [47187.793809] BTRFS: sdd1 checksum verify failed on 610385920 wanted
> 8942E6D9 found 974441F7 level 0
> [47187.803132] BTRFS: read error corrected: ino 1 off 610385920 (dev
> /dev/sdd1 sector 1208544)
> [47187.803321] BTRFS: read error corrected: ino 1 off 610390016 (dev
> /dev/sdd1 sector 1208552)
> [47187.803500] BTRFS: read error corrected: ino 1 off 610394112 (dev
> /dev/sdd1 sector 1208560)
> [47187.804405] BTRFS: read error corrected: ino 1 off 610398208 (dev
> /dev/sdd1 sector 1208568)
> [47293.372124] BTRFS: sdd1 checksum verify failed on 30064640 wanted
> E532B4CB found 1F6BB1CF level 0
> [47293.392624] BTRFS: read error corrected: ino 1 off 30064640 (dev
> /dev/sdd1 sector 75104)
> [47293.392931] BTRFS: read error corrected: ino 1 off 30068736 (dev
> /dev/sdd1 sector 75112)
> [47293.393143] BTRFS: read error corrected: ino 1 off 30072832 (dev
> /dev/sdd1 sector 75120)
> [47293.393367] BTRFS: read error corrected: ino 1 off 30076928 (dev
> /dev/sdd1 sector 75128)
> [47415.432548] BTRFS: sdc1 checksum verify failed on 483311616 wanted
> 2337659C found FE72CF24 level 0
> [47415.806725] BTRFS: sdc1 checksum verify failed on 483311616 wanted
> 2337659C found FE72CF24 level 0
> [47415.816027] BTRFS: read error corrected: ino 1 off 483311616 (dev
> /dev/sdc1 sector 960352)
> [47415.816199] BTRFS: read error corrected: ino 1 off 483315712 (dev
> /dev/sdc1 sector 960360)
> [47415.816372] BTRFS: read error corrected: ino 1 off 483319808 (dev
> /dev/sdc1 sector 960368)
> [47415.816838] BTRFS: read error corrected: ino 1 off 483323904 (dev
> /dev/sdc1 sector 960376)
> [47593.204118] BTRFS: sdd1 checksum verify failed on 131497984 wanted
> C1B6312B found E5650C5B level 0
> [47593.212026] BTRFS: read error corrected: ino 1 off 131497984 (dev
> /dev/sdd1 sector 273216)
> [47593.212182] BTRFS: read error corrected: ino 1 off 131502080 (dev
> /dev/sdd1 sector 273224)
> [47593.212355] BTRFS: read error corrected: ino 1 off 131506176 (dev
> /dev/sdd1 sector 273232)
> [47593.212526] BTRFS: read error corrected: ino 1 off 131510272 (dev
> /dev/sdd1 sector 273240)
> [48026.762834] BTRFS: sdc1 checksum verify failed on 856752128 wanted
> 640450F found B927668E level 0
> [48026.802163] BTRFS: read error corrected: ino 1 off 856752128 (dev
> /dev/sdc1 sector 1689728)
> [48026.802810] BTRFS: read error corrected: ino 1 off 856756224 (dev
> /dev/sdc1 sector 1689736)
> [48026.803040] BTRFS: read error corrected: ino 1 off 856760320 (dev
> /dev/sdc1 sector 1689744)
> [48026.803240] BTRFS: read error corrected: ino 1 off 856764416 (dev
> /dev/sdc1 sector 1689752)
> [48157.352847] BTRFS: sdd1 checksum verify failed on 721502208 wanted
> F58E06B7 found 28CBAC0F level 0
> [48157.363809] BTRFS: read error corrected: ino 1 off 721502208 (dev
> /dev/sdd1 sector 1425568)
> [48157.364175] BTRFS: read error corrected: ino 1 off 721506304 (dev
> /dev/sdd1 sector 1425576)
> [48157.364341] BTRFS: read error corrected: ino 1 off 721510400 (dev
> /dev/sdd1 sector 1425584)
> [48157.364559] BTRFS: read error corrected: ino 1 off 721514496 (dev
> /dev/sdd1 sector 1425592)
> [48717.558596] BTRFS: sdd1 checksum verify failed on 425967616 wanted
> 8B488C01 found 52F6044C level 0
> [48717.576610] BTRFS: read error corrected: ino 1 off 425967616 (dev
> /dev/sdd1 sector 848352)
> [48717.576856] BTRFS: read error corrected: ino 1 off 425971712 (dev
> /dev/sdd1 sector 848360)
> [48717.577651] BTRFS: read error corrected: ino 1 off 425975808 (dev
> /dev/sdd1 sector 848368)
> [48717.577877] BTRFS: read error corrected: ino 1 off 425979904 (dev
> /dev/sdd1 sector 848376)
> [48740.429310] BTRFS: sdd1 checksum verify failed on 570064896 wanted
> 4B343BE0 found BD92455F level 0
> [48740.437713] BTRFS: read error corrected: ino 1 off 570064896 (dev
> /dev/sdd1 sector 1129792)
> [48740.437935] BTRFS: read error corrected: ino 1 off 570068992 (dev
> /dev/sdd1 sector 1129800)
> [48740.438098] BTRFS: read error corrected: ino 1 off 570073088 (dev
> /dev/sdd1 sector 1129808)
> [48740.438271] BTRFS: read error corrected: ino 1 off 570077184 (dev
> /dev/sdd1 sector 1129816)
> [48746.809311] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 6E0D3115 level 0
> [48746.810381] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 6E0D3115 level 0
> [48746.819699] BTRFS: sdd1 checksum verify failed on 588447744 wanted
> F90C810B found 24492BB3 level 0
> [48746.819733] ------------[ cut here ]------------
> [48746.819746] WARNING: CPU: 0 PID: 31802 at fs/btrfs/super.c:259
> __btrfs_abort_transaction+0x43/0x110()
> [48746.819747] BTRFS: Transaction aborted (error -5)
> [48746.819748] Modules linked in: iscsi_trgt(O)
> [48746.819755] CPU: 0 PID: 31802 Comm: kworker/u2:11 Tainted: G
> O 3.16.5-gentoo-20150103-0758 #1
> [48746.819757] Hardware name: /MS-6570, BIOS 6.00 PG 11/07/2003
> [48746.819762] Workqueue: btrfs-extent-refs btrfs_extent_refs_helper
> [48746.819764] c9a9fe74 c9a9fe74 c9a9fe34 c14d581f c9a9fe64 c10333d0
> c15bcea0 c9a9fe90
> [48746.819767] 00007c3a c15c9417 00000103 c11c8723 c11c8723 fffffffb
> f4c1a000 c14fd856
> [48746.819771] c9a9fe7c c103341e 00000009 c9a9fe74 c15bcea0 c9a9fe90
> c9a9fea8 c11c8723
> [48746.819775] Call Trace:
> [48746.819783] [<c14d581f>] dump_stack+0x16/0x18
> [48746.819787] [<c10333d0>] warn_slowpath_common+0x70/0x90
> [48746.819791] [<c11c8723>] ? __btrfs_abort_transaction+0x43/0x110
> [48746.819793] [<c11c8723>] ? __btrfs_abort_transaction+0x43/0x110
> [48746.819796] [<c103341e>] warn_slowpath_fmt+0x2e/0x30
> [48746.819799] [<c11c8723>] __btrfs_abort_transaction+0x43/0x110
> [48746.819805] [<c11e2cc4>] btrfs_run_delayed_refs.part.74+0xd4/0x1d0
> [48746.819808] [<c11e2e4d>] delayed_ref_async_start+0x6d/0x80
> [48746.819810] [<c1222558>] normal_work_helper+0x88/0x120
> [48746.819813] [<c12227db>] btrfs_extent_refs_helper+0xb/0x10
> [48746.819816] [<c1044081>] process_one_work+0xd1/0x300
> [48746.819819] [<c1044779>] worker_thread+0xe9/0x430
> [48746.819821] [<c1044690>] ? create_and_start_worker+0x40/0x40
> [48746.819823] [<c1044690>] ? create_and_start_worker+0x40/0x40
> [48746.819828] [<c1048cf5>] kthread+0x95/0xb0
> [48746.819831] [<c14da0c0>] ret_from_kernel_thread+0x20/0x30
> [48746.819834] [<c1048c60>] ? kthread_worker_fn+0xb0/0xb0
> [48746.819835] ---[ end trace 554ae9ecb6239dc6 ]---
> [48746.819838] BTRFS: error (device sdd1) in
> btrfs_run_delayed_refs:2806: errno=-5 IO failure
> [48746.819840] BTRFS info (device sdd1): forced readonly
^ permalink raw reply [flat|nested] 6+ messages in thread
end of thread, other threads:[~2015-01-08 15:02 UTC | newest]
Thread overview: 6+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2015-01-04 14:50 BTRFS: Transaction aborted (error -5) Dyweni - BTRFS
2015-01-04 23:24 ` Chris Murphy
2015-01-06 14:58 ` Dyweni - BTRFS
2015-01-06 15:04 ` Dyweni - BTRFS
2015-01-06 15:15 ` Dyweni - BTRFS
2015-01-08 15:02 ` Dyweni - BTRFS
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.