* Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
@ 2017-04-03 10:11 Robert Krig
2017-04-03 13:50 ` Robert Krig
2017-04-03 14:08 ` Hans van Kranenburg
0 siblings, 2 replies; 14+ messages in thread
From: Robert Krig @ 2017-04-03 10:11 UTC (permalink / raw)
To: linux-btrfs
Hi guys, I seem to have run into a spot of trouble with my btrfs partition.
I've got 4 x 8TB in a RAID1 BTRFS configuration.
I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs
progs version v4.7.3
Server has 8GB of Ram.
I was running duperemove using a hashfile, which seemed to have run out
space and aborted. Then I tried a balance operation, with -dusage
progressively set to 0 1 5 15 30 50, which then aborted, I presume that
this caused the fs to mount readonly. I only noticed it somewhat later.
I've since rebooted, and I can mount the filesystem OK, but after some
time (I presume caused by reads or writes) it once again switches to
readonly.
I tried unmounting/remounting again and running a scrub, but the scrub
aborts after some time.
Here is the output from the kernel when the partition crashes:
Apr 03 11:32:57 atlas kernel: BTRFS info (device sda): The free space
cache file (37732863967232) is invalid. skip it
Apr 03 11:33:46 atlas kernel: BTRFS critical (device sda): corrupt leaf,
slot offset bad: block=38666170826752, root=1, slot=157
Apr 03 11:33:46 atlas kernel: ------------[ cut here ]------------
Apr 03 11:33:46 atlas kernel: WARNING: CPU: 0 PID: 17810 at
/home/zumbi/linux-4.9.13/fs/btrfs/extent-tree.c:6961
__btrfs_free_extent.isra.69+0x152/0xd60 [b
Apr 03 11:33:46 atlas kernel: BTRFS: Transaction aborted (error -5)
Apr 03 11:33:46 atlas kernel: Modules linked in: xt_multiport
iptable_filter ip_tables x_tables binfmt_misc cpufreq_userspace
cpufreq_conservative cpufreq_
Apr 03 11:33:46 atlas kernel: ppdev lp parport autofs4 btrfs xor
raid6_pq dm_mod md_mod fuse sg sd_mod ahci libahci libata crc32c_intel
scsi_mod fan therm
Apr 03 11:33:46 atlas kernel: CPU: 0 PID: 17810 Comm: mc Not tainted
4.9.0-0.bpo.2-amd64 #1 Debian 4.9.13-1~bpo8+1
Apr 03 11:33:46 atlas kernel: Hardware name: ASUS All Series/H87M-E,
BIOS 0703 10/30/2013
Apr 03 11:33:46 atlas kernel: 0000000000000000 ffffffff97d29cd5
ffffb8ab4bb53a50 0000000000000000
Apr 03 11:33:46 atlas kernel: ffffffff97a778a4 0000154c080b2000
ffffb8ab4bb53aa8 ffff8908ad438b40
Apr 03 11:33:46 atlas kernel: ffff890951b96000 0000000000000000
ffff89086c3d4000 ffffffff97a7791f
Apr 03 11:33:46 atlas kernel: Call Trace:
Apr 03 11:33:46 atlas kernel: [<ffffffff97d29cd5>] ? dump_stack+0x5c/0x77
Apr 03 11:33:46 atlas kernel: [<ffffffff97a778a4>] ? __warn+0xc4/0xe0
Apr 03 11:33:46 atlas kernel: [<ffffffff97a7791f>] ?
warn_slowpath_fmt+0x5f/0x80
Apr 03 11:33:46 atlas kernel: [<ffffffffc05853e2>] ?
__btrfs_free_extent.isra.69+0x152/0xd60 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffffc0589d06>] ?
__btrfs_run_delayed_refs+0x466/0x1360 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffffc05ca364>] ?
set_extent_buffer_dirty+0x64/0xb0 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffffc058dacf>] ?
btrfs_run_delayed_refs+0x8f/0x2b0 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffffc05a314f>] ?
btrfs_should_end_transaction+0x3f/0x60 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffffc05b00fa>] ?
btrfs_truncate_inode_items+0x63a/0xde0 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffffc05b1852>] ?
btrfs_evict_inode+0x4a2/0x5f0 [btrfs]
Apr 03 11:33:46 atlas kernel: [<ffffffff97c1fa06>] ? evict+0xb6/0x180
Apr 03 11:33:46 atlas kernel: [<ffffffff97c13d98>] ?
do_unlinkat+0x148/0x300
Apr 03 11:33:46 atlas kernel: [<ffffffff97ffb3fb>] ?
system_call_fast_compare_end+0xc/0x9b
Apr 03 11:33:46 atlas kernel: ---[ end trace 2a45c2819ff7b785 ]---
Apr 03 11:33:46 atlas kernel: BTRFS: error (device sda) in
__btrfs_free_extent:6961: errno=-5 IO failure
Apr 03 11:33:46 atlas kernel: BTRFS info (device sda): forced readonly
Apr 03 11:33:46 atlas kernel: BTRFS: error (device sda) in
btrfs_run_delayed_refs:2967: errno=-5 IO failure
Apr 03 11:33:50 atlas kernel: BTRFS warning (device sda): failed setting
block group ro, ret=-30
Apr 03 11:33:50 atlas kernel: BTRFS warning (device sda): failed setting
block group ro, ret=-30
Apr 03 11:33:52 atlas kernel: BTRFS warning (device sda): failed setting
block group ro, ret=-30
Apr 03 11:33:53 atlas kernel: BTRFS warning (device sda): Skipping
commit of aborted transaction.
Apr 03 11:33:53 atlas kernel: BTRFS: error (device sda) in
cleanup_transaction:1850: errno=-5 IO failure
Apr 03 11:33:53 atlas kernel: BTRFS info (device sda): delayed_refs has
NO entry
Apr 03 11:33:54 atlas kernel: BTRFS warning (device sda): failed setting
block group ro, ret=-30
I tried running a btrfs-debug-tree -b 38666170826752 /dev/sda
btrfs-progs
v4.7.3
leaf 38666170826752 items 199 free space 1506 generation 1248226 owner
2
fs uuid
8c4f8e26-3442-463f-ad8a-668dfef02593
chunk uuid
1f04f64e-0ec8-4b39-83d9-a2df75179d3e
item 0 key (23416295448576 EXTENT_ITEM 36864) itemoff 16230
itemsize
53
extent refs 1 gen 671397 flags
DATA
extent data backref root 5 objectid 4959957 offset 0
count
1
item 1 key (23416295485440 EXTENT_ITEM 8192) itemoff 16177
itemsize
53
extent refs 1 gen 972749 flags
DATA
extent data backref root 5 objectid 7328099 offset 0
count
1
item 2 key (23416295493632 EXTENT_ITEM 12288) itemoff 16124
itemsize
53
extent refs 1 gen 797708 flags
DATA
extent data backref root 5 objectid 5842103 offset
1966080 count
1
item 3 key (23416295505920 EXTENT_ITEM 8192) itemoff 16071
itemsize
53
extent refs 1 gen 1244513 flags
DATA
extent data backref root 44107 objectid 28528 offset
974848 count
1
item 4 key (23416295514112 EXTENT_ITEM 8192) itemoff 16034
itemsize
37
extent refs 1 gen 625327 flags
DATA
shared data backref parent 38666872045568 count
1
item 5 key (23416295522304 EXTENT_ITEM 16384) itemoff 15997
itemsize
37
extent refs 1 gen 625327 flags
DATA
shared data backref parent 38666872045568 count
1
item 6 key (23416295538688 EXTENT_ITEM 49152) itemoff 15944
itemsize
53
extent refs 1 gen 585321 flags
DATA
extent data backref root 5 objectid 4742401 offset
393216 count
1
item 7 key (23416295587840 EXTENT_ITEM 8192) itemoff 15907
itemsize
37
extent refs 1 gen 625327 flags
DATA
shared data backref parent 38666872045568 count
1
this goes on and on. I can provide the entire output if thats helpful.
Any ideas on what I could do to fix the partition? Is it fixable, or is
it a lost cause?
Thanks guys.
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 10:11 Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block" Robert Krig
@ 2017-04-03 13:50 ` Robert Krig
2017-04-03 14:09 ` Hans van Kranenburg
2017-04-03 14:08 ` Hans van Kranenburg
1 sibling, 1 reply; 14+ messages in thread
From: Robert Krig @ 2017-04-03 13:50 UTC (permalink / raw)
To: linux-btrfs
On 03.04.2017 12:11, Robert Krig wrote:
> Hi guys, I seem to have run into a spot of trouble with my btrfs partition.
>
> I've got 4 x 8TB in a RAID1 BTRFS configuration.
>
> I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs
> progs version v4.7.3
>
> Server has 8GB of Ram.
>
>
> I was running duperemove using a hashfile, which seemed to have run out
> space and aborted. Then I tried a balance operation, with -dusage
> progressively set to 0 1 5 15 30 50, which then aborted, I presume that
> this caused the fs to mount readonly. I only noticed it somewhat later.
>
> I've since rebooted, and I can mount the filesystem OK, but after some
> time (I presume caused by reads or writes) it once again switches to
> readonly.
>
> I tried unmounting/remounting again and running a scrub, but the scrub
> aborts after some time.
>
>
I've compiled the newest btrfs-tools version 4.10.2
This is what I get when running a btrfsck -p /dev/sda
hecking filesystem on
/dev/sda
UUID:
8c4f8e26-3442-463f-ad8a-668dfef02593
incorrect offsets 8590
1258314415
bad block
38666170826752
ERROR: errors found in extent allocation tree or chunk
allocation
Speicherzugriffsfehler
For the non-german speakers: Speicherzugriffsfehler = Memory Access Error
Dmesg shows this:
Apr 03 15:47:05 atlas kernel: btrfs[9140]: segfault at 9476b99e ip
000000000044c459 sp 00007fff556b4b10 error 4 in
btrfs[400000+9d000]
^ permalink raw reply [flat|nested] 14+ messages in thread* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 13:50 ` Robert Krig
@ 2017-04-03 14:09 ` Hans van Kranenburg
0 siblings, 0 replies; 14+ messages in thread
From: Hans van Kranenburg @ 2017-04-03 14:09 UTC (permalink / raw)
To: Robert Krig, linux-btrfs
On 04/03/2017 03:50 PM, Robert Krig wrote:
>
>
> On 03.04.2017 12:11, Robert Krig wrote:
>> Hi guys, I seem to have run into a spot of trouble with my btrfs partition.
>>
>> I've got 4 x 8TB in a RAID1 BTRFS configuration.
>>
>> I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs
>> progs version v4.7.3
>>
>> Server has 8GB of Ram.
>>
>>
>> I was running duperemove using a hashfile, which seemed to have run out
>> space and aborted. Then I tried a balance operation, with -dusage
>> progressively set to 0 1 5 15 30 50, which then aborted, I presume that
>> this caused the fs to mount readonly. I only noticed it somewhat later.
>>
>> I've since rebooted, and I can mount the filesystem OK, but after some
>> time (I presume caused by reads or writes) it once again switches to
>> readonly.
>>
>> I tried unmounting/remounting again and running a scrub, but the scrub
>> aborts after some time.
>>
>>
>
>
> I've compiled the newest btrfs-tools version 4.10.2
>
> This is what I get when running a btrfsck -p /dev/sda
>
> hecking filesystem on
> /dev/sda
>
> UUID:
> 8c4f8e26-3442-463f-ad8a-668dfef02593
>
> incorrect offsets 8590
> 1258314415
>
> bad block
> 38666170826752
>
>
>
> ERROR: errors found in extent allocation tree or chunk
> allocation
> Speicherzugriffsfehler
>
> For the non-german speakers: Speicherzugriffsfehler = Memory Access Error
>
> Dmesg shows this:
>
> Apr 03 15:47:05 atlas kernel: btrfs[9140]: segfault at 9476b99e ip
> 000000000044c459 sp 00007fff556b4b10 error 4 in
> btrfs[400000+9d000]
That's probably because the tool does not verify if the numbers in the
fields make sense before using them.
--
Hans van Kranenburg
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 10:11 Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block" Robert Krig
2017-04-03 13:50 ` Robert Krig
@ 2017-04-03 14:08 ` Hans van Kranenburg
2017-04-03 14:20 ` Robert Krig
1 sibling, 1 reply; 14+ messages in thread
From: Hans van Kranenburg @ 2017-04-03 14:08 UTC (permalink / raw)
To: Robert Krig, linux-btrfs
On 04/03/2017 12:11 PM, Robert Krig wrote:
> Hi guys, I seem to have run into a spot of trouble with my btrfs partition.
>
> I've got 4 x 8TB in a RAID1 BTRFS configuration.
>
> I'm running Debian Jessie 64 Bit, 4.9.0-0.bpo.2-amd64 kernel. Btrfs
> progs version v4.7.3
>
> Server has 8GB of Ram.
>
>
> I was running duperemove using a hashfile, which seemed to have run out
> space and aborted. Then I tried a balance operation, with -dusage
> progressively set to 0 1 5 15 30 50, which then aborted, I presume that
> this caused the fs to mount readonly. I only noticed it somewhat later.
The balance probably did not cause the issue, but it ran across the
invalid metadata page, while digging around in the filesyste and then
choked on it.
> I've since rebooted, and I can mount the filesystem OK, but after some
> time (I presume caused by reads or writes) it once again switches to
> readonly.
>
> I tried unmounting/remounting again and running a scrub, but the scrub
> aborts after some time.
>
>
> Here is the output from the kernel when the partition crashes:
>
> Apr 03 11:32:57 atlas kernel: BTRFS info (device sda): The free space
> cache file (37732863967232) is invalid. skip it
> Apr 03 11:33:46 atlas kernel: BTRFS critical (device sda): corrupt leaf,
> slot offset bad: block=38666170826752, root=1, slot=157
> [...]
Note: The root=1 is a lie? Looking at the output of btrfs-debug-tree
below, this is definitely a tree block of tree 2, not 1. I have seen
this more often, but not looked at the code yet. Maybe some bug in
assembling the error message?
> I tried running a btrfs-debug-tree -b 38666170826752 /dev/sda
>
> btrfs-progs
> v4.7.3
>
> leaf 38666170826752 items 199 free space 1506 generation 1248226 owner
> 2
>
> fs uuid
> 8c4f8e26-3442-463f-ad8a-668dfef02593
>
> chunk uuid
> 1f04f64e-0ec8-4b39-83d9-a2df75179d3e
>
> item 0 key (23416295448576 EXTENT_ITEM 36864) itemoff 16230
> itemsize
> 53
>
> extent refs 1 gen 671397 flags
> DATA
>
> extent data backref root 5 objectid 4959957 offset 0
> count
> 1
>
> [...]
The corruption is at item 157. Can you attach all of the output, or
pastebin it?
> this goes on and on. I can provide the entire output if thats helpful.
Yes. The corruption is in item 157, and then from the point of the
itemoff value. This is the offset of the item data in the metadata page.
See https://btrfs.wiki.kernel.org/index.php/On-disk_Format#Leaf_Node
> Any ideas on what I could do to fix the partition? Is it fixable, or is
> it a lost cause?
Memory corruption, not on disk corruption.
So, either a bitflip, or garbage which ended up on this memory location
for whatever reason or a bug in whatever part of the kernel, a pointer
in another module gone wonky, etc, which we might learn more about after
seeing more of the output.
--
Hans van Kranenburg
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 14:08 ` Hans van Kranenburg
@ 2017-04-03 14:20 ` Robert Krig
2017-04-03 14:25 ` Robert Krig
2017-04-03 14:44 ` Hans van Kranenburg
0 siblings, 2 replies; 14+ messages in thread
From: Robert Krig @ 2017-04-03 14:20 UTC (permalink / raw)
To: Hans van Kranenburg, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 362 bytes --]
On 03.04.2017 16:08, Hans van Kranenburg wrote:
> On 04/03/2017 12:11 PM, Robert Krig wrote:
> The corruption is at item 157. Can you attach all of the output, or
> pastebin it?
>
I've attached the entire log of btrfs-debug-tree. This was generated
with btrfs-progs 4.7.3
If it makes a difference, I can try it again with the newest version of
btrfs-progs?
[-- Attachment #2: tree.log --]
[-- Type: text/plain, Size: 27959 bytes --]
btrfs-progs v4.7.3
leaf 38666170826752 items 199 free space 1506 generation 1248226 owner 2
fs uuid 8c4f8e26-3442-463f-ad8a-668dfef02593
chunk uuid 1f04f64e-0ec8-4b39-83d9-a2df75179d3e
item 0 key (23416295448576 EXTENT_ITEM 36864) itemoff 16230 itemsize 53
extent refs 1 gen 671397 flags DATA
extent data backref root 5 objectid 4959957 offset 0 count 1
item 1 key (23416295485440 EXTENT_ITEM 8192) itemoff 16177 itemsize 53
extent refs 1 gen 972749 flags DATA
extent data backref root 5 objectid 7328099 offset 0 count 1
item 2 key (23416295493632 EXTENT_ITEM 12288) itemoff 16124 itemsize 53
extent refs 1 gen 797708 flags DATA
extent data backref root 5 objectid 5842103 offset 1966080 count 1
item 3 key (23416295505920 EXTENT_ITEM 8192) itemoff 16071 itemsize 53
extent refs 1 gen 1244513 flags DATA
extent data backref root 44107 objectid 28528 offset 974848 count 1
item 4 key (23416295514112 EXTENT_ITEM 8192) itemoff 16034 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 5 key (23416295522304 EXTENT_ITEM 16384) itemoff 15997 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 6 key (23416295538688 EXTENT_ITEM 49152) itemoff 15944 itemsize 53
extent refs 1 gen 585321 flags DATA
extent data backref root 5 objectid 4742401 offset 393216 count 1
item 7 key (23416295587840 EXTENT_ITEM 8192) itemoff 15907 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 8 key (23416295596032 EXTENT_ITEM 4096) itemoff 15854 itemsize 53
extent refs 1 gen 625327 flags DATA
extent data backref root 5 objectid 1123021 offset 6029312 count 1
item 9 key (23416295600128 EXTENT_ITEM 4096) itemoff 15801 itemsize 53
extent refs 1 gen 975337 flags DATA
extent data backref root 5 objectid 7334929 offset 0 count 1
item 10 key (23416295604224 EXTENT_ITEM 57344) itemoff 15748 itemsize 53
extent refs 1 gen 572974 flags DATA
extent data backref root 5 objectid 4430156 offset 0 count 1
item 11 key (23416295661568 EXTENT_ITEM 106496) itemoff 15695 itemsize 53
extent refs 1 gen 585319 flags DATA
extent data backref root 5 objectid 4742398 offset 2490368 count 1
item 12 key (23416295768064 EXTENT_ITEM 4096) itemoff 15642 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5769382 offset 12288 count 1
item 13 key (23416295772160 EXTENT_ITEM 4096) itemoff 15589 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5769383 offset 4096 count 1
item 14 key (23416295776256 EXTENT_ITEM 4096) itemoff 15536 itemsize 53
extent refs 1 gen 585370 flags DATA
extent data backref root 5 objectid 4742594 offset 1310720 count 1
item 15 key (23416295780352 EXTENT_ITEM 8192) itemoff 15499 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 16 key (23416295788544 EXTENT_ITEM 151552) itemoff 15446 itemsize 53
extent refs 1 gen 992062 flags DATA
extent data backref root 5 objectid 7458028 offset 0 count 1
item 17 key (23416295940096 EXTENT_ITEM 4096) itemoff 15393 itemsize 53
extent refs 1 gen 1027477 flags DATA
extent data backref root 5 objectid 7508879 offset 4096 count 1
item 18 key (23416295944192 EXTENT_ITEM 4096) itemoff 15340 itemsize 53
extent refs 1 gen 1023977 flags DATA
extent data backref root 5 objectid 7496365 offset 20480 count 1
item 19 key (23416295948288 EXTENT_ITEM 36864) itemoff 15287 itemsize 53
extent refs 1 gen 516177 flags DATA
extent data backref root 5 objectid 3897818 offset 12976128 count 1
item 20 key (23416295985152 EXTENT_ITEM 45056) itemoff 15234 itemsize 53
extent refs 1 gen 444976 flags DATA
extent data backref root 5 objectid 3591929 offset 12320768 count 1
item 21 key (23416296030208 EXTENT_ITEM 8192) itemoff 15197 itemsize 37
extent refs 1 gen 516186 flags DATA
shared data backref parent 23313632870400 count 1
item 22 key (23416296038400 EXTENT_ITEM 524288) itemoff 15144 itemsize 53
extent refs 1 gen 444976 flags DATA
extent data backref root 5 objectid 3533012 offset 5242880 count 1
item 23 key (23416296562688 EXTENT_ITEM 4096) itemoff 15091 itemsize 53
extent refs 1 gen 1027477 flags DATA
extent data backref root 5 objectid 7508880 offset 4096 count 1
item 24 key (23416296566784 EXTENT_ITEM 4096) itemoff 15038 itemsize 53
extent refs 1 gen 1027477 flags DATA
extent data backref root 5 objectid 7508881 offset 4096 count 1
item 25 key (23416296570880 EXTENT_ITEM 4096) itemoff 14985 itemsize 53
extent refs 1 gen 993013 flags DATA
extent data backref root 5 objectid 4049898 offset 0 count 1
item 26 key (23416296574976 EXTENT_ITEM 69632) itemoff 14932 itemsize 53
extent refs 1 gen 625149 flags DATA
extent data backref root 17685 objectid 592 offset 4587520 count 1
item 27 key (23416296644608 EXTENT_ITEM 32768) itemoff 14879 itemsize 53
extent refs 1 gen 625279 flags DATA
extent data backref root 5 objectid 3808516 offset 2621440 count 1
item 28 key (23416296677376 EXTENT_ITEM 12288) itemoff 14842 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 29 key (23416296689664 EXTENT_ITEM 12288) itemoff 14805 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 30 key (23416296701952 EXTENT_ITEM 20480) itemoff 14752 itemsize 53
extent refs 1 gen 585364 flags DATA
extent data backref root 5 objectid 4742489 offset 0 count 1
item 31 key (23416296722432 EXTENT_ITEM 4096) itemoff 14699 itemsize 53
extent refs 1 gen 910568 flags DATA
extent data backref root 5 objectid 6957613 offset 32768 count 1
item 32 key (23416296726528 EXTENT_ITEM 4096) itemoff 14646 itemsize 53
extent refs 1 gen 828772 flags DATA
extent data backref root 5 objectid 6265461 offset 4096 count 1
item 33 key (23416296730624 EXTENT_ITEM 16384) itemoff 14609 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 34 key (23416296747008 EXTENT_ITEM 4096) itemoff 14556 itemsize 53
extent refs 1 gen 1244513 flags DATA
extent data backref root 44107 objectid 28528 offset 1138688 count 1
item 35 key (23416296751104 EXTENT_ITEM 4096) itemoff 14503 itemsize 53
extent refs 1 gen 585370 flags DATA
extent data backref root 5 objectid 4742594 offset 1441792 count 1
item 36 key (23416296755200 EXTENT_ITEM 4096) itemoff 14453 itemsize 50
extent refs 2 gen 624508 flags DATA
shared data backref parent 32429980205056 count 1
shared data backref parent 28836725522432 count 1
item 37 key (23416296759296 EXTENT_ITEM 4096) itemoff 14400 itemsize 53
extent refs 1 gen 583347 flags DATA
extent data backref root 5 objectid 4734798 offset 1179648 count 1
item 38 key (23416296763392 EXTENT_ITEM 4096) itemoff 14347 itemsize 53
extent refs 1 gen 1244513 flags DATA
extent data backref root 44107 objectid 28528 offset 1159168 count 1
item 39 key (23416296767488 EXTENT_ITEM 4096) itemoff 14294 itemsize 53
extent refs 1 gen 1244513 flags DATA
extent data backref root 44107 objectid 28528 offset 1171456 count 1
item 40 key (23416296771584 EXTENT_ITEM 4096) itemoff 14244 itemsize 50
extent refs 2 gen 629924 flags DATA
shared data backref parent 45571208593408 count 1
shared data backref parent 45571208511488 count 1
item 41 key (23416296775680 EXTENT_ITEM 8192) itemoff 14191 itemsize 53
extent refs 1 gen 1187993 flags DATA
extent data backref root 5 objectid 8476605 offset 0 count 1
item 42 key (23416296783872 EXTENT_ITEM 4096) itemoff 14138 itemsize 53
extent refs 1 gen 1151507 flags DATA
extent data backref root 5 objectid 8416212 offset 4096 count 1
item 43 key (23416296787968 EXTENT_ITEM 16384) itemoff 14101 itemsize 37
extent refs 1 gen 602054 flags DATA
shared data backref parent 28836268982272 count 1
item 44 key (23416296804352 EXTENT_ITEM 4096) itemoff 14019 itemsize 82
extent refs 2 gen 1240825 flags DATA
extent data backref root 44096 objectid 24899 offset 0 count 1
extent data backref root 44104 objectid 24899 offset 0 count 1
item 45 key (23416296808448 EXTENT_ITEM 32768) itemoff 13982 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 45570366078976 count 1
item 46 key (23416296841216 EXTENT_ITEM 40960) itemoff 13929 itemsize 53
extent refs 1 gen 624759 flags DATA
extent data backref root 5 objectid 748960 offset 0 count 1
item 47 key (23416296882176 EXTENT_ITEM 4096) itemoff 13892 itemsize 37
extent refs 1 gen 624759 flags DATA
shared data backref parent 25308725788672 count 1
item 48 key (23416296886272 EXTENT_ITEM 98304) itemoff 13839 itemsize 53
extent refs 1 gen 585320 flags DATA
extent data backref root 5 objectid 4742399 offset 262144 count 1
item 49 key (23416296984576 EXTENT_ITEM 94208) itemoff 13786 itemsize 53
extent refs 1 gen 585320 flags DATA
extent data backref root 5 objectid 4742400 offset 262144 count 1
item 50 key (23416297078784 EXTENT_ITEM 4096) itemoff 13733 itemsize 53
extent refs 1 gen 625327 flags DATA
extent data backref root 5 objectid 1123022 offset 3194880 count 1
item 51 key (23416297082880 EXTENT_ITEM 4096) itemoff 13680 itemsize 53
extent refs 1 gen 625327 flags DATA
extent data backref root 5 objectid 1123023 offset 6021120 count 1
item 52 key (23416297086976 EXTENT_ITEM 8192) itemoff 13643 itemsize 37
extent refs 1 gen 624827 flags DATA
shared data backref parent 38666806181888 count 1
item 53 key (23416297095168 EXTENT_ITEM 8192) itemoff 13590 itemsize 53
extent refs 1 gen 1244513 flags DATA
extent data backref root 44107 objectid 28528 offset 1228800 count 1
item 54 key (23416297103360 EXTENT_ITEM 8192) itemoff 13553 itemsize 37
extent refs 1 gen 602064 flags DATA
shared data backref parent 33638567919616 count 1
item 55 key (23416297111552 EXTENT_ITEM 8192) itemoff 13500 itemsize 53
extent refs 1 gen 910566 flags DATA
extent data backref root 5 objectid 6956267 offset 0 count 1
item 56 key (23416297119744 EXTENT_ITEM 8192) itemoff 13447 itemsize 53
extent refs 1 gen 956803 flags DATA
extent data backref root 5 objectid 7206718 offset 0 count 1
item 57 key (23416297127936 EXTENT_ITEM 12288) itemoff 13410 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 58 key (23416297140224 EXTENT_ITEM 8192) itemoff 13373 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32430505574400 count 1
item 59 key (23416297148416 EXTENT_ITEM 4096) itemoff 13320 itemsize 53
extent refs 1 gen 1246375 flags DATA
extent data backref root 5 objectid 8561372 offset 150167552 count 1
item 60 key (23416297152512 EXTENT_ITEM 49152) itemoff 13267 itemsize 53
extent refs 1 gen 585322 flags DATA
extent data backref root 5 objectid 4742402 offset 393216 count 1
item 61 key (23416297201664 EXTENT_ITEM 8192) itemoff 13214 itemsize 53
extent refs 1 gen 585370 flags DATA
extent data backref root 5 objectid 4742594 offset 131072 count 1
item 62 key (23416297209856 EXTENT_ITEM 4096) itemoff 13161 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5769384 offset 61440 count 1
item 63 key (23416297213952 EXTENT_ITEM 4096) itemoff 13108 itemsize 53
extent refs 1 gen 910568 flags DATA
extent data backref root 5 objectid 6957614 offset 57344 count 1
item 64 key (23416297218048 EXTENT_ITEM 28672) itemoff 13071 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 65 key (23416297246720 EXTENT_ITEM 8192) itemoff 13034 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32429648658432 count 1
item 66 key (23416297254912 EXTENT_ITEM 12288) itemoff 12981 itemsize 53
extent refs 1 gen 1244681 flags DATA
extent data backref root 44108 objectid 47426 offset 1232896 count 1
item 67 key (23416297267200 EXTENT_ITEM 4096) itemoff 12928 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 524288 count 1
item 68 key (23416297271296 EXTENT_ITEM 4096) itemoff 12875 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 655360 count 1
item 69 key (23416297275392 EXTENT_ITEM 8192) itemoff 12822 itemsize 53
extent refs 1 gen 1245205 flags DATA
extent data backref root 5 objectid 9180661 offset 389120 count 1
item 70 key (23416297283584 EXTENT_ITEM 4096) itemoff 12769 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 786432 count 1
item 71 key (23416297287680 EXTENT_ITEM 16384) itemoff 12732 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 72 key (23416297304064 EXTENT_ITEM 12288) itemoff 12695 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 73 key (23416297316352 EXTENT_ITEM 8192) itemoff 12658 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 33638161399808 count 1
item 74 key (23416297324544 EXTENT_ITEM 4096) itemoff 12605 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 917504 count 1
item 75 key (23416297328640 EXTENT_ITEM 4096) itemoff 12552 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1048576 count 1
item 76 key (23416297332736 EXTENT_ITEM 4096) itemoff 12499 itemsize 53
extent refs 1 gen 1246375 flags DATA
extent data backref root 5 objectid 8561372 offset 149889024 count 1
item 77 key (23416297336832 EXTENT_ITEM 20480) itemoff 12446 itemsize 53
extent refs 1 gen 797708 flags DATA
extent data backref root 5 objectid 5842102 offset 1179648 count 1
item 78 key (23416297357312 EXTENT_ITEM 4096) itemoff 12393 itemsize 53
extent refs 1 gen 826505 flags DATA
extent data backref root 5 objectid 6149831 offset 15597568 count 1
item 79 key (23416297361408 EXTENT_ITEM 4096) itemoff 12340 itemsize 53
extent refs 1 gen 826505 flags DATA
extent data backref root 5 objectid 6149831 offset 15859712 count 1
item 80 key (23416297365504 EXTENT_ITEM 4096) itemoff 12287 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866301 offset 110592 count 1
item 81 key (23416297369600 EXTENT_ITEM 4096) itemoff 12234 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866302 offset 163840 count 1
item 82 key (23416297373696 EXTENT_ITEM 4096) itemoff 12181 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866303 offset 98304 count 1
item 83 key (23416297377792 EXTENT_ITEM 4096) itemoff 12128 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1179648 count 1
item 84 key (23416297381888 EXTENT_ITEM 4096) itemoff 12075 itemsize 53
extent refs 1 gen 1077452 flags DATA
extent data backref root 5 objectid 7630809 offset 32768 count 1
item 85 key (23416297385984 EXTENT_ITEM 16384) itemoff 12022 itemsize 53
extent refs 1 gen 797708 flags DATA
extent data backref root 5 objectid 5842102 offset 1703936 count 1
item 86 key (23416297402368 EXTENT_ITEM 45056) itemoff 11969 itemsize 53
extent refs 1 gen 625327 flags DATA
extent data backref root 5 objectid 218629 offset 83623936 count 1
item 87 key (23416297447424 EXTENT_ITEM 45056) itemoff 11932 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 88 key (23416297492480 EXTENT_ITEM 16384) itemoff 11895 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 89 key (23416297508864 EXTENT_ITEM 81920) itemoff 11858 itemsize 37
extent refs 1 gen 624922 flags DATA
shared data backref parent 23316189806592 count 1
item 90 key (23416297590784 EXTENT_ITEM 12288) itemoff 11821 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 91 key (23416297607168 EXTENT_ITEM 4096) itemoff 11784 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 33638161399808 count 1
item 92 key (23416297611264 EXTENT_ITEM 4096) itemoff 11731 itemsize 53
extent refs 1 gen 604738 flags DATA
extent data backref root 5 objectid 156284 offset 0 count 1
item 93 key (23416297615360 EXTENT_ITEM 12288) itemoff 11694 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 94 key (23416297627648 EXTENT_ITEM 86016) itemoff 11641 itemsize 53
extent refs 1 gen 585320 flags DATA
extent data backref root 5 objectid 4742400 offset 1835008 count 1
item 95 key (23416297713664 EXTENT_ITEM 32768) itemoff 11588 itemsize 53
extent refs 1 gen 585345 flags DATA
extent data backref root 5 objectid 4742444 offset 1572864 count 1
item 96 key (23416297746432 EXTENT_ITEM 4096) itemoff 11535 itemsize 53
extent refs 1 gen 583891 flags DATA
extent data backref root 5 objectid 4735844 offset 1310720 count 1
item 97 key (23416297750528 EXTENT_ITEM 4096) itemoff 11482 itemsize 53
extent refs 1 gen 1077452 flags DATA
extent data backref root 5 objectid 7630813 offset 5951488 count 1
item 98 key (23416297754624 EXTENT_ITEM 16384) itemoff 11445 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 99 key (23416297771008 EXTENT_ITEM 4096) itemoff 11392 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866304 offset 114688 count 1
item 100 key (23416297775104 EXTENT_ITEM 4096) itemoff 11339 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866305 offset 110592 count 1
item 101 key (23416297779200 EXTENT_ITEM 4096) itemoff 11286 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866306 offset 98304 count 1
item 102 key (23416297783296 EXTENT_ITEM 4096) itemoff 11233 itemsize 53
extent refs 1 gen 583343 flags DATA
extent data backref root 5 objectid 4734792 offset 1179648 count 1
item 103 key (23416297787392 EXTENT_ITEM 4096) itemoff 11180 itemsize 53
extent refs 1 gen 910568 flags DATA
extent data backref root 5 objectid 6957615 offset 217088 count 1
item 104 key (23416297791488 EXTENT_ITEM 4096) itemoff 11143 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 105 key (23416297795584 EXTENT_ITEM 4096) itemoff 11106 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 106 key (23416297799680 EXTENT_ITEM 4096) itemoff 11053 itemsize 53
extent refs 1 gen 1246375 flags DATA
extent data backref root 5 objectid 8561372 offset 149909504 count 1
item 107 key (23416297803776 EXTENT_ITEM 4096) itemoff 11000 itemsize 53
extent refs 1 gen 583343 flags DATA
extent data backref root 5 objectid 4734792 offset 1310720 count 1
item 108 key (23416297807872 EXTENT_ITEM 4096) itemoff 10947 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1310720 count 1
item 109 key (23416297811968 EXTENT_ITEM 4096) itemoff 10894 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1441792 count 1
item 110 key (23416297816064 EXTENT_ITEM 49152) itemoff 10841 itemsize 53
extent refs 1 gen 585323 flags DATA
extent data backref root 5 objectid 4742403 offset 393216 count 1
item 111 key (23416297865216 EXTENT_ITEM 4096) itemoff 10788 itemsize 53
extent refs 1 gen 1246375 flags DATA
extent data backref root 5 objectid 8561372 offset 150163456 count 1
item 112 key (23416297869312 EXTENT_ITEM 4096) itemoff 10735 itemsize 53
extent refs 1 gen 585369 flags DATA
extent data backref root 5 objectid 4742571 offset 0 count 1
item 113 key (23416297873408 EXTENT_ITEM 45056) itemoff 10698 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 38666872045568 count 1
item 114 key (23416297918464 EXTENT_ITEM 12288) itemoff 10661 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32430574272512 count 1
item 115 key (23416297930752 EXTENT_ITEM 12288) itemoff 10608 itemsize 53
extent refs 1 gen 797708 flags DATA
extent data backref root 5 objectid 5842104 offset 1966080 count 1
item 116 key (23416297943040 EXTENT_ITEM 4096) itemoff 10555 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866307 offset 139264 count 1
item 117 key (23416297947136 EXTENT_ITEM 4096) itemoff 10502 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1572864 count 1
item 118 key (23416297951232 EXTENT_ITEM 4096) itemoff 10449 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1703936 count 1
item 119 key (23416297955328 EXTENT_ITEM 4096) itemoff 10396 itemsize 53
extent refs 1 gen 828772 flags DATA
extent data backref root 5 objectid 6265462 offset 4096 count 1
item 120 key (23416297959424 EXTENT_ITEM 12288) itemoff 10343 itemsize 53
extent refs 1 gen 663454 flags DATA
extent data backref root 5 objectid 4933595 offset 0 count 1
item 121 key (23416297971712 EXTENT_ITEM 49152) itemoff 10290 itemsize 53
extent refs 1 gen 585323 flags DATA
extent data backref root 5 objectid 4742404 offset 1966080 count 1
item 122 key (23416298020864 EXTENT_ITEM 4096) itemoff 10237 itemsize 53
extent refs 1 gen 585369 flags DATA
extent data backref root 5 objectid 4742571 offset 1048576 count 1
item 123 key (23416298024960 EXTENT_ITEM 4096) itemoff 10184 itemsize 53
extent refs 1 gen 585369 flags DATA
extent data backref root 5 objectid 4742571 offset 1179648 count 1
item 124 key (23416298029056 EXTENT_ITEM 20480) itemoff 10147 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 125 key (23416298049536 EXTENT_ITEM 12288) itemoff 10110 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32430574272512 count 1
item 126 key (23416298061824 EXTENT_ITEM 4096) itemoff 10057 itemsize 53
extent refs 1 gen 828772 flags DATA
extent data backref root 5 objectid 6265463 offset 4096 count 1
item 127 key (23416298065920 EXTENT_ITEM 8192) itemoff 10020 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 128 key (23416298074112 EXTENT_ITEM 4096) itemoff 9967 itemsize 53
extent refs 1 gen 910568 flags DATA
extent data backref root 5 objectid 6957616 offset 118784 count 1
item 129 key (23416298078208 EXTENT_ITEM 8192) itemoff 9930 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 130 key (23416298086400 EXTENT_ITEM 4096) itemoff 9877 itemsize 53
extent refs 1 gen 606107 flags DATA
extent data backref root 5 objectid 1125648 offset 311296 count 1
item 131 key (23416298090496 EXTENT_ITEM 12288) itemoff 9840 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32430102347776 count 1
item 132 key (23416298102784 EXTENT_ITEM 4096) itemoff 9787 itemsize 53
extent refs 1 gen 585369 flags DATA
extent data backref root 5 objectid 4742571 offset 1310720 count 1
item 133 key (23416298106880 EXTENT_ITEM 16384) itemoff 9750 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 134 key (23416298123264 EXTENT_ITEM 4096) itemoff 9697 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1835008 count 1
item 135 key (23416298127360 EXTENT_ITEM 4096) itemoff 9644 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 1966080 count 1
item 136 key (23416298131456 EXTENT_ITEM 45056) itemoff 9591 itemsize 53
extent refs 1 gen 585340 flags DATA
extent data backref root 5 objectid 4742428 offset 655360 count 1
item 137 key (23416298176512 EXTENT_ITEM 12288) itemoff 9538 itemsize 53
extent refs 1 gen 585369 flags DATA
extent data backref root 5 objectid 4742571 offset 131072 count 1
item 138 key (23416298188800 EXTENT_ITEM 16384) itemoff 9501 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477099704320 count 1
item 139 key (23416298205184 EXTENT_ITEM 16384) itemoff 9464 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 140 key (23416298221568 EXTENT_ITEM 20480) itemoff 9411 itemsize 53
extent refs 1 gen 992064 flags DATA
extent data backref root 5 objectid 7459264 offset 0 count 1
item 141 key (23416298242048 EXTENT_ITEM 4096) itemoff 9374 itemsize 37
extent refs 1 gen 623391 flags DATA
shared data backref parent 38667183308800 count 1
item 142 key (23416298246144 EXTENT_ITEM 4096) itemoff 9321 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 2097152 count 1
item 143 key (23416298250240 EXTENT_ITEM 4096) itemoff 9268 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 2228224 count 1
item 144 key (23416298254336 EXTENT_ITEM 12288) itemoff 9231 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32430574272512 count 1
item 145 key (23416298266624 EXTENT_ITEM 4096) itemoff 9178 itemsize 53
extent refs 1 gen 910568 flags DATA
extent data backref root 5 objectid 6957617 offset 57344 count 1
item 146 key (23416298270720 EXTENT_ITEM 45056) itemoff 9125 itemsize 53
extent refs 1 gen 585340 flags DATA
extent data backref root 5 objectid 4742429 offset 524288 count 1
item 147 key (23416298315776 EXTENT_ITEM 12288) itemoff 9072 itemsize 53
extent refs 1 gen 585369 flags DATA
extent data backref root 5 objectid 4742571 offset 1441792 count 1
item 148 key (23416298328064 EXTENT_ITEM 8192) itemoff 9035 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 149 key (23416298336256 EXTENT_ITEM 4096) itemoff 8982 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866308 offset 143360 count 1
item 150 key (23416298340352 EXTENT_ITEM 4096) itemoff 8929 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866309 offset 86016 count 1
item 151 key (23416298344448 EXTENT_ITEM 4096) itemoff 8876 itemsize 53
extent refs 1 gen 800339 flags DATA
extent data backref root 5 objectid 5866310 offset 106496 count 1
item 152 key (23416298348544 EXTENT_ITEM 4096) itemoff 8823 itemsize 53
extent refs 1 gen 795227 flags DATA
extent data backref root 5 objectid 5768364 offset 2359296 count 1
item 153 key (23416298352640 EXTENT_ITEM 8192) itemoff 8786 itemsize 37
extent refs 1 gen 625327 flags DATA
shared data backref parent 32477101621248 count 1
item 154 key (23416298360832 EXTENT_ITEM 45056) itemoff 8733 itemsize 53
extent refs 1 gen 599414 flags DATA
extent data backref root 5 objectid 4723703 offset 0 count 1
item 155 key (23416298405888 EXTENT_ITEM 8192) itemoff 8696 itemsize 37
extent refs 1 gen 604205 flags DATA
shared data backref parent 33638575620096 count 1
item 156 key (23416298414080 EXTENT_ITEM 4096) itemoff 8643 itemsize 53
extent refs 1 gen 599985 flags DATA
extent data backref root 5 objectid 4050533 offset 12288 count 1
item 157 key (23416298418176 EXTENT_ITEM 4096) itemoff 8590 itemsize 53
extent refs 1 gen 910568 flags DATA
extent data backref root 5 objectid 6957618 offset 53248 count 1
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 14:20 ` Robert Krig
@ 2017-04-03 14:25 ` Robert Krig
2017-04-04 4:02 ` Robert Krig
2017-04-03 14:44 ` Hans van Kranenburg
1 sibling, 1 reply; 14+ messages in thread
From: Robert Krig @ 2017-04-03 14:25 UTC (permalink / raw)
To: Hans van Kranenburg, linux-btrfs
On 03.04.2017 16:20, Robert Krig wrote:
>
> On 03.04.2017 16:08, Hans van Kranenburg wrote:
>> On 04/03/2017 12:11 PM, Robert Krig wrote:
>> The corruption is at item 157. Can you attach all of the output, or
>> pastebin it?
>>
>
> I've attached the entire log of btrfs-debug-tree. This was generated
> with btrfs-progs 4.7.3
>
> If it makes a difference, I can try it again with the newest version of
> btrfs-progs?
I forgot to mention that btrfs-debug-tree also segfaults with a "memory
access error"
I'm gonna run a extensive memory check once I get home, since you
mentioned corrupt memory might be an issue here.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 14:25 ` Robert Krig
@ 2017-04-04 4:02 ` Robert Krig
2017-04-04 13:29 ` Brian B
2017-04-04 16:52 ` Chris Murphy
0 siblings, 2 replies; 14+ messages in thread
From: Robert Krig @ 2017-04-04 4:02 UTC (permalink / raw)
To: Hans van Kranenburg, linux-btrfs
On 03.04.2017 16:25, Robert Krig wrote:
>
> I'm gonna run a extensive memory check once I get home, since you
> mentioned corrupt memory might be an issue here.
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majordomo@vger.kernel.org
> More majordomo info at http://vger.kernel.org/majordomo-info.html
I ran a memtest over a couple of hours with no errors. Ram seems to be
fine so far.
I've looked at the link you provided. Frankly it looks very scary. (At
least to me it does)
But I've just thought of something else.
My storage array is BTRFS Raid1 with 4x8TB Drives.
Wouldn't it be possible to simply disconnect two of those drives, mount
with -o degraded and still have access (even if read-only) to all my data?
E.g. I could use the two removed drives as a backup and rebuild my array
from there. Since I'm kind of playing with the idea of turning it into a
MD RAID5 and only use btrfs on specific lvm volumes which need it.
The one thing that slightly worries me with this idea is, I don't know
if there is a way to tell which datablocks are on which drives. If I've
understood btrfs raid1 correctly it simply ensures that there is at
least a copy of each block on a different device.
Would my idea work? Or could it be that I can only safely remove one
drive, since the other drives might contain blocks from any of the other
drives?
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-04 4:02 ` Robert Krig
@ 2017-04-04 13:29 ` Brian B
2017-04-04 13:48 ` Hugo Mills
2017-04-04 13:52 ` Austin S. Hemmelgarn
2017-04-04 16:52 ` Chris Murphy
1 sibling, 2 replies; 14+ messages in thread
From: Brian B @ 2017-04-04 13:29 UTC (permalink / raw)
To: Robert Krig, Hans van Kranenburg, linux-btrfs
[-- Attachment #1.1: Type: text/plain, Size: 663 bytes --]
On 04/04/2017 12:02 AM, Robert Krig wrote:
> My storage array is BTRFS Raid1 with 4x8TB Drives.
> Wouldn't it be possible to simply disconnect two of those drives, mount
> with -o degraded and still have access (even if read-only) to all my data?
Just jumping on this point: my understanding of BTRFS "RAID1" is that
each file (block?) is randomly assigned to two disks of the array (no
matter how many disks are in the array). So if you remove two disks,
you will probably have files that were "assigned" to both of those
disks, and will be missing.
In short, you can't remove more than one disk of a BTRFS RAID1 and still
have all of your data.
[-- Attachment #2: OpenPGP digital signature --]
[-- Type: application/pgp-signature, Size: 801 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-04 13:29 ` Brian B
@ 2017-04-04 13:48 ` Hugo Mills
2017-04-04 13:52 ` Austin S. Hemmelgarn
1 sibling, 0 replies; 14+ messages in thread
From: Hugo Mills @ 2017-04-04 13:48 UTC (permalink / raw)
To: Brian B; +Cc: Robert Krig, Hans van Kranenburg, linux-btrfs
[-- Attachment #1: Type: text/plain, Size: 1153 bytes --]
On Tue, Apr 04, 2017 at 09:29:11AM -0400, Brian B wrote:
> On 04/04/2017 12:02 AM, Robert Krig wrote:
> > My storage array is BTRFS Raid1 with 4x8TB Drives.
> > Wouldn't it be possible to simply disconnect two of those drives, mount
> > with -o degraded and still have access (even if read-only) to all my data?
> Just jumping on this point: my understanding of BTRFS "RAID1" is that
> each file (block?) is randomly assigned to two disks of the array (no
Arbitrarily assigned, rather than randomly assigned (there is a
deterministic algorithm for it, but it's wise not to rely on the exact
behaviour of that algorithm, because there are a number of factors
that can alter its behaviour).
> matter how many disks are in the array). So if you remove two disks,
> you will probably have files that were "assigned" to both of those
> disks, and will be missing.
>
> In short, you can't remove more than one disk of a BTRFS RAID1 and still
> have all of your data.
Indeed.
Hugo.
--
Hugo Mills | Some days, it's just not worth gnawing through the
hugo@... carfax.org.uk | straps
http://carfax.org.uk/ |
PGP: E2AB1DE4 |
[-- Attachment #2: Digital signature --]
[-- Type: application/pgp-signature, Size: 836 bytes --]
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-04 13:29 ` Brian B
2017-04-04 13:48 ` Hugo Mills
@ 2017-04-04 13:52 ` Austin S. Hemmelgarn
1 sibling, 0 replies; 14+ messages in thread
From: Austin S. Hemmelgarn @ 2017-04-04 13:52 UTC (permalink / raw)
To: Brian B, Robert Krig, Hans van Kranenburg, linux-btrfs
On 2017-04-04 09:29, Brian B wrote:
> On 04/04/2017 12:02 AM, Robert Krig wrote:
>> My storage array is BTRFS Raid1 with 4x8TB Drives.
>> Wouldn't it be possible to simply disconnect two of those drives, mount
>> with -o degraded and still have access (even if read-only) to all my data?
> Just jumping on this point: my understanding of BTRFS "RAID1" is that
> each file (block?) is randomly assigned to two disks of the array (no
> matter how many disks are in the array). So if you remove two disks,
> you will probably have files that were "assigned" to both of those
> disks, and will be missing.
>
> In short, you can't remove more than one disk of a BTRFS RAID1 and still
> have all of your data.
>
That understanding is correct. From a functional perspective, BTRFS
raid1 is currently a RAID10 implementation with striping happening at a
very large granularity.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-04 4:02 ` Robert Krig
2017-04-04 13:29 ` Brian B
@ 2017-04-04 16:52 ` Chris Murphy
2017-04-04 16:55 ` Chris Murphy
1 sibling, 1 reply; 14+ messages in thread
From: Chris Murphy @ 2017-04-04 16:52 UTC (permalink / raw)
To: Robert Krig; +Cc: Hans van Kranenburg, Btrfs BTRFS
On Mon, Apr 3, 2017 at 10:02 PM, Robert Krig
<robert.krig@render-wahnsinn.de> wrote:
>
>
> On 03.04.2017 16:25, Robert Krig wrote:
>>
>> I'm gonna run a extensive memory check once I get home, since you
>> mentioned corrupt memory might be an issue here.
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majordomo@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>
>
> I ran a memtest over a couple of hours with no errors. Ram seems to be
> fine so far.
Inconclusive. A memtest can take days to expose a problem, and even
that's not conclusive. The list archive has some examples of where
memory testers gave RAM a pass, but doing things like compiling the
kernel would fail.
>
> I've looked at the link you provided. Frankly it looks very scary. (At
> least to me it does)
> But I've just thought of something else.
>
> My storage array is BTRFS Raid1 with 4x8TB Drives.
> Wouldn't it be possible to simply disconnect two of those drives, mount
> with -o degraded and still have access (even if read-only) to all my data?
man mkfs.btrfs
Btrfs raid1 supports only one device missing, no matter how many drives.
Mounting -o ro,degraded is probably permitted by the file system, but
chunks of the file system and certainly your data, will be missing. So
it's just a matter of time before copying data off will fail.
I suggest trying -o ro with all drives, not a degraded mount, and
copying data off. Any failures should be logged. Metadata errors are
logged without paths, whereas data corruption included path to the
affected file. This is easier than scraping the file system with btrfs
restore.
If you can't mount ro with all drives, or ro,degraded with just one
device missing, you'll need to use btrfs restore which is more
tolerant of missing metadata.
--
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-04 16:52 ` Chris Murphy
@ 2017-04-04 16:55 ` Chris Murphy
2017-04-05 6:07 ` Robert Krig
0 siblings, 1 reply; 14+ messages in thread
From: Chris Murphy @ 2017-04-04 16:55 UTC (permalink / raw)
To: Chris Murphy; +Cc: Robert Krig, Hans van Kranenburg, Btrfs BTRFS
On Tue, Apr 4, 2017 at 10:52 AM, Chris Murphy <lists@colorremedies.com> wrote:
> Mounting -o ro,degraded is probably permitted by the file system, but
> chunks of the file system and certainly your data, will be missing. So
> it's just a matter of time before copying data off will fail.
** Context here is, more than 1 device missing.
--
Chris Murphy
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-04 16:55 ` Chris Murphy
@ 2017-04-05 6:07 ` Robert Krig
0 siblings, 0 replies; 14+ messages in thread
From: Robert Krig @ 2017-04-05 6:07 UTC (permalink / raw)
Cc: Btrfs BTRFS
On 04.04.2017 18:55, Chris Murphy wrote:
> On Tue, Apr 4, 2017 at 10:52 AM, Chris Murphy <lists@colorremedies.com> wrote:
>
>
>> Mounting -o ro,degraded is probably permitted by the file system, but
>> chunks of the file system and certainly your data, will be missing. So
>> it's just a matter of time before copying data off will fail.
> ** Context here is, more than 1 device missing.
>
Thanks you guys for all your help and input.
I've ordered two new drives to backup all my data. I have a cloud backup
in place, but 13TB takes a while to upload :-)
I think I'm gonna abandon btrfs as the main fs for my home server. I'm
just gonna set up a separate LVM volume for storing snapshots and
backups, since I use btrfs on all my single disk machines.
Thanks again everyone.
^ permalink raw reply [flat|nested] 14+ messages in thread
* Re: Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block"
2017-04-03 14:20 ` Robert Krig
2017-04-03 14:25 ` Robert Krig
@ 2017-04-03 14:44 ` Hans van Kranenburg
1 sibling, 0 replies; 14+ messages in thread
From: Hans van Kranenburg @ 2017-04-03 14:44 UTC (permalink / raw)
To: Robert Krig, linux-btrfs
On 04/03/2017 04:20 PM, Robert Krig wrote:
>
>
> On 03.04.2017 16:08, Hans van Kranenburg wrote:
>> On 04/03/2017 12:11 PM, Robert Krig wrote:
>> The corruption is at item 157. Can you attach all of the output, or
>> pastebin it?
>>
>
> I've attached the entire log of btrfs-debug-tree. This was generated
> with btrfs-progs 4.7.3
Meuh,
item 156 key (23416298414080 EXTENT_ITEM 4096) itemoff 8643 itemsize 53
item 157 key (23416298418176 EXTENT_ITEM 4096) itemoff 8590 itemsize 53
8590 + 53 = 8643.
I don't get what's invalid about that.
"incorrect offsets 8590 1258314415"
if (btrfs_item_offset_nr(buf, i) !=
btrfs_item_end_nr(buf, i + 1)) {
ret = BTRFS_TREE_BLOCK_INVALID_OFFSETS;
fprintf(stderr, "incorrect offsets %u %u\n",
btrfs_item_offset_nr(buf, i),
btrfs_item_end_nr(buf, i + 1));
goto fail;
}
Ah, ok, so the corruption is in item 158, but it's reported as
corruption in item 157.
There's no really simple tool right now to fix this manually. We can
also try to dd 16kiB of metadata from disk, fix it, and write it back.
We've been doing that before, it's a bit of work, but it can succeed.
Here's more instructions:
https://www.spinics.net/lists/linux-btrfs/msg62459.html
So, if you're the adventurous type...
But then again, if this is really memory failure, there might be other
errors all around the fs, which you didn't hit while reading back the
data yet.
Also note that btrfs does not protect you against this, also not for
data in files that gets corrupted in memory before it's written out
(which contains the checksum step).
> If it makes a difference, I can try it again with the newest version of
> btrfs-progs?
No, that code hasn't been touched in over 5 years.
--
Hans van Kranenburg
^ permalink raw reply [flat|nested] 14+ messages in thread
end of thread, other threads:[~2017-04-05 6:07 UTC | newest]
Thread overview: 14+ messages (download: mbox.gz / follow: Atom feed)
-- links below jump to the message on this page --
2017-04-03 10:11 Need some help: "BTRFS critical (device sda): corrupt leaf, slot offset bad: block" Robert Krig
2017-04-03 13:50 ` Robert Krig
2017-04-03 14:09 ` Hans van Kranenburg
2017-04-03 14:08 ` Hans van Kranenburg
2017-04-03 14:20 ` Robert Krig
2017-04-03 14:25 ` Robert Krig
2017-04-04 4:02 ` Robert Krig
2017-04-04 13:29 ` Brian B
2017-04-04 13:48 ` Hugo Mills
2017-04-04 13:52 ` Austin S. Hemmelgarn
2017-04-04 16:52 ` Chris Murphy
2017-04-04 16:55 ` Chris Murphy
2017-04-05 6:07 ` Robert Krig
2017-04-03 14:44 ` Hans van Kranenburg
This is an external index of several public inboxes,
see mirroring instructions on how to clone and mirror
all data and code used by this external index.